Evaluating Multilingual, Context-Aware Guardrails: Evidence from a Humanitarian LLM Use Case
Summary
This technical post evaluates multilingual, context-aware guardrails in humanitarian LLM deployments, comparing English and Farsi responses under identical policies. Using the any-guardrail framework with FlowJudge, Glider, and AnyLLM across 60 scenarios (30 English, 30 Farsi translations), the authors reveal language- and model-dependent scoring gaps, hallucinations in guardrail reasoning, and actionable recommendations for improving multilingual safety through search capabilities, repeated evaluations, and language-aware policies.