arXiv
Cultural localization finds country labels flatten social advice
A country label can make an AI sound local while its social advice is actually a bland global average. The authors compared four language models with human ratings of 150 everyday scenarios across societies. In a São Paulo clinic waiting room, a digital receptionist could adjust advice to local judgments if models are tested against representative local evidence.
An assistant can often tell whether an action seems broadly acceptable. What it cannot reliably do is explain how that judgment changes from one society to another. A chatbot may be given a country name and still offer advice that sounds local only because it has smoothed many different views into one safe average. For someone asking about eye contact, conversational distance, or another ordinary interaction, that average can be reassuring and wrong.
What cultural advice depends on
Everyday norms are not fixed rules. The same act can be acceptable in one setting and inappropriate in another. Large language models learn patterns from text, however, rather than watching representative populations judge behavior directly. Translation can change an answer without making it agree with the people who speak that language. Naming a country can create the appearance of local knowledge while leaving the model dependent on broad or stereotyped associations.
To measure this gap, the authors compared four language models with human ratings of 150 everyday scenarios across 90 societies. The human benchmark came from 25,422 people surveyed between July 2023 and May 2024. These were convenience samples, not nationally representative ones, so the benchmark is not a perfect map of any society. It is still a way to ask a more precise question than whether a model understands etiquette in general: does it reproduce the differences between societies?
When a country label is not enough
The answer was often no. The four models treated societies as much more alike than the survey data did. Their estimated differences between societies were less than half the size measured in the human ratings, even when the models were directly told which society to consider.
The models also struggled to match the pattern of local answers. Their average scenario-level cross-cultural correlations with the survey ratings ranged from 0.23 to 0.33. Depending on the model, between 49% and 66% of scenarios had a correlation below 0.30. A correlation here is a measure of whether two sets of answers rise and fall together: a low value means the model often missed which societies judged an action more or less acceptable.
The difficulty was not the same for every situation. The models distinguished societies relatively better for kissing and flirting than for ordinary behaviors such as talking or reading a newspaper. That matters because cultural adaptation is not only about dramatic or obviously sensitive topics. It also concerns the small actions a system may comment on every day.
Prompting the models in local survey languages improved accuracy modestly, but it did not remove the compression of cultural differences. When no society was named, the model’s unconditioned profile was closest to its United States profile in three of the four models, and second closest in the fourth. A neutral-sounding answer was therefore not necessarily a worldwide answer. It often leaned toward one society while hiding that fact.
What locally tested AI would look like
If this result holds, a digital receptionist in a clinic waiting room in São Paulo would do more than switch languages. It could give advice about eye contact, conversational distance, and acceptable behavior based on local judgments, because those judgments had been used to test the system rather than replaced by a national label.
That future requires representative local data and models able to reproduce differences between societies instead of smoothing them away. It also gives developers a clearer test for claims of cultural sensitivity: not whether a model responds in Portuguese or mentions Brazil, but whether its advice follows the behavior-level evidence of the people it is meant to serve.
Until that happens, systems that advise, teach, moderate, or interact socially across borders need human and local validation for everyday-norm judgments. A country name and a translated prompt are not proof that social advice is locally reliable.