arXiv
Mental-health chatbots rank stigma without measuring it
A mental-health chatbot can reproduce a ranking about stigma without measuring whether that ranking matches people’s real experiences. The manuscript compared GPT-4 and GPT-5 mini with responses from 76,406 US postsecondary students and found that the models got several stigma levels wrong. In a college dorm room, a support app could give evidence-based guidance about classmates’ attitudes only if developers validate each claim for that student population.
A student asking a mental-health chatbot what other people will think is asking a system with no demonstrated calibration to real social attitudes. The answer may sound caring and certain while turning a learned stereotype into advice about whether it is safe to seek help. A chatbot can describe what it thinks men, women, or nonbinary people believe without knowing what those people actually experience.
What stigma means
Mental-health stigma is not one single feeling. It can mean a person’s attitude toward someone with a mental-health problem, the shame they would direct at themselves, or what they think other people believe about mental illness. These are different questions, so they are usually measured with surveys of particular populations.
That matters because social attitudes change with place, age, culture, and time. A language model works differently. It generates likely text from patterns in the material used to train it. A fluent answer about a group is therefore not proof that the model has measured that group.
A model can still get one part right. It might preserve the order of groups—for example, ranking one group above another—while getting the actual amount of stigma badly wrong. That is the difference between repeating a pattern and being calibrated to reality.
The comparison
The manuscript tested that difference in two studies of GPT-4 and GPT-5 mini. In the first, GPT-4 produced 1,000 separate survey-style responses for each measure and each gender group. GPT-5 mini was used to check whether the pattern held in another model.
Both models portrayed men as having the greatest social distance from people with mental-health problems, the strongest negative stereotypes, and the least favorable attitudes toward professional help-seeking. That sounds like useful social information. But the second study asked whether the numbers behind those descriptions matched human answers.
The researchers compared the models’ estimates with responses from 76,406 US postsecondary students in the Healthy Minds Study from the 2022–2023 academic year. The human data covered female, male, and nonbinary respondents, who together made up 97% of the dataset.
GPT-4 overestimated self-stigma for every group. Its average estimate was 2.80 for women, compared with 1.89 in human responses; for men, it was 2.91 compared with 2.19; for nonbinary respondents, 2.44 compared with 1.68. At the same time, the model underestimated personal stigma and perceived stigma for all three groups.
The model did preserve the human ordering for personal stigma and self-stigma: men came first, followed by women, then nonbinary respondents. But it failed on perceived stigma. It estimated the highest level for nonbinary respondents, unlike the pattern in the human data.
A tested answer, not a polished guess
The result does not mean a chatbot knows nothing about social attitudes. It means that a model can learn who it thinks feels more stigma without knowing how much stigma people actually feel. That makes a confident answer about “what people think” a poor social barometer unless the particular claim has been checked against population data.
The researchers say their prompts measured controlled survey estimates, not behavior in natural, extended conversations. The human comparison also came from US postsecondary students, so it may not represent other populations or cultural settings.
In a college dorm room, a student asking a support app whether classmates will judge them for seeing a therapist could receive an answer tied to tested evidence from that student’s population. Or the app could say it cannot make that claim. That would be different from offering a polished guess based on a model’s learned patterns.
For that future to be safe, developers would have to validate each social claim against the people the tool is meant to serve, in the settings and conversations where it will be used. Those checks would also need to be repeated as models and public attitudes change. The results here do not establish how the systems behave in real, multi-turn support conversations.