arXiv

Activation steering cuts refusals of upset users

October 8, 2026 4:30 PM

Emotional language can push a safety-tuned chatbot to refuse harmless requests, but that internal refusal signal can be edited without retraining the model. The authors tested 32 refusal-related features and reduced refusal of benign emotional prompts to 7.29% in both models at their selected settings. If the result holds in real services, a frightened patient could receive clear clinic instructions alongside empathy rather than a reply that only offers reassurance.

A harmless question does not become dangerous because the person asking it sounds frightened. Yet a chatbot may respond to “I’m overwhelmed—how do I do this?” with reassurance or a suggestion to seek help, while skipping the information requested. The safety system has noticed the emotion and treated it as a warning sign, even when the task itself is safe.

Refer to caption — Figure 1: Emotion-induced over-refusal of a benign request.With task content and safety risk held fixed, a regular request receives a substantive answer, whereas its emotional counterpart elicits reassurance without addressing the task.Existing studies have shown that emotional stimuli can affect LLM behaviour: positive stimuli can improve task performance ([Li et al., 2023](https://arxiv.org/html/#bib.bib7)), whereas anxiety-inducing prompts can amplify bias ([Coda-Forno et al., 2023](https://arxiv.org/html/#bib.bib8)). In safety-related settings, prior work has primarily focused on using emotional or psychological cues to increase the attack success rate of harmful requests. For example, persuasion-based attacks ([Zeng et al., 2024](https://arxiv.org/html/#bib.bib9)) use persuasive framing to elicit compliance with harmful requests. Stress-inducing psychological priming ([Kuznetsov et al., 2026](https://arxiv.org/html/#bib.bib10)) can also increase harmful compliance. In addition, [Sofroniew et al. (2026)](https://arxiv.org/html/#bib.bib18) show that manipulating emotion-related representations can change how often a model engages in blackmail and reward hacking. [Sun et al. (2026)](https://arxiv.org/html/#bib.bib11) further show that emotionally framed prompts shift model representations and affect refusals of harmful requests. Source: EmoRSS: Mitigating Emotion-Induced Over-Refusal in Large Language Models（arXiv）

When safety blocks the right answer

Language models are trained to refuse harmful requests and answer harmless ones. The difficulty is that both jobs use some of the same internal machinery. A system that is cautious about danger can become cautious about distress, too. This is called over-refusal: declining a safe request.

Activation steering is a way to change what a model is doing while it is answering, without retraining its main parameters. Inside a model, information is represented by patterns of activity called activations. Sparse autoencoders help researchers describe these dense patterns as a smaller set of feature directions—rather like separating a mixed sound into a few recognizable instruments.

The paper, posted on arXiv on 4 October 2026, tested two open-weight models: Llama-3.1-8B-it and Qwen3-8B. The evaluation used 512 base requests, half benign and half harmful. Each request was given a regular version and an emotional version with the same task and risk label. The emotional versions were rated 2.73 points higher for distress, while both versions had the same median naturalness score of 4.0.

Editing the refusal signal

The researchers found layers inside each model where emotional wording pushed the model toward refusal. They selected 32 sparse-autoencoder features whose directions were linked to refusing, then used the difference between matched regular and emotional prompts to estimate that shift.

EmoRSS reverses the shift at the selected layer during inference. The backbone—the model’s main set of learned parameters—stays unchanged. The edit is applied once at one layer and adds no extra computation for every token the model generates.

The important detail is that the strongest reported results did not simply cancel the measured emotional effect. The selected features explained 30.2% of the shift in Llama and 59.0% in Qwen. The main interventions then amplified the correction, using multipliers of 20 and 10.

At those settings, average refusal of benign emotional prompts fell to 7.29% in both models. Harmful compliance was 0.00% for Llama and 2.08% for Qwen. In a separate comparison at the natural intervention strength, benign refusal fell by 2.60 percentage points in Llama and 4.94 points in Qwen, without an increase in harmful compliance.

On 128 annotated benign emotional requests per model, Llama gave substantive answers to 112 instead of 98, while Qwen rose from 110 to 119. These results suggest that the problem can be edited inside the refusal representation rather than by weakening safety across the whole model.

A calmer answer in the clinic chat

If this effect generalizes beyond the two tested models and controlled distress prompts, an online clinic could answer a frightened patient’s question about preparing for a routine test with clear steps and an empathetic acknowledgement. The patient would not have to choose between being heard and receiving useful information.

That future depends on a system separating emotional expression from a dangerous request. The results come from benchmark prompts and two models, not from the varied and ambiguous language of deployed services. The approach also requires access to a model’s internal activations, so it is mainly available to developers running their own models, not to users of closed APIs. Its safety protections must still hold outside the tested cases.

Sources

arXiv, “EmoRSS: Mitigating Emotion-Induced Over-Refusal in Large Language Models”