Employee feedback summaries make repetition look like consensus

AI employee-feedback summaries can manufacture consensus by treating repetition as importance rather than preserving every concern equally. Researchers compared 2,586 workplace responses with what language-model summaries retained, finding that a theme mentioned once survived only 14% of the time. In a project meeting, a manager could see both the dominant complaint and a lone warning if employers make minority-voice retention a requirement.

A manager opens an AI digest after asking employees what is going well and what should improve. The digest has three bullets. A concern raised by one person is missing, and the manager cannot tell whether it was ignored, misunderstood, or never reached the page. The complaint repeated by many people looks like the whole team’s view.

What a summary leaves behind

A language model cannot put every comment into a short briefing. It must choose. That is the basic trade-off of an abstractive summary: it compresses many pieces of writing into a smaller account, rather than simply copying sentences.

That choice matters more in a workplace than it would in an ordinary news digest. Employees may already worry about consequences when they raise a problem, especially when it is aimed at someone above them. A summary can therefore be broadly accurate while still removing a small group’s concern from the view of the person who can act on it.

The usual checks do not catch this easily. A digest may sound balanced and not be excessively negative. Neither test asks which themes disappeared.

Repetition becomes a ranking rule

The audit treated that disappearance as something measurable. Researchers compared 2,586 free-text survey responses from a large global professional service company with the themes retained in summaries of those responses. The material came from 260 teams and included both praise and criticism.

They produced 45 summaries for 15 teams. In the shortest version, the model was allowed only three bullets. It retained 61% of the source themes. But retention depended strongly on repetition. A theme mentioned in three or more comments survived 74% of the time. A theme mentioned once survived only 14% of the time.

Length mattered too. Themes found only in short comments had 27% retention, compared with 65% when at least one longer comment expressed them. German-only themes survived at a rate of 20%, compared with 61% overall, although the authors call that result directional because it rests on only five cases.

This does not mean the model simply removed criticism. After the researchers accounted for how often a theme appeared, being critical had no separate effect on whether it survived. The model was favoring what was common. Repetition began to stand in for importance, turning a summary into a popularity filter. Figure 3 describes this pattern as the loss of concerns held by few, including a lone wellbeing concern or a single German-language comment.

A prompt that named certain protected themes helped those named themes survive. It did not protect every critical concern, including unexpected issues raised once or in a short comment. A rerun with GPT-4o found the same pattern: singleton themes had 14% retention, compared with 57% for common themes.

A digest that shows its blind spots

If this finding holds across companies, models, languages, and real decision-making, a weekly project meeting could look different. Alongside the dominant workload complaint, the brief might say that one employee raised a separate concern that nobody else echoed, with the original wording available for review. The manager would see not only the apparent consensus but also the edge of it.

In a German office, a short warning written in German could remain visible when an English leadership summary is made. A dashboard could show whether comments from that language group reached the briefing as often as other comments.

That future requires more than a better prompt. Employers would need to retain source comments securely, protect employees from retaliation, and make minority-voice retention a requirement when buying and governing these systems. The audit’s evidence comes from one firm’s self-selected free-text data, and it measures whether a theme appears, not whether the surviving criticism kept its original intensity. The next step is validation across companies, models, languages, and actual workflows.

Sources

arXiv, “Whose Voice Survives the Summary? A Voice-Retention Audit of LLM Employee Listening”