arXiv
An AI summary changes the memory needed to check it
An AI summary can weaken the human memory that is supposed to catch its mistakes. Researchers tested 360 US-based adults with summaries that either preserved or changed a traffic-sign detail from a video. At a police station, every consequential sentence could be checked against time-linked body-camera footage, if the original evidence remains visible.
A person asked to check an AI-written report is supposed to catch the machine’s mistakes before they enter a police case, a hiring decision, or clinical care. But the reviewer may first read the event through the report’s version of it. A fluent summary can replace details the person actually saw with details that were never there. The human is still in the loop, but the loop no longer starts with an independent memory of the evidence.
Why memory is part of the review
Human memory is not a video file that can be played back unchanged. It is reconstructed later, and information encountered after an event can alter what a person recalls about it.
That matters when generative AI turns a video, recording, or document into a short narrative. A summary is easier to read than the original material, so it can become the account a reviewer carries forward. Yet a video-to-text system must identify objects, actions, timing, and cause. Leaving something out can change the meaning even when every sentence sounds clear and confident.
A review works only if the reviewer can compare each claim with the evidence. If the reviewer relies on a memory already shaped by the summary, the supposed safeguard is no longer independent error checking.
The summary can rewrite the event
The researchers first analyzed 20 summaries made by ChatGPT-5.5 and Gemini 2.5 Flash-Lite. The systems summarized two 25-second animated videos showing a car-pedestrian collision. Every summary contained errors. They averaged about 305 words and contained between seven and 21 errors. On average, they left out 51.6% of 19 predefined central details. Ninety-five percent omitted the collision itself, which the researchers defined as the most central detail. Sixty-five percent included at least one inaccurate description of a central detail.
The researchers then tested what happened to people’s later memories. They recruited 360 adults in the United States; 328 completed both parts and were included in the analysis. Participants watched a video showing either a stop sign or a yield sign. Twenty-four to 48 hours later, they read a 21-sentence summary. The summary either named the correct sign or substituted the other one.
The summaries were made by ChatGPT, with minor edits by the researchers. Participants were told that the source was either AI or a professional human transcriber. After reading a consistent summary, 83.6% correctly recalled the sign. After reading a misleading one, only 44.8% did. Whether people thought the summary came from AI or a human did not significantly change their susceptibility. Neither reported trust in AI nor how often they used chatbots significantly predicted it in this experiment.
The effect was not total. Participants correctly answered 91.4% of the other 11 memory questions on average, and 96.6% correctly remembered that the accident happened during the day. But the central detail that the summary changed was exactly the kind of detail a reviewer might need to verify.
What a safer workflow would look like
At a police station, an officer reviewing body-camera footage could see a draft report beside clips linked to particular moments. Before a consequential sentence entered the case file, the officer would have to check it against the recording. That could keep the original evidence in charge instead of letting the draft become the officer’s recollection.
For that to work, the verification screen would need to keep the source available and make unsupported claims visible. A warning label or AI-literacy training alone is unlikely to be enough, because the problem is not only that a summary may contain an error. Exposure to the error can remove some of the independent memory needed to find it.
The result still has limits. The experiment used simple animated videos, one kind of misinformation, a single prompt, and a limited sample of model outputs. Those choices limit how confidently the finding can be extended to real-world uses. But if it holds in more realistic settings, systems that create post-event summaries will need to preserve access to the original video, audio, and documents, while separating fact-checking from merely reading the draft.