October 19, 2026
Technology Watch

From the laboratory to industry: following new technology on its way

← Front page

arXiv

Multimodal fake news makes fitting images no proof

Image drawn by AI

A convincing picture and polished wording are no longer reliable signs that a social-media claim is true. The researchers turned 4,507 real articles into paired true and false posts, keeping real names and events while changing the central facts and adding tailored images. If verification tools also trace sources and check claims, a neighborhood health warning could be examined before its chart and image send it racing through a group.

A social-media post can look like evidence without proving anything. It may use the name of a real institution, a precise number, a familiar chart, and a picture that seems to show the event. Yet the claim at the center can be false. A real image may have been taken from another context, while a made-up image can be designed to fit the words. Moderators cannot examine every polished post as carefully as a human fact-checker, so systems that mainly look for awkward writing or a matching picture are easy to mislead.

Why a matching image is not proof

Text-only language models have already made it easier to write convincing false stories. Adding an image makes the story harder to check, because the words and picture can support each other. But “fits the story” is not the same as “is true.” A photograph can be genuine but unrelated. An illustration can be artificial but clearly presented as an illustration. To verify a post, a system generally needs evidence outside it: the original reporting, records showing where an image came from, or information from an independent trusted source.

That is the difference between spotting a strange-looking post and checking a claim. The first can be done by noticing patterns. The second asks whether the event happened and whether the picture really belongs to it.

Refer to caption
Figure 4: An example failure case in multimodal fake-news detection.To assess whether MLLMs make fake-news decisions in a human-like manner, we conduct an additional human annotation study on 300 news items, sampling 100 items from each dataset with a balanced split of 50 true and 50 fake items. Following the same structure as CoT models, annotators first judge four verification dimensions—factual accuracy of text, misleading language in text, image–text alignment, and image authenticity—and then provide a final veracity judgment (true/fake). These dimensions serve as diagnostic cues for analyzing the reasoning process rather than as independent definitions of news veracity. In particular, image authenticity must be considered jointly with textual factuality and cross-modal evidence, since legitimate news may contain AI-generated illustrations, while fake news may reuse authentic but out-of-context images. The annotation protocol is described in Appendix [C](https://arxiv.org/html/#A3 "Appendix C Human Annotation ‣ Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?"), and the interface is shown in Fig. [6](https://arxiv.org/html/#A3.F6 "Figure 6 ‣ Appendix C Human Annotation ‣ Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?"). Source: Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?（arXiv）

A factory for believable false posts

The researchers built a pipeline that starts with a real news article and produces two short social-media posts: one that faithfully summarizes it and one that changes an important fact. A language model keeps anchors such as names, institutions, numbers, and the event itself, then reverses the central claim. An image generator creates a picture to support the false version. A separate model checks the result for changed facts, readable language, image realism, fit between image and text, and a suitable social-media format.

The team made these paired posts from 4,507 copyright-free articles covering science, health, and entertainment. This turns a scattered problem into a repeatable test. For example, a real account of the Rosetta mission’s planned soft crash into Comet 67P can become a false story about an unplanned collision, while still naming the European Space Agency, Rosetta, and the comet. The details that make the post feel grounded are not removed; they are used to carry the lie.

The researchers then tested 16 multimodal models, which can work with both text and images. Their paper reports that GPT-5.5 reached 99.21 average F1 and accuracy in this benchmark when given a structured prompt. But the benchmark comes from one generation pipeline and only three domains. A model may be recognizing the habits of that pipeline rather than checking the truth of the claim.

Refer to caption
Figure 2: An agent framework for generating multimodal social media fake newsIn this paper, we propose a multi-agent framework to generate multimodal social media fake news grounded in real-world events. As shown in Fig. [1](https://arxiv.org/html/#S0.F1 "Figure 1 ‣ Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?"), the fake news post generated by our framework comprises a short story and a corresponding image that counter the true news narrative. For example, given a Nature news article about the Rosetta mission, our framework reverts the successful deliberate ”soft crash into Comet 67P” into a fabricated failure story of an ”unplanned collision”, while retaining key factual anchors such as ESA, Rosetta, and Comet 67P to ensure narrative plausibility and deceive readers. Source: Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?（arXiv）

What a better check would look like

In a neighborhood Facebook group, someone might see a polished health warning with an official-looking chart. A trustworthy platform would not treat the chart, tone, or image as proof. It would show where the claim came from and send it for verification before it spread widely. That would require checking the text and the image together, including whether an authentic image has been reused out of context.

The paper’s human review used four separate questions: Is the text factually accurate? Is its language misleading? Does the image fit the text? Does the image appear authentic? Those questions are useful clues, not substitutes for evidence. The authors say their evaluation is limited by the available copyright-free material, and the code and data will be shared only with registered researchers for ethical research. The larger test remains whether detection systems can keep working when the posts come from new generators, prompts, campaigns, and real reused images.

Sources

Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?