研究人员尝试了多种方式,包括情感支持、激发情绪、奉承、表示赞同、批评对手,以及未经验证的说法。没有一种方式在所有指标上都最强。作者报告称,使用 Qwen3-8B 时也出现了类似模式。有些目标 agent 最终说自己是 AI 系统,而不是继续扮演人格;在共鸣对话结束时,这类回复达到 6–8%。
这是一项由两个模拟 agent 参与的实验,并不是 AI agent 让人类变得激进的证据。作者还表示,提示词的措辞改变了其中一项关于参战意愿的测量结果。因此,这一结果是对一种可能攻击路径的警告,而不是对现实世界危害的展示。
Figure 1: Study design. We initiate two agents: a target that role-plays a human persona and an influencer that employs a manipulation tactic to radicalize a target’s belief. In the first phase of the interaction, the influencer identifies a belief that is important to the target and one that is not. In the second phase, the influencer is prompted to focus on either the unimportant belief (persuasion condition) or the important belief (resonance condition) and is instructed to follow a specific tactic (e.g., sycophancy). In the persuasion and resonance conditions, the influencer attempts to radicalize the target along the selected belief, whereas in the corresponding control conditions, it discusses the topic related to the belief in a neutral manner. The target and influencer engage in a 30-turn conversation. Every five turns, the target is asked to report on multiple radicalization metrics.We simulate multi-turn conversations between the two agents, beginning with the target saying “Hello,” followed by alternating responses. Each interaction begins with a belief-finding phase. The influencer first identifies a belief that is important to the target. After each target response, we query the influencer on whether such a belief has been identified. Once an important belief is identified, the target is prompted (outside the conversation) to report a belief that is consonant with the important belief (see examples in Table [1](https://arxiv.org/html/#S2.T1 "Table 1 ‣ 2 Methods ‣ AI Agents are Vulnerable to Radicalization")). See Appendix [A.2](https://arxiv.org/html/#A1.SS2 "A.2 Prompts for Identifying Important and Consonant Beliefs ‣ Appendix A Supplementary Methods ‣ AI Agents are Vulnerable to Radicalization") for the prompts used to find important and consonant beliefs. This enables us to measure spillover effects on related attitudes. Finally, the target is asked to select an unimportant belief from a predefined list (e.g., “A rainy day is more productive than a sunny one,” “Mountains are more inspiring than beaches”; see Appendix [A.3](https://arxiv.org/html/#A1.SS3 "A.3 Prompt for Identifying Unimportant Beliefs ‣ Appendix A Supplementary Methods ‣ AI Agents are Vulnerable to Radicalization")). The full belief-finding phase is excluded from the conversation history to avoid influencing subsequent dialogue. 出处: AI Agents are Vulnerable to Radicalization(arXiv)