2026年10月19日
Technology Watch

从实验室到产业,追踪新技术的路径

← 返回首页

arXiv

一个化身可以在沉默中保持存在感

AI 绘制的示意图

一个化身无需等待新的提示,只要延续自己的记忆、情绪、想法和预定行动,就能在沉默中保持社交存在感。论文作者构建了一个意向性智能体,在沉默期间每隔 15 秒更新一次这一状态。在虚拟诊所的候诊室里,如果系统保持连贯、不凭空编造事件,临床医生化身就能在患者整理思绪时保持专注的表情。

如今,人们说话时,化身可以点头,也可以从视频或音频信号中复制面部动作。但对话一旦停下来,它通常就没什么可依据的了。它可能会僵住、恢复成中性表情,或者重复一套预设动作。系统会对最新信号做出反应,却无法清楚说明已经发生了什么、它在期待什么,或者它的表情应该继续、改变还是停止。

信号之间,化身需要什么

面部反应并不是一个简单的翻译问题。同一句话,根据对话情境的不同,可能应该得到微笑、担忧的表情,或者完全不表现出反应。现有系统通常会学习声音、视频和面部动作之间的统计联系。有些系统可以生成多种可能的反应,但系统内部的历史通常被存储为一种隐藏状态,其中各个部分没有人可以检查或调整的名称。

新的框架为这一状态划分出四个有名称的部分:内在思维流、情绪状态、行动状态和激活的记忆。情绪状态采用了三个常见概念——感受是积极还是消极、感受有多强,以及它暗示了多大程度的控制力或支配力。这些数值可以帮助引导面部表情,但它们本身并不能准确说明哪些面部肌肉应该运动。

作者还做了一个谨慎的区分。内在思维流是作为下一次更新的工程记录而使用的文本。它不能证明化身具有意识或人类式的思维。

沉默的心跳

论文最核心的变化,是不再把每一个面部动作都视为对新提示的直接回应。在沉默期间,一个定时的心跳会启动下一次更新。在测试系统中,这个心跳每 15 秒出现一次。语言模型会考虑保留下来的想法、情绪、行动和记忆,然后生成下一个内部想法以及一个可能的行动。情感模型会把这一结果转化为面部动作,并最终生成面孔。

这样一来,化身就能在不假装发生了新事情的情况下保持活跃。它的指令会把期待与观察分开,并禁止加入没有依据的外部事件。如果它的期待与观察结果不一致,下一步想法就应该处理这种不匹配。因此,系统可以决定表现出反应,也可以保持沉默但继续活动。

与不透明的单一输出相比,这种设计更容易检查。开发者可以分别查看或修改记忆中的上下文、感受、期待或预定行动。但这种设计也比直接的面部动作模型更昂贵、更慢,因为语言模型循环必须持续运行。作者称,控制内部动态的方程式是启发式规则,并不是某个完整概率模型的结果。

结果显示出改进,但并不能证明它具有类似人类的行为。在 REACT 2025 基准测试中,完整系统在盲测的人类评估中得分高于等待事件或只使用手写规则的版本。它的平均评分为 5.527,而记录下来的人类反应为 5.195;作者表示,经过校正后,这一差异没有达到统计显著性。在一项 30 分钟、没有输入的测试中,98.33% 的更新没有凭空生成没有依据的外部事件。但仍有一些更新生成了这类事件。

Refer to caption
Figure 2: The overall architecture of the intentional agent.For engineering implementation purposes, the intentional agent maintains an Inner Thought Flow (ITF) TtT\_{t} as part of its structured internal state. We use ITF as an engineering representation of the intermediate state used for subsequent generation; we do not treat it as evidence that the agent has human consciousness or cognition. ITF is mainly prompt-driven at inference, and the base LLM is not fine-tuned to learn it. During silence, updates are initiated by a timed heartbeat rather than new external events. The thought generator updates from retained information without a new external event; the prompt keeps expectation separate from observation, prohibits unsupported external events, and requires prediction mismatches to be addressed in the subsequent thought. At each update, the retained state is combined with current perceptual and prediction-error information; the thought generator produces the next thought, after which the action generator produces a possible action state. This mechanism is somewhat similar to the framework by Liu et al. [Liu et al. (2025)](https://arxiv.org/html/#bib.bib7). However, their approach still relies on the passive triggering of inner thoughts by external events as a starting point to address the optimisation of dialogue flows. In contrast, the intentional agent focuses on how the agent initiates intentional actions towards the external world. Under our internal dynamics framework, this implies how the agent acts upon the environment even when it is situated alone. Here, TtT\_{t}, emotion vector EtE\_{t}, action vector AtA\_{t}, and activated memory content MtM\_{t} together constitute the retained internal state St←{Tt,Et,At,Mt}S\_{t}\\leftarrow\\{T\_{t},E\_{t},A\_{t},M\_{t}\\}. External perception NtN\_{t} is supplied separately to the current update and can be empty. Conventional temporal models usually encode interaction history in a learnt hidden vector whose individual dimensions have no predefined meanings. In our framework, information used for subsequent updates is organised into explicit, named components rather than compressed into a single opaque hidden representation. These components can therefore be inspected and changed separately. At each update, the base LLM used as the thought generator receives the current state to generate the next thought, after which the action generator produces the next action state. The updated thought and action then enter the affective and facial generation process. Because TtT\_{t} largely contains working memory, the memory module primarily stores long-term memory. During the generation of Tt+1T\_{t+1}, TtT\_{t} and the possible NtN\_{t} are inputted into the memory module to calculate cosine similarity for extracting the most similar memory elements. Subsequently, a LLM integrates the text of the memory element itself, its corresponding temporal attributes, and potential executed actions into a descriptive text segment MtM\_{t}. After this, TtT\_{t} is stored by the memory module. 出处: Beyond End-to-End Black Box Mapping: An Intentional Agent Framework for Cognitive-driven Facial Reaction Generation(arXiv)
Refer to caption
Figure 6: Qualitative frame-by-frame comparison. In this exchange, the speaker presents a difficult-task dilemma, asking what happens when more time is spent on the harder task, while the listener begins formulating a response. Ground Truth shows natural temporal variation, whereas Event-Triggered and Heuristic-Only are more repetitive or neutral. Full Architecture generates a continuous, context-sensitive trajectory that evolves towards a mildly tense or concerned expression as the dilemma unfolds, without reproducing the ground-truth sequence frame by frame. Expression amplitudes are scaled for visual clarity.### F.7 Comparison with Existing Automatic Metrics 出处: Beyond End-to-End Black Box Mapping: An Intentional Agent Framework for Cognitive-driven Facial Reaction Generation(arXiv)

安静的化身可能会做什么

在虚拟诊所的候诊室里,患者可以停下来整理思绪,而临床医生化身则保持恰当的专注或柔和表情,不会僵住或重复机械的点头。在语言学习终端,学习者可以说到一半停下来,不必再次触发辅导员;化身的面孔可以安静地保持投入,而不是每轮对话结束后都重置。

这种未来不只取决于一个令人信服的演示。内部状态必须在长得多的对话中保持连贯,而且更新成本必须低到足以用于实际场景。系统还必须避免把沉默变成虚构的意图,反应速度也要足够快,不能让延迟分散注意力;它的行为还必须可控,并符合文化习惯。作者进行的长时间沉默测试发现,完整系统在之后的想法中保留已陈述事实的比例为 62.08%,远高于只保留想法的版本,但仍远非完美。只有记得自己为什么沉默,而且不编造这个原因,化身才能在沉默中保持存在感。

来源

arXiv,《超越端到端黑箱映射:一个用于认知驱动面部反应生成的意向性智能体框架》