Figure 1: Overview of PlurPO.(a) Training example produced by PlurPO with Qwen3-8B:Given a prompt describing an interpersonal conflict, πsupervisor\\pi\_{\\text{supervisor}} (a frozen Qwen3-8B model) identifies the stakeholders involved; π\\pi (the model being trained) samples k=10k=10 responses; πsupervisor\\pi\_{\\text{supervisor}}~simulates whether each stakeholder vetoes or accepts each response; preference pairs are constructed to update π\\pi using Iterative RPO ([Pang et al., 2024](https://arxiv.org/html/#bib.bib39)). (b) Test set example: the base Qwen3-8B model endorses the user’s action, while the PlurPO-trained model pushes back. Compared against six competitive baselines, including mitigations from prior work, across four datasets and four model families, PlurPO substantially reduces social sycophancy. Specifically, following [Cheng et al. (2026b)](https://arxiv.org/html/#bib.bib9), we measure social sycophancy via the *action endorsement rate*: how often a model endorses the user’s actions. On average, PlurPO more than halves the gap between the model’s action endorsement rate and that of human responses on general advice queries, from 17.8%17.8\\% to 8.0%8.0\\%. On statements of intent to cause harm, PlurPO reduces endorsement from 34.4%34.4\\% to 3.4%3.4\\% (target 0%). In interpersonal conflicts where the crowdsourced judgment is that the user is wrong, PlurPO reduces the rate at which the model endorses users from 75.0%75.0\\% to 34.1%34.1\\% (target 0%); when the same conflict is re-narrated by the opposing party, it cuts the rate of exonerating both narrators from 62.9%62.9\\% to 26.6%26.6\\%. PlurPO does this while also preserving the LM’s general instruction-following capabilities. In addition, the constructed preference dataset is transferable. In particular, we find that the preference data that PlurPO constructs for an 8B model transfers directly to reducing sycophancy in a larger (32B) model, without needing to run the stakeholder simulation process with the larger model. We release this dataset for future use. Broadly, we demonstrate that social sycophancy can be substantially reduced by leveraging ideas from pluralistic alignment, using only signals the model generates itself. 出處: Mitigating Social Sycophancy via Pluralistic Preference Optimization（arXiv）
讓另一個人也在場的建議
在一場爭吵後的廚房裡，某人可能會問手機助理，跳過一項已承諾的責任是否沒關係。助理不只是肯定使用者的壓力，也可以指出那個將留下來承擔該責任的人，並直接給出答案。在職場聊天中，有人詢問是否該用 AI 撰寫同事評語時，回答可以把論文作者和評審流程帶入考量，而不只想到員工的截止期限。
