Abstract
Alignment training on individual LLM agents often fails to transfer to multi-agent teams, but designating a safety lead and having a coordinator publish safety specifications can restore single-agent alignment.
Previous work has shown that alignment training on individual LLM agents can fail to transfer to multi-agent settings. We investigate the underlying mechanisms for this failure, and explore interventions to preserve alignment transfer to multi-agent LLM teams. We focus on two software-engineering tasks (sepsis triage, a news recommender) and ten consulting-proposal generation tasks. We present four findings: (1) The single-vs-team safety gap does not show a clear trend with more capable models. We find that Anthropic's Mythos Preview shows the largest gap on one of our tasks but the smallest on another; (2) Many natural interventions, such as having agents critique each other's work or coordinate in a group chat, do not reliably bring agent teams back to single-agent alignment behavior; (3) diffusion of responsibility substantially accounts for the safety gap: agents in teams are less likely to proactively check for system-level safety failures and more likely to ignore, rationalize, or deflect responsibility for issues when they arise; (4) having a designated safety lead agent ensure system-level safety, and having a coordinator agent publish a safety specifications document are effective at restoring single-agent alignment. We also release magelab, a multi-agent LLM orchestration and experimentation framework. Our findings can help practitioners design agent teams that successfully preserve the behaviors expected of aligned individual agents.
Community
Diffusion of responsibility explaining the safety gap sounds very human, and it's good to see that a named safety lead helps. Did the safety lead keep working as the team got bigger, or did the effect fade?
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Scaling Inference-Time Compute with Teams of Coding Agents (2026)
- Decentralized Multi-Agent Coordination and Communication in Shared Codebases (2026)
- MULTILOOP: Evolving Multi-Agent Harnesses for Complex Information-Seeking Tasks (2026)
- AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs (2026)
- ARIA: Runtime Inspection and Amendment for Multi-Agent Systems (2026)
- Agents Are Systems, Not Models: Rethinking Agentic Evaluation (2026)
- Agent Architecture Cost-Success Trade-offs: A Controlled Evaluation across Enterprise-Style SOPs (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper