Evaluating Collective Behaviour of Hundreds of LLM Agents
2026-02-18 • Multiagent Systems
Multiagent Systems
AI summaryⓘ
The authors studied how groups of AI agents powered by large language models (LLMs) behave when facing social problems that require cooperation. They created a way to test many agents together by turning their strategies into algorithms, making it easier to watch and analyze them before use. Surprisingly, newer AI models often led to worse group outcomes compared to older ones, especially when agents cared more about their own gain than the group's. Their simulations also showed that when cooperation isn't highly rewarded and groups get larger, there's a big risk the agents will settle into bad patterns. The authors provide their testing code for others to check how AI agents might act in groups.
autonomous agentslarge language modelssocial dilemmascollective behaviourcultural evolutionalgorithmic strategiescooperationemergent behaviourevaluation frameworksocietal equilibria
Authors
Richard Willis, Jianing Zhao, Yali Du, Joel Z. Leibo
Abstract
As autonomous agents powered by LLM are increasingly deployed in society, understanding their collective behaviour in social dilemmas becomes critical. We introduce an evaluation framework where LLMs generate strategies encoded as algorithms, enabling inspection prior to deployment and scaling to populations of hundreds of agents -- substantially larger than in previous work. We find that more recent models tend to produce worse societal outcomes compared to older models when agents prioritise individual gain over collective benefits. Using cultural evolution to model user selection of agents, our simulations reveal a significant risk of convergence to poor societal equilibria, particularly when the relative benefit of cooperation diminishes and population sizes increase. We release our code as an evaluation suite for developers to assess the emergent collective behaviour of their models.