LargeAgentSystems.org
For AI to go well, we need a new science of billion-scale systems.
As millions of AI agents permeate human economies, societies, and cultures, systemic risks from instability, inequality, and disempowerment are growing. Meeting these challenges demands a concerted cross-disciplinary effort. We're bringing together people working towards pro-human outcomes as the world transitions to large-scale, mixed systems of humans and AI, which we call, “large agent systems.”
Part One
The problem.
Framing
A new type of system.
Human systems were designed for humans. Large-scale changes to the participant mix on large systems has historically led to sudden, unforeseen, severe systemic failures - including the GFC, the 2010 Flash Crash, and political polarisation, partially attributed to foreign interference using bot farms.
In the age of agentic AI, those changes could be catastrophic. A humanity disempowered by its tools may be unable to meaningfully change course when economic incentives turn against it. A rapid concentration of power could upend social contracts, leading to prolonged instability and diminishing the world's ability to respond to other threats. A collection of distributed agents could develop superintelligence, with unknowable consequences.
Trajectory
Agentic systems are growing fast.
Founded largely in the past year, early examples of purely-AI large agent systems have hundreds of thousands of participants.
Growth index, log scale (100 = start of range)
Each source is indexed to 100 at its first tracked point in the selected range, since not every source has been measured for the same length of time.
Source: Gigascale-Labs/las-usage-stats, scraped daily.
Live deployments are also accelerating.
The number of companies implementing large agent systems and agent infrastructure is growing.
Organizations, cumulative by type, 2020–present
Cumulative count of catalogued organizations by type. 18 additional catalogued organizations have no recorded founding year or type and aren't reflected above.
Scale
Large systems are different.
Single-Agent
- -One agent.
- -Focused on alignment, interpretability, and control.
- -Historically dominant focus of AI safety and governance.
- -Strong political and regulatory attention.
“The problem of accidents in machine learning systems, defined as unintended and harmful behavior that may emerge from poor design of real-world AI systems.”
Amodei et al., 2016 →Multi-Agent System
- -Two to dozens of agents.
- -Focused on communication, coordination, and monitoring.
- -A focus of safety and governance research since 2021.
- -Emerging regulatory attention.
“Today, AI systems are beginning to autonomously interact with one another and adapt their behaviour accordingly, forming multi-agent systems.”
Hammond et al., 2025 →Large Agent System
- -Thousands to billions of agents.
- -Focused on aggregate outcomes, system mechanisms, scalable safety.
- -Technical research emerging since 2025.
- -Some political attention due to impact on human employment.
Threat Models
Threat models in large agent systems.
Gradual Disempowerment
“We argue that this dynamic could lead to an effectively irreversible loss of human influence over crucial societal systems, precipitating an existential catastrophe through the permanent disempowerment of humanity.”
Kulveit et al., 2025 →Systemic Instability
“Our current trajectory points toward a spontaneous emergence of a vast and highly permeable AI agent economy, presenting us with opportunities for an unprecedented degree of coordination as well as significant challenges, including systemic economic risk and exacerbated inequality.”
Tomašev et al., 2025a →Inequality
“We define this emerging challenge as ‘agentic inequality’: disparities in power, opportunity, and outcomes arising from unequal access to, and capabilities of, AI agents.”
Sharp et al., 2025 →Collective Superintelligence
“The alternative AGI emergence hypothesis, where general capability levels are first manifested through coordination in groups of sub-AGI individual agents with complementary skills and affordances, has received far less attention.”
Tomašev et al., 2025b →Partially Observable Systems
“Current interpretability techniques, developed primarily for static models, show limitations when applied to agentic systems.”
Zhu et al., 2026 →Power Concentration
“Historically unprecedented levels of automation could concentrate the power to get stuff done, by reducing the value of human labour, empowering small groups with big AI workforces, and potentially giving one AI developer a huge capabilities advantage.”
Hadshar, 2025 →Outdated Models
“The model implies tail-loss amplification of 18–54%, economically significant relative to Basel III countercyclical buffers.”
Meng & Chen, 2026 →Part Two
The approach.
Large agent systems are an object of study. Different fields see the problem in different ways. To keep large agent systems pro-human, we need to develop a common ground.
Disciplines
Large agent problems are highly cross-disciplinary.
The appropriate lens for a given system problem depends on:
- System type - production economy, social network, labour market, financial system.
- Participant mix - purely AI, or a mix of humans and AI.
- Observability - whether aggregates, agent interactions, and agents themselves are accessible to a monitor.
Along each axis, different knowledge can be brought to bear.
Focus Areas
Focus areas.
Monitoring
Monitoring large agent systems presents several new challenges, including scaling via federated interpretability, behavioural interpretability for partially-observable agents, and privacy-preserving interpretability.
Steering
Actively intervening in large systems avoids bad outcomes, when the system is too unconstrained to design against failure.
Simulation
Large agent systems are too complex to predict many behaviours, so we simulate outcomes instead.
Redesign
Modifying system mechanisms, entry rules, etc. to keep outcomes pro-human.
Research Agendas
Groups in the area.
Google DeepMind
Tomašev, Franklin & Osindero
DeepMind's running thread on the AI agent economy.
DEXAI – Icaro Lab
Bisconti, Pierucci & Galisai
Microfoundations of macro safety.
Cooperative AI Foundation × GovAI
Hammond & Chan
Multi-agent risk and coordination infrastructure.
Org Map
Org map.
A map of the people, organizations, and companies working on large agent systems.
Open the mapPart Three
Take action.
In the face of potentially irreversible risks, the time to start working on large agent systems is now. We highlight some pressing open problems and barriers the field faces in the near-term.
Open Questions
Open questions.
- 01
Lorem ipsum dolor sit amet, consectetur adipiscing elit?
Excerpt from paper - placeholder.
- 02
Ut enim ad minim veniam, quis nostrud exercitation ullamco?
Excerpt from paper - placeholder.
- 03
Duis aute irure dolor in reprehenderit in voluptate velit?
Excerpt from paper - placeholder.
- 04
Excepteur sint occaecat cupidatat non proident sunt in culpa?
Excerpt from paper - placeholder.
Barriers
Barriers to overcome.
Collective Action Problem
No single company has incentive to solve LAS safety.
Lack of Data
Pure-AI LAS are few; agent presence on mixed systems is hard to identify; and large-scale simulation papers rarely publish their simulation data.
Inter-disciplinary Collaboration
LAS calls for highly interdisciplinary specialised teams, drawing on AI safety, social sciences, and scaling engineering. It takes time and contact to connect fields who don't usually talk, particularly going outside the academy.
Streetlight Effects
LAS work is new, highly specialised, and outside the curriculum of major AI safety training programmes like ARENA, making it costlier for new people to enter the space than to work on established agendas like mechanistic interpretability or single-agent evals.
Community
Join the community.
We run a Slack community for AI safety researchers, social scientists, and policy and strategy experts working on large agent systems.
Request an invite