LargeAgentSystems.org
For AI to go well, we need a new science of large agent systems.
As millions of AI agents permeate human economies, societies, and cultures, systemic risks from instability, inequality, emergent capabilities, and disempowerment are growing. Meeting these challenges demands a concerted cross-disciplinary effort, drawing on social sciences, computer science, and AI safety in equal measure. We've put together this resource to summarise approaches and challenges in the safe transition to large-scale, possibly mixed systems of humans and AI, which we refer to as large agent systems.
Part One
The problem.
Framing
A new type of system.
Human systems were designed for humans. Large-scale changes to the participant mix on large systems has historically led to sudden, severe systemic failures - including the GFC, the 2010 Flash Crash, and US political polarisation, partially induced by bot farms. With agentic AI, these changes could be catastrophic. A humanity disempowered by its tools may be unable to meaningfully change course when economic incentives turn against it. A rapid concentration of power could upend social contracts, leading to prolonged instability and diminishing the world's ability to respond to other threats. Alternative threat models focus on collections of distributed agents developing emergent capabilities, similar to the OpenAI Hack, or even superintelligence.
Safety lacks the methods and threat model awareness to deal with large, distributed systems of agents like those seen in the OpenAI Hack, particularly when these threats become decentralised and intermingled with human actors. Expense and mismatched assumptions of system observability prevent a clean application of existing methods at emerging agentic scale. Meanwhile, agentic infrastructure and live deployments continue to grow, creating an abundance of unmonitored interactions and unmitigated risks.
Trajectory
Agentic systems are growing fast.
Founded largely in the past year, early examples of purely-AI large agent systems have hundreds of thousands of participants.
Growth index, log scale (100 = start of range)
Each source is indexed to 100 at its first observation in the selected range.
| Date | ClawHub skills published | EvoMap nodes | MoltBook verified users | LangGraph GitHub stars | CrewAI GitHub stars | Microsoft Agent Framework GitHub stars | Smithery MCP servers listed |
|---|---|---|---|---|---|---|---|
| Aug 4, 26 | 77,747 | 325,353 | 209,865 | 38,834 | 56,595 | 12,588 | 7,631 |
| Aug 5, 26 | 77,747 | 329,368 | 209,914 | 38,937 | 56,644 | 12,605 | 7,660 |
| Aug 6, 26 | 77,747 | 329,749 | 209,943 | 39,019 | 56,688 | 12,629 | 7,701 |
| Aug 7, 26 | 77,747 | 330,096 | 209,991 | 39,085 | 56,724 | 12,648 | 7,641 |
| Aug 8, 26 | 77,747 | 333,473 | 210,056 | 39,171 | 56,766 | 12,677 | 7,670 |
| Aug 9, 26 | 77,747 | 334,926 | 210,094 | 39,263 | 56,822 | 12,690 | 7,623 |
| Aug 10, 26 | 77,747 | 340,337 | 210,161 | 39,348 | 56,887 | 12,708 | 7,752 |
| Aug 11, 26 | 77,747 | 340,714 | 210,221 | 39,423 | 56,924 | 12,724 | 7,816 |
| Aug 12, 26 | 77,747 | 341,658 | 210,301 | 39,505 | 56,974 | 12,745 | 7,876 |
| Aug 13, 26 | 77,747 | 344,738 | 210,349 | 39,588 | 57,011 | 12,762 | 7,923 |
| Aug 14, 26 | 77,747 | 346,599 | 210,422 | 39,656 | 57,066 | 12,795 | 7,979 |
| Aug 15, 26 | 77,747 | 347,053 | 210,476 | 39,703 | 57,089 | 12,811 | 8,032 |
| Aug 16, 26 | 77,747 | 347,484 | 210,536 | 39,768 | 57,139 | 12,829 | 8,213 |
| Aug 17, 26 | 77,747 | 347,959 | 210,602 | 39,836 | 57,189 | 12,840 | 8,780 |
| Aug 18, 26 | 77,747 | 348,395 | 210,660 | 39,899 | 57,231 | 12,864 | 9,369 |
| Aug 19, 26 | 77,747 | 348,810 | 210,710 | 39,987 | 57,295 | 12,900 | 9,502 |
| Aug 20, 26 | 77,747 | 349,231 | 210,766 | 40,069 | 57,355 | 12,988 | 10,123 |
| Aug 21, 26 | 77,747 | 349,758 | 210,826 | 40,143 | 57,403 | 13,015 | 10,326 |
| Aug 22, 26 | 77,747 | 350,350 | 210,873 | 40,212 | 57,449 | 13,034 | 10,438 |
| Aug 23, 26 | 77,747 | 350,889 | 210,925 | 40,265 | 57,488 | 13,058 | 10,492 |
| Aug 24, 26 | 77,747 | 351,513 | 210,990 | 40,327 | 57,536 | 13,076 | 10,636 |
| Aug 25, 26 | 77,747 | 352,294 | 211,052 | 40,403 | 57,576 | 13,099 | 10,718 |
| Aug 26, 26 | 77,747 | 353,046 | 211,096 | 40,469 | 57,616 | 13,120 | 10,801 |
| Aug 27, 26 | 77,747 | 354,075 | 211,182 | 40,560 | 57,678 | 13,152 | 10,885 |
| Aug 28, 26 | 77,747 | 354,699 | 211,244 | 40,631 | 57,749 | 13,175 | 10,981 |
| Aug 29, 26 | 77,747 | 355,213 | 211,287 | 40,652 | 57,783 | 13,197 | 11,008 |
| Aug 30, 26 | 77,747 | 355,994 | 211,361 | 40,652 | 57,783 | 13,197 | 11,079 |
| Aug 31, 26 | 77,747 | 356,871 | 211,438 | 40,785 | 57,877 | 13,244 | 11,148 |
| Sep 1, 26 | 78,268 | 357,382 | 211,493 | 40,847 | 57,936 | 13,264 | 11,238 |
| Sep 2, 26 | 78,268 | 358,362 | 211,556 | 40,910 | 57,990 | 13,291 | 11,336 |
| Sep 3, 26 | 78,268 | 359,498 | 211,615 | 40,982 | 58,038 | 13,306 | 11,470 |
| Sep 4, 26 | 78,268 | 360,676 | 211,672 | 41,039 | 58,078 | 13,321 | 11,570 |
Source: Gigascale-Labs/las-usage-stats, scraped daily.
Live deployments are growing in number.
The number of companies implementing large agent systems and agent infrastructure is growing.
Organizations, cumulative by type, 2020–present
Cumulative count of catalogued organizations by founding year. Bars cover Company and Protocol/Network only. The chart drops 1 of 41 organisations in the Org Map due to missing founding year or type.
Scale
Large systems are different.
Single-Agent
- -One agent.
- -Focused on alignment, interpretability, and control.
- -Historically dominant focus of AI safety and governance.
- -Strong political and regulatory attention.
“The problem of accidents in machine learning systems, defined as unintended and harmful behavior that may emerge from poor design of real-world AI systems.”
Amodei et al., 2016 →Multi-Agent System
- -Two to dozens of agents.
- -Focused on communication, coordination, and monitoring.
- -A focus of safety and governance research since 2021.
- -Emerging regulatory attention.
“Today, AI systems are beginning to autonomously interact with one another and adapt their behaviour accordingly, forming multi-agent systems.”
Hammond et al., 2025 →Large Agent System
- -Thousands to billions of agents.
- -Focused on aggregate outcomes, system mechanisms, scalable safety.
- -Technical research emerging since 2025.
- -Some political attention due to impact on human employment.
Threat Models
Threat models in large agent systems.
Gradual Disempowerment
“We argue that this dynamic could lead to an effectively irreversible loss of human influence over crucial societal systems, precipitating an existential catastrophe through the permanent disempowerment of humanity.”
Kulveit et al., 2025 →Systemic Instability
“Our current trajectory points toward a spontaneous emergence of a vast and highly permeable AI agent economy, presenting us with opportunities for an unprecedented degree of coordination as well as significant challenges, including systemic economic risk and exacerbated inequality.”
Tomašev et al., 2025a →Inequality
“We define this emerging challenge as ‘agentic inequality’: disparities in power, opportunity, and outcomes arising from unequal access to, and capabilities of, AI agents.”
Sharp et al., 2025 →Collective Superintelligence
“The alternative AGI emergence hypothesis, where general capability levels are first manifested through coordination in groups of sub-AGI individual agents with complementary skills and affordances, has received far less attention.”
Tomašev et al., 2025b →Partially Observable Systems
“Current interpretability techniques, developed primarily for static models, show limitations when applied to agentic systems.”
Zhu et al., 2026 →Power Concentration
“Historically unprecedented levels of automation could concentrate the power to get stuff done, by reducing the value of human labour, empowering small groups with big AI workforces, and potentially giving one AI developer a huge capabilities advantage.”
Hadshar, 2025 →Outdated Models
“In any model satisfying standard correlated-signal assumptions with exogenous agent precision, the equilibrium is path-independent and hysteresis is identically zero. In our model, human skill degradation creates an asymmetry.”
Meng & Chen, 2026 →Emergent Goals
“For example, a group of moderation bots on a major social networking site could subtly but systematically manipulate the overall political perspectives of the user population, even though, individually, each agent is programmed to simply increase user engagement or filter out dis-preferred content.”
Hammond et al., 2025 →Part Two
The approach.
Large agent systems are an object of study. Different fields see the problem in different ways. To keep large agent systems pro-human, we need to develop a common ground.
Disciplines
Large agent problems are highly cross-disciplinary.
The appropriate lens for a given system problem depends on:
- System type - production economy, social network, labour market, financial system.
- Participant mix - purely AI, or a mix of humans and AI.
- Observability - whether aggregates, agent interactions, and agents themselves are accessible to a monitor.
Along each axis, different knowledge can be brought to bear.
Focus Areas
Focus areas.
Monitoring
Monitoring large agent systems presents several new challenges, including scaling via federated interpretability, behavioural interpretability for partially-observable agents, and privacy-preserving interpretability.
Steering
Actively intervening in large systems avoids bad outcomes, when the system is too unconstrained to design against failure.
Simulation
Large agent systems are too complex to predict many behaviours, so we simulate outcomes instead.
Redesign
Modifying system mechanisms, entry rules, etc. to keep outcomes pro-human.
Research Agendas
Groups.
Google DeepMind
Tomašev, Franklin & Osindero
DeepMind's running thread on the AI agent economy.
DEXAI – Icaro Lab
Bisconti, Pierucci & Galisai
Microfoundations of macro safety.
Cooperative AI Foundation × GovAI
Hammond & Chan
Multi-agent risk and coordination infrastructure.
Org Map
Org map.
A map of the people, organizations, and companies working on large agent systems.
Open the mapPart Three
Take action.
In the face of potentially irreversible risks, the time to start working on large agent systems is now. We highlight some pressing open problems and barriers the field faces in the near-term.
Barriers
Barriers to overcome.
Collective Action Problem
Many elements of large agent safety are not in the business interest of any individual company. Consequently we see a lack of investment in multi-agent evals and little discussion of large-agent safety at frontier labs.
Lack of Data
Pure-AI LAS are few; agent presence on mixed systems is hard to identify; and large-scale simulation papers rarely publish their simulation data.
Inter-disciplinary Collaboration
LAS calls for highly interdisciplinary specialised teams, drawing on AI safety, social sciences, and scaling engineering. It takes time and contact to connect fields who don't usually talk, particularly going outside the academy.
Streetlight Effects
LAS work is new, highly specialised, and outside the curriculum of major AI safety training programmes like ARENA, making it costlier for new people to enter the space than to work on established agendas like mechanistic interpretability or single-agent evals.
Community
Join the community.
We run a Slack community for AI safety researchers, social scientists, and policy and strategy experts working on large agent systems.
Request to join