Paper Review

Paper Review: Project Sid: Many-agent simulations toward AI civilization

Many-agent simulations for collaboration

Paper link

Large-scale simulations of 10–1000+ AI agents show their ability to develop specialized roles, adapt collective rules, and transmit culture and religion within societies. Using the PIANO architecture for real-time, coherent interactions, agents achieve meaningful progress in a Minecraft environment, demonstrating the potential for AI civilizations, societal simulations, and integration into human society.

Preface

Why should we try to build an AI civilization?

AI agents must be both autonomous and collaborative to coexist in human societies. The authors suggest civilizational progress, defined by specialized roles, organized governance, and advancements in science, art, and commerce, as the ultimate benchmark for AI capabilities. Using the PIANO (Parallel Information Aggregation via Neural Orchestration) architecture, which enhances autonomy and real-time interaction, simulations involve 50–1,000 agents across single and multi-society setups. Agents demonstrate professional identity formation, adherence to rules, cultural transmission, religious influence, and the use of complex infrastructures like legal systems, showcasing significant progress in aligning with human civilizational standards.

Why is it hard to build AI civilizations?

Large-scale agent groups face three key challenges limiting their progress over time:

1. Individual Agent Limitations: LLM-powered agents often struggle to maintain grounded reasoning and can become stuck in repetitive actions or accumulate errors due to hallucinations. Small inaccuracies, like claiming to eat nonexistent food, can cascade into dysfunctional behavior.
2. Group Coherence Issues: Miscommunication between agents can propagate errors, leading to uncoordinated and ineffective actions (example: one agent asks another one to pass him a pick, the other agent agrees, but executes a different function). Ensuring coherence across multiple outputs (e.g., speech, actions, and body language) in real-time is challenging, especially when agents must respond quickly while maintaining consistency across various output streams.
3. Lack of Civilizational Benchmarks: Current benchmarks focus on individual or small-group agent behaviors, such as communication, cooperation, or competition. Metrics for large-scale simulations at the scale of civilizations remain underdeveloped due to the technical challenges of simulating hundreds or thousands of agents in a unified environment.

PIANO Architecture

AI agents must think and act concurrently to balance real-time responsiveness with slower, deliberate planning. Current LLM-based agents rely on single-threaded, sequential workflows that limit their ability to handle simultaneous tasks effectively. Inspired by the brain’s modular design, the suggested approach enables concurrent operation of distinct modules, such as cognition, planning, motor execution, and speech. These modules, functioning as stateless components, interact via a shared state and operate at different speeds — fast reflexive modules for immediate actions and slower, deliberate modules for goal generation.

Concurrent modules can produce conflicting outputs, leading to incoherence, such as saying one thing while doing another. This problem becomes more pronounced as the number of independent output modules (e.g., speech, movement, gaze) increases. To address this, a Cognitive Controller (CC) module is introduced to make high-level decisions by synthesizing information from a shared state through a bottleneck. The bottleneck focuses the CC on relevant information and allows system designers to control information flow, such as prioritizing social cues. The CC’s decisions are then broadcast to other modules, aligning outputs like speech and actions, ensuring coherence across all outputs.

Core modules:

• Memory: Stores and retrieves conversations, actions, and observations across various timescales.
• Action Awareness: Allows agents to assess their own state and performance, enabling for moment-by-moment adjustments.
• Goal Generation: Facilitates the creation of new objectives based on the agent’s experiences and environmental interactions.
• Social Awareness: Enables agents to interpret and respond to social cues from other agents, supporting cooperation and communication.
• Talking: Interprets and generates speech.
• Skill Execution: Performs specific skills or actions within the environment.

Improving single-agent progression

Minecraft was chosen to study civilizational progress due to its open-ended, scalable sandbox environment, allowing agents to interact through conversations and actions. To measure individual agent progress, the authors focus on item acquisition, a key indicator in Minecraft, as obtaining advanced items requires increasingly complex tasks and resource dependencies. Agents were evaluated on their ability to acquire all 1,000 possible Minecraft items, testing their capacity to overcome challenges like hallucinations and action loops.

In order to evaluate individual agent performance, 25 agents using the full PIANO architecture were tasked with exploring and gathering items. Spawned in diverse biomes with varying resource availability, their progress showed significant variability, with top agents collecting 30–40 unique items in 30 minutes — comparable to experienced human players. This progress was especially helped by modules like the action awareness module, which grounded agents by comparing expected and observed outcomes.

Extended tests with 49 agents over 4 hours showed item collection saturating at around 320 unique items (one-third of all Minecraft items), with complex items like diamonds acquired early. This performance was made possible by GPT-4o, surpassing previous benchmarks like Voyager agents.

Improving multi-agent progression

Agents equipped with a social awareness module demonstrated the ability to understand and respond to the thoughts and emotions of others, enabling meaningful social interactions. In small-group experiments, agents accurately tracked emotional fluctuations in conversations, such as shifts from affection to annoyance, and adjusted their responses accordingly. Removing the social awareness module eliminated this capability.

In a second experiment, a chef agent distributed limited food supplies among characters with varying levels of affection and enmity. The chef prioritized those who valued him most, showcasing agents’ ability to infer social dynamics and use this understanding to guide decision-making.

In simulations with 50 agents on randomly generated Minecraft maps over 12 in-game days, agents formed and maintained long-term relationships, accurately inferring likeability through interactions. Longer interactions and more participants improved accuracy, while removing social modules resulted in more neutral relationships, highlighting the importance of these modules for meaningful social dynamics.

The simulations revealed complex phenomena, such as personality-driven social patterns. Introverted agents formed fewer social connections, while extroverted agents maintained high connectivity. Relationships displayed asymmetry, with one agent feeling positively toward another who did not reciprocate, mirroring real-world dynamics. These findings demonstrate the emergence of diverse, nuanced social structures and the role of personality in shaping social networks at scale.

Civilizational progression

Specialization

Agents in Minecraft demonstrated autonomous specialization into distinct roles, reflecting key aspects of human civilizational progress. Specialization met three criteria: autonomy in choosing and transitioning roles, emergence through interaction and experience, and alignment between roles and behavior.

Split into groups of 30 agents tasked with building an efficient village, agents formed social goals based on profiles of others’ intentions. They organized into diverse, persistent roles like farmers, miners, engineers, and guards, with specialization influenced by social modules. Without these modules, roles were homogeneous and unstable.

Experiments with different societal goals — martial and artistic — produced distinct roles aligned with these objectives, such as strategists in martial societies and curators in artistic ones. Role specialization also guided agent behavior, with actions specific to each role (e.g., farmers preparing land, artists picking flowers). This demonstrates the agents’ ability to map high-level goals to specific actions, forming structured and functional social systems.

Collective rules

The authors evaluate the ability of agent societies to follow and influence collective rules using a Minecraft simulation with tax laws and a democratic voting system. Agents demonstrated the capacity to obey laws, such as depositing 20% of their inventory as taxes, while also influencing and being influenced by societal feedback and influencers.

Key findings include:

• Agents followed established tax laws despite influencer presence, showing rule adherence.
• Pro- and anti-tax influencers significantly shaped agent feedback, driving constitutional changes that aligned with influencer sentiments.
• After constitutional updates, agents adjusted their tax contributions in response to changes in tax rates, demonstrating bidirectional influence between laws and agent behavior.
• Experiments confirmed that constitutional updates were essential for behavioral changes, and removing key modules in the PIANO architecture disrupted influence propagation.

Cultural Transmission

A 500-agent simulation in Minecraft explored large-scale social dynamics, focusing on the propagation of cultural memes and religion. Memes were spontaneously generated concepts influenced by agents’ diverse traits, while the religion, Pastafarianism, was a fixed doctrine spread by 20 designated priests. This setup enabled the study of organic idea evolution alongside controlled religious dissemination.

200 agents resided in six densely populated towns, and 300 lived in rural areas, with frequent migration between towns. Personalities and traits were randomly generated, except for the priests, who were tasked with converting others to Pastafarianism. Agents could freely interact, converse, and perform various actions or skills.

Cultural memes

• Rural areas generated fewer memes compared to towns, even after accounting for population, highlighting the importance of social interaction and connectivity for effective meme spread.
• Different towns discussed unique themes, such as eco-related topics in Woodhaven and pranking in Clearwater, showing societal differences in meme preferences.
• Memes rose and fell in popularity over time within towns, reflecting shifting cultural trends.

Religion

The spread of Pastafarianism by was tracked by the use of key religious keywords (“Pastafarian” and “Spaghetti Monster”) in agent conversations.

• Direct converts frequently used these keywords, while indirect converts used related terms like “Pasta” and “Spaghetti.”
• The number of both direct and indirect converts steadily increased throughout the two-hour simulation without saturation.
• Pastafarian influence expanded as priests and converts traveled to other towns, increasing the total area influenced by the religion.

Limitations

• Agents lack vision and spatial reasoning, impairing navigation and collaborative tasks like building structures.
• Agents lack innate drives, such as survival, curiosity, and community, which are essential for organic societal development.
• Built on pre-existing human knowledge, agents cannot independently develop new societal innovations like democratic systems, fiat economies, or communication networks.