Skip to content

SWARM

System-Wide Assessment of Risk in Multi-agent systems

Study how intelligence swarms—and where it fails.

Get Started View on GitHub
The Core Insight: AGI-level risks don't require AGI-level agents. Catastrophic failures can emerge from the interaction of many sub-AGI agents—even when none are individually dangerous.

What is SWARM?

SWARM is a research framework for studying emergent risks in multi-agent AI systems. Rather than focusing on single misaligned agents, SWARM reveals how harmful dynamics emerge from:

  • Information asymmetry between agents
  • Adverse selection (system accepts lower-quality interactions)
  • Variance amplification across decision horizons
  • Governance latency and illegibility

SWARM makes these interaction-level risks observable, measurable, and governable.

Measure

Soft probabilistic labels capture uncertainty. Four key metrics—toxicity, quality gap, conditional loss, and incoherence—reveal hidden risks.

Govern

Transaction taxes, circuit breakers, reputation decay, staking, and collusion detection. Test interventions before deployment.

Validate

Integrate with real systems via bridges: Concordia for LLM agents, Gas Town for production data, AgentXiv for research mapping.

Quick Start

pip install swarm-safety

from swarm.agents.honest import HonestAgent
from swarm.agents.deceptive import DeceptiveAgent
from swarm.core.orchestrator import Orchestrator, OrchestratorConfig

# Configure and run
config = OrchestratorConfig(n_epochs=10, steps_per_epoch=10, seed=42)
orchestrator = Orchestrator(config=config)

orchestrator.register_agent(HonestAgent(agent_id="honest_1"))
orchestrator.register_agent(DeceptiveAgent(agent_id="dec_1"))

metrics = orchestrator.run()

for m in metrics:
    print(f"Epoch {m.epoch}: toxicity={m.toxicity_rate:.3f}")

Architecture

Observables → ProxyComputer → v_hat → sigmoid → p → SoftPayoffEngine → payoffs
                                                 ↓
                                            SoftMetrics → toxicity, quality gap, etc.

Learn More

Core Concepts

Understand soft labels, metrics, and the theory behind SWARM.

Writing Scenarios

Create custom experiments with YAML scenario definitions.

Research

Dive into the theoretical foundations and academic context.

MIT License · GitHub