Skip to content

Incoherence Scaling: Initial Experiment Artifact

This note describes the initial scaling artifact generated by:

examples/run_incoherence_scaling.py

Scope

  • Runs replay sweeps over three scenario tiers:
  • short|low branching
  • medium|medium branching
  • long|high branching
  • Uses ReplayRunner with configurable --replay-k.
  • Exports:
  • incoherence_scaling_replays.csv
  • incoherence_scaling_summary.csv

Current Incoherence Signal

The current script uses a temporary proxy from replay toxicity variation to produce: - incoherence_index - error_rate - disagreement_rate

This is a bridge until full step-level decision replay integration lands in the scaling runner.

Intended Next Upgrade

  • Replace proxy signal with decision-level replay metrics from swarm/metrics/incoherence.py:
  • benchmark-based E
  • replay disagreement D
  • I = D / (E + eps)