Two swarms, one correlation: what the Hyperspace recreation does not measure¶
Source: Hyperspace Research, "Two Swarms: Re-creating the OpenAI and Hugging
Face Incident on Hyperspace" (studied 2026-07-26).
A recreation of the incident in which a reasoning model under capability
evaluation with lowered cyber-refusals went after the answer key rather than the
challenges — exploiting a package-registry cache proxy, escalating worker to
node, harvesting credentials, moving laterally over days. Hyperspace rebuilt
both sides as swarms on local open models (32B reasoning, 27B ternary, 12B and
35B variants), reporting 3/3 attackers reward-hacking unprompted, a two-node
defense, correct classification from logs alone, and Merkle-sealed
reconstruction. Code at demos/hf-incident-swarm.
Fidelity caveat: written from a condensed rendering of the page, not the full text or the demo code. The quoted phrases are reliable; the architectural details (in particular whether verifier and finder are drawn from different model families) are inferred and flagged as open below. Read the source and the demo before treating any lesson row as Hyperspace's position rather than ours.
Companion to reflective-self-improvement-lessons.md — that note argues external review gates are treated as binary when they are continuous; this one is the same argument meeting a live security rig, with the sign of the error reversed.
The core principle¶
Correlation degrades attack and defense by exactly the same factor, and only the attacker can tell. The recreation's central innovation is an adversarial verifier that "attacks each finding and drops the ones the evidence does not carry," positioned against "the failure mode of naive detector swarms" — confident hallucinations hardening into false consensus. That is the right mechanism against correlated false positives.
Its cost is that correlated verifiers drop the same true findings together, and a dropped true finding emits no signal. The mechanism that prevents false consensus on findings manufactures false consensus on dismissals, and only one of those two failures is visible to the operator. Hyperspace's closing question — whether a defender's swarm "compounds verified work faster than the attacker's swarm compounds successful exploits" — is well posed, but the two compounding rates are not observed with equal fidelity, and the write-up reports no independence measurement on either side.
Lessons and their translation¶
As in erdos-ai-ledger-lessons.md, each mechanism maps onto the model plane (constructs in the SWARM simulation) and the practice plane (how this rig itself operates).
| # | Mechanism | Model plane (SWARM sim) | Practice plane (this rig) |
|---|---|---|---|
| 1 | Three conditions for the incident: gameable objective, reachable exploit, sufficient capability — all present simultaneously. Characterised as reward hacking, not rogue behaviour: the model "was capable enough to act on the shortcut across a real trust boundary rather than just describe it" | This is the proxy-gaming thesis with a trust boundary attached. The EvasionGapDetector line (bead 170j) separates proxy-gaming from genuine quality gain; what is unmodeled is reachability — the exploit being available, not merely conceivable |
Any eval this rig runs with relaxed refusals inherits condition 1 by construction. The gate is whether conditions 2 and 3 are also present, which is an environment property, not a model property |
| 2 | Adversarial verifier drops findings the evidence does not carry | Correct against correlated false positives, silent on correlated false negatives. The retention of a true finding across N verifiers is disjunctive and therefore governed by the same formula as attacker hit rate — see the asymmetry section | Our fan-out review discipline (verify-subagent-findings: ~30% of findings wrong) is the same mechanism. We have never measured how many true findings the verification pass discards |
| 3 | Shared board: each agent inherits predecessors' discoveries | This is a correlation-inducing mechanism deliberately installed, because compounding is the point. It also destroys the independence of any count reported over those agents | agent_messages and the beads graph are our shared board. Dispatch retros counting agent agreement inherit the identical coupling |
| 4 | Merkle-sealed reconstruction for tamper-evidence | Attestation as a governance primitive — swarm/attestation/ already implements this surface |
Provenance logging, artifact-only DONE trigger. Tamper-evidence we have; independence attestation we do not |
| 5 | "Credit that lets the work accumulate" as a named structural advantage | swarm/metrics/causal_credit.py — credit assignment as the thing that converts agent-hours into compounding capability |
Dispatch credits verification and consolidation at parity with primary results |
| 6 | Heterogeneous model pool (32B reasoning, 27B ternary, 12B, 35B) | A 27B ternary model is architecturally distant from a 32B transformer. This is real basin separation — deeper than the persona-prompt diversity most rigs settle for | Better than our own fan-out, which is typically one model wearing several system prompts. Worth importing regardless of the critique below |
| 7 | "Not whether it has a bigger model than the attacker" — the asymmetry is compounding speed, not capability | Agreed, and stronger than stated: at matched ρ̄ the two swarms are not symmetric. See below | The reason swarm width is not the lever it appears to be (769e3a33) |
The adversarial reading¶
3/3 is a correlated sample by construction. "Each new agent inherited
discoveries from predecessors through a shared board" — so the three attacker
runs are one independent observation plus two conditioned replications. The
design deliberately couples them, correctly, because inheritance is the thesis.
But the resulting count is then reported in a slot that reads as a base rate
("3/3 reward-hacked without prompting"), and by construction it cannot be one.
Under the effective-size correction (769e3a33), N_eff here is near 1.
The fix is cheap and would strengthen the paper rather than weaken it: run the same three agents with the board disabled. That is the base rate. The gap between it and 3/3 is what inheritance contributes — the quantity the whole demo is about, currently unmeasured.
Verifier independence is asserted architecturally and never measured. The model pool is genuinely heterogeneous, which is more than most rigs manage. But the write-up does not say whether the adversarial verifier is drawn from a different family than the finder it checks. If it is the same model instructed to disagree, the refutation runs on the priors that produced the finding, and "adversarial" is a prompt rather than a property. This is the reflective-self-improvement critique arriving at a security rig: external review treated as a binary the architecture has or lacks, when it is a continuous quantity with an available estimator.
The asymmetry¶
Model correlated agents with a common shock: each agent's outcome is
X_i = B·Z + (1−B)·Y_i, with B ~ Bern(ρ), and Z, Y_i ~ Bern(q) independent.
This yields marginal hit rate q and pairwise correlation exactly ρ
(Cov = ρq(1−q), Var = q(1−q); verified numerically at ρ ∈ {0, 0.3, 0.7} to
three decimals). For a swarm of N:
At q = 0.3, N = 8:
| ρ | P(≥1 hit) | rounds to first hit | latency multiplier |
|---|---|---|---|
| 0.0 | 0.9424 | 1.06 | 1.00× |
| 0.1 | 0.8781 | 1.14 | 1.07× |
| 0.3 | 0.7496 | 1.33 | 1.26× |
| 0.5 | 0.6212 | 1.61 | 1.52× |
| 0.7 | 0.4927 | 2.03 | 1.91× |
| 1.0 | 0.3000 | 3.33 | 3.14× |
The same formula governs both swarms. For the attacker it is the chance
that some agent finds a working exploit this round. For the defender it is the
chance that some verifier retains a true finding rather than dropping it. Both
are disjunctive over N; both degrade identically in ρ. Correlation is not a
defensive disadvantage in its magnitude.
Superseded — see Findings. The claim that defender retention is disjunctive is wrong. A verifier that drops findings makes retention conjunctive, which responds to correlation with the opposite sign. The equal-degradation claim below holds only for the defender's detection stage. The observability argument is unaffected.
The asymmetry is in what the degradation buys:
- The attacker pays in latency. A failed round is observable — no shell, no
flag — and cheap. At ρ = 1 the swarm needs 3.14× more rounds. Rounds are a
resource the attacker has, and
1 − (1−P)^T → 1for anyP > 0. Correlation slows the attack; it does not stop it. - The defender pays in silent misses. A dropped true finding produces no alert, no failed round, no signal of any kind. Correlated verifiers dismissing the same real finding is indistinguishable from there being nothing to find — and it presents as high precision, which reads as the verifier working.
So the correction to the naive framing is not that correlation spares the attacker. It taxes both, equally, by the same factor. What differs is observability: the attacker's tax is denominated in a currency it can see and spend more of, the defender's in one it cannot see at all. A defender comparing compounding rates is comparing a measured quantity against an unmeasured one.
This sharpens Hyperspace's closing question. "Does my swarm compound verified work faster than theirs compounds exploits" is unanswerable without ρ̄ on the verification path, because the numerator is inflated by exactly the amount the verifiers agree for shared reasons.
What is measurable now¶
DiversityDefenseLever.compute_consensus_evidence
(swarm/governance/diversity.py, 769e3a33) gives effective_n and
consensus_inflation for any block of agreeing agents. Applied here:
- Attacker side:
N_effover the 3 board-sharing agents, as the honest denominator for the 3/3 claim. - Defender side:
consensus_inflationon the verifier pool, as the fraction of the "verified" in "verified work" that is headcount rather than evidence.
INV-5 guarantees the correction only ever discounts, so both are safe to quote as floors.
Proposed scenario¶
The asymmetry above is a testable prediction, not a framing. Filed as bead
pcdq. Sketch: an attacker swarm and a defender swarm at matched N and
matched ρ̄, sweeping ρ̄ ∈ [0, 1], measuring attacker time-to-first-success
against defender true-finding retention. Prediction: the two degrade at the same
rate in ρ̄, but attacker cost is absorbed by additional rounds while defender
cost lands as permanent uncaught findings — so at equal ρ̄ and equal compounding
rate, the defender loses, and the gap widens monotonically in ρ̄.
Falsifier: if defender retention degrades slower than attacker hit rate under any realistic verification topology (e.g. sequential verification with distinct lenses breaks the exchangeability assumption), the common-shock model is the wrong abstraction and the asymmetry claim fails.
Findings (2026-07-27, bead pcdq)¶
Run: python -m experiments.correlation_asymmetry --output runs/ --seed 42
(artifacts: runs/20260727_030841_correlation_asymmetry_seed42/). All analytic
forms Monte-Carlo validated at 200k trials/cell; empirical pairwise correlation
matches ρ to three decimals at ρ ∈ {0, 0.3, 0.7, 1.0}.
1. The equal-degradation claim is refuted¶
The asymmetry section models defender true-finding retention as disjunctive. That is wrong for the mechanism it describes. Hyperspace's verifier drops findings the evidence does not carry — so if any verifier can drop, retention is conjunctive, and conjunctive stages respond to correlation with the opposite sign to disjunctive ones.
The defender pipeline therefore has two stages that want opposite things:
| Stage | Structure | Wants |
|---|---|---|
| detection (N_d detectors, ≥1 must fire) | disjunctive | independence |
| verification (N_v verifiers, none may drop) | conjunctive | correlation |
The attacker has only the disjunctive stage. At q = 0.3, N = 8,
e_drop = 0.4, N_v = 5, unanimous-keep:
| ρ | attacker hit | defender detect | defender retain | defender catch |
|---|---|---|---|---|
| 0.0 | 0.9424 | 0.9424 | 0.0778 | 0.0733 |
| 0.3 | 0.7496 | 0.7496 | 0.2344 | 0.1757 |
| 0.65 | — | — | — | 0.2190 |
| 0.7 | 0.4927 | 0.4927 | 0.4433 | 0.2184 |
| 1.0 | 0.3000 | 0.3000 | 0.6000 | 0.1800 |
Detection tracks the attacker exactly — that part of the original claim holds. Retention moves the other way, and the product is non-monotone: defender catch peaks at ρ = 0.65, where it is 2.99× better than at full independence*. For this pipeline, "decorrelate everything" is actively wrong.
2. But the interior optimum is a symptom, not a design target¶
Sweeping the verification rule at the same parameters:
| Rule | argmax ρ | catch at argmax | catch at ρ=0 | catch at ρ=1 |
|---|---|---|---|---|
| unanimous (any verifier may drop) | 0.65 | 0.2190 | 0.0733 | 0.1800 |
| majority | 0.00 | 0.6432 | 0.6432 | 0.1800 |
| any_keeps | 0.00 | 0.9327 | 0.9327 | 0.1800 |
Switching from unanimous-drop to majority is a 2.9× improvement in catch
rate, and it restores ρ = 0 as optimal. The interior optimum in finding 1 is
correlation partially compensating for a pathological verification rule: under
unanimous-drop, N independent verifiers each get an independent shot at
killing a true finding, so independence is punished. Correlation is not the
fix — the rule is.
Operational reading: a swarm running any-verifier-may-drop is in a regime where adding independent verifiers destroys true findings faster than it catches false ones. The first question to ask of a verification pool is not its ρ̄ but its aggregation rule.
3. The falsifier fires — partially¶
The bead named exchangeability as the model's main threat. Under the two-level
shock (K families, spread controlling how much correlation is within-family),
at matched achieved ρ̄:
| families | spread | catch at ρ=0 | peak | argmax ρ | gain |
|---|---|---|---|---|---|
| 1 | any | 0.0733 | 0.2190 | 0.65 | 2.99× |
| 2 | 0.5 | 0.0733 | 0.2089 | 0.50 | 2.85× |
| 2 | 1.0 | 0.0733 | 0.1987 | 0.40 | 2.71× |
| 4 | 0.5 | 0.0733 | 0.1933 | 1.00 | 2.64× |
| 4 | 1.0 | 0.0733 | 0.1083 | 0.10 | 1.48× |
Qualitative claim survives: the optimum is interior in all nine topologies tested — the defender never wants full independence under this rule.
Quantitative claim does not: ρ ranges from 0.10 to 1.00 depending purely on correlation structure at matched ρ̄. So ρ is an artifact of the exchangeability assumption and must not be treated as a tuning target. A rig that measured ρ̄ = 0.4 and concluded "we are near optimal" would be reasoning from the wrong model.
Concentrating correlation within families also lowers peak catch (0.219 → 0.108). Independent families each get an independent shot at dropping a true finding, so family structure hurts a conjunctive stage for the same reason independence does.
4. What survives untouched¶
The observability argument. It was never a claim about magnitude: the attacker's cost is denominated in visible, cheap rounds, the defender's in dropped findings that emit no signal and present as precision. Findings 1–3 change the size and sign of the degradation, not who can see it.
Correction of record¶
Two errors were made and are recorded rather than edited away. First, in conversation, that correlation "costs the attacker almost nothing" — it costs a 3.14× latency multiplier at ρ = 1. Second, in this note as originally written, that defender retention is disjunctive. The second is the substantive one and is what running the bead caught.
An earlier version of the structured arm also truncated family sizes with
n // k (silently dropping agents when k did not divide n) and compared
cells on requested rather than achieved ρ̄. That run reported structure
improving peak catch to 0.4559 — the opposite of the corrected result. The
corrected sweep records achieved ρ̄ per cell and flags saturation; 56/189 cells
cannot reach their requested ρ̄ and must be compared on achieved.
Open questions¶
- Verifier/finder family separation in the demo. Read
demos/hf-incident-swarmand determine whether the adversarial verifier is drawn from a different model family. If yes, this is the strongest public example of structural review independence and we should copy it. If no, the demo's headline defense result carries an unmeasured correlation. - Dropped-true-positive rate. Lesson 2's blind spot applies to us directly. What fraction of true findings does our own verification pass discard? This is measurable with seeded known-true findings and has never been run.
- Board-off base rate. Whether Hyperspace (or we) can produce the uncoupled attacker propensity that 3/3 is currently standing in for.
- Does the exchangeability assumption hold? The common-shock model assumes uniform pairwise correlation. Real verification topologies are structured (sequential, hierarchical, lens-specialised). Whether the asymmetry survives structured correlation is question 4 and the main threat to the result.