13 min read

From One Research Agent
to a Swarm: Parallel
Trading Signal Discovery

Running twenty agents in parallel instead of one sequential researcher doesn't make your search better — by default it just makes your multiple-testing problem twenty times bigger. Here's the architecture that keeps it honest, including the bug it caught and the result it overruled.

The obvious way to speed up an agentic research loop is to stop running experiments one at a time. Spin up twenty Claude Code subagents, give each one the same feature set, let them each propose a trading signal, backtest all twenty, keep the best one. It's tempting because it's easy to build and the first result usually looks great.

It's also, without additional structure, just p-hacking with extra parallelism. Twenty independent searches against the same historical data will produce at least one candidate that clears a Sharpe threshold by chance alone — that's not a discovery, it's the expected behavior of a multiple-testing problem at n=20. Marcos López de Prado's "probability of backtest overfitting" work exists precisely because this failure mode is so easy to fall into and so hard to notice from inside the search. Building a swarm that's actually trustworthy — not just fast — means designing the coordination and the validation funnel as carefully as the agents themselves. This is a real architecture built and run this way, including a bug it caught and a promising-looking result it correctly overruled.

Wave-Based, Not Blind-Parallel

The naive version spawns N agents with identical instructions and no visibility into each other. Every agent independently reinvents the two or three most obvious ideas, and the swarm's total output is a handful of genuinely distinct hypotheses buried in a pile of near-duplicates. The fix is conditioning each wave on what the previous wave already tried:

  • Each subagent gets the same feature vocabulary (a fixed, documented whitelist — never raw column access) and the same hard constraints as the single-agent loop.
  • Each subagent also gets a summary of what's already in the ledger — signal names, hypotheses, and outcome — so proposing a near-copy of an existing entry is visibly pointless before any backtest runs.
  • A mechanical similarity check (Jaccard overlap between a candidate's fired events and every ledger entry's fired events, not a semantic or textual comparison) catches near-duplicates the agent didn't recognize as duplicates itself.

This turns "run more agents" from a volume lever into a coverage lever. The goal of a wave isn't more candidates, it's more of the search space actually explored — which means the second wave should look different from the first, informed by what the first one found and discarded.

A Proposal Format an Agent Can't Break Out Of

The single biggest engineering constraint in a multi-agent research swarm isn't the search logic, it's the boundary around what an agent-proposed signal is allowed to do. Letting twelve subagents each write and execute arbitrary Python against your data pipeline is a wide, unaudited surface — and unnecessary, because a trading signal is fundamentally a small set of comparisons against a fixed feature vocabulary. The proposal schema reflects exactly that, and nothing more:

{
  "name": "illustrative_coil_breakout",
  "hypothesis": "A close breaking a short-term high out of a tight recent
                  range, on above-average volume, catches continuation
                  earlier than a slower breakout confirmation would.",
  "combine": "AND",
  "conditions": [
    {"feature": "breakout_10", "op": ">=", "value": 1},
    {"feature": "range_5d",    "op": "<",  "value": 4.0},
    {"feature": "rvol",        "op": ">",  "value": 1.75}
  ]
}

Every field is validated mechanically before anything runs: feature must be in a fixed whitelist, op must be one of a handful of comparison operators, and the compiled signal is built with plain dict indexing and boolean masks — never eval(), never a code string the agent wrote directly. An agent can propose a bad hypothesis. It cannot propose arbitrary code. That distinction is what makes running a dozen of them unattended a reasonable thing to do at all.

A Funnel, Not a Leaderboard

Every candidate that survives the redundancy check goes through the same staged funnel, in order, and most of them don't make it to the end:

Stage What it checks Why it's cheap-first
1 — Screen Minimum trade count, coverage across CPCV paths, fraction of paths with a positive result Fast descriptive stats — rejects the majority before any expensive step
2 — Redundancy Event overlap against production signals and every other surviving candidate A signal that fires on the same dates as an existing one adds cost, not information
3 — Portfolio test Does adding this signal to the existing book actually improve the combined result, incrementally A signal can be individually fine and still be a net negative once correlated with what's already there
4 — Full CPCV + dual deflated Sharpe Real purged cross-validation, plus two separate deflated-Sharpe checks — see below Only run on whatever's left after 1–3, because it's the expensive step

Ordering the funnel cheap-to-expensive is what makes running a dozen agents in parallel computationally reasonable — most candidates get eliminated at stage 1 or 2, in seconds, and only a handful ever reach the full CPCV pass that takes real compute.

The data every stage of this funnel runs against

None of this works without a survivorship-bias-free, point-in-time universe — a candidate signal backtested against today's Nasdaq-100 constituent list is being validated against a universe that never existed. The NDX PIT Dataset is 4,957 trading days × 272 tickers of exactly that.

The Bug That Cost a Wave — and What It Taught

An early wave of twelve subagents was given a feature vocabulary doc that described three features as percentage-scaled when the underlying pipeline actually left them as raw fractions — an unglamorous, entirely human documentation error. The practical effect: seven of the eight candidates that referenced those features required thresholds that were numerically impossible to hit (a "return greater than 15" condition against a feature actually scaled 0–1 fires on nothing), and one more used a feature in a direction that was mathematically excluded by its own definition. Eight of twelve candidates were silently dead on arrival, and the screening stage caught it immediately — zero fired events is about as unambiguous a stage-1 failure as exists.

The fix was mechanical once diagnosed: correct the vocabulary documentation, re-scale the feature computation to match what was actually documented, and re-spawn the affected agents with the corrected reference. The broader lesson is less about this specific bug and more about what caught it: a cheap, fast, mandatory stage-1 screen turns an expensive silent failure (garbage candidates quietly making it deep into a validation pipeline) into an immediate, visible one. That property matters more as the number of parallel agents grows, not less — the more candidates a wave produces, the more valuable it is that bad ones fail loudly and early rather than consuming compute at stage 4.

Two Deflated Sharpes, Answering Two Different Questions

The last stage runs the same Bailey–López de Prado deflated Sharpe ratio calculation twice, against two different sample sizes, because they answer genuinely different questions:

  • Across the CPCV paths of the selected model — is the specific strategy that came out of this process robust across independent out-of-sample windows, or did it get lucky on a subset of them?
  • Across every trial the whole search process ran — using the total candidate count (every proposal that reached a scoreable stage, kept or discarded) as the sample size for the deflated-Sharpe formula's multiplicity correction. This asks a different question entirely: given how many things were actually tried, is the process that produced this candidate trustworthy, or did something look good purely because enough independent attempts were made?

These two numbers can and do disagree. A candidate can look excellent evaluated only against its own CPCV paths — because that check has no way to know how many other candidates were tried and discarded to arrive at it — while looking far weaker once the deflated-Sharpe formula is told the true size of the search that produced it. In one full run of this architecture, exactly that happened: the strongest individual candidate held up cleanly in isolation, but the combined portfolio result that looked most attractive after a simplified validation pass showed a real capacity and position-crowding confound once tested under the production book's actual concurrent-demand limits, and its regime consistency didn't hold across every out-of-sample window — two paths covering a high-volatility period got worse, not better. The search-level deflated Sharpe reflected that honestly where the per-path number alone would not have.

The recommendation out of that run wasn't "deploy the best candidate." It was "tighten the funnel and re-derive" — not a failure of the swarm, the correct output of one. A validation step that only ever produces "ship it" isn't doing its job; the entire point of running the dual deflated-Sharpe check is that it's allowed to overrule a result the earlier, cheaper stages of the funnel made look promising.

What Actually Transfers to Other Search Problems

None of the specific trading logic here is the reusable part — the reusable part is the shape of the discipline, and it applies to any domain where you're using an agent swarm to search a large hypothesis space against a validation metric that can be overfit:

  • Fence the search surface. Agents propose within a fixed, whitelisted vocabulary — never arbitrary code — the same principle as the single-agent loop's protected files, applied to a schema instead of a file boundary.
  • Condition each wave on prior waves. A swarm that can't see its own history just re-explores the same obvious territory N times.
  • Cheap screens before expensive ones. Order the funnel so bad candidates die in milliseconds, not after a full validation pass.
  • Multiplicity-correct the final answer. However good the winning candidate looks in isolation, the validation has to account for how many candidates were actually tried to find it — or it isn't measuring what you think it's measuring.
  • Let the process say no. If every run of your swarm ends in "ship it," the validation step isn't doing anything the agents wouldn't have claimed on their own.
The core argument

What separates a research swarm from parallelized p-hacking

  • A structured, no-eval() proposal schema — agents can propose a bad hypothesis, never arbitrary code
  • Ledger-conditioned waves and mechanical redundancy checks, so parallelism buys coverage, not duplicate noise
  • A cheap-to-expensive funnel that kills bad candidates in milliseconds, not after a full CPCV pass
  • Two deflated-Sharpe checks — one for the selected model, one for the search process that found it
  • The honest output of a real run is sometimes "don't deploy yet" — and that's the check working, not failing