Building an Autonomous Trading
Strategy Research Loop
with Claude Code

Most people use Claude Code for trading research the same way they'd use it for any coding task: they describe an idea, the model writes a backtest, they read the Sharpe ratio, and they move on. That's fine for one experiment. It falls apart the moment you want to run fifty of them, because nothing stops the agent from quietly changing the thing that makes a backtest honest — the universe construction, the label definition, the validation split — while chasing a better number.

The fix isn't a better prompt. It's an architecture that makes cheating structurally impossible: separate the parts of the pipeline that must never change from the one part the agent is allowed to touch, give it a metric that can't be gamed by luck, and let it run in a loop that forgets nothing important between batches. crucible is a real, working, open-source implementation of that idea for Nasdaq-100 signal research — not a toy demo, the actual harness this site's own methodology work uses. This walks through how it's built and why each constraint exists.

The Boundary That Makes This Safe

The repo has five files that matter. Four of them the agent is explicitly told never to touch:

This is the single most important design decision in the whole system, and it has nothing to do with prompting. An agent that can edit the cross-validation code can always find a split that makes any strategy look good — that's not malice, it's exactly what "improve this number" optimization pressure produces when nothing constrains the search space. Fence the validation infrastructure off entirely and the only lever left is the actual signal rule. Diffs stay small, `git diff strategy.py` tells you exactly what changed, and a bad experiment reverts with `git checkout -- strategy.py` in one command.

The interface the agent has to satisfy is a plain function contract, not a framework to learn:

get_signals(train_features, train_labels, test_features) → dict[date, Series]

# train_features / test_features: MultiIndex (date, ticker), CS z-scored columns
# train_labels: +1 (upper barrier hit), -1 (lower barrier hit), 0 (time expired)
# return: {test_date: Series({ticker: signal})}
#   signal > 0  = open an independent long signal for that ticker/date
#   signal magnitude = conviction weight (meta-label probability, Kelly
#   fraction, or just 1.0 for unweighted) — same-day exits are averaged,
#   weighted by this value
#
# Exit is handled entirely by the fixed triple-barrier box in backtest.py —
# the agent never touches position sizing, rebalance cadence, or exposure caps.

A Metric That Survives Being Optimized Against

The second failure mode of naive agentic research is Goodhart's law with extra steps: tell the agent to maximize Sharpe ratio and it will eventually find the one lucky train/test split that produces a great Sharpe ratio on a strategy that doesn't actually work. crucible's harness runs Combinatorial Purged Cross-Validation — C(6,2) = 15 independent out-of-sample paths from six time blocks — and reports the full distribution, not a point estimate:

oos_sharpe:      0.79    ← mean Sharpe across all 15 CPCV paths
oos_sharpe_std:  0.09    ← distribution tightness — the real signal
folds_passed:    13/15
num_signals:     1240    ← accepted signal events across CV splits
upper_hit_rate:  0.47    ← fraction hitting the profit barrier first
avg_signal_ret:  0.0031  ← mean triple-barrier return, after costs
profit_factor:   1.12    ← gross signal gains / gross signal losses
hit_rate_std:    0.04    ← path-to-path hit-rate stability

The quantity the agent is actually asked to improve is oos_sharpe / oos_sharpe_std — quality, not magnitude. A strategy at 0.80 ± 0.08 beats one at 1.10 ± 0.60, even though the second number looks better on a pitch deck. Underneath that, a set of hard discard rules gates every keep decision regardless of the quality score:

• Max drawdown worse than -35% → discard
• oos_sharpe_std > 0.5 → discard, too regime-dependent
• num_signals < 100 → discard, too sparse to trust
• upper_hit_rate < 0.45 → discard, not enough barrier hits to be a real edge
• avg_signal_ret <= 0 → discard, no net edge after costs
• profit_factor < 1.05 → discard, losses overwhelm gains
• hit_rate_std > 0.10 → discard, the label edge isn't stable across regimes

None of these are soft suggestions in a system prompt. They're checked mechanically against the CPCV output before anything gets committed. An agent that finds a strategy with a beautiful Sharpe and a 40% drawdown doesn't get to argue for it — the rule fires and the experiment is logged as a discard.

The Loop: Stateless Batches, Git as the Checkpoint

Each research batch is a fresh Claude Code invocation with no memory of previous batches. It reads results.tsv (the full experiment history), strategy.py (current best), and program.md (the search space and priority order) from disk, proposes exactly one change, backtests it, and either commits or reverts:

# Manual — good for the first few experiments while you calibrate the harness
claude "read CLAUDE.md and program.md, then run 3 experiments"

# Autonomous — 10 batches × 3 experiments = 30 experiments, unattended
bash run_research.sh 10

Statelessness is deliberate, not a limitation. If the agent carried context forward across batches, thirty batches in you'd be debugging a conversation, not a codebase — and worse, the agent could talk itself into a narrative ("this feature family seems promising, let me try six variants") that looks like research but is really just increasing the number of implicit hypotheses tested against the same data. Forcing every batch to re-derive its plan from results.tsv keeps each experiment an independent, auditable decision. The append-only log is the actual audit trail — nothing in it ever gets rewritten, discards included.

A single experiment inside a batch follows a fixed sequence: pick one change from the priority tiers in program.md (parameter tuning before new features — cheaper to test, less overfitting surface), edit strategy.py, run a fast walk-forward pass first, and only escalate to the full 15-path CPCV if the fast pass clears the discard gates. That ordering matters for the same reason it matters in any expensive search: don't pay for the full validation until a cheap screen says it's worth it.

What "Economically Real" Means to the Loop

The search space isn't unbounded either. program.md asks the agent to justify each change with a one-sentence economic rationale before making it, and orders the search by tier — CUSUM sampling threshold, profit/stop multiples, and holding period first (cheap to test, low overfitting surface), feature-level changes second. This mirrors the same discipline this site's own survivorship bias and methodology writing keeps coming back to: a signal that "works" in one lucky backtest window isn't evidence of anything until it survives a validation scheme designed to catch exactly that failure. An agent with unlimited feature-engineering freedom and a single train/test split will always find something that looks like an edge. The constraint isn't there to slow the agent down for its own sake — it's there because the constraint is the research discipline. Remove it and you haven't made the agent smarter, you've made the search less trustworthy.

Where a Single Agent Stops Being Enough

This loop is genuinely good at what it does — sequential, disciplined, one hypothesis fully validated before the next one starts. It's also slow in a specific way: thirty batches of three experiments each is ninety sequential ideas, run one after another, each waiting on the previous CPCV pass to finish before the next begins. For a single well-scoped research question that's the right pace. It's the wrong pace for a different kind of question: "what does the whole space of plausible tactical signals for this strategy look like, and which corners of it are actually worth pursuing?"

That's a breadth problem, not a depth problem, and it's tempting to solve it by just running more agents in parallel — spin up twenty Claude Code sessions, let each one propose a signal, take whatever comes back with the best number. That approach fails for a reason that should be familiar by now: run twenty independent searches against the same data with no coordination between them, and you've built a multiple-testing problem, not a research program. Twenty agents each finding one thing that beats a backtest by chance is exactly what you'd expect from pure noise at that sample size — the same overfitting risk the single-agent loop's discard rules exist to prevent, just distributed across more compute.

Making parallel agent research actually trustworthy — rather than just p-hacking at higher throughput — needs its own set of constraints: agents that can see what's already been tried before proposing something new, a way to check a new signal isn't just a redundant restatement of one already in production, and a validation step that accounts for how many things were actually tried, not just how the one you kept performed in isolation. That's a different architecture from crucible's sequential loop, built and tested on this site's own production signal book — the next piece walks through it, including where it actually caught a real bug and where the honest conclusion was "don't deploy yet."