A worked session

One idea taken from cost check to holdout, including the part where it fails.

This is the btc-ma family as it actually happened, in the order the tools were called. It ends in a rejection, which is the useful part.

1. Cost, before anything else#

cost_model_refresh()
cost_breakeven(trades_per_day=4)

A moving-average crossover on 1-hour bars trades a few times a day. At 120 bps round trip and 4 trades/day the annual drag is roughly 1,752%. That is already brutal, but a slow MA crossover on 1h bars does not trade four times a day — it trades a few times a month. The check passes, at low frequency only. Frequency is now a constraint on the design, not a free parameter.

2. Data#

data_ingest_universe(universe="coinbase-liquid-22", timeframe="1h")
job_status(job_id="...")
data_quality_report(universe="coinbase-liquid-22")

3. Split, once#

split_register(
  train   = ["2021-01-01", "2024-06-30"],
  test    = ["2024-07-01", "2025-06-30"],
  holdout = ["2025-07-01", "2026-06-30"],
)

4. Register and validate#

strategy_register(family="btc-ma", spec={"fast": 20, "slow": 60, ...})
feature_validate(strategy_id="...")

feature_validate checks lag conventions. Features carry lag=1 by default: a moving average over closes through bar i is known at bar i's close, not before it. Getting this wrong produces a beautiful backtest of a strategy that cannot be traded.

5. The single-asset result#

backtest_submit(strategy_id="...", symbol="BTC-USD", segment="train")
backtest_result(run_id="...")

Sharpe 0.9497. Best of 28 recorded training runs.

This is the number that would get published. It is also the maximum of 28 draws, so on its own it means very little — which the ledger already knows:

experiment_ledger(family="btc-ma")   # N = 28 and rising

6. The cross-sectional result#

backtest_cross_section(strategy_id="...", universe="...", segment="cross_section")

One trial. Ten assets.

Pooled Sharpe -1.376. Three of ten positive. Best: AERO-USD at 1.19.

The gap between 0.9497 on one asset and -1.376 across ten is the finding. Nothing else in this session matters as much.

7. Why it failed#

The Spearman correlation between per-asset Sharpe and per-asset trade count was -0.61. Assets that traded less scored better.

That is not a signal that needs tuning. If the ranking tracks turnover, the thing being measured is cost, and the "edge" is an artefact of which assets happened to trade least during the sample. Widening the sweep cannot fix it — it would just find the configuration that trades least, which is the configuration closest to not trading at all.

8. What not to do next#

The tempting move is sweep_submit over a few thousand parameter combinations to find something that survives. Consider what that costs:

sweep_submit(...)   # 2,000 configurations = 2,000 trials

N goes to 2,028. Every deflated Sharpe computed afterwards — for this family and for any fork of it — is corrected against 2,028 trials. MinBTL at N=2,028 is far beyond the available history. You would be spending the family's future to buy a number you already have a mechanical explanation for.

9. What to do instead#

family_fork(parent="btc-ma", justification="...")

or research something else. A fork inherits the parent's trial history, so this is not an escape hatch — it is a way of recording that you are changing hypothesis rather than continuing to search the same space.

The honest outcome here was to stop. That is recorded in the ledger too.

10. If it had worked#

gate_check(strategy_id="...")
strategy_freeze(strategy_id="...")
holdout_evaluate(strategy_id="...")

One attempt, ever. gate_check first — it costs nothing and tells you whether the in-sample evidence justifies spending it.

Read this page as Markdown: /docs/workflow.md