# The trial ledger

> What counts as a trial, why the count only rises, and why you cannot supply it yourself.

## What a trial is

A trial is one *evaluated configuration*. Concretely:

| Action | Trials |
|---|---|
| `backtest_submit` on one asset | 1 |
| `backtest_cross_section` over 22 assets | **1** |
| `sweep_submit` over 2,000 configurations | **2,000** |
| Re-running an identical spec you already ran | 0 — deduplicated by hash |
| `holdout_evaluate` | 0 — but it consumes the one-shot |

The asymmetry between rows two and three is the most important thing on this
page. One specification tested across many assets is **one hypothesis
evaluated against a larger sample**. Two thousand specifications tested to
find the best one is **two thousand hypotheses**, and the maximum of two
thousand noisy draws is large even when every underlying edge is zero.

## Why the count only rises

The trials table is append-only, enforced by database triggers that refuse
`DELETE` and `UPDATE`. There is no supported path — through the MCP tools or
otherwise — to lower N.

This is not distrust of the user. It is a design response to how research
actually degrades:

- Sessions span weeks and many conversations.
- Context windows get compacted; what was tried is forgotten.
- Sweeps get re-run after a parameter tweak nobody wrote down.
- The person computing the deflated Sharpe is often not the person who ran
  the sweep.

By the time a correction is applied, the honest N is unrecoverable by memory.
So the server remembers instead.

## You cannot pass N

```
stats_deflated_sharpe(strategy_id="...")     # no n_trials parameter exists
```

There is no `n_trials` argument. The server reads the ledger for the family
that strategy belongs to. This is the single load-bearing constraint in the
whole system: an API that accepted N would be an API where the correction is
whatever the caller wants it to be.

## Forks inherit their history

```
family_fork(parent="btc-ma", justification="...")
```

A fork starts a new family but **carries the parent's trial count forward**.
A sibling fork was fitted on the same data over the same hypothesis space, so
its trials are part of your multiple-testing burden whether or not you
rename the family.

`family_fork` requires a written justification. It is recorded. The intent is
not to make forking hard — sometimes a genuinely new idea grows out of an old
family — but to make it deliberate and auditable rather than a reflex when a
hurdle gets uncomfortable.

## Reading your own count

```
experiment_ledger(family="btc-ma")
```

Returns N, the Sharpe distribution across recorded runs, and the extremes.
Read-only, append-only underneath.

N is per-tenant under row-level security and monotonic. No number in this
documentation is your N — a figure baked into a document would be stale for
whoever produced it and simply false for everyone else.

## The recorded example

The server's own `btc-ma` family shows what the ledger is for.

Twenty-eight recorded training runs on Bitcoin produced a best Sharpe of
**0.9497**. Taken alone that is a publishable-looking number. But it is the
maximum of 28 draws, and the deflated Sharpe against N=28 is a very different
statistic from the raw one.

Then the same specification was run cross-sectionally over ten assets — one
trial — and pooled to **-1.376**, with 3 of 10 positive and the best being
AERO-USD at 1.19. The Spearman correlation between per-asset Sharpe and
per-asset trade count was **-0.61**: assets that traded less scored better,
so the ordering tracked turnover and its cost rather than signal.

Every figure above is recomputable from that run's stored `metrics.per_asset`.
That is the standard the ledger holds itself to — an earlier version of the
server's own instructions quoted a pooled -1.86 over 22 assets and named BTC
as the distribution maximum, and none of it matched any recorded run. The
ledger held one cross-sectional run, over ten assets, and BTC was not among
them. The text was wrong; the ledger was not.