Updated Sep 9, 2026 · Week 1 board.
A trained model in Studio comes back with six checks rather than one score. Each check lands green, amber or red. The report stops at the first red and names it, so a broken model tells you which part broke.
Six checks and where their lines fall
| Check | Green | Amber | Red | What a red is telling you |
|---|---|---|---|---|
| Sample size | 1,000 graded picks or more | 200 to 999 | Under 200 | The sample is too small, so every number under it is noisy. |
| Calibration error | Under 0.05 | 0.05 to 0.15 | Over 0.15 | Stated probabilities do not mean what they look like. |
| Robustness | Under 0.02 | 0.02 to 0.04 | Over 0.04 | The record swings with which weeks you trained on. |
| Vegas correlation | Under 0.92 | 0.92 to 0.97 | Over 0.97 | You are buying the line instead of beating it. |
| Recent 4w ATS | Above 52% | 50% to 52% | Below 50% | The rolling four-week record sits under break-even. |
| Edge half-life | Over 8 weeks | 4 to 8 weeks | Under 4 weeks | Half the edge is gone inside a month. |
Read them in that order, because that is the order the report walks. It raises its banner on the first red it meets and then stops looking. A model can fail two checks and only ever hear about one.
What the middle four are watching
Sample size and calibration error take most of the attention. The other four are worth a sentence each, because a red on any of them names something you can act on.
Robustness measures how far the spread record moves when the training weeks change. Steady across those splits means the model learned the game, while a wide swing means it learned the particular weeks it was handed.
Vegas correlation measures how closely the model tracks the market price. Recent 4w ATS is the rolling four-week record against the spread, and it goes red below 50%, which is the point where the model is losing to a coin.
Edge half-life measures how long before half the edge is gone. Under 4 weeks the report tells you to retrain on a short cycle or expect the decay to run ahead of you. Over 8 weeks the edge is durable enough to leave alone between runs.
An unknown check cannot raise the banner
A check with no number behind it comes back unknown. The report says so in plain words, and an unknown check is barred from raising the banner. A report with no banner is therefore not the same thing as a clean report.
Count the numbers before you read a quiet card as good news. Six figures means the report graded everything it has a figure for. Four figures and two blanks means two of the six went unmeasured, and either of them was free to be the worst thing about the model.
Case one, a red on sample size
A model trained against 140 graded picks lands red on the very first check. The report’s reason is that the sample is too small for the numbers under it to mean much. Nothing below that line is worth arguing with yet.
The change to make sits in Scope. Widening the season window feeds the model more games, and more games is the only thing that moves this check. Train again, then read the count before you read anything else.
Case two, a red on Vegas correlation
A model scoring 0.985 here is over the 0.97 line by a distance. The report’s reason is that the predictions are buying the line instead of beating it. The model works, in the narrow sense that it agrees with the market. It has no disagreement left to bet.
The change to make sits in Features. If the market price option is switched on, take it out and train again. Feed a model the market’s own number and it will land right beside that number, which leaves nothing to bet.
Case three, a red on robustness
Say the spread record swings 0.06 across the training splits, which is over the 0.04 line. The report’s reason is that the model is sensitive to which weeks it trained on. A record that changes when the weeks change is not a record of anything durable.
The change to make sits in Transforms and Features. Fewer inputs, and a decay that leans on recent games, both cut the number of ways a model can fit the particular weeks it happened to see. Take one input out, train again, and read this line rather than the win rate.
The checks and the publish gates are different lists
Six green checks do not open the publish button. Publishing runs its own list of six: every core slot decided, no slot left empty, a sport recorded on the target, a finished training run, 20 graded forward picks and a closing-line measure. The first four land while you build. The last two land only after the games play.
That split is worth holding onto, because the two lists answer different questions. The gates ask whether the model is finished. The checks ask whether it is worth anything. The last gate has a free counterpart in the closing-line value tool, which compares a price you took against the close.
Where the report lives
The six checks render on a saved model’s card, in a block of the card given over to trust. The build board does not carry them, which is why a first-time builder can train a model and never read one. Save the model, open its card, and the checks are there.
That fixes the order of the work. Training comes first, saving second, and reading the card last. A model you trained and never saved has no card to carry a report.
Reading the report a second time
Every repair changes more than the check it was aimed at. Widening the season window lifts sample size, and it moves calibration error, robustness and the four-week record along with it, because all four are measured on the games the model saw. Read the whole list again after each training run rather than only the line you were working on.
The Receipts Drawer
Work top to bottom and stop at the first red, because that is what the report itself does. A blended score would hide which of the six broke, and the broken one is the story. Sample size sits first for a reason, since every check under it rests on the same games. The public track record runs the same discipline in the open, printing calibration next to the cover rate for graded models.
The desk treats an amber on sample size differently from an amber on calibration error. Waiting fixes the first one, because the sample grows every week that picks grade. Nothing fixes the second until the Calibration slot changes and the model trains again. Watch which of the two you are holding before you decide to wait on it.
FAQ
What does the banner on my model card mean? It names the first check that came back red. The report walks the six in a fixed order and stops there, so a second red never gets a banner of its own.
Is 200 graded picks enough? It is enough to leave red and reach amber, and green waits for 1,000. Between those two the report treats the record as directional rather than settled.
Does a card with no banner mean the model is sound? A check with no number behind it reads unknown, and an unknown check cannot raise a banner. Count the figures first. Six of them means the report graded everything it can.
Does a red check stop me publishing? The trust checks and the publish gates are separate lists. Publishing waits on decided slots, a finished training run and 20 graded forward picks. A red trust check is a reason to stop, and the button does not enforce it.
