Release notes are where model marketing either becomes engineering or gets exposed as copy. A useful entry lets another person identify the exact artifact, rebuild its inputs, run the same untouched evaluation, and see what changed from the prior version. Anything less is a launch post.
Provenance tier: empty release ledger. No source-linked model run is attached, so no model name, weight file, benchmark, training count, release date promise, ATS record, or performance claim is published.
The current ledger is intentionally empty
This page previously read like a populated changelog without carrying the artifacts needed to verify it. That is backwards. A model version, benchmark, weight footprint, or monthly score belongs in release notes only when a durable source points to the exact bytes and evaluation that produced it.
The empty state is not “nothing happened.” It is “no release clears the evidence contract yet.” The cure is concrete: attach the model artifact, preprocessing manifest, source references, immutable evaluation window, result file, and reproduction command. Then add the entry without rewriting the earlier gap.
What every release must identify
Start with identity: model name, semantic version, artifact checksum, target, sport, market, and the commit or registry version that owns the release. A human-readable nickname is not enough. Two artifacts with the same label but different preprocessing are different releases.
Next, identify the data: source name, retrieval or snapshot reference, event key, feature cutoff, target settlement rule, missing-value policy, deduplication rule, and every transformation that can alter a row. Keep the manifest beside the weights so the model cannot outlive the meaning of its inputs.
Training and evaluation windows must be separate
Publish the start and end of the training window, tuning window, and untouched evaluation window, plus eligible row counts for each. Explain exclusions. A test window selected after results were visible is not untouched, and a row count without eligibility rules cannot be reproduced.
For time-ordered sports data, splits should move forward through time. Random folds may still be useful for diagnostics, but they do not replace a forward evaluation that resembles deployment.
Name the baseline before the result
A score has no meaning without a comparison. The baseline may be a market-derived probability, a simple historical estimate, or the previous production model. State how it was built, confirm it uses the same rows, and publish its result next to the candidate.
Do not switch baselines between releases merely because another one is easier to beat. If the baseline contract changes, mark that as a release-note event and restate older comparisons only in a separately versioned analysis.
Use the metric that matches the target
Probability models can be evaluated with proper scoring rules and calibration. Margin or total models need error metrics tied to their numeric target. A betting selector needs the actual market quote and settlement rule used at decision time. These layers should appear separately so a good probability model is not confused with an untested betting rule.
When a release makes spread selections, report the ATS win-loss record, percentage, window, and sample size, with pushes declared. Do not publish a return calculation unless every wager carries its real price and the computation is attached. Missing prices mean the financial question remains unanswered.
Calibration belongs beside discrimination
A model can rank games well while attaching poor probabilities. Release notes should show whether predicted probabilities line up with observed frequencies across pre-declared bands, along with the count in each band. Sparse bands should look sparse, not be smoothed into confidence.
Calibration should be measured on the untouched evaluation window or a later live ledger. Recalibrating on the same rows and then praising the corrected curve is training, not validation.
Publish failure modes before users find them
Every model has boundaries. Name the leagues, market states, missing inputs, early-season conditions, lineup uncertainties, or data delays that can invalidate a prediction. State what the runtime does when those conditions occur: pass, fall back to a declared baseline, or return a typed error with a cure.
A release note should also say what the model does not cover. Silence is not support. If props, live markets, or playoff games were absent from evaluation, the artifact should not imply competence there.
Changelogs need causal discipline
For each version, list the smallest meaningful changes: source revision, feature addition, bug repair, architecture change, calibration change, or operational fix. Tie each claimed improvement to a controlled comparison on the same evaluation contract. If several things changed together, say the effect cannot be attributed to one of them.
Keep no-change releases and failed candidates in the ledger. A candidate that did not clear the bar is useful evidence about the idea. Deleting it encourages the same experiment to be rediscovered and over-sold later.
Rollbacks are releases too
A rollback entry names the version being left, the version restored, the trigger, the evidence, and the operational steps. Preserve both artifacts. The point is not to erase the failed release but to make the production state and the historical lesson unambiguous.
When a data correction changes an old evaluation, publish a correction entry with both old and new results. Do not silently edit the original line. An append-only ledger is easier to audit than a polished history.
Reproduction must start from clean inputs
A reproduction guide should declare runtime versions, dependency lockfile, source snapshot, preprocessing command, model command, evaluation command, expected output files, and accepted numerical tolerance. It should work without private state unless the release clearly labels the missing dependency.
The verifier should compare artifact checksums and row counts before comparing scores. Matching a headline metric on different rows is not reproduction.
How community remixes should be compared
A remix is a new artifact. Give it a new identity, preserve the parent version, and evaluate it on the same untouched contract before drawing a comparison. Once the shared holdout has influenced design, move both models to a fresh future window.
Use Workshop for controlled variants, Build for a new model contract, and the model leaderboards for public records with declared windows and samples. A leaderboard row should link back to the release artifact, not substitute for it.
The release template
Each future entry should contain: identity, sources, windows and row counts, feature contract, architecture, baseline, untouched result, calibration or error summary, eligible betting record when applicable, known failure modes, changelog, rollback path, checksums, and reproduction steps. Missing sections stay visible as missing.
That structure is deliberately repetitive. Consistency lets a reader compare versions without decoding a new marketing format every month.
What would populate the ledger
What to watch: the first artifact bundle that includes source snapshot, preprocessing manifest, immutable weights, untouched evaluation output, checksums, and a successful clean reproduction. Until that chain exists, the release ledger stays empty.
Model calibration from graded predictions
Calibration points render only when a verified source binds prediction probabilities to settled outcomes for the same observations.
Expected value from graded outcomes
Expected-value cells render only when a verified source binds observed win outcomes to the price paid for the same bets.
