The record behind the record
Decisions log
Every substantive decision behind this product — model changes, gate rules, withdrawn calls, bugs and what they cost — argued in full at the time it was made. The log is append-only: a decision that proves wrong gets a new entry saying so, never an edit. Pages across the site cite these entries by number; this is where the numbers resolve.
This is a near-verbatim render of a live internal document. The internal log is append-only — superseding an entry means writing a new one, never editing the old. For this public copy: 6 entries are withheld in full, each leaving a numbered stub stating its category, and the operator is named as such. Nothing else is edited — suppliers, costs where they survive, mistakes and all. (The first public render also role-named the suppliers; that was dropped the same day as cosmetic — the data provider's CDN is in every fixture page's own markup, so the redaction protected nothing.) The policy is itself versioned code in the repository (
scripts/publish_decisions.py).
§17: "Every deviation from this spec gets an entry: what changed, why, and what it affects. When the model or thresholds change, record the model version so historical published edges remain interpretable against the model that produced it."
§0: "Do not re-open settled decisions mid-build; if a decision proves wrong, change it explicitly and note it here."
Entries are append-only and numbered. Superseding an entry means adding a new one that says so, not editing the old one.
D-001 — Next.js 16, not 15
Date: 2026-08-06 · Phase: 0 · Spec: §3 technical stack
§3 specifies "Next.js 15 (App Router), TypeScript, React Server Components".
create-next-app@latest resolves to 16.3.0.
Decision: take 16.
Why: every §3 requirement — App Router, RSC, SSR for the §8.5 SEO surface, ISR for the §10 caching rules — is unchanged in 16. Pinning a greenfield project one major behind buys nothing and costs an upgrade later, during a fortnight where §14 says the only question that matters is whether the model beats closing lines.
Affects: apps/web. Watch for API differences against training data —
Next 16 ships an AGENTS.md into apps/web pointing at
node_modules/next/dist/docs/, which is the authority, not memory.
D-002 — Python source stays 3.10-compatible; CI and production run 3.12
Date: 2026-08-06 · Phase: 0 · Spec: §3 technical stack
§3 specifies Python 3.12. The build machine has 3.10.9.
Decision: requires-python = ">=3.10", no PEP 695 generics, no 3.11+
stdlib. CI (.github/workflows/ci.yml) runs 3.12.
Why: Phase 0 is ingest, which uses nothing 3.12-specific. Blocking the build on a Python install is a poor trade against starting odds capture, and §4.2 is unambiguous that every hour not capturing is lost forever.
Affects: pyproject.toml, all of model/. Revisit at Phase 2 — scipy
and statsmodels fitting is where a version difference could actually bite.
Install 3.12 locally before then.
D-003 — Every reference table carries a provider column
Date: 2026-08-06 · Phase: 0 · Spec: §4.4
The §4.4 DDL has provider_id but no provider.
Decision: unique key is (provider, provider_id), not provider_id alone.
Why: §14 Phase 0 debugs ingest against API-Football and Phase 1 switches to
Sportmonks. Manchester United is id 33 in one feed and id 14 in the other.
Without the provider column those collide, and §9.6 — "no scraped data in the
shipped product, ever", all production data from the licensed feed — becomes
unverifiable, because there would be no way to tell which rows came from where.
With it, WHERE provider = 'sportmonks' is a checkable guarantee.
Affects: every reference table, IdMap, every adapter.
D-004 — Tables and a package the spec does not list
Date: 2026-08-06 · Phase: 0 · Spec: §3.1, §4.3, §4.4, §6.4, §10
Added, each because a spec requirement had nowhere to live:
| Addition | Required by |
|---|---|
raw_payloads |
§4.3 "all raw API responses written to raw_payloads (JSONB) before parsing" |
ingest_checkpoints |
§4.3 "job-level checkpointing so a killed backfill resumes rather than restarts" |
ingest_runs |
§10 "any ingest job failure raises an alert" |
edge_rejections |
§6.4 "Log every rejection with its reason. A silent rejection is a bug you will never find." |
odds_polls |
see D-006 |
model/core/ package |
config, DB and the rate-limited HTTP client are shared by ingest, project and publish; duplicating them guarantees drift |
Affects: packages/db/src/schema/ingest.ts, model/core/.
D-005 — odds_snapshots is not partitioned at creation
Date: 2026-08-06 · Phase: 0 · Spec: §4.4, §10
§4.4 says "Partition by month once it grows"; §10 says "once above ~20M rows".
Decision: create it unpartitioned. Ship the partitioning migration ready to run, do not run it.
Why: declarative partitioning complicates every subsequent migration and Drizzle does not model it. The spec itself says "once", not "from the start". With D-006 in place the 20M threshold is months away rather than a fortnight.
Affects: packages/db/sql/partition-odds-snapshots.sql. Add a row-count
check to the freshness job before this is forgotten.
D-006 — odds_snapshots stores on change, not on every poll
Date: 2026-08-06 · Phase: 0 · Spec: §4.2, §4.4, §6.3, §10
§4.4 says "time series — never overwrite, always insert". It does not say whether an unchanged price is re-inserted every poll.
Decision: insert only when the price differs from the last price observed
for that exact selection at that book. Record every poll — changed or not — in
odds_polls.
Why (volume): one fixture is roughly 30 players × 5 markets × 2 sides × 8 books ≈ 2,400 selections. At the §4.2 cadence that is ~144 sweeps per fixture, so ~350k rows per fixture and ~17M rows a week across five leagues. The §10 partition threshold would arrive inside a fortnight, and almost every row would be a duplicate of the one before it.
Why (lossless): change detection compares against the last observed value. If a price moved and returned between two polls, polling never saw it either way. Nothing is lost that inserting-always would have captured.
Why odds_polls: with store-on-change, "newest snapshot" stops meaning
"how fresh is this price" — a price that has not moved in two hours is current,
not stale. §6.3's 30-minute freshness rule and §10's last-updated timestamp
read odds_polls. Reading odds_snapshots instead would suppress every edge
on a quiet market, which is the opposite of the intent.
Affects: model/ingest/jobs/snapshot_odds.py, odds_polls,
model/ingest/freshness.py, and any future value-board query. Any code
asking "how fresh are these odds?" must query odds_polls.
D-007 — GitHub Actions cannot deliver the §4.2 near-kickoff cadence
Date: 2026-08-06 · Phase: 0 · Spec: §3 job scheduling, §4.2, §6.6
§3 chooses GitHub Actions cron. §4.2 asks for a 5-minute sweep inside T-60m.
Problem: the Actions scheduler has a 5-minute floor, is documented as best-effort, and routinely delays runs 10–30 minutes under load — worst at peak hours, which for UK football is precisely Saturday afternoon and midweek evenings.
Consequence: not a missed cron. A missing closing line. §6.6 makes CLV the metric and §15 judges the whole trial on it, so a scheduler that skips the last hour before kickoff undermines the one thing the fortnight is meant to measure.
Decision: own the clock. propboard worker
(model/ingest/worker.py) runs the whole §4.2 cadence in-process, deployed as
a single always-on container (Dockerfile, fly.toml). The Actions workflows
stay as a fallback for everything except the near sweep.
Status: CLOSED, 2026-08-06, by D-013.
Affects: .github/workflows/ingest-odds.yml, and the credibility of the
trial verdict.
D-013 — The ingest worker owns its own clock
Date: 2026-08-06 · Phase: 0 · Spec: §3 job scheduling, §4.2
Closes D-007.
Decision: a long-running process (propboard worker) schedules the §4.2
cadence itself, deployed as exactly one Fly machine.
Four properties, each chosen against a specific failure:
| Property | Failure it prevents |
|---|---|
Ticks computed from a fixed epoch (now // interval), not last_run + interval |
A 90s job on a 300s interval drifts 90s per cycle; within a day the near sweep runs at arbitrary offsets |
| Missed ticks do not queue | A stalled worker resuming would fire three consecutive sweeps into the rate limit for prices already stale |
| A raising job is logged and retried next tick, never fatal | A worker that exits on the first transient error captures nothing until a human notices (§10) |
| SIGTERM finishes the running job first | Fly sends SIGTERM on deploy; dying mid-sweep leaves a poll unrecorded |
Why one machine only: two workers double every sweep and write each price
change twice, corrupting the time series §1.1 calls the moat. fly.toml has no
[http_service] block, deliberately — adding one lets Fly autostop the machine
on idle, which for a scheduler means it stops capturing odds precisely because
it had nothing else to do.
Why the near sweep is first in the schedule list: under a rate limit, the far sweep (up to 60 fixtures across 72h) would otherwise starve the fixtures about to kick off — the ones whose closing line §6.6 actually needs.
Cost: one shared-cpu-1x/256MB machine. Cheap against a CLV verdict that cannot be trusted.
D-008 — npm workspaces, not pnpm
Date: 2026-08-06 · Phase: 0 · Spec: §3.1
pnpm is not installed on the build machine; npm 11 is.
Decision: npm workspaces. The §3.1 layout is unchanged.
Why: one fewer global install for a two-package monorepo. Revisit if
node_modules duplication becomes a problem, which at this size it will not.
D-009 — Immutability enforced by database triggers, not convention
Date: 2026-08-06 · Phase: 0 · Spec: §1.1, §4.4, §8.8
§4.4 says published_edges is "Append only. Never DELETE, never UPDATE except
to fill closing_price / result after settlement." §4.2 says odds_snapshots is
"never overwrite, always insert".
Decision: enforce both with BEFORE DELETE / BEFORE UPDATE triggers
(packages/db/sql/published-edges-immutability.sql). Settlement columns are
the only permitted update on published_edges, and each may be written once.
Why: §1.1 sells "timestamped, immutable" and §8.8 promises "no filtering that could hide losing periods". A guarantee that depends on everyone remembering not to run a DELETE is not a guarantee — it is an intention. This also blocks the subtler failure: quietly re-scoring a settled bet.
Affects: requires a manual psql -f after the initial migration; Drizzle
does not generate triggers. See README setup step 4.
D-010 — Normalised fixture_status enum alongside the raw provider string
Date: 2026-08-06 · Phase: 0 · Spec: §4.4
§4.4 has fixtures.status with no stated vocabulary.
Decision: a fixture_status enum
(scheduled|live|finished|postponed|cancelled|abandoned|unknown) plus
provider_status holding the raw feed string.
Why: providers disagree (NS/FT/AET/PST vs Sportmonks' state
objects) and every job filters on status. An unrecognised status maps to
unknown, never scheduled — a fixture wrongly marked scheduled would be
polled for odds forever, whereas one marked unknown shows up in a query and
gets fixed.
D-011 — Placeholder shrinkage constants are flagged unfitted
Date: 2026-08-06 · Phase: 0 · Spec: §5.4, §12
§5.4: "k is fitted per market by cross-validation (expect roughly 8–15
matches of prior weight). Fit k — do not guess it."
Decision: thresholds.yaml ships k_by_market values in the 8–15 range so
the pipeline runs end to end, under an explicit shrinkage.fitted: false flag.
Why: the numbers are needed for the code path to exist; they are not
fitted, and publishing against unfitted k would violate §0's
non-negotiable — an edge the model cannot defend.
Action for Phase 2: the publish job must refuse to run while
shrinkage.fitted is false. This is asserted in test_config.py.
D-012 — tackles_won is never populated
Date: 2026-08-06 · Phase: 0 · Spec: §6.5
Neither adapter fills tackles_won: API-Football reports tackles.total only,
and the Sportmonks lineup-details block does not separate attempted from won.
Decision: leave it NULL. Do not derive it, do not approximate it.
Why: §6.5 warns that books settle tackles on different definitions and
requires docs/settlement-sources.md to record which. A derived
tackles_won would be a number nobody could defend against any book's terms,
which is exactly what §0 forbids.
Affects: the player_tackles market cannot be published until
settlement-sources.md has a verified row for the book in question.
D-014 — Fixed effects with shrunk rates, not an estimated GLMM
Date: 2026-08-06 · Phase: 2 · Spec: §5.4, §5.5
§5.5 step 2 asks for "a hierarchical negative binomial per market: player random effect, position fixed effect, referee random effect, opponent-possession covariate, minutes as exposure/offset".
Decision: implement the fixed-effects form, with the empirical Bayes shrunk player rate (§5.4) and the shrunk referee rate entering as covariates, rather than estimating random effects with a GLMM.
Why: the shrinkage in §5.4 is doing the same job a random effect would —
pulling a small-sample player toward their stratum mean — and §5.4 calls it
"the most important modelling rule" in its own right. Doing it explicitly is
faster to fit, far easier to cross-validate k against, and directly
explainable in §8.3's row expansion, which has to show "per-90 with sample
size" and "a plain-English explanation of the drivers". A GLMM's shrinkage is
implicit in the fitted variance components and cannot be shown to a user as a
number.
What this gives up: the random-effect form would estimate the shrinkage
strength from the data as part of the likelihood rather than by a grid search
over k, and would propagate uncertainty in the player effect into the
predictive distribution. Both are real improvements.
When to revisit: after §5.6's baseline comparison passes. §14 step 9 stops the project if the model does not beat a season-average baseline, and a GLMM that fails that test is no more useful than a GLM that fails it. Improve the model only once there is something worth improving.
Affects: model/fit/negbin.py, model/fit/crossval.py.
D-015 — Integer prop lines are priced with an explicit push
Date: 2026-08-06 · Phase: 2 · Spec: §6.1
§6.1 gives:
P(over line) = 1 - CDF(floor(line))
P(under line) = CDF(floor(line))
Problem: that is exact for half-lines, which cannot push. For an integer
line it is not. "Over 2.0 fouls" is a void at exactly 2, but CDF(2) includes
P(X=2), so the formula assigns the push mass to the under.
Consequence: an overstated under probability is an overstated edge on the under. §0's non-negotiable is never displaying an edge the model cannot defend, and this one could not be defended against the bookmaker's own settlement rules.
Decision: line_probabilities returns (over, under, push) and handles
integer lines explicitly. Half-lines return push=0.0 and match §6.1 exactly,
so nothing changes for the common case.
Affects: model/core/distributions.py. The +EV engine must use
line_probabilities, never 1 - cdf(floor(line)) inline.
D-016 — In the Sportmonks feed, a missing player stat means ZERO
Date: 2026-08-06 · Phase: 1 · Spec: §5.2, §6.5
Observed: Sportmonks omits zero-valued statistics from a player's details
array entirely. A player who committed no fouls has no type_id 56 row at all.
Only 12 of 40 lineup rows in a sample fixture carried a Fouls entry.
Verified, not assumed. Summed the player-level values per team and compared against the team-level total for the same fixture, across three fixtures and eight statistics:
| Stat | Agreement |
|---|---|
| Fouls | 6/6 exact |
| Shots on target | 6/6 exact |
| Yellow cards | 6/6 exact |
| Saves | 6/6 exact |
| Interceptions | 6/6 exact |
| Tackles | 4/6 exact, 2 off by one |
| Shots total | 5/6 exact, 1 off by one |
Decision: for a player with a Minutes Played row (type_id 119), a missing
detail row for a stat in ZERO_WHEN_ABSENT is recorded as 0. A player with no
minutes row did not appear and produces no stat line at all.
Why this matters more than it looks: the ingest layer's standing rule is
that absence means unknown, never zero (see PlayerStatLine). That rule is
correct for API-Football, which always returns the key with an explicit null.
It is exactly wrong for Sportmonks. Had it stood, training_rows would have
dropped every player who committed no foul, and the model would have been
fitted only on foul-committers — inflating every projection in the product.
The two adapters legitimately differ; the canonical contract is the same and
each adapter is responsible for translating its own feed's convention.
Scope limit: only stats observed being emitted at player level default to
zero. shots_blocked and red_cards were not observed and stay None — "not
emitted for this player" and "never emitted at all" are indistinguishable from
one fixture, and a guessed zero is the confident wrong number §0 forbids.
Affects: model/ingest/providers/sportmonks.py, and every rate in §5.2.
D-017 — [withheld]
Withheld from the public render — commercial detail — data-plan economics. The internal log is append-only and this entry exists there unedited; the number is preserved here so the sequence stays checkable.
D-018 — Only one ingest job may run at a time, enforced by an advisory lock
Date: 2026-08-06 · Phase: 1 · Spec: §4.3, §4.2
Observed: running verify-provider while sync-history was mid-backfill
produced 429 from /fixtures/between/... retrying in 1198.0s — a 20-minute
Retry-After from the provider.
Cause: §4.3's token bucket limits each process. The provider's budget is per account. Two processes each staying politely under 2,000 calls/hour together request 4,000, and the provider is counting the account.
Why this is worse than it sounds: the backoff is correct behaviour and the client handled it properly — but a 20-minute stall on a matchday means the 5-minute near-kickoff sweep sits idle through precisely the window §6.6 needs the closing line from. The cost of a careless second command is not a slow command; it is a missing closing line.
Decision: run_job takes a Postgres session-scoped advisory lock
(pg_try_advisory_lock) before starting. A job that cannot get it fails fast,
naming what is already running, rather than queueing behind it and burning the
budget on retries.
Why an advisory lock: Postgres releases it when the connection closes, so a crashed or killed job cannot wedge the pipeline — no stale lock file, no manual cleanup. It also works across machines, which matters once the worker runs on Fly and someone runs a command locally.
Note: this validates D-013's single-worker design. Jobs inside the worker run sequentially in one process and never contend. The lock exists for the other case: an ad-hoc command run while the worker is live.
Affects: model/ingest/jobs/_base.py. Pass exclusive=False only for a
job that makes no provider calls.
D-019 — The warehouse, not the checkpoint, decides what has been ingested
Date: 2026-08-07 · Phase: 1 · Spec: §4.3, §6.4
Observed: two consecutive backfill passes reported 0 rows, 901 api calls
and 0 rows, 926 api calls, then continued as if progressing.
Cause, and it was three bugs stacked:
sync_referencehad run before the Championship was added to the plan, so no Championship players existed. Every stat line for that league failed to resolve aplayer_id.upsert_player_statsdropped unresolvable rows atlog.debug— invisible. §6.4: "Log every rejection with its reason. A silent rejection is a bug you will never find." That warning was about the +EV engine; it applies just as hard here.- The fixture was checkpointed as done anyway. The data was therefore lost permanently, and every later pass would skip it while reporting success.
Decisions:
sync_historysyncs squads per season, before that season's fixtures. A 2024/25 fixture is full of players who have since left; the current squad list cannot resolve them.- A fixture is checkpointed only once its rows land. Zero rows written, or any unresolved stat line, leaves it unmarked for the next pass.
doneis derived fromplayer_fixture_stats, not from the checkpoint. The checkpoint is a cache; the warehouse is the truth. A checkpoint claiming a fixture is done when no rows exist is detected, logged, and ignored — so the corruption already written by the earlier version self-heals.- Completed seasons are skipped without re-listing. Re-fetching every team and fixture across five leagues cost ~900 calls per pass before doing any work, which on a bounded-pass loop is most of the trial budget spent on nothing.
The general lesson: a resume marker that records intent rather than outcome will eventually claim work that never happened. Derive progress from the data wherever the data can answer the question.
Affects: model/ingest/jobs/sync_history.py,
model/ingest/repository.py.
D-025 — k is fitted against calibration, not log loss
Date: 2026-08-07 · Phase: 2 · Spec: §5.4, §5.6, §6.2
§5.4 says "k is fitted per market by cross-validation" but never says against
what. Log loss was the original choice, and it is wrong for this product.
Evidence. After the D-023 fix, four of seven markets failed §5.6, every one of them in a 0.7–1.0 predicted bucket of 60–300 rows, none of them anywhere in the bulk:
| Market | worst bucket | predicted | observed |
|---|---|---|---|
player_fouls_drawn |
0.7–0.8 | 0.75 | 0.48 |
player_fouls |
0.7–0.8 | 0.74 | 0.43 |
player_shots |
0.9–1.0 | 0.93 | 0.64 |
player_tackles |
0.8–0.9 | 0.84 | 0.64 |
Log loss is dominated by the tens of thousands of rows in the middle and barely
registers a few hundred in the tail. So the fitted k optimised a region that
was never the problem.
Direct proof it matters: player_fouls at k=130 calibrated at 0.026 and
passed; at k=30, the log-loss optimum, it calibrates at 0.309 and fails. Same
model, same data, different objective, opposite verdict.
Why calibration is the right objective here: §6.2 converts model probability straight into a price. A model well-calibrated on average and over-confident where it is most confident produces its largest fake edges exactly where it recommends the biggest stakes. That is §0's failure mode precisely.
The degenerate optimum, and the guard against it. Shrink hard enough and every player is assigned the stratum prior: perfectly calibrated, and useless for finding an edge. So calibration is the objective and discrimination is a constraint —
- take every
kcalibrating within 0.01 of the best; - discard any giving up more than 2% of the best log loss;
- among survivors, take the lowest log loss.
Lexicographic rather than a weighted blend, because the two metrics share no
scale and any weight would be arbitrary. If nothing satisfies both, the search
returns the best-calibrating k and lets the §5.6 gate refuse — it is the
gate's job to say no, not the search's job to pretend.
Affects: model/fit/crossval.py.
D-023 — Rate covariates enter the log-link model as logs
Date: 2026-08-07 · Phase: 2 · Spec: §5.5, §5.6
The bug: rate_shrunk — a per-90 rate — was passed into a log-link model
as a level. A log link means a covariate contributes exp(beta * x), so the
expected count was exponential in the player's rate. With the fitted beta of
1.02, a player rated 2.0 fouls per 90 was scaled by exp(2.04) ≈ 7.7, and one
rated 3.0 by exp(3.06) ≈ 21. The real relationship is proportional.
The fix: pass log(rate). Then exp(beta * log(rate)) = rate^beta, and a
beta near 1 reproduces proportionality. Same for the opponent-conceded and
referee rates.
How it was found — and how it was nearly missed. Two earlier fixes were
attempted on guesses and neither moved anything: switching the dispersion
estimator to MLE (alpha 0.19 → 0.21), and narrowing the gate to §6.3's
publishable band (0.157 → 0.155). Only after writing
scripts/diagnose_calibration.py and looking at predictions by decile did the
signature become obvious:
| decile of mu | predicted/observed |
|---|---|
| 0 (lowest) | 0.45 |
| 9 (highest) | 1.23 |
| overall mean | 1.02 |
Accurate on average, far too spread out at the extremes. That is a link-scale error, and nothing else looks like it. Look at the data before theorising.
Result: log loss improvement over baseline 30.7% → 33.9%; MAE 0.696 → 0.652; in-band calibration error 0.441 → 0.170, and 0.026 once noise-driven buckets stopped counting (D-024). All three moved together, which a real fix does and a guess does not.
Still open: the diagnostic also showed players in the 0–15 minute band
foul at roughly twice the rate the linear minutes offset predicts — late
substitutes commit more fouls per minute. §6.4's min_projected_minutes: 20
excludes them from publication, so it is not urgent, but the exposure offset in
§5.5 assumes minutes are interchangeable and they are not.
D-024 — A calibration bucket must be significant, not merely large
Date: 2026-08-07 · Phase: 2 · Spec: §5.6
§5.6 requires observed frequency to track predicted "within tolerance". The gate originally let any bucket with 30+ observations fail it.
Problem: at n=30 the binomial standard error is about 0.09 — nearly the
whole 0.10 tolerance. A bucket that size can breach the tolerance on noise
alone. The gate was failing player_fouls on a 43-row bucket whose miss was
2.3 standard errors: suspicious, but not evidence.
Decision: a bucket can only fail the gate if it has at least 30 observations and misses by at least 3 standard errors. Practical significance and statistical significance are different things and §5.6's gate needs both.
Why strict: a gate that fails on noise is a gate people learn to override, and §13 wires this one into CI as a publication blocker. A false failure there costs more than a marginal true one.
Not a relaxation of the tolerance. The 0.10 threshold is untouched; this governs which buckets are allowed to speak. After the D-023 fix the worst significant bucket misses by 0.026.
D-022 — Calibration is gated on §6.3's publishable band
Date: 2026-08-07 · Phase: 2 · Spec: §5.6, §6.3
§5.6 requires calibration to hold across probability deciles. §6.3 refuses to publish any edge whose model probability falls outside [0.10, 0.90], calling those "usually a data error, not value".
Decision: calibration_holds gates on buckets intersecting the publishable
band. The overall figure is still computed and reported.
Why: failing on a bucket §6.3 rejects by construction would block a model that is sound everywhere the product actually operates. The tolerance inside the band stays exactly where §5.6 put it — this narrows where the gate applies, never how strict it is.
Honesty note: this was introduced while chasing a calibration failure and it did not fix it (0.155 in-band against 0.157 overall). It is kept because the reasoning is right, not because it helped. The failure it was reaching for is real and remains open.
D-021 — The odds capture window is three weeks, not 72 hours
Date: 2026-08-07 · Phase: 1 · Spec: §4.2
§4.2 sets the far sweep at T-72h. That assumes a running season, where every fixture worth pricing is already inside three days.
Observed: bookmakers are pricing fixtures a fortnight out right now. An EPL opening-weekend fixture 14 days away returned 22 quotes across six books (bet365, Unibet, bwin, 1xbet, Dafabet, Sbo) on 1X2, over/under and BTTS.
Consequence of leaving it at 72h: given D-017 — the season starting after the trial ends — a 72-hour window captures nothing at all for the entire trial. Live markets would have been moving, unrecorded, the whole time.
Decision: far window widened to 21 days; sync_fixtures horizon widened to
24 days to match, since a fixture cannot be polled if it is not in the table.
Why the cost is near zero: store-on-change (D-006) means a market that is not moving writes one row and then nothing. An illiquid market two weeks out costs a handful of rows, and one API call per fixture per sweep.
Why the benefit is not: §4.2 — "odds history cannot be backfilled. Every hour not capturing is data lost forever." Widening the window is the only decision here that buys something unrecoverable.
Related fix: verify-provider no longer declares §16.1 answered when the
inspected fixture is more than 72 hours out. Books open player props 1-3 days
before kickoff, so "no props on a fixture 14 days away" is not evidence of
anything — it now reports INCONCLUSIVE and exits 2 rather than condemning the
subscription on a null result.
Affects: model/ingest/jobs/snapshot_odds.py,
model/ingest/jobs/sync_fixtures.py, model/ingest/verify_provider.py.
D-020 — Ingest jobs establish their own reference data
Date: 2026-08-07 · Phase: 1 · Spec: §4.2, §6.4
Observed: after the Championship was added to the plan, every one of its fixtures was skipped and 17,114 stat lines went unwritten.
Cause: sync_reference had run before the league existed, so leagues had
no row for it. upsert_fixtures skips any fixture whose league will not
resolve, so no Championship fixture was ever inserted, so no stat line could
find its fixture. The job reported success throughout.
Two diagnostic failures made this expensive to find:
- The unresolved-row counter recorded only the player id, so a whole
investigation went after squad completeness — the wrong cause entirely. It
now names which id failed (
fixture:/player:/team:/opponent:). A diagnostic that does not discriminate is barely a diagnostic. - The per-pass budget counted successes rather than attempts, so a league whose fixtures all failed ran past the limit unchecked, spending ~1,100 API calls in one pass on fixtures that could not possibly be written.
Decision: sync_history upserts the league and the season's squads itself
before touching fixtures, rather than assuming sync_reference has been run
recently enough. Jobs establish what they depend on.
Also: placeholder player rows are now created for anyone appearing in a lineup but absent from the squad list — mid-season departures, loanees and youth call-ups are normal, and the API call that fetched the fixture has already been spent. Discarding the appearance to keep the players table tidy is the wrong trade.
Affects: model/ingest/jobs/sync_history.py,
model/ingest/repository.py.
D-026 — A squad is who plays for the club now, not who used to
Date: 2026-08-09 · Phase: 2 · Spec: §5.3, §5.4, §6.4
Observed: the match page showed Rayan Aït Nouri projected for Wolverhampton Wanderers while registered to Manchester City, Sam Johnstone while at Crystal Palace, Ladislav Krejčí while at Girona. Across the 98 fixtures in the projection window, 543 of 2,596 squad rows — 21% — belonged to players since registered elsewhere.
Cause: PLAYER_HISTORY_SQL selects everyone with 10+ completed appearances
for a team_id. That answers "who has played for this club", which is a
different question from "who plays for it now", and nothing downstream noticed
the difference.
Why it is worse than 21% of rows being unbettable: those players were also
inside _stratum_prior, the mean every remaining team-mate is shrunk toward
(§5.4). A departed squad does not just add junk rows — it moves the projection
of every player who stayed. Confirmed on refitting the same fixture: the
leading fouls projection moved 1.03 → 1.06 once the filter was in.
Decision: cut the squad to who can actually play, from two sources in order:
- A squad named for the fixture (
lineups) — authoritative, because it is who the club says is available. Currently unused:lineupsholds zero rows, for any fixture, of any type. MODEL.md's note that "onlyconfirmedrows exist so far" describes the API-Football era and is no longer true of the table; corrected there. players.current_team_id, when the reference data is fresh enough to carry it —MAX_REFERENCE_AGE_DAYS = 14, chosen against the transfer window rather than the sync cadence.
Three fallbacks, each keeping everyone rather than filtering on something
that cannot be trusted: stale reference data, a named squad too small to be a
complete capture, and a surviving squad below MIN_SQUAD_SIZE = 5. The
direction is deliberate. A projection for someone who has moved is visibly
wrong and can be argued with; an empty squad is silent, and §1.5's honesty rule
cuts against the failure nobody can see.
§6.4 applies here, not just to the +EV engine. Every exclusion is counted by
reason on ProjectionResult.skipped_players and surfaced on the cycle result —
including the fallbacks, which record zero when they fire and find nothing, so
"the reference was stale but nobody had moved" stays distinguishable from "the
reference was fine". D-019 is the same lesson one layer down: a drop logged at
debug cost an investigation that chased entirely the wrong cause.
Verified: re-ran projections over the live window. 28,175 rows written, 8,575 player-market rows dropped as departed, no fallback triggered. Fixture 24891 went from 69 projected players to 56, with none registered elsewhere.
Affects: model/project/job.py, model/project/cycle.py,
model/tests/test_project_squad.py. The web side keeps its departedTo
marker (apps/web/lib/queries.ts) because the fallback paths can still emit
these rows, and when they do the page should say so.
D-027 — The web layer may sum a stored distribution, and nothing else
Date: 2026-08-09 · Phase: 2 · Spec: §3.1, §6.1, §8.3
§3.1: "Next.js only reads projections and edges — it never computes them. This separation is deliberate and must not be blurred."
The match page needs two things that look like computation:
overProbability(dist, line)— P(count > line), for the "Over 1.5" column and for colouring the ladder.discrete_distributiongives the tail bucket the exact remainder so the payload sums to 1; adding a subset of it re-derives nothing, it reads the model's own number at a coarser grain.perNinety(mu, minutes)— the model's offset is exactlylog(minutes / 90), somu = mu_per_90 × minutes / 90is an identity. A unit change, not an estimate.
Decision: both are allowed, in apps/web/lib/model.ts, and nothing beyond
them is.
The guard that makes the first one safe. The ladder closes with an open
n+ bucket holding all remaining mass. If the line sits at or above that
bucket's floor, the mass straddles it and no subset sum can separate the sides.
overProbability returns null there and the column renders a dash. Splitting
that bucket would mean inventing a number — the same failure D-015 identified
when the push mass on an integer line was being handed silently to the under.
Lines are half-lines only, for the same reason: MarketMeta.line ends in .5
for every market, because an integer line can push and the stored dist has
nowhere to put the void.
Affects: apps/web/lib/model.ts, apps/web/app/match/[id]/page.tsx.
D-028 — Trends filter on a match window, not a date range
Date: 2026-08-09 · Phase: 2 · Spec: §8.5
§8.5 lists "date range" among the trends filters.
Decision: a per-player match window — last 5, 10, 20 — instead.
Why: a calendar range gives a regular eleven matches and a rotated squad player three, then prints both hit rates in the same column as though they were comparable. "62% over 1.5" means something different at n=11 and n=3, and the table has no way to say so. A match window gives every row the same denominator, which is the condition under which a hit rate can be read at all.
The window counts QUALIFYING matches. recency is ranked inside the
filtered set, so with the venue and role filters on, "last 5" is his last five
home starts — not his last five matches, some of which were home starts. The
page says which, because describing a different query from the one that ran
would have a reader comparing two rows and drawing the wrong conclusion about
both.
Also added: team, venue (home/away), started-only, minimum appearances and minutes-per-appearance, and a sort control. The appearance floor is clamped to the window — "last 5 matches" with "at least 10 appearances" is a HAVING that can never be satisfied, which returns an empty table that looks broken rather than over-constrained.
Affects: apps/web/lib/trends.ts, apps/web/app/trends/players/.
D-029 — The stratum prior travels on the fitted model
Date: 2026-08-09 · Phase: 2 · Spec: §5.4, §5.5, §17
Model version: v20260809-1258
Observed: a goalkeeper averaging 2.5 saves per 90 projected at 0.77.
Cause — two mismatches between fitting and projection, stacked:
Fitting (fit/cli.py) |
Projection (project/job.py) |
|
|---|---|---|
| Prior | league:position_group — §5.4 as written |
minutes-weighted mean across the whole squad |
| Observed rate | {stat}_career_p90, the mean of per-match per-90 rates |
sum(stat)*90/sum(minutes), minutes-weighted |
Only keepers record saves, so a squad-wide prior is ~25 outfielders' zeros: 0.229 against the goalkeeping 2.674. The second mismatch is smaller but systematic — the projection rate ran 24.9% lower than the fitted one on average, and the gap tracked appearance length exactly (+78% for players averaging 16-30 minutes, +0% for ever-presents).
The projection job's own docstring already warned about this class of error for the coefficients — "a different model wearing the same version stamp". It had happened one level down, in the covariate the coefficients are applied to, and it meant every §5.6 figure on record described a model that was not the one being served.
Decision: stop deriving the prior twice.
- One implementation,
features/shrinkage.py::build_stratum_priors. - Its output is persisted on
fitted_models.stratum_priorsand read back by the projection job, so there is no second derivation to disagree with. §17 needs this for the same reason it needs the coefficients. PLAYER_HISTORY_SQLcomputes the observed rate the waycareer_p90does.- A model carrying no stored priors is refused, not projected from a guess.
resolve_prior widens on absence — league-and-position, then league, then
global — and build_stratum_priors withholds any stratum under 50 rows. That
answers MODEL.md's open question 4 by a general rule instead of a special case:
the 23 players with no position_group produce ~26 rows per league, fall under
the floor, and land on the league prior. A stored zero does not widen —
midfielders really do record no saves, and treating that as missing would shrink
them toward the goalkeeping mean.
Result: mean goalkeeper saves projection 0.723 → 2.756. All seven gates
re-passed. player_saves calibration error moved 0.000 → 0.064, still inside
the 0.100 tolerance — the old 0.000 was calibration of a distribution nobody was
being shown.
Found while fixing it: _estimate_alpha floored only at alpha <= 0, so a
value in (0, 1e-6) passed and then rounded to exactly 0.000000 in
fitted_models.alpha's numeric(10,6) storage. player_tackles did this and
killed a projection run mid-flight after four markets had been written. Now
floored at MIN_ALPHA = 0.01, and the projection job skips a market with
non-positive alpha rather than taking the run down with it.
Operational finding, and it cost an hour: propboard worker holds its code
in memory. After the fix landed and the model was refitted, the worker kept
producing runs from the old code against the new model — mean GK saves 0.723
against 2.756, in adjacent run ids. The web layer reads max(run_id), so the
site served the stale numbers while the corrected ones sat one row away. A model
or code fix is not live until the worker is restarted.
Affects: model/features/shrinkage.py, model/fit/cli.py,
model/fit/persistence.py, model/fit/negbin.py, model/project/job.py,
packages/db/migrations/0003_eminent_saracen.sql, scripts/run_projections.py.
D-030 — The four match markets already in the feed are ingested
Date: 2026-08-09 · Phase: 4 · Spec: §8.2, §4.4
Observed: the Sportmonks odds endpoint returns 68 distinct market names
per fixture. MARKET_NAME_TO_CODE covered three of them. Everything else —
including four markets §8.2 names for the homepage filter bar — was parsed,
matched against the map, found missing, and dropped.
Decision: map Double Chance, Draw No Bet, Asian Handicap and
Goal Line, and seed the market rows.
Three things this turned up that the map alone would not have:
- Double chance labels name the clubs — "Fulham or Draw", "Draw or Chelsea", "Fulham or Chelsea". Resolved positionally on the word "draw" rather than by matching team names, which would mean threading the fixture's participants into the odds parser and would break on any spelling difference between the two feeds.
odds_sidehad no value for it. Double chance covers two of three results and cannot be expressed as home/draw/away, so the enum gainedhome_or_draw,home_or_away,draw_or_away. Found by a failed insert mid-run, which is the right place to find it — the enum is doing its job.- Asian handicap lines are MIRRORED, not shared. The same bet is quoted
-0.50to the home side and+0.50to the away side. A line selector that filtered both sides on one value would show a price against a line nobody offered.bestFornegates for the away side on a mirrored market, and the selector lists home-side lines only so the reader is not offered two spellings of the same choice.
goal_line is a separate code from ou_goals, not a duplicate: the Asian
goal line carries quarter lines (2.25, 2.75) that settle as half-win/half-push,
where "Goals Over/Under" is the plain half-line market. Same number, different
settlement — and §6.5's whole point is that settlement definition decides
whether two things are the same market.
Result: 3,071 prices captured on the first sweep. asian_handicap 1,908 across 47 fixtures, goal_line 968 across 47, double_chance 117 across 39, draw_no_bet 78 across 39.
Affects: model/ingest/providers/sportmonks.py, model/config/leagues.yaml,
packages/db/src/schema/enums.ts, migration 0005_thankful_firebrand.sql,
apps/web/app/_components/fixtures-board.tsx, apps/web/app/page.tsx.
D-031 — Crests, logos and photos come from the feed, and only from the feed
Date: 2026-08-09 · Phase: 4 · Spec: §9.6, §11
Decision: store the licensed feed's image_path on players, teams and
leagues, and render it. Do not re-host it, and do not source an image from
anywhere else.
Why the URL and not the bytes: re-hosting means a copy to keep in sync and a licensing question nobody has answered. The URL is what the feed gives us and what the licence covers.
§9.6 is enforced by configuration, not by intention. "No scraped data in
the shipped product, ever" is a rule that decays the moment it depends on
everyone remembering it, so next.config.ts allowlists exactly one image host.
A wildcard there would let any URL that reached the database render on a page.
Backfilled without a single API call. §4.3 requires every raw response be
written to raw_payloads before parsing "so re-parsing never requires
re-fetching". This is the first time that promise has been cashed: teams came
out of fixture participants, players out of squad entries, leagues out of the
leagues endpoint. Coverage after: players 5,208/5,210, teams 122/123,
leagues 5/5.
A missing image renders as initials, never as a broken image. Coverage will
never be total — a player created from a lineup he appeared in (D-020) has no
squad row and therefore no photo. A 404ing <img> reads as a broken site; a
bordered square with two letters reads as a feed with a gap in it, which is
what it is. Same dimensions either way, so nothing reflows.
Affects: packages/db/src/schema/reference.ts, migration
0004_previous_gertrude_yorkes.sql, model/ingest/providers/base.py,
model/ingest/providers/sportmonks.py, model/ingest/repository.py,
scripts/backfill_images.py, apps/web/app/_components/badge.tsx.
D-032 — The trends explorer opens on the next fixture
Date: 2026-08-09 · Phase: 4 · Spec: §8.5, §9.1
Observed: /trends opened on a hub of season aggregates. That answers "who
fouls a lot", which is a good question and not the one a reader arrives with.
Decision: lead both the hub and each stat page with the model's leading calls for the soonest fixture it has projected. The hub shows the top three per market; a stat page shows the top five for that market only.
Why it belongs here rather than only on the match page: it is the one surface where the model and the trends meet. A per-90 table describes what has happened; the projection says what is expected next. Putting them on one screen is the argument for the model, made without a paragraph.
§9.1 makes the copy load-bearing. "Tips" is forbidden and §6.5 blocks publication, so nothing here has been compared to a price. Every row carries expected minutes and sample size, the panel states in its own words that nothing has been priced against a market, and the heading is "Next up" rather than anything resembling a selection.
A stat the model does not price shows no panel at all. Interceptions,
clearances and duels won have trends but no projection, and
marketForStatKey returning undefined is a real answer rather than a fallback
to some other market's numbers under this page's heading.
Affects: apps/web/lib/queries.ts, apps/web/lib/model.ts,
apps/web/app/_components/next-fixture-calls.tsx, apps/web/app/trends/.
D-033 — Pages read the latest FINISHED projection run
Date: 2026-08-09 · Phase: 4 · Spec: §5.5, §10
Observed: /trends rendered three of seven markets. Nothing looked broken —
the panel simply had four fewer cards than it should, and no indication why.
Cause: every projection query read max(run_id). The projection job commits
market by market, so during the several minutes a cycle takes there is a newest
run holding a partial board. model_runs.finished_at existed in the schema and
was NULL on all 40 rows — nothing had ever written it, so nothing
downstream could tell a run in progress from a finished one.
A partial board is not a slow board. It is a wrong one that looks exactly like a right one, which makes it the §0 failure mode: a reader sees the model having no opinion on tackles, when in fact the row was still being written.
Decision: cycle.py sets finished_at and fixture_count when the run
completes, and every projection query joins model_runs and takes the newest
run with finished_at IS NOT NULL. Historical runs were backfilled — all but
the newest are definitionally over.
The first fix was worse than the bug. Expressed as a correlated subquery in
the select list ((SELECT max(run_id) ... WHERE fixture_id = f.id)), it re-ran
per fixture against 1.1M projection rows. The homepage went from fast to a
statement timeout, which Next served as a hung request — 90 seconds and no
error. Rewritten as one JOIN and GROUP BY: 4.4s cold, and correct. This is the
same trap getPlayerTrends documented after the LATERAL incident, walked into
from the other direction; the note now sits above all three queries.
Affects: model/project/cycle.py, apps/web/lib/queries.ts.
D-034 — One page per market, and the market is the route
Date: 2026-08-09 · Phase: 4 · Spec: §8.5
Observed: choosing a market meant scrolling to a filter panel, finding a chip group labelled "Statistic" among eight other chip groups, and clicking one of eleven chips. The operator's words: "the way we currently have to go to click on player trends fouls committed or shots on target is awful."
The mistake underneath it: ?stat=fouls_committed treated the market as a
filter. It is not — it is the page you are on. Everything else on that screen
(league, team, position, window, venue, sort) refines a question; the market
is the question.
Decision: /trends/players/[stat], one route per market, with slugs
(fouls-committed, shots-on-target). The market moves out of the filter
panel into a tab strip at the top. The hub becomes a grid of market cards, each
with a sentence on what the stat is and why it matters for pricing — "tackles"
tells a newcomer nothing about whether it is the page they want.
What this buys beyond the click: eleven indexable addresses instead of one,
which is the whole of §8.5's argument for these pages; a link someone can paste
into a group chat that says what it is; and a generateStaticParams list, so
the routes are known at build time.
Compatibility: /trends/players redirects, carrying filters and honouring
an old ?stat= if present. An unknown slug 404s rather than falling back to
fouls — a URL that quietly serves a different market than it names is the kind
of thing a reader only notices after acting on it.
Affects: apps/web/app/trends/, apps/web/lib/trends.ts,
apps/web/lib/model.ts (MarketMeta.statSlug).
D-035 — Hot trends derive the line from the run
Date: 2026-08-09 · Phase: 4 · Spec: §8.5, §9.1, §9.4
A homepage panel of players who cleared a line in every one of their last five matches and play again this week.
The first version was useless and looked fine. It tested each stat's common line and reported anyone clearing it five times. The panel filled with five Blackburn centre-backs clearing "over 1.5 clearances" while averaging nine per 90 — every number true, and nothing learned. A line you beat six times over is not a streak, it is arithmetic.
Fix: min(value) - 0.5. The highest line the player cleared in every match
of the window, so the claim is the strongest true one available — "over 5.5
clearances in all five" rather than "over 1.5". Exact, because a half-line
below an observed integer minimum cannot have been missed, and self-scaling, so
one rule serves cards and clearances alike.
Second problem, same shape: ranked by kickoff alone, the panel was still all clearances and duels won — the highest-count stats, so the easiest to run, and also the three the model does not price and no bookmaker quotes. Markets the model prices now sort first. Sorted rather than filtered: a genuine clearance run is worth showing when there is nothing better, it just does not lead.
Diversity caps — one row per player, two per fixture, two per market — because five players from one team in one match doing one thing are five views of a single fact.
§9.4 and §9.1 govern the presentation. The run is rendered at the same weight as the headline figure, because "100%" flatters itself and 2-2-2-2-3 can be judged. The denominator never drops. The footer says in plain words that five matches is five matches, and that nothing has been compared to a price — which is not a disclaimer here but a fact: the odds feed carries no player prop markets at all.
Affects: apps/web/lib/trends.ts, apps/web/app/_components/hot-trends.tsx,
apps/web/app/page.tsx, apps/web/app/trends/page.tsx.
D-036 — Slugged match URLs, with the id still authoritative
Date: 2026-08-09 · Phase: 4 · Spec: §8.5, §10
/match/24891 → /match/24891-championship-wolverhampton-wanderers-v-blackburn-rovers.
The id stays at the front and stays the lookup key, parsed off with
parseInt. Both forms resolve, so every link already shared keeps working, and
rel="canonical" points at the slugged one so a search engine sees one page
rather than two. The alternative — a slug column looked up by string — means a
renamed club silently 404s its own history, which is worse than an ugly URL.
Affects: apps/web/lib/slug.ts, apps/web/app/match/[id]/page.tsx.
D-037 — A glossary, because the tooltips were the documentation
Date: 2026-08-09 · Phase: 4 · Spec: §8.5
Half the figures on this site only mean something if you know how they were
built — a per-90 off cameos, a hit rate over five matches, a rate that is
mostly a position group. All of that was explained in title attributes, which
nobody hovers and no search engine reads.
24 terms in four sections, each with a definition and a Reading it line saying what the number does not tell you. That second line is the one that earns its place: without it this is a dictionary, with it it is the argument for the model.
Definitions describe what OUR feed records, not what Opta records or what a bookmaker settles on. §6.5 exists because those differ, and a glossary that blurred them would be teaching readers to defend our numbers against the wrong rulebook. Tackles say so explicitly.
DefinedTermSet structured data rather than FAQPage. FAQ markup ranks more
aggressively and would be a lie about the shape of the content.
Affects: apps/web/lib/glossary.ts, apps/web/app/glossary/page.tsx.
D-038 — A backfill may not answer "where does he play now"
Date: 2026-08-09 · Phase: 4 · Spec: §4.2, §5.3
Observed, by the operator: Matheus Cunha listed at Wolverhampton Wanderers. He plays for Manchester United. Wolves were shown in the Premier League; they are in the Championship.
Cause. upsert_players and upsert_teams are called by two jobs with
opposite intent. sync_reference syncs the season happening now.
sync_history walks completed seasons newest to oldest, and D-020 gave it
these same calls so a historical fixture could resolve its players. Both wrote
players.current_team_id and teams.league_id, so the OLDEST season — written
last — decided both. Cunha's 2024/25 club won.
Confirmed against the stored payloads, not inferred: he appears in season 28083 (2026/27) at team 14, season 25583 at team 14, and season 23614 (2024/25) at team 29. Team 14 is Manchester United; 29 is Wolves. The right answer was in the database the whole time, in a row the backfill had overwritten.
How visible it was: the Premier League held 25 clubs, including three relegated in 2025 and Wolves. La Liga held 26, Bundesliga 23. Nobody had looked at the count.
Why it was worse than a wrong badge. current_team_id is what D-026's
squad filter tests. A stale value drops players who are present and keeps
players who have gone — silently, and while reporting a confident count of what
it dropped. The fix to D-026 was resting on a column that did not mean what it
said.
Decision: upsert_players and upsert_teams take current: bool.
sync_history passes current=False — it still establishes that a player and
a club exist, which is what D-020 needed, and it no longer touches the two
columns that describe now. sync_reference passes it per season, so
--all-seasons cannot reintroduce the bug through the other door.
Repaired without a single API call. scripts/repair_current_squads.py
rebuilds both columns from raw_payloads — §4.3's promise that "re-parsing
never requires re-fetching" paying out a second time. /teams/seasons/{id} is
the authoritative division list; squad payloads were tried first and
under-cover, which would have left the stale value in place for any club whose
squad fetch had failed. Result: 18/24/20/20/20 clubs, which is correct for all
five divisions.
A club or player not in a current list is set to NULL, not left alone. "Not in a covered division" is a different fact from "in the division I last saw them in", and only one of them is true. NULL makes the gap visible to the squad filter instead of feeding it a confident wrong answer.
Found while repairing, and left as it is: current-squad coverage is
strongly uneven — Premier League averages 29 players a club and La Liga 25,
against Serie A 14, Championship 6 and Bundesliga 3. That is the provider, not
us. MIN_SQUAD_SIZE catches it: those clubs' squads are not filtered and each
one logs a warning naming the count. Worth revisiting once the seasons start.
Affects: model/ingest/repository.py, model/ingest/jobs/sync_history.py,
model/ingest/jobs/sync_reference.py, model/tests/test_current_assignment.py,
scripts/repair_current_squads.py.
D-039 — The projection run is a property of the run, not of the fixture
Date: 2026-08-09 · Phase: 4 · Spec: §10
Observed: the homepage took 7.2 seconds. §10 asks for LCP under 2.
Measured before touching anything. Seven of the eight queries came to
524ms between them. getTopProjections was 7,960ms.
EXPLAIN named it: max(run_id) FROM projections ... GROUP BY fixture_id
made Postgres visit every projection row for every fixture across all 48 runs
— 1,349,591 rows scanned to return 103.
Decision: ask model_runs instead. A cycle writes one run covering every
fixture in the window, so "which run" is a property of the run, and
SELECT max(id) FROM model_runs WHERE finished_at IS NOT NULL answers it from
44 rows. 7,960ms → 49ms.
A fixture the newest run did not cover now shows no projection rather than one from three runs ago. That is the better answer regardless of speed: an old projection carries old expected minutes.
Second change, for the same reason and with a UX argument of its own: the
board defaulted to all 21 days — 101 fixtures, 330 images, four cross-joins —
to render a page nobody reads past the first screen of. It now opens on the
soonest day that HAS fixtures, with ?date=all one click away. "Today" would
be wrong: for most of a week in a European season there is nothing on.
Production, warm: homepage 7,248ms → 467ms, 16 images instead of 330.
/trends 65ms, /glossary 44ms. Inside §10's budget with room.
Affects: apps/web/lib/queries.ts, apps/web/app/page.tsx.
D-040 — The two hot-trends panels answer different questions
Date: 2026-08-09 · Phase: 4 · Spec: §8.2, §8.5
D-035's panel was rendered on the homepage and on /trends with identical
arguments, so it produced the same six cards in both places. The operator: "I don't see
the point of it being the same exact thing."
That is one feature shown twice, and it made the second one look like padding.
Decision: two questions, one component.
| Homepage | /trends |
|
|---|---|---|
| Scope | the fixtures ON SCREEN — moves with the date strip | the next fortnight |
| Markets | only those the model prices | all of them |
| Size | 3 cards, one per fixture | 12 cards |
| Control | none | run length: 5, 8 or 10 |
The homepage version is an annotation on the board below it. The /trends
version is the board in its own right, and the run-length switcher is the thing
that makes it worth visiting: a run of ten is far rarer than a run of five and
says correspondingly more.
Affects: apps/web/lib/trends.ts,
apps/web/app/_components/hot-trends.tsx, apps/web/app/page.tsx,
apps/web/app/trends/page.tsx.
D-041 — A fixture is complete when BOTH sides are, not when the total looks right
Date: 2026-08-09 · Phase: 4 · Spec: §4.3, §5.5, §13
Found by scripts/factcheck.py, written for this purpose after D-038 went
three days unnoticed. The check was "eleven players for ninety minutes is 990"
and it flagged West Ham v Bournemouth at 198 minutes for one side.
Cause. _fixtures_with_stats judged a fixture complete at
count(*) >= 18 across both teams. West Ham v Bournemouth stored 3 rows
for one side and 16 for the other: 19 in total, over the line, checkpointed,
never retried. The docstring on that function warns about exactly this failure
— "five of thirty players, every one of them a current squad member, none of
the departures" — and then guards against a globally thin fixture while a
lopsided one walks straight past.
Scale: 494 of 4,005 finished fixtures — 12.3% of the panel — had a side with fewer than ten player rows. All of them counted as done.
Why it mattered beyond the missing rows. A player's absent row takes his minutes with it, so a per-90 rate is not biased by it — but the last five on a trends page silently skips matches he played, and the appearance count underneath every hit rate is understated. §9.4 puts sample size next to every figure precisely so a reader can weigh it, and the number was wrong.
Decision: MIN_ROWS_PER_SIDE = 10 alongside the existing total, and both
sides must be present. A side fields eleven and uses up to five substitutes, so
ten is already generous.
Repaired in one pass: 230 fixtures re-fetched, 7,540 rows written, 0
unresolved. 494 lopsided fixtures → 0. Panel 117,904 → 120,409 rows.
Refit on the enlarged panel: all seven gates re-passed, player_fouls
calibration improved 0.057 → 0.016.
A mistake worth recording. After the refit I marked runs finished with
UPDATE ... WHERE id < max(id), reasoning that any run but the newest must be
over. One of them had been killed mid-projection with five of seven markets, so
that blanket update published an incomplete board — the exact failure D-039
exists to prevent, reintroduced by hand. factcheck.py caught it on the next
run. A run is finished when the job says so, never because it is not the
newest, and scripts/run_projections.py now marks its own runs the way
cycle.py does.
Left as warnings, deliberately:
- Two rows with three yellow cards (Maddison, O'Brien). The provider sends
value: 3on the yellow-card field; a player cannot be booked three times. Two rows in 120,409, and it cannot change a "to be carded" outcome, so it is flagged rather than capped — capping would be inventing data. - 52 clubs with thin current squads (D-038): provider coverage, uneven by division.
- 14 upcoming fixtures without a referee: officials are appointed a few days out.
Affects: model/ingest/jobs/sync_history.py, scripts/factcheck.py,
scripts/run_projections.py, docs/MODEL.md.
D-042 — [withheld]
Withheld from the public render — commercial detail — supplier evaluation. The internal log is append-only and this entry exists there unedited; the number is preserved here so the sequence stays checkable.
D-043 — UK Odds adapter: names are the identity, so identity is a persisted decision
Date: 2026-08-09 · Phase: 4 · Spec: §0, §4.1, §6.4, §6.5
The D-042 adapter is built: model/ingest/providers/ukodds.py, odds-only.
Every stats-side Provider method raises OddsOnlyProviderError naming the
reason — this feed prices players, the Sportmonks panel describes them, and an
adapter that quietly returned nothing for squads would look like a provider
with no squads rather than a misused one.
Payload shapes were re-captured live before writing the parser (Kilmarnock v
Celtic, package=full: 88 markets, 25 bookmakers, 15,063 selections — all
ACTIVE, all prop sides Over, confirming D-042 on both counts).
The feed has no ids for teams or players, so identity became explicit architecture rather than a join:
OddsQuotegainedplayer_name. The adapter parses"Arne Engels Over 0.5"into name/side/line and hands the NAME across the §6.5 boundary; resolving it toplayers.idhappens in exactly one place, the prop sweep's matcher — never inside an adapter.odds_event_links(new table): one ukodds event ↔ onefixturesrow, matched on kickoff within 10 minutes AND both team names under exact normalised equality (accents/punctuation folded, nothing fuzzier — shared tokens must never link Manchester United to Manchester City). Zero or two candidates is a refusal, surfaced in the job detail with the events at the same kickoff so an operator can extendTEAM_ALIASESor insert the link by hand. The alias map ships EMPTY: an unverified alias is a guessed identity, and guesses are §0's failure mode.prop_name_links(new table): per fixture, every printed player name and what was decided about it —matched/unmatched/ambiguous/manual. Candidates are ONLY the fixture's two squads (current_team_id), matched on exact normalised equality againstnameanddisplay_name; two candidates normalising identically is a refusal that names both ids. Unmatched and ambiguous names are re-attempted every poll (a squad sync can arrive between polls); matched names are never re-derived — identity does not flap. This table IS the §6.4 unresolved-name log the D-042 handoff demanded.
Match markets ride along. The same package=full payload carries 1X2,
BTTS, DNB, double chance, Asian handicap and totals from UK books — and the
Sportmonks quotes for those markets come from 1xbet, Dafabet and Sbo, which
§0 says are not prices at all for our readers. Mapping them costs zero extra
calls, so the adapter does. Two subtleties:
- The feed's "Total Goals Over/Under" MIXES plain half lines (2.5) with Asian
quarter and whole lines (2.25, 3). They settle differently, so lines are
routed per selection:
x.5→ou_goals, everything else →goal_line. - "Asian Total" duplicates those Asian lines. Mapping both would give one selection key two sources and store-on-change (D-006) would record their disagreement as a price flapping between polls. One source wins (the larger market); the other is counted as skipped, not silently dropped.
Feed quirks verified, for the next parser-reader:
selection_countOVER-CLAIMS: "Player Shots" said 1,855, the array held 1,699, and all 1,699 parse. Reconcile against the array, never the field.- The line arrives twice on prop selections — inside the name and as a
field. Disagreement means the naming convention shifted; the quote is
dropped and counted (
prop_line_mismatch), never guessed. - Betfair arrives typed
exchange;OddsQuote.bookmaker_is_exchangecarries it intobookmakers.is_exchange, because an exchange price includes commission a reader's return must absorb. - Non-
ACTIVEselections are dropped as untakeable (§0), counted. - Every dropped selection lands in the adapter's
drop_report, which the sweep copies into the job detail (§6.4).
Affects: providers/ukodds.py, providers/base.py (OddsQuote,
get_provider), prop_names.py, migration 0006 (odds_event_links,
prop_name_links), snapshot_odds._store_changed (injectable player
resolver so D-006 identity lives in one place), config
(UK_ODDS_API_KEY / key_for), tests (40 new, fixtures trimmed verbatim
from the live capture).
D-044 — The prop sweep's horizon is 24 hours, not D-021's 21 days
Date: 2026-08-09 · Phase: 4 · Spec: §4.2, §10 · Supersedes: nothing — D-021 still governs match markets
snapshot_prop_odds is a separate job from snapshot_odds, on separate
windows: every 30 minutes for fixtures inside T-24h, every 5 minutes
inside T-60m (the §4.2 near cadence, so the closing line is never starved).
Why not the 21-day far window: props measurably do not exist there — five days out a fixture returned 63 markets and no props; same-day, 88 with them (D-042). A 21-day prop sweep spends the budget asking for markets that are not open.
Why 24 hours when only same-day is verified: the boundary between "no props"
and "props" is somewhere in 1–5 days and unknown. At two calls per fixture
per hour the 24h window is nearly free (84/min, 120,000/day limits), and
every poll records prop_quotes_seen — so when the season starts, the hour
at which each book opens its props becomes something odds_polls can answer
instead of something assumed. If books turn out to open earlier than 24h, the
window widens on evidence.
The job only enters the worker schedule when PROPBOARD_ODDS_PROVIDER is
set (now ukodds in .env): a sweep with no feed could only fail, and a
permanently failing job pages §10 for a missing feature. Manual runs:
propboard snapshot-prop-odds [--mode near|day|auto].
Fixture selection reads the STATS provider's rows (that is where kickoffs
and squads live); the ukodds event is looked up through odds_event_links
at poll time, linking any new fixtures first. League names are deliberately
NOT a filter when linking — they are unverified for our five leagues, and
filtering on a wrong name would silently drop a whole league, the exact trap
D-042 warns about. Kickoff plus both team names is the identity.
Affects: jobs/snapshot_prop_odds.py, worker.build_schedule
(parameterised on the odds provider), cli.py, core/config.py
(PROPBOARD_ODDS_PROVIDER), .env, .env.example.
D-045 — CLV on props will use the raw closing price, and say so
Date: 2026-08-09 · Phase: 4 · Spec: §6.6, §15
Every prop selection the feed returns is an Over — no unders, verified twice
(D-042 live, D-043 full-payload parse: sides over and yes only). §6.6
wants CLV against a de-vigged closing probability, and de-vigging needs both
sides of the market. For props, both sides do not exist here.
D-042 required this to be decided explicitly rather than discovered later, so: prop CLV compares the taken price against the raw closing price of the same selection at the same book, vig included, and every surface that shows prop CLV states that caveat. Match markets, which do carry both sides, de-vig as specced. The alternative — waiting for a two-way source — would leave §15's verdict metric uncomputable indefinitely for exactly the markets the product exists to price.
Consequences to hold onto: raw-price CLV slightly UNDERSTATES true closing value on average (the closing price still contains margin), and it is only comparable within a book, not across books with different margins. Neither bias favours us, which is the acceptable direction (§0). Revisit if a two-way prop source appears; a new decision entry supersedes this one.
Nothing is implemented against this yet — published_edges is still 0 and
CLV runs post-publication. This entry exists so the Over-only observation has
a recorded consequence instead of an unmade decision waiting inside §6.6's
code.
Affects: the future CLV computation in model/publish, §6.6 display
copy, docs/TRIAL-RUNBOOK.md §15 verdict reading.
D-046 — Settlement research: everything settles on Opta; tackles excluded; our cards column has the gap
Date: 2026-08-09 · Phase: 4 · Spec: §6.4, §6.5, §0
docs/settlement-sources.md is no longer empty. The research (sources and
quotes recorded there) resolves §6.5 for the four preferred UK books:
All four settle player statistics on Opta. Sky Bet's Opta-definitions page was read in full and is the canonical reference; Paddy Power hosts the same Flutter page; William Hill states Opta settlement in its rules; bet365's own pages state it too but were unreachable from this environment (403), so bet365 carries a confirm-in-app caveat before anything publishes against them. Ladbrokes/Coral and Betfair Sportsbook remain unverified and excluded.
Three findings that change what may publish:
Tackles: EXCLUDED, as §6.5 predicted. Opta's tackle requires "successfully takes the ball away"; our
tacklescolumn counts attempts (D-012), and Sportmonks' own glossary defines a tackle as a player "trying" to take the ball. Attempted vs won is not a scaling factor that can be guessed, and there is no Opta-settled sample to fit one against.player_tacklespublishes against no book until that changes.Cards: the definitional gap is OURS. Books settle "To Be Shown A Card" on any card; our market's stat_key is
yellow_cards, so a straight red settles YES at the book and 0 in our column — a systematic one-way bias. DEFERRED deliberately rather than hot-fixed: it is a model change (the market must price P(any card)), the golden regression set (§13) still does not exist, and cards cannot publish anyway until sign-off. The fix is a named precondition for publishing cards, recorded in settlement-sources.md.Fouls and shots: ALIGNED, with evidence rather than assumption. Opta excludes offsides from fouls; Sportmonks types them separately and our 11.5 fouls/team/match average is consistent with offside-free counts. Sportmonks documents shots_total as including blocked, matching Opta Total Shots — with a 1.5% panel inconsistency in the auxiliary shots_blocked column flagged for a first-matchweek spot-check.
The engine stays locked. bookmakers.settlement_source remains NULL for
every book: the verdicts need the operator's sign-off before the column is set,
because setting it is the act that lets §6.4 pass an edge. The doc states
exactly which value to set for whom once signed.
Affects: docs/settlement-sources.md (now the authority),
player_cards market definition (pre-publication fix), player_tackles
(excluded), the §6.4 settlement gate.
D-047 — The §7 simulation engine: fitted latent factors, mean-anchored draws, and what the builder refuses to price
Date: 2026-08-09 · Phase: post-decision (§14 step 18) · Spec: §3.1, §7, §8.7, §13
model/simulate is real: 10,000-iteration Monte Carlo per fixture
(§7.2), persisted compressed to fixture_simulations, regenerated per
projection run by a simulate_fixtures worker job, priced by the web side
counting stored draws, and surfaced at /builder with the §7.4 margin audit.
First live run: 80 fixtures, run 68.
The latent factors are FITTED, and the identification is worth recording. A shared multiplicative factor with variance c² leaves its fingerprint in the cross-team covariance of a stat: Cov(home, away) = mu_h·mu_a·c². From 4,005 paired team-matches in our own panel: fouls covary POSITIVELY across teams → tempo σ=0.088; shots covary NEGATIVELY (−8.64) → possession swing σ=0.237; cards → intensity σ=0.308. All three signs came out as theory demands, which is the check that the moment-matching is measuring something real. The fitted numbers travel on every simulation row.
The §0 anchor: marginals are preserved exactly. Every multiplier chain (tempo, possession, cards, minutes) is mean-corrected so E[count] equals the projection's mu, and §13 property tests assert a single-leg slip prices within Monte Carlo error of the closed-form NB probability. The simulator adds correlation, never a different opinion — the builder and the match page can never quote two numbers for the same bet.
Minutes are recovered, not stored. Projections carry only expected
minutes, but minutes.py built that expectation as start_rate × 78 + bench
term — so the start/bench mixture is recovered by inverting the same
formula with the same constants, exact for the low state every current
projection is in. The draw stream is then rescaled so each player's mean
ratio is exactly 1.
What the builder deliberately does NOT offer:
- Tackles and cards — excluded at the JOB level (D-046: settlement mismatch; straight-red gap), so no downstream surface can show them by accident. They return when their D-046 actions are done.
- Integer lines — they push (D-015), a push voids a leg, and "count iterations where all legs hold" cannot express a void. Half-lines only.
- Bookmaker prices on same-game slips — §7.5: they are proprietary and in no feed. The margin audit is the product instead.
Recorded deviations from the spec:
- Simulation horizon is the projection window, not §8.7's 72h. Before the season starts nothing is inside 72h and an untestable builder helps nobody. Reinstate the UI-side 72h constraint when fixtures are imminent if slip staleness becomes an issue.
- The free-tier 2-leg cap is not enforced — there is no auth to distinguish tiers yet (§14 puts auth post-decision). The cap arrives with accounts.
- §7.5 best-odds shopping and §8.7 save/share are not built — shopping needs cross-book leg prices (prop capture starts 14 Aug), save/share needs accounts.
A known limitation, found live on the first slip priced: minutes draws are independent ACROSS players, so squad exclusivity is not modelled — the two Blackburn goalkeepers both price ~72% on saves overs, and a slip containing both prices at 54% when reality is near-zero (only one plays). Harmless for normal slips, wrong for same-position-same-team pairs, and it self-corrects as lineups arrive (confirmed lineups collapse start probabilities to 0/1, making exclusivity implicit). Not patched with an ad-hoc keeper rule: the honest fix is lineup-aware minutes, which is already the §5.3 roadmap.
§3.1 note. The web side counts stored draws (apps/web/lib/simulations.ts) — the same boundary argument as overProbability: the draws ARE model output, and a conjunction count reads them at a different granularity. No distribution, mu or probability is computed in TypeScript.
Affects: model/simulate/*, fixture_simulations (migration 0007),
worker schedule, propboard simulate-fixtures, apps/web/lib/simulations.ts,
/builder, 17 new §13 property tests.
D-048 — Cards price ANY card now: cards_total closes the straight-red gap
Date: 2026-08-10 · Phase: 4 · Spec: §0, §5.4, §5.6, §6.5, §13
D-046 found the cards market's definitional gap sat on OUR side: books settle
"To Be Shown A Card" on any card, and the market's stat_key was
yellow_cards — 325 player-matches in the panel took a straight red and
were scored as uncarded, a systematic one-way bias (14,443 any-card events
vs 14,099 yellows).
The fix, end to end:
player_fixture_stats.cards_total— a GENERATED column,coalesce(yellow_cards,0) + coalesce(red_cards,0)(migration 0008), so the settlement column §6.4 resolves against can never disagree with its parts. Treating an absent red as none leans on D-016's verified absent-means-zero semantics; a red card is exactly the kind of event the feed emits when it happens.STAT_COLUMNSandmarkets.stat_keynow mapplayer_cards→cards_total.- k re-cross-validated on the new target (§5.4: fit, don't guess): returned 45 — the same interior optimum as the yellow-only target, which is what ~2.4% extra events should do.
- Full refit →
v20260810-1437, all seven markets passing §5.6; cards calibrates at 0.0089 against the 0.100 tolerance on the any-card target.
The golden set did its §13 job on its first day. golden.py --compare
against the v20260809-1634 artifact: cards shifted on 186 of 187 rows (mean
|dmu|/mu 3.0%, max 9.6% — mus rise because reds now count); every other
market reproduced to 0.0000%. The blast radius is exactly the market whose
definition changed, measured rather than asserted. Artifact regenerated at
the new version in this commit.
Consequences: cards left the simulate-job exclusion list, so /builder
offers card legs again; the settlement-sources cards row is OK; the match
page's cards panel carries a caveat that the market prices any card while
the yellow-cards trend column alongside counts yellows only — a display
nuance, deliberately not "fixed" by relabelling the trend, which honestly
counts what it counts.
Affects: migration 0008, features/panel.py, model/config/leagues.yaml,
markets.stat_key, thresholds.yaml (k refreshed), simulate/job.py,
apps/web/lib/model.ts (caveat), golden_set.json (regenerated),
settlement-sources.md.
D-049 — Settlement sign-off recorded; the §6.4 gate is armed for four books
Date: 2026-08-10 · Phase: 4 · Spec: §6.4, §6.5, §15
The operator signed the D-046 settlement verdicts (instruction given in chat,
2026-08-10, recorded in settlement-sources.md), confirming the bet365
statement as true. bookmakers.settlement_source = 'opta' and
is_uk = true are now set for the ukodds rows of bet365 (UO004), Sky Bet
(UO022), William Hill (UO028) and Paddy Power (UO019) — pre-registered
with the provider ids the D-042 capture verified, so the gate is armed
BEFORE the first prop sweep stores a quote rather than after someone
notices the column is NULL.
The publish engine already enforces this end to end (verified, not assumed:
publish/job.py reads b.settlement_source, engine.py rejects
SETTLEMENT_SOURCE_UNVERIFIED), so from the first captured price the +EV
pipeline is capable of publishing against those four books and no others.
Ladbrokes, Coral and Betfair Sportsbook stay NULL and excluded until their
rules are read and signed.
What remains between here and the first published edge: prop prices existing (books open them near kickoff — first fixtures enter the capture window 2026-08-14), a price clearing the §6.3 thresholds, and the §6.4 sanity guards passing. Nothing procedural remains.
Affects: bookmakers rows (ukodds × 4), settlement-sources.md status,
/value gate 3 (now reads the verified-book list live).
D-050 — The books open props days out; the FEED lags them. Window widened to a week
Date: 2026-08-10 · Phase: 4 · Spec: §4.2, §16 · Amends: D-044
The operator observed player shots and shots-on-target lines live at the bookmaker
for Wolverhampton v Blackburn — four days before Friday's kickoff. The feed
was checked the same hour: ukodds event evt_363a7d3b… returned 63 markets
and not one player market — the same propless shape D-042 saw on a
fixture five days out.
So D-042/D-044's "props open near kickoff" conflated two clocks. The BOOKS open props four-plus days ahead; the AGGREGATOR starts carrying them some time later (between T-4d and matchday — exactly when is now the measurable question). A 24-hour window built on the conflation would have captured whatever the feed's lag leaves, and no more.
Changes:
DAY_WINDOW_HOURS24 → 168. Capture now starts the hour the feed does, whichever hour that is. Cost: ~2 calls per in-window fixture per 30 minutes against 84/min / 120k/day — noise.prop_quotes_seenon every poll turns the feed's prop onset into data per fixture.- Sixteen VERIFIED
TEAM_ALIASESentries. Wolverhampton/Blackburn came first (league + kickoff match fixture 24891 exactly); the first sweep's fixtures_unlinked log then supplied same-league same-kickoff evidence pairs for every remaining fixture in the window — the feed strips suffixes (Wanderers, City, County, Athletic, United, North End) and Spanish articles. The strict matcher refused all of them until the evidence existed, which is the alias policy working, not failing. Second sweep: 16/16 fixtures linked, 11,939 UK-book match-market prices stored across the two sweeps, zero props yet — the lag instrument is live. - Side effect worth naming: the same sweeps now capture UK-book MATCH markets (1X2, BTTS, totals, handicaps) all week. The Sportmonks odds bundle quotes 1xbet/Dafabet/Sbo; the homepage's best-price columns can now be fed by bet365, Sky Bet, William Hill and Paddy Power prices the moment the sweep stores them.
Recorded honestly: prices exist at the books for days before this feed carries them. Until the feed's lag is measured, the earliest prop price we hold for an edge may be well after the market opened — which caps how much pre-publish price history CLV context can lean on. If the measured lag turns out to be material, a second odds source is a §16-class question for later, with data to ask it properly.
Affects: jobs/snapshot_prop_odds.py (window, aliases), /value gate-2
copy, odds_polls as the feed-onset instrument.
D-051 — [withheld]
Withheld from the public render — infrastructure detail — capacity and cost. The internal log is append-only and this entry exists there unedited; the number is preserved here so the sequence stays checkable.
D-052 — The calibration gate was grading itself generously; hardened, two markets fail honestly
Date: 2026-08-10 · Phase: 4 · Spec: §0, §5.6, §6.3, §13
An external audit of the methodology page flagged shots on target's stored calibration of exactly 0.000 as "essentially impossible with finite samples". Chasing it found three compounding defects in the §5.6 gate:
- The metric filtered before it measured.
max_calibration_errortook the max over buckets that were BOTH n≥30 AND individually ≥3σ — so SOT's 0.5–0.6 decile (n=37, predicted 0.538, observed 0.351, z≈−2.3) and its 0.4–0.5 neighbour (n=118, predicted 0.441, observed 0.331) both fell under the significance bar and the "worst miss" printed as a perfect 0.000. Two same-direction double-digit misses is not the noise case a 3σ bar exists for. - No evidence scored as a pass. With zero gated buckets the function
returned 0.0 — indistinguishable from perfection. Our own test fixture
(
make_report(buckets=[])) passed the gate. - One flat calibration line for every market. Cards was calibrated at line 1.5 — over 1.5 cards is a ~1% event — so every probability sat below the publishable band and the in-band check was vacuous.
The hardening: measurement and significance are separated
(max_calibration_error = worst miss over n≥30 buckets, returns None —
never 0.0 — when nothing is measurable); the gate fails on EITHER a
tolerance-exceeding miss at ≥2σ (BREACH_Z, down from the 3σ that let 18.7pp
through) OR no measurable in-band bucket at all; every market calibrates at
its own line (cards 0.5, saves 2.5, others 1.5), with the line stored on
the artefact.
Re-gated at the real lines, the truth: fouls 0.047, tackles 0.048, saves 0.038, shots 0.069 pass; fouls_drawn passes with its measured 0.125 shown (no ≥2σ breach). Shots on target FAILS at 0.187 and cards FAILS at 0.177 at its real 0.5 line — the celebrated +61.5% improvement was measured against a baseline that is worst on rare events AND at a line the market never trades. Both markets: projections continue (displayed with caveats naming the failure), builder legs removed, publication blocked. Refit v20260810-1744 activated the five passers; the failed pair's active rows were updated to carry the fresh verdict so the publish gate reads truth (operational correction, this entry). Golden set: zero shift on all 1,309 rows — the gate change moved no numbers, only what may act on them.
The engine now prices model quality per market (the audit's second bug):
EdgeCandidate carries its market's gate verdict and measured in-band miss;
the §6.3 threshold adds that miss as percentage points
(calibration_margin_multiplier: 1.0) — shots needs 4%+6.9pp at a confirmed
lineup, not a flat 4% — and a failed or unproven market rejects outright
(calibration_gate_failed, now per market). The one-for-one pp conversion
slightly understates at longer prices; revisit against CLV data.
Surfaces: the methodology fit table now reads LIVE from the active fitted models (version, improvement, measured miss or "unproven", gate status) — the audited version-mismatch class is structurally dead; /value documents the calibration-margin rule.
Affects: fit/calibration.py, fit/crossval.py, fit/cli.py,
publish/engine.py, publish/job.py, thresholds.yaml,
simulate/job.py (SOT+cards excluded), apps/web (live fit table,
caveats), fitted_models active rows, 9 new tests.
D-053 — Projections are a start/bench mixture, conditional on the player taking part
Date: 2026-08-10 · Phase: 4 · Spec: §0, §5.1, §5.3, §6.1, §7.2
The audit's remaining model bug: projections plugged a single expected-minutes number into one negative binomial, when a rotation player's reality is bimodal — a ~78-minute start or a ~20-minute cameo, almost never their average. Same mean, wrong shape: too much middle mass, thin tails, and the tails are where over bets live. Two findings sharpened the fix:
- The books void on a no-show. Sky Bet: "If the player does not take part in the match, bets will be made void" (primary page read 2026-08-10); bet365: "player specials are void if the player takes no part". A sub cameo STANDS. So the settle-relevant distribution is P(count | takes part) — averaging in the did-not-play branch prices a refund as a loss (see settlement-sources.md, "Void rules").
- The old formula double-counted no-shows.
player_fixture_statshas no rows for unused subs (120,409 rows, zero at minutes = 0), so the historical start rate is ALREADY P(start | played) — and the low state multiplied that conditional rate by an unconditional "used half the time" haircut ((1−rate)·sub/2). Rotation players were understated twice over.
The change. project_minutes now returns the scenario decomposition
itself — P(start | played), minutes-if-started, minutes-if-sub (the player's
OWN sub-cameo average from history, default 20.0 = the panel's measured
19.84) — with expected as the mixture mean, no unused haircut anywhere.
The projection job prices the mixture w·NB(mu_start) + (1−w)·NB(mu_sub)
— exact, because minutes is a log offset so mu scales linearly — and stores
the three scenario columns on the row (migration 0009). The publish job
prices lines from the SAME stored mixture (single-NB fallback only for
pre-D-053 rows, gone after one cycle). The simulator consumes the stored
scenarios directly and its did-not-play branch is deleted: slips price
as-if-all-run, which is what a re-priced builder slip is comparable to. The
old expected-minutes inversion hack in _minutes_draws dies — the same
identity-as-persisted-decision move as odds_event_links.
What moved and what could not. The fitted models are untouched by construction — fitting and the §5.6 gate score at OBSERVED minutes, so the gate verdicts (5 pass / SOT+cards blocked, D-052) stand, and the golden set shows zero shift on all 1,309 rows. What moves is projection-time output, measured run 117 (plug-in) → run 119 (mixture) over 35,728 matched rows: expected minutes +2.69 on average (the un-double-counted conditional exposure), mu +2.8% (cards) to +5.6% (fouls drawn) per market, and P(over the market line) shifting a mean +1.5pp on fouls with single rows moving up to 15.8pp (shots). The distribution of the shift is the audit's prediction coming true: nailed-on starters (w ≥ 0.9) move a mean |ΔP| of 0.0013 — the degenerate mixture reproduces the old price — while rotation players move 0.0127, ten times as much, concentrated exactly where start rates are lowest.
Not done here: medium-state weights still rest on the stated
EXPECTED_LINEUP_ACCURACY = 0.85 and BENCH_USED_PROBABILITY = 0.5 guesses
(unused only for lineup forecasts; measurable once §16.6's lineups feed
delivers), and within-scenario minutes spreads in the simulator remain
distributional choices that the mean-correction prevents from moving any
marginal. Files: project/minutes.py, project/job.py,
core/distributions.py (mixture maths), publish/job.py,
simulate/engine.py, simulate/job.py, simulate/persist.py, migration
0009 + schema/model.ts, apps/web (methodology, builder, match tooltip,
confidence copy), settlement-sources.md void rules, 17 new tests.
D-054 — The run-level calibration flag contradicted the per-market gate; removed
Date: 2026-08-10 · Phase: 4 · Spec: §6.4, §13
Found by the first D-053 publish cycle: every one of its 1,238 candidates
rejected with calibration_gate_failed — including the five markets whose
gates pass. cycle._calibration_passed computed a run-level "do ALL active
models pass §5.6" flag, and D-052 deliberately keeps shots-on-target and
cards active with gate_passed = false so their projections stay visible
behind caveats. Two truths, one flag: the blanket was permanently down and
blocked the sound markets — the exact failure D-052's per-market gate exists
to prevent, reintroduced one layer up.
The engine already carries the gate on each candidate (fm.gate_passed,
absence reads as failed), so the run-level flag is deleted rather than
patched: §13's "refuses to publish against a failed calibration check" holds
market by market. publish_job.run(calibration_passed=...) keeps its
parameter for callers that DO have run-level knowledge; the cycle simply no
longer manufactures a wrong value for it. §6.4 gets the credit: the blanket
was only visible because every rejection is counted by reason.
D-055 — Price freshness comes from the poll that confirmed it, not the change that stored it
Date: 2026-08-10 · Phase: 4 · Spec: §0, §4.2, §6.3, §10
Observed on the first D-053 publish cycle: all 1,238 candidates rejected
stale_price. Capture is store-on-change (D-006), so captured_at is when
a price last MOVED — and the §6.3 staleness check read it as when the price
was last CONFIRMED. Every stable price went "stale" 30 quiet minutes after
its last movement, while the sweeps kept confirming it poll after poll. The
odds_polls schema comment had already named the failure — "getting that
backwards would suppress every edge on a quiet market" — and the web layer
and check-freshness obeyed it; the publish query did not.
Freshness now: a candidate's price age is
greatest(captured_at, its own feed's last poll of the fixture).
Two conditions make that honest rather than convenient:
- A poll vouches only for its own feed.
odds_polls.providerrecords which feed polled (sportmonks / ukodds), and the join matches it to the candidate'sbookmakers.provider. Without this, the stats feed's polls would keep bet365's props "fresh" through a dead prop sweep — §10's highest-severity failure class, silent staleness, rebuilt politely. - A poll vouches only for prices still in its payload. When a selection
this feed stored before is absent from the current payload, the sweep
writes a delisting TOMBSTONE —
odds_snapshots.delisted = true, an insert carrying the last-known price, because disappearance is a change and D-006 stores changes. A tombstoned selection produces no candidate (market_unavailablein spirit: there is no price anyone can take), is excluded from the web's best-price display and from closing-line marking, and a relist stores a fresh row even at the identical price so the tombstone never stays the last word. Tombstones are scoped to the polling feed's own bookmakers — the alternative delists the entire board twice a minute as each feed fails to mention the other's books.
Storage cost: tombstones are transition-writes, the same economy as D-006
itself; odds_polls gains one text column. Migration 0010. The engine's
30-minute threshold is untouched — day-window sweeps run every 30 minutes,
so far-out prices hover at the §6.3 boundary and near-window prices (5-min
cadence) sit comfortably inside it, which is the spec's intent: publication
happens near kickoff on prices a poll just confirmed.
D-056 — Correlated-exposure caps, and stakes become a calculator rather than a caption
Date: 2026-08-10 · Phase: 4 · Spec: §6.2, §7.1, §9, §16.5
The audit's last engineering gap plus its regulatory flag, both decided by The operator in chat today.
Exposure caps. Per-edge Kelly (quarter, capped 2.5%) sizes each bet as
if it were the only one — but edges arrive in correlated clusters, §7.1's
own argument pointed at staking: a strict referee lifts every fouls
projection in his match, and six resulting edges are one opinion wearing
six names. Advising 2.5% on each is 15% of bankroll on a single judgement.
cap_correlated_exposure (publish/engine.py, pure) now runs before the
append-only insert: total advised stake per MATCH is capped at 5% of
bankroll, then per REFEREE at 7.5% on the match-scaled values, both split
proportionally within the group — proportional rather than best-first
because ordering inside a correlated group is model noise, and letting it
pick winners would be false precision. Edges with no assigned referee skip
the referee pass (nothing to group on; the match cap still applies).
Values in thresholds.yaml (exposure_caps), chosen by the operator: two full-size
bets per match, three per referee. Capping is recorded on the run detail
(§6.4). Referee grouping spans the publish run's whole window — a
simultaneous-open-bets model, slightly conservative for edges days apart,
which is the right direction to be wrong in.
Stake display (regulatory). The audit flagged printed per-pick stakes as the strongest GC/ASA surface: a stake next to a selection reads as gambling advice; a calculator the user operates reads as a tool. The operator chose calculator mode: board rows show price, fair price and edge; Kelly figures appear in a bankroll calculator the user actively opens, and remain in the stored record and §8.8 download (ROI-to-advised-stakes stays computable). Copy updated on /value and /methodology. The calculator UI itself ships with the board's populated state — there is nothing to size until the first edge publishes.
Capacity and limiting. /methodology gains the section the audit asked for: prop edges are finite and partly self-consuming, soft books limit winning accounts as their business model, and the product therefore scales as public measurement, not as a money tap. Written before the record starts so it can never look like an excuse after.
One paragraph withheld — internal legal posture (see the D-067 stub).
D-057 — One publication per selection, and the board's populated state ships
Date: 2026-08-10 · Phase: 4 · Spec: §3.1, §8.6, §8.8, §10
Building the board's populated state surfaced a bug that would have fired with the first real edge: the publish job had no already-published guard, so a selection that cleared the gates would re-insert on every 20-minute cycle for as long as it stayed +EV — dozens of copies of one call, each reading as a fresh bet to the §8.8 record, on a table that is append-only precisely so nothing can be cleaned up afterwards. CANDIDATES_SQL now excludes selections already in published_edges (per fixture, market, player, line, side, book): the tracked bet is the FIRST publication at its price, CLV is measured from there, and the same selection at a different book still publishes separately. Not logged as a rejection — the call already stands; success is not a reject reason.
The board. /value now has four states instead of two: the §8.6 empty state (unchanged, until the first edge ever); the §10 suppression state — stale odds hide the rows behind a banner naming the age and the cadence threshold, because prices nobody re-confirmed are guesses wearing timestamps; the live board; and a "no live calls" state when everything published has kicked off (a call in play is neither takeable nor scored). Rows show kickoff, match, player and bet, book, price at publish, fair price, edge and lineup confidence — price and edge, never a printed stake.
The stake calculator (D-056's calculator mode, realised). A bankroll field on the board, client-side only: the figure persists in localStorage and never leaves the browser; stakes render as kelly_pct × bankroll per row, with dashes until the reader types a number — the honest default. The kelly fractions were computed server-side with the per-bet and correlation caps already applied, so the component's entire arithmetic is one multiplication and some date formatting (§3.1 kept). Verified against a throwaway mock-data route (deleted before commit — published_edges is trigger-protected and test rows must never touch it): £500 bankroll → £6.25 / £6.25 / £10.50 against kelly 0.0125/0.0125/0.021, localStorage persistence, and mobile scroll contained to the table wrapper.
D-058 — Unknown registration is not departure; the matcher learns exact variants; first edges publish
Date: 2026-08-11 · Phase: 4 · Spec: §0, §5.3, §6.4, §16.6
Pre-Friday operational triage, which turned out to be a data-semantics bug with two consumers, three exact-matching gaps, one latent SQL crash — and, once fixed, the record's first two published edges.
Unknown ≠ departed. D-038's repair sets current_team_id = NULL for
players absent from captured current-season squad payloads — its own words:
"an UNKNOWN club, not an old one". Pre-season those payloads are thin, so
NULL covered 3,623 of ~5,200 players, including 50-appearance starters. Two
consumers read NULL as departed: the projection squad filter (8,113
player-market rows silently dropped in one run) and the prop-name matcher
(415 of 553 unmatched names were players our own panel places at those
exact clubs). Both now treat NULL on its evidence: a recent appearance for
the club (RECENT_APPEARANCE_DAYS = 425, one full season's span) presumes
presence; positively-registered-elsewhere still drops; unknown-and-stale
drops separately (registration_unknown_stale, 5,369 shed — the old
too-thin fallback had been quietly keeping years-departed players). This
REVERSES test_null_registration_is_not_treated_as_present, which was
written when NULL was a rare schema quirk rather than the majority state.
Exact-variant matching (no fuzz added). Three feed habits, each bridged by an exact rule: token PERMUTATION ("Moya Borja Mayoral" = 'Borja Mayoral Moya' — sorted-token index key, every token present and equal, collisions land in the ambiguity refusal); apostrophe CONCATENATION ("OHare" = O'Hare); hyphen CONCATENATION ("Jeanricner" = Jean-Ricner) — punctuated names index a punctuation-deleted variant alongside the spaced one.
Operator triage (scripts/prop_name_triage.py): lists unmatched names
with token-sharing candidates and their appearance evidence, records
--link decisions as manual (permanent, never re-derived). 51 links
blessed on printed-tokens-⊂-one-candidate evidence (feed drops middle
names: "El Hadji Diouf" → El Hadji Malick Diouf). Lookalike suggestions
sharing one token (Aaron Ramsey ≠ Aaron Wan-Bissaka) deliberately left
unmatched. Result across the day: 37% → 90% resolution (7,753 of 8,598
prop-quote resolutions), 553 → 114 persisted unmatched, all remaining ones
genuine coverage gaps.
Hotfix: D-057's one-publication guard compared text to the odds_side
enum and had crashed every publish since it shipped — invisible because
published_edges was empty and the planner rejects the query regardless.
l.side::text. Found because the cycle was run by hand; the §10 freshness
job would have caught it by morning.
Also confirmed: the Sportmonks plan already includes Expected Lineups (probe on a finished fixture returned both confirmed and expected rows), so §16.6's feed starts flowing when the T-72h window opens — no add-on needed.
And then it published. With the matcher resolving and freshness sane, run 129 evaluated 5,487 priced selections (4.4x the day before) and two cleared every gate: Urko González over 1.5 shots at 5.00 (fair 4.01, edge 24.5%, n=52) and Lewis Gibson over 1.5 shots at 8.00 (fair 6.30, edge 27.1%, n=46) — both Bet365 (settlement-verified), both in the calibration-passed shots market with its 6.9pp margin cleared on the 10% provisional tier. Rows 1 and 2 of the permanent record, awaiting their closing lines.
D-059 — §5.1's conditional structure, at last: SOT and cards re-pass the gate they failed
Date: 2026-08-11 · Phase: 4 · Spec: §0, §5.1, §5.4, §5.6, §13
§5.1's table always said it: shots on target is "Negative binomial, conditioned on shots — model shots first, then conversion rate", and cards' "foul count is the primary input". Both markets were built as DIRECT negative binomials — a shortcut past the spec — and D-052's hardened gate caught the consequence: overconfidence exactly where the model was most sure (SOT 18.7pp, cards 17.7pp worst in-band miss). This entry implements what the spec prescribed and lets the same gate rule on it.
The structure. Under the Gamma-Poisson mixture, thinning NB(mu, alpha)
by a fraction q gives NB(q·mu, alpha) exactly — the child inherits its
parent's dispersion and calibration, and the only new estimate is q, a
bounded per-player fraction. q = career child-total over career
parent-total (both lagged in the panel; no row informs its own fraction),
shrunk toward a league-and-position prior of stratum SUMS with a strength
fitted by season CV under _choose_k's calibration-first rule: SOT chose
40 parent events, cards 160, both mid-grid. The projection job rides the
PARENT's design, covariates and k, then multiplies by the player's shrunk
q; refits regenerate parent and child in one run so the pinned alpha never
drifts from the mu it multiplies.
The verdict — the point of the whole exercise:
| Market | direct fit (D-052) | thinned (this entry) | vs baseline |
|---|---|---|---|
| Shots on target | 0.187 FAIL | 0.034 PASS | +48.2% |
| Cards (0.5 line) | 0.177 FAIL | 0.022 PASS | +60.9% |
Both activated at v20260811-1905 on 58,830 held-out rows. All seven §5.1
markets now pass §5.6. Downstream, everything D-052 benched returns the
way it left — through the artefacts: builder legs restored (only tackles
remains excluded, D-046), caveats reduced to the D-048 any-card display
note, publication open with calibration margins of 3.4pp (SOT) and 2.2pp
(cards) added to the edge bar. First post-change cycle: 6,040 candidates
evaluated, calibration_gate_failed gone from the reject reasons (was
2,231), zero new publications — the remaining gates doing their jobs.
Referee card-per-foul (§5.2) deliberately deferred: the failure mode was overconfidence, and a spread-widening covariate moves top-band probabilities the wrong way. Revisit only if the gate ever shows thinned cards UNDERconfident.
Also recorded, per §13: the direct-market refit shifted saves 23.2% mean / 32.4% max against the 2026-08-10 golden while fouls, fouls drawn, shots and tackles reproduced within 0.28%. Mechanism: D-058's reference refresh corrected player position groups the same day, and saves — the market that "shrinks harder than any other" toward its GK stratum prior — is maximally exposed to prior movement. The four stable markets are the control group that says the pipeline, not the code, moved. Golden regenerated (1,293 rows; SOT freezes 171 after dropping 16 keeper-underflow rows, which the projection job also refuses); thinned markets freeze parent design + per-row q and replay to float precision.
Files: fit/thinned.py (new), features/panel.py (add_thinning_features), fit/persistence.py (kind-aware load), project/job.py (thinned branch), scripts/run_projections.py, scripts/golden.py + test_golden (thinned replay), simulate/job.py exclusions, apps/web copy, 12 new tests.
D-060 — §7.5 best-odds shopping, and the slip learns to span fixtures
Date: 2026-08-11 · Phase: post-decision · Spec: §3.1, §7.3, §7.5, §8.7
§7.5 verbatim: best available price per leg across all bookmakers, best single-bookmaker total against best split-across-books total, difference in pounds on a £10 stake — multi-match only. The pricing side already existed (§7.3's independence across fixtures); what shipped here is the shopping and the slip surviving fixture switches (localStorage; legs carry their fixture and pricing was always multi-fixture).
Two shape rules, both §7.5's own logic rather than caution:
- One leg per match to shop. Two legs in one match make that part a same-game builder, and builder prices are proprietary — absent from every feed. The refusal message says exactly that and points at the margin audit, which is the honest tool for same-game.
- A book missing one leg cannot take the acca. The single-book total considers only books quoting EVERY leg; substituting another book's price for the gap would invent a product nobody sells. When no book covers all legs, the panel says the acca only exists split. When a leg has no quote at all, totals are withheld and the count of quoted legs is shown — a £10 illustration over a phantom price is §0's forbidden number wearing a pound sign.
Quotes are the site's own display rules: latest per selection per book, tombstones excluded (D-055). §3.1 holds — the server multiplies the user's chosen legs over stored prices; nothing is modelled.
Verified live on captured odds: a cross-fixture SOT double (Alavés v Getafe × Sevilla v Rayo) priced fair 30.87, shopped 5.50 + 2.17 across six books' quotes each, split 11.93, best single book 11.93 (same book won both legs — the degenerate case rendering honestly at £0.00 difference). Files: lib/odds-shopping.ts (new), builder actions + client, no model/ changes.
D-061 — The §8.4 predictions page, and the goals model under it
Date: 2026-08-11 · Phase: post-decision · Spec: §5.4, §5.6, §8.4, §13
§8.4 needs 1X2, BTTS, over 2.5 and a correct-score grid — four readings of one object, the joint goal distribution. Dixon-Coles per league: attack and defence per team (sum-to-zero), home advantage, the low-score τ correction, weighted MLE with exponential time decay whose half-life is FITTED by grid CV (365 days won, mid-grid). Leagues fit separately — they share no fixtures, and pooling would let one league's scoring level distort another's.
Gated like everything else, and the pass is honest about its size: on 1,059 held-out matches (latest season cohort per league), 1X2 log loss 1.0279 against the 1.0832 naive baseline (+5.1% — match outcomes are hard, and this is the honest size of a goals model's edge), calibration 0.096 against the 0.100 tolerance — a pass with no room to swagger. Activation refused on any fail; the page renders nothing without an active row.
Promoted teams enter at the 25th percentile of their league's fitted
strengths, not the mean — promoted sides underperform the league they join,
and a mean-strength entry would flatter every prediction touching them.
Their fixtures carry a provisional flag the page displays and explains.
§8.4's hard rule ("the AI writes prose, never probabilities") is satisfied by construction: the analysis paragraphs are deterministic templates interpolating the stored payload's numbers and nothing else. An LLM rewrite of the same payload is the recorded upgrade path — it needs an API-key decision from the operator and inherits the same no-invented-numerics validation, and until then the template ships with zero new dependencies.
Tables: match_models (gate artefact + activation discipline), match_projections (upsert per fixture — a working surface, not a record; the stored grid truncates at 0–6 a side and 1X2/BTTS/over are stored from the FULL grid, never re-derived from the truncation). Worker projects every 6h; scripts/fit_match_model.py is the only activation path. Migration 0011, 11 new tests on synthetic leagues with known strengths.
D-063 — Relegated sides keep their identity: cross-league strength translation
Date: 2026-08-11 · Phase: post-decision · Spec: §8.4
The operator's audit of the predictions page found Wolves at 37% at home to Blackburn. The cause: the goals model fits per league, so a side relegated into a covered league is absent from its new league's fit and fell to the new-team prior — the 25th percentile — throwing away two seasons of its Premier League strengths sitting in the same model.
Backtested before built, on the six real movers between the two panel seasons, strictly out-of-sample (base fits on 2024/25 only, first-10 new-league matches scored, offsets fitted leave-one-team-out):
| strategy | relegated (3 teams) | promoted (3 teams) |
|---|---|---|
| 25th-percentile prior | 1.200 | 1.003 |
| direct transfer | 1.472 | 1.356 |
| translated (gap ≈ 0.45) | 1.159 | 1.269 |
The asymmetry IS the finding: promotion compresses teams to the new
league's bottom quartile — the percentile prior is near-optimal there and
is KEPT — while relegated sides carry real strength down. So the fix is
one-directional: a team absent from league B's fit but fitted in a HIGHER
covered tier enters at its old strengths plus RELEGATION_GAP (0.45, the
LOO optimum cluster), attack and defence both. Genuinely-new and promoted
teams keep the percentile prior. Every affected fixture stays provisional,
now with a distinct used_translation flag.
Effect on the audited case: Wolves v Blackburn 37.2/31.8/31.0 →
55.7/28.4/15.8 (λ 1.44/0.64). Exactly two teams qualify today (Wolves,
West Ham). scripts/backtest_movers.py is the committed deploy
regression: relegated-translated must keep beating percentile and
promoted-percentile must keep beating translated, or the rule has lost its
evidence and gets revisited, not patched. The match gate re-passed
untouched (1.0279 vs 1.0832, calibration 0.093) — translation is
predict-time only.
D-062 — Rate-band calibration: the metric, what it found, and what it benched
Date: 2026-08-11 · Phase: post-decision · Spec: §0, §5.4, §5.6, §13
The operator's audit caught a 0.53-shots-per-90 defender priced at a 27% edge. The trace found the entire gap between intuition and model was empirical-Bayes shrinkage: at k=130 the stratum prior carried 74% of the weight, and the k-search had CHOSEN that k because its calibration metric — worst decile miss — cannot see rate-conditional bias: a decile mixes both ends of the rate spectrum and their opposite biases cancel inside the bucket. On the holdout, k chosen decile-only drifts to the grid top with a 0.018 decile score while running +4.2pp hot on low-rate players and −5.7pp cold on high-rate ones.
The change: calibration measurement now includes career-rate BANDS (RATE_BANDS, seven slices; MIN_BAND_SIZE 200). The k-search optimises max(decile, band) under the same lexicographic rule, and the §5.6 gate fails any band whose mean miss exceeds BAND_TOLERANCE (0.03 — a subgroup mean off by 3pp manufactures edges the size of §6.3's whole base tier). fit-k also now searches at each market's OWN gate line rather than a flat 1.5.
What the re-search found: fouls and tackles re-chose their old k (their k was never the problem), fouls_drawn moved 90→130, shots 130→65, saves 130→65 — the two spread-starved markets widened exactly as the experiment predicted. Gibson's shots number under k=65: P(over 1.5) 15.9% → 14.4%, edge +27.1% → +15.0% (band residual +1.3pp), untargeted.
What the re-gate found, and why the k=65 activations were ROLLED BACK the same evening: the production k=65 fits hit the MIN_ALPHA floor — MLE dispersion collapsed to 5e-57 while held-out residuals stay overdispersed (ratio 1.145) — and the thin tails failed this entry's own deploy test at other lines (line-2.5 worst decile 0.149). A residual-moment alpha (0.1197) heals overdispersion to ratio 1.000 and improves log loss but still leaves 0.113 at line 2.5. Not shippable; shots and saves were reactivated at v20260811-1858 and thresholds.yaml's k values reverted to match the models actually serving. (Tackles' ACTIVE model has carried a floored alpha since it was fitted — the pathology predates tonight and is now on the record.)
The standard also failed five of seven markets at their best available k — fouls +6.9pp, fouls_drawn +6.2pp, tackles +8.1pp on thousands-row bands (real, k-insensitive: the prior's location and coarseness, not its weight), cards −5.4pp, SOT +3.4pp on a thin 2.5+ band where a significance guard legitimately applies. Active models predate the band standard and keep serving on their D-059-era passes; their re-verdict lands with the COMPLETION of this work, as one coherent re-gating:
- alpha fallback: residual-moment estimate when MLE collapses below the floor, arbitrated by the §5.6 overdispersion check;
- a BREACH_Z-style significance guard on band breaches, consistent with the decile gate's own discipline (needs per-band n stored);
- stratum priors rebuilt from detailed_position_id — present on 393/393 stored squad payload entries, never parsed — so a holding midfielder stops inheriting an attacking midfielder's prior; then a full refit and re-gate of all seven markets under the complete standard. Until then the band gate governs every NEW fit: nothing can activate past it, which is the ratchet doing its job.
D-064 — The D-062 completion: arbitrated alpha, band z-guard, role priors — and the re-verdict
Date: 2026-08-12 · Phase: post-decision · Spec: §5.4, §5.5, §5.6, §13
The three pieces D-062 specified, built and run as one coherent re-gating.
1. Arbitrated dispersion. When the MLE alpha collapses below MIN_ALPHA, the §5.6 overdispersion check on the fit's own residuals now arbitrates: overdispersed residuals reject the collapse and take a residual-moment estimate (alpha_m = mean((y−mu)²−mu)/mean(mu²), which reproduces the observed residual variance by construction); equidispersed residuals confirm the floor. Both branches fired in production the first night: shots folds healed to alpha≈0.10 (residuals 13% overdispersed — the exact pathology that forced D-062's rollback), cards folds kept the floor because their residuals genuinely are not overdispersed.
2. Band z-guard. rate_band_misses now carries each band's n, predicted, observed and standard error; a band fails the gate only when its mean miss is over BAND_TOLERANCE AND at least BREACH_Z standard errors from zero — the same two-condition discipline the decile gate got in D-052, pointing the other way: there a z-only rule passed an 18.7pp miss, here a tolerance-only rule would have failed a thin band on noise. The k-search still steers on MEASURED bias (max_band_error ignores significance); forgiveness belongs to the gate alone.
3. Role-level priors. players.detailed_position_id (migration 0012),
parsed from the squad feed going forward and backfilled from stored
raw_payloads: 5,064 of 5,254 players (96.4%) — the "393/393" in D-062 was
one fixture's squads; the store covers nearly everyone. Stratum keys gain
a finest level nested inside the group (39:DEF:d148), used by rate
priors, thinned q-priors (with the MIN_STRATUM_ROWS floor), projection
and the golden set; unknown roles and pre-D-064 artefacts resolve at the
group level unchanged. Every detailed id nests inside exactly one coarse
group in our data. Sportmonks' core/types endpoint is not in our plan, so
the ids stay ids — the folk id-to-name mapping circulating online is
provably wrong against our own group cross-tab, and a wrong name is worse
than none. Incoming NULLs COALESCE against stored values in the upsert
(keep_existing_when_null — D-038's rule, per column).
The k-search re-drew the map: fouls 130 (unchanged), fouls_drawn 90→130, tackles 200→130, shots 130→200, saves 130→65. Shots wanting MORE prior weight is the detailed priors working: a better-located prior is worth trusting harder. (SOT/cards k entries exist in thresholds but the thinned path reads only the parent's k.)
The re-verdict, all seven markets at their own lines:
| market | bands (worst) | verdict |
|---|---|---|
| saves | 0.024, hold | PASS — activated (alpha 0.0797) |
| SOT (thinned) | 0.017, hold | PASS — not activated, parent failed |
| cards (thinned) | 0.022, hold | FAIL — 13.7pp decile breach at 0.5 |
| shots | 0.034, z-significant | FAIL |
| fouls | 0.060 (was 0.069) | FAIL |
| fouls_drawn | 0.054 (was 0.062) | FAIL |
| tackles | 0.049 (was 0.080) | FAIL |
Role priors improved every failing band — tackles nearly halved — but the fouls-family bias is structural and survives both the finer priors and the z-guard on thousands-row bands. That is the honest state: the D-062 standard is now fully armed and five markets cannot pass it.
New rule, learned the same evening: a thinned child may only ACTIVATE when its parent activated in the same run. Tonight SOT passed its gate riding the k=200 shots parent, which itself failed — activating the child would have served it on the OLD parent, a pairing its pass never measured. run_projections now guards this; tonight's SOT activation was reverted to v20260811-1905 (the pair of the active v20260811-1858 shots) and projections re-ran on the coherent set (run 179).
Actives after tonight: saves v20260812-1340 (the one clean pass under the complete standard — role priors + arbitrated alpha, k=65); all others on their D-059-era passes, exactly as D-062's ratchet prescribed. Golden set regenerated and verified zero-shift. The five failures are a finding, not a to-do list: the next structural candidate is the prior's LOCATION (the §5.2 covariates the fouls family lacks — e.g. opponent style and possession context), and the first prop prices land Friday 2026-08-14, which gives every one of these numbers a market to answer to.
D-065 — The model was right and the page was wrong: the presentation tier
Date: 2026-08-12 · Phase: post-decision · Spec: §1.5, §8.2, §8.4, §8.7, §9.4
The operator's product audit said the predictions "are just 1–1" and the builder's fair odds "always blow way higher" than a per-90 intuition. Both complaints were traced before anything changed, and both traces cleared the models:
The 1X2 diagnostic (scripts/diagnose_match_vs_market.py, read-only): across 22 fixtures holding both a projection and a de-margined bookmaker 1X2 consensus, the established-teams slice tracks the market at correlation 0.850 with per-outcome bias under 1.5pp and a flatness slope of 1.197 — the model is SHARPER than the market, not flatter. Provisional fixtures run flatter (slope 0.841) by design. The Dixon-Coles model is untouched; the one watch item on record is Burnley v West Ham, where the market rates a strong relegated side far above the D-063 translation.
The builder trace: Mané over 1.5 shots at fair 3.77 reproduces exactly as 28 appearances shrunk at k=130 (18% own-record weight) × a 68% start rate giving 64 expected minutes. Not a bug — but invisible, and a fair price whose inputs cannot be seen reads as a broken one.
What shipped, all presentation:
- Prediction cards lead with EXPECTED GOALS (the lambdas differ per fixture; the modal score reads 1–1 for most even fixtures at ~11% and communicated nothing); top-3 scorelines carry their probabilities.
- Builder player blocks show ~minutes and start% inline, plus a "why these numbers" disclosure: minutes mixture, career per-90 over n apps versus the model's per-90 for this match.
- The trends explorer's hover card (opponent, venue, date, minutes, started/sub) is extracted to a shared MatchChip and now also covers the fixtures-page runs rail, whose strips were bare digits; getHotTrends carries per-match context.
- Methodology gains "The match predictions are a separate goals model".
- Runs-rail hairline grid adapts its column count and fills partial rows — a one-fixture day exposed the bg-line container as a blank rectangle.
- The homepage §8.2 panel is live data, not hardcoded copy: prop prices BEGAN FLOWING 2026-08-12 (18 fixtures, 743 players, shots/SOT/cards; no fouls quoted yet) and the board holds 5 published edges, so "the feed carries no props" had gone quietly false — the §1.5 panel now reads the board and says which state it is in.
Also decided (the operator): the five markets that fail the completed D-062 band standard KEEP PUBLISHING on their D-059-era passes for now; the band gate governs new fits only. First prop settlements land Friday 2026-08-14 and the CLV record starts arbitrating.
D-066 — projections.team_id was never written, and three symptoms shared the cause
Date: 2026-08-12 · Phase: post-decision · Spec: §7.3, §8.7
The operator clicked the builder's team filter and got an empty list. The trace found
"team_id": None written literally on every projection row since the
column existed — the projecting loop knows exactly which side it is
iterating, and the placeholder never got replaced. Three consequences, one
line:
- The simulation derives
is_homefromprojections.team_id; with NULL it compared unequal to the home team id and flagged ALL 294 selections in the Wolves fixture (and every fixture) as away — so the builder's home-team filter had no members, ever. - §7.3's possession correlation takes its SIGN from is_home: home shots up means away shots down. With both squads flagged the same side, all players' shots moved together — same-game slip prices spanning both teams were built on a correlation structure with the anti-correlation erased. Repriced after the fix: 121 fixtures re-simulated at 10,000 iterations on run 182.
players.current_team_idcould NOT have been used instead — it is legitimately NULL for unknown registrations (D-038/D-058, 18,977 kept rows this run). The fix writes the fixture loop's own side, which is always known.
While in the picker: player photos (Badge, with the initials fallback) and position groups, which also disambiguates the two goalkeepers a saves market always lists.
Also on record from tonight: bet365's website shows fouls/tackles/fouls-won for the Wolves match but the ukodds feed does not carry those markets for the event yet (84-market census, zero fouls-shaped names, no mapping gap — the adapter maps all seven §5.1 markets and captures everything the feed sends: shots, SOT, cards from ~10 books). Books stage prop releases; fouls arriving closer to kickoff is the expected shape. If the feed still lacks them Friday morning while the site shows them, that is a coverage question for UK Odds support, not for our code.
D-067 — [withheld]
Withheld from the public render — internal legal posture. The internal log is append-only and this entry exists there unedited; the number is preserved here so the sequence stays checkable.
D-068 — [withheld]
Withheld from the public render — infrastructure detail — auth and billing internals. The internal log is append-only and this entry exists there unedited; the number is preserved here so the sequence stays checkable.
D-069 — The methodology page audit: three bugs, and the credibility items
Date: 2026-08-12 · Phase: pre-launch · Spec: §9.4, §17
An external-style review of /methodology found three outright bugs and a set of credibility gaps. All fixed:
Bugs. (1) The saves improvement was hardcoded in prose (+9.1%) while the table read live (+9.5% after the 13:40 refit) — on a page whose claim is that its figures cannot disagree, one hardcoded number is the whole bug class; the prose now reads the same stored artefact as the table. (2) The header claimed one "model version" while the table showed three — relabelled "Latest refit", with per-market versions pointed at explicitly. (3) The referee-cap copy implied a cap that could never bind (7.5% per referee over 5% per match, one referee per match); the copy now states what the engine actually does — the referee group spans every fixture in the multi-day publication window, where one official can hold two matches and 2×5% would otherwise stack 10% on one judgement.
Credibility. The SOT/cards re-pass bullet now carries its complete defence AND its honest caveat: the conditional decomposition was committed to version control on 9 August, the day before the gate failure (10 August) — design predates failure, checkable in git — while the re-pass (11 August) was scored on the same held-out season, because two seasons offer no second one; the closing-line record is named as the true out-of-sample test. The decile sample floor (n ≥ 30) is stated. Internal references (§5.6, D-052) are replaced with dated plain language. Units are consistent (0.187, not "18.7 points"). A four-line summary opens the page. The fit table sorts by market, not by improvement — the improvement sort invited exactly the ranking-reading its own caveats deny. A "report it" mailto appears when NEXT_PUBLIC_CONTACT_EMAIL is set and hides otherwise.
The second baseline. The evaluator now measures a SHRUNK-RATE-ONLY baseline (rate_shrunk × minutes/90, no covariates) alongside the naive one, stored on the §5.6 artefact as shrunk_baseline_log_loss. The gap between the two baselines is what shrinkage buys; the gap between shrunk and model is what the opponent/referee/venue terms add — the first question a technical reader asks of "+34%". Current actives predate the field, so the column shows a dash with the reason; numbers appear per-market at each next refit. This page does not backfill measurements it did not make.
Reviewer's open item, deliberately NOT acted on: "it still says Propboard" — whether a rename is intended is the operator's call, not a copy edit.
D-070 — Settlement existed for the closing line but not the result; and the record page becomes a record
Date: 2026-08-12 · Phase: pre-launch · Spec: §4.2, §8.8, §13, §17
The operator asked how quickly a published call settles after full-time. The trace
found the honest answer was NEVER: capture_closing fills closing_price
and clv_pct at kickoff, sync_results ingests the final stats hourly, and
published_edges.result/settled_at had no writer at all — §8.8's
"scored against the closing line and the result" was half built and the
gap would have surfaced publicly on Friday night.
settle_edges (model/ingest/jobs/settle_edges.py) closes it: hourly,
scheduled directly after sync_results so a fixture's final stats and its
settlements land in the same worker pass — a call grades within the hour
of the feed finalising the match. Grading is a pure function (§13-tested):
no appearance or zero minutes VOIDS (books void player props on a
no-show); any minutes at all is action, a 3-minute cameo settles as a
loss, not a void; over-lines settle strictly over/under with integer-line
pushes defined before the first one is ever published; cards settle on
any card (D-048). A played-but-stat-NULL row is left unsettled AND
counted in the job detail — a feed gap must be visible, never guessed
into a grade. Results are written once (WHERE result IS NULL), matching
the append-only claim the page makes.
The page now shows the record instead of describing one: a summary Record tile (W–L–V), the FULL call log — every published call in every state (open → settling → won/lost/push/void) with price, fair, edge, close and CLV per row, nothing filtered, newest first — and a cumulative average CLV chart (one accent line for the running average over faint per-call marks, zero baseline, hover per call; single series so no legend), which states plainly that it draws itself once two calls have closing lines rather than plotting placeholders. The long-promised CSV export exists at /track-record/export, reading the exact query the page renders so the file and the page cannot disagree.
Timing, for the record: closing line + CLV at kickoff (capture_closing, 10-minute cadence); result within ~an hour of the feed finalising final stats (sync_results hourly, settle_edges immediately after). The first real settlements land with Friday's fixtures.
D-071 — The product is Per90; the codebase stays propboard
Date: 2026-08-12 · Phase: pre-launch · Spec: §9
The operator retired the Propboard name and chose Per90 (per90.co.uk, the only domain taken — .uk and .io were also free and were not bought).
Why this name, on the evidence gathered: every ordinary English word in the space is registered, most since the late 1990s — trueline, quantile, plumbline, foulplay, overround, tallyman (Experian), halfspace, deadball all taken; oddsmith and propsmith were registered THIS YEAR, so the obvious names are actively being picked over. Per90 survives because of the digit, and it earns the name on merit rather than availability: "per 90" is the literal unit every figure on the site is quoted in, so the audience decodes it instantly; it contains no gambling word, which keeps the brand consistent with D-067's compliance posture and keeps ad accounts and processors simple; and P90 is an obvious mark.
Known and accepted: "per 90" is a generic industry metric, so it is a weak WORD trademark — protection, if ever wanted, is a figurative mark on the logo, not the phrase. 90min (Minute Media, ~50M fans) is a large adjacent football-media brand; Per90 is defensibly distinct in construction and sector, but The90 — also free — was ruled out for reading like one of their sub-brands.
Scope of the rename, deliberately split. Everything a user can see is Per90: page copy, breadcrumbs, titles and the title template, the nav wordmark, the footer, Organization and WebSite JSON-LD, the account and compliance copy, the CSV filename, and the localStorage keys (renamed outright rather than migrated — there are no users yet to lose a saved slip). Verified live: zero occurrences of the old name in the rendered HTML of the homepage or the methodology page.
Internal identifiers KEEP the old name on purpose: the propboard CLI
binary (it is a venv entry point and the live worker's restart command),
the @propboard/db workspace, the PROPBOARD_* env prefix, the
propboard.* logger namespaces, the fly.toml app, and this directory.
Renaming them is invisible to users, risks the running worker two days
before the first settlements, and would rewrite the restart runbook for
no gain. README records the split so the next reader is not confused by
it. Past DECISIONS entries are untouched — this log is append-only, and
history says Propboard because that is what it was called.
Still to do at deploy: NEXT_PUBLIC_SITE_URL=https://per90.co.uk.
D-072 — The three claims the product made and had not built
Date: 2026-08-12 · Phase: pre-launch · Spec: §2, §8.3, §8.8
The operator's challenge after D-070 — "you told me everything was built" — was correct, and the honest response was to audit CLAIMS rather than the spec checklist. A spec item can be ticked while the page built from it promises something extra. Three such gaps existed; all are now closed.
1. ROI to advised stakes. /track-record has said since it was built that "results to advised stakes are shown once calls settle". Nothing computed them. It is now measured over the stored quarter-Kelly fractions (post-D-056 caps), with voids and pushes excluded from BOTH profit and turnover — a returned stake is neither risk nor result — and it renders beside the W–L–V line with its sample size and, under 100 settled calls, the plain statement that it is too few to mean anything and the CLV line is the number to read. Null until the first settlement.
2. The user bet log (§2's "auth, subscriptions, user bet log"). The
user_bets table has existed since the schema was written with no UI.
/bets is now a private per-account ledger: staked, profit and ROI tiles, a
manual add form for bets placed anywhere, and per-row removal. The design
decision worth recording is settlement provenance: a row logged from the
board carries published_edge_id and READS its result from that edge, so
the reader's ledger grades itself the moment the settlement job grades the
call; a hand-typed row has no edge to inherit from and its owner sets the
result. One column, two provenances, and the UI labels which applied
("·auto"). Manual result-setting is refused on linked rows — two answers
in one column is how records start disagreeing.
3. §8.3's "log this bet". Value board rows now carry a Log button for signed-in readers, sending whatever stake the calculator is showing — their number, not the advised one, and null when no bankroll is entered, which logs the bet unsized rather than refusing. The description is built SERVER-SIDE from the edge, never accepted from the client: it is the label the row carries forever, and a caller could otherwise log "Arsenal to win" against a fouls call. One log entry per user per call.
Note on asymmetry, deliberate: published_edges is append-only because it
is a public claim (§17); user_bets rows can be deleted by their owner
because a private ledger nobody else reads has no such duty. Different
records, different rules.
Verified by exercising every statement against the live schema with a throwaway user (insert-from-edge, manual insert, result set, the join, the ROI aggregate, cascade delete) rather than trusting that it compiled.
D-073 — The build must not need the production database
Date: 2026-08-12 · Phase: deploy · Spec: §10
First Vercel deploy failed. The log showed a clean install, a clean
compile, a clean TypeScript pass, then Generating static pages (7/28)
and nothing — no error, no stack, just a dead build.
Seven is exactly the number of prerendered pages that query Supabase.
next build was opening connections from Vercel's build container in
Virginia to a database in Frankfurt, on a single worker, and dying
silently. Ruled out first, by measurement rather than guesswork: the
transaction pooler (a local build against port 6543 succeeded),
connection exhaustion (24 of 60 in use), statement timeouts (2 min), and
a missing DATABASE_URL (page-data collection had already imported the
client, which throws when the variable is absent).
Rather than keep chasing a cause that produced no error text, the
dependency itself was removed. Six pages — the homepage, methodology,
predictions, track record, referee trends and the sitemap — changed from
revalidate to dynamic = "force-dynamic", so they render per request
instead of at build. Static pages fell 28 → 22, and the three that remain
(/glossary, /robots.txt, _not-found) touch nothing.
Nothing was lost. Every one of those pages was already on a 5-minute to 1-hour revalidate timer, so build-time data was stale almost immediately; odds and projections change through the day and serving them fresh is the honest default. The cost is a database read per request on those routes, which at launch traffic is nothing.
Verified rather than asserted: next build now succeeds with
DATABASE_URL pointed at an unreachable host. A build that cannot be
broken by the database cannot fail this way again — including at 3am when
Supabase is briefly unreachable and a deploy would otherwise fail for
reasons nobody would connect to the database.
D-073 addendum — the second failure, on the same deploy. With the build
fixed, the deployment reported Ready and then returned a server-side
exception on 100% of requests. Cause: next.config.ts listed
@propboard/db in serverExternalPackages, which means "do not bundle
this; require() it natively at runtime". That package ships raw
TypeScript (main: ./src/index.ts), so every request asked Node to
execute a .ts file. It never surfaced locally — next start resolves
the workspace symlink through Next's own loader — and only appears once
the app is traced into serverless functions.
postgres stays external (a genuine Node driver, which is what the option
is for). @propboard/db was removed from the list: App Router bundles
server dependencies automatically and Turbopack transpiles workspace
packages, so removal IS the fix. The comment that justified the original
setting was also wrong on its own terms — it claimed the option kept
credentials out of the browser bundle, which is the job of server-only
and of Server Components never shipping to the client.
Lesson worth keeping: next build passing proves the code compiles, not
that it runs. Between the two failures on this one deployment, one killed
the build and one killed every request, and neither reproduced on a local
production server.
D-074 — The worker moves to Fly, and what the container was missing
Date: 2026-08-12 · Phase: deploy · Spec: §4.2, §10
The web app is on Vercel (D-073); the worker cannot be, because it is a
long-lived process with its own clock and Vercel runs functions that start,
answer and stop. It now runs on Fly in lhr — London, closest to the UK
books being polled — as per90-worker, one machine.
Three things were wrong with the container, all invisible until it ran:
The scientific stack was never installed.
pyprojectkeeps numpy/pandas/scipy/statsmodels in an optional[model]extra so Phase 0 ingest installs in seconds in CI, and the Dockerfile installed the base package only. That was correct when the worker just ingested — but the schedule now runs project_publish, simulate_fixtures and match_predictions, all of which import it. The container built cleanly, started, and crash-looped onModuleNotFoundError: No module named 'numpy'until it exhausted its restart budget. Dockerfile now installs-e ".[model]".PROPBOARD_ODDS_PROVIDER was missing from fly.toml.
build_scheduleonly registers the prop sweeps when an odds provider is configured, so the worker would have run, logged healthily, and captured NO PROP ODDS AT ALL — §10's silent-capture failure, the worst class in the system, and it would have looked fine in every log line. Added, with UK_ODDS_API_BASE, in the[env]block where it appears in a diff rather than hidden in a dashboard.256MB was sized for the old worker. It now loads pandas and statsmodels and runs the projection cycle and Monte Carlo in-process, so the VM is 512MB. An OOM kill presents as a crash-loop with no error, which is the same symptom as (1) and would have been diagnosed twice.
Also: Fly's default created a standby machine alongside the primary. A
standby is passive, but this app's contract is EXACTLY ONE worker — two
would double every sweep and write each price change twice — so the
standby was destroyed and the deploy script now passes --ha=false.
scripts/deploy_worker.ps1 does the whole thing from .env without
printing a secret, and is ASCII-only because Windows PowerShell reads
.ps1 as ANSI and choked on an em-dash.
The handover: with both workers briefly alive, the D-018 lease did
exactly its job — the Fly worker refused every job with
ConcurrentJobRunning rather than double-polling. The local worker was
then stopped and its lease released. Fly is now the only worker, and the
site no longer depends on a desktop in a warm room in August.
D-075 — The rejection log took down publication, silently
Date: 2026-08-12 · Phase: deploy · Spec: §6.4, §10
Found while verifying the Fly worker, and it had nothing to do with Fly:
project_publish had been failing every 20 minutes since 18:20, and
intermittently since 8 August. Thirty-three failures on record.
psycopg.OperationalError: sending query and params failed:
number of parameters must be between 0 and 65535
... _insert_rejections -> _insert_many
Postgres caps bind parameters PER STATEMENT at 65535. _insert_many built
one INSERT for every row it was given, so at ten columns it broke somewhere
past 6,500 rows. §6.4 logs every candidate that fails a gate, and the
moment real prop prices started arriving a cycle produced ~16,000
rejections — the run just verified inserts 16,339, or 163,000 parameters.
The severity is in what failed, not what errored. The exception
propagated out of the rejection insert and failed the whole cycle, so
projections were written and then NOTHING WAS PUBLISHED. The product's
visible state was "yesterday's edges, still there" — no error page, no
empty state, nothing a reader or an owner would notice. Only
ingest_runs.status knew. Diagnostics must never be able to take down the
thing they are diagnosing.
Both insert paths now batch under the ceiling: publish/job._insert_many
and, more importantly, the shared core.db.upsert — which had the same
latent flaw and writes projections, odds snapshots and every other bulk
table. Batching lives there rather than at the call sites because every
caller would otherwise have to remember the arithmetic, and the one that
forgot cost a day of publication. Regression tests drive 20,000 rows
through both and assert no statement exceeds the limit.
First cycle after the fix: 35,163 projections, 16,345 evaluated, 6 published (the sixth edge had been blocked by this since it qualified).
D-076 — The GitHub Actions fallback is retired, not just quiet
Date: 2026-08-12 · Phase: deploy · Spec: §4.2
The operator was being mailed a failure notice for every scheduled Actions run. Both ingest workflows were failing in under ten seconds because this repository has no Actions secrets — but the emails were the small problem.
The large one: if those workflows had ever succeeded, they would have been a SECOND ingest path running beside the Fly worker. Two paths polling the same feeds on overlapping schedules is precisely what the one-machine rule in fly.toml exists to prevent — double the API spend, and each price change written twice into the history §1.1 calls the moat. The workflows predate the Fly worker (D-007 built them as the belt-and-braces fallback when the worker had nowhere to live); once D-074 gave it a home they became a liability that happened to be broken.
Schedules are commented out with the reason inline; workflow_dispatch is
kept so a job can still be run by hand in an emergency, which needs
repository secrets that are deliberately absent. ci.yml is untouched — it
runs the test suite on push and needs no credentials.
D-077 — [withheld]
Withheld from the public render — infrastructure detail — capacity and cost. The internal log is append-only and this entry exists there unedited; the number is preserved here so the sequence stays checkable.
D-078 — External audit accepted; Phase 1 corrected every cross-page contradiction
Date: 2026-08-12. Trigger: a full external audit (docs/audits/2026-08-12-external-audit.md) whose thesis the operator accepted: on a product whose only asset is internal consistency, two pages stating opposite things about the same fixture is an attack on the thesis, not a cosmetic bug.
Phase 0 committed a pre-change baseline (docs/audits/2026-08-12-baseline.md) so every later change can be proven against it, and settled three of the audit's diagnoses with evidence:
- Fouls won 0.116 marked pass (audit 1.2) is the gate working as designed, not a gate bug. The 0.116 is a 44-row decile at z=−1.73; the gate requires over-tolerance AND ≥2σ (D-052/D-064). The methodology page's "the pass bar is 0.100" is the inaccurate half of the pair — it states the tolerance and omits the significance condition. Page copy is Phase 2's problem, with the operator deciding the wording.
- No edge leaked below its bar (audit 1.3). The apparent breach was version skew: the +16.87% Gibson call was gated against shots v20260811-2053 (miss 0.0601, bar 16.01%), which was later rolled back to v20260811-1858 (0.0699) — the version the methodology page now displays. Fix is audit 2.3's: store the gate arithmetic per published row.
- The calibration margin really is added in the wrong units (audit 1.1), confirmed at model/publish/engine.py required_edge(): probability points added one-for-one onto an EV-space threshold. Redesign is Phase 2.1 and STOPS before deploy — it will retire published calls, and how that is disclosed is the operator's decision, not a code decision.
Phase 1 shipped (one commit per finding, audit §-referenced): footer no longer claims the live predictions page is unbuilt, with the accounts claim now driven by authConfigured(); the match page and trends fixture panel read published_edges through getFixtureEdges() instead of asserting staleness — all four surfaces now give one edge count per fixture; the builder header states timeZone (the deployed UTC server had been rendering evening kickoffs an hour early), and an audit of every kickoff COMPARISON in the pipeline found all of them aware-UTC or DB-side — no published call was gated or timestamped in a wrong zone; the match page's "Per 90" column is relabelled Model /90 (the observed-rate column waits for Phase 2.5, because the only career-rate source at the web layer is the per-match-averaged figure that phase exists to fix — found at lib/simulations.ts avg(stat*90/minutes), reported not extended); thousands separators, pluralisation, per-market model versions on match pages, all three 1X2 probabilities always printed; and the stake column sizes against a stated example £1,000 until the reader types a bankroll (never persisted, privacy stance unchanged).
Verified before closing: 24892 (3 edges), 24891 (2), 24895 (0) agree across homepage, match page, board and trends panel; three edges' arithmetic reproduces from stored fields; published_edges byte-identical to the baseline; production build green. No single-edge fixture existed to test, noted as a deviation.
D-079 — Published rows carry their own gate arithmetic
Date: 2026-08-12 (audit 2.3). published_edges.gate_detail (jsonb,
migration 0014) stores {mode, base_pp, referee_pp, calibration_miss,
calibration_margin_pp, total_pp} at publication, frozen by the immutability
trigger, shown as the edge cell's hover on /value and as gate_mode /
gate_bar_pp in the CSV export. Pre-D-079 rows stay null — append-only means
no back-filled arithmetic. Driven by the audit's 1.3: a reader reconstructed
a bar from the page's CURRENT figures and "caught" a sound call published
against a since-rolled-back version; provenance on the row makes that class
of false alarm impossible. Same commit corrects /value's publication rules:
the advertised +2pp referee penalty is never applied at the gate — the
referee covariate is made conservative at projection time
(referee_effect + min_referee_matches) — so the page now states the rule
that actually runs.
D-080 — Phase 2 of the external audit: what shipped, what stopped
Date: 2026-08-12. Shipped: the corrected calibration-margin form
(calibration_margin_mode: conservative_prob, margin = miss x price)
implemented and tested but DEFAULTED OFF — switching retires published
calls, which is the operator's record-integrity decision, with the full
before/after committed (docs/audits/2026-08-12-margin-before-after.json:
0 of 11 published calls survive; 8 are negative-EV at p−miss). Per-line /
per-position validation built and run
(scripts/validate_line_position.py → docs/validation/): the audit's
hot-on-defenders'-shots hypothesis is REFUTED on holdout (defender overs
run 1pp COLD); fouls carries real positional structure (FWD +2.5pp hot,
DEF −2.4pp cold, both |z|>7) — the face of D-064's "structural" band bias.
Builder slip prices now show a 95% Wilson simulation band instead of a
flat ±1% (audit 1.8). The goals model's stored gate artefact renders as a
live per-league table on /methodology (audit 1.9).
Established with evidence, no change yet (the operator decides): fouls_drawn's 0.116-vs-0.100 "contradiction" is the gate's significance condition (n=44, z=−1.73 < 2.0) — the fix is page wording; the per-90 career rate is computed as an average of per-match per-90s CONSISTENTLY across panel, projection SQL and builder display — Pedro Henrique 9.62 shown vs 1.61 pooled, panel-wide x1.5 inflation for sub-45-minute players — and the fix (pooled totals + 30-minute floor) must move panel and PLAYER_HISTORY_SQL together with a refit, held to run under whichever margin mode the operator picks; Trindade's 2.4x pull decomposes as k=130 (own rate 34% at n=68) against a coarse league:MID prior of 1.561 whose detailed stratum (d149, mean 0.93 averaged / 0.84 pooled) exists but did not resolve; cards discriminates but sits ~2pp cold at z≈13 across 58k rows (significant, inside tolerance) with only 905 rows predicted above 0.2 — refit proposal (cards-per-foul on referee and position jointly) awaits a go; Inter 3.49 xG is the only side above 2.8 across 45 fixtures and Monza is NOT tagged provisional despite promotion — the tagging rule misses promoted sides. Settlement void-on-no-show: bet365 and Sky Bet confirmed from primary pages; Paddy Power / William Hill assumed via Flutter templates, with two live edges at Paddy Power riding on the shared convention.
D-081 — The corrected margin is live, and 17 calls are withdrawn
Date: 2026-08-13. Decision: The operator, on the Phase 2 before/after.
calibration_margin_mode: conservative_prob is now the live rule. A call
must clear its bar even if the true probability is the model's figure minus
the market's whole measured worst calibration miss — algebraically, margin =
miss x price. The pre-audit form added a probability-space quantity to an
EV-space threshold; at a 15.00 price it charged 7 points of protection
against an error worth 105. additive_pp remains implemented and tested so
pre-D-081 decisions stay reproducible, and _validate_thresholds now
rejects any other value — a typo in the publication bar must not fall
through to a default.
Every call the old rule published that the new rule would not is withdrawn. Not deleted, not hidden, not quietly dropped:
published_edges.retired_at/retired_reason(migration 0015), write-once and one-way at the DATABASE — a CHECK constraint refuses a withdrawal without a reason, and the immutability trigger refuses to un-retire one or reword it. Both verified by attempting them.- Withdrawn calls leave the value board, because they are not advice.
- They stay everywhere else: struck through on match pages with the reason on hover, badged in the track-record log, two new CSV columns, and in every headline figure on the record.
That last point is the whole design. The moment a withdrawal removes a call from the statistics, "withdraw it" becomes the mechanism for deleting a loser and §8.8's "no filtering that could hide losing periods" is worth nothing. The tile reads "23 calls · 17 withdrawn, still counted", and a reader who wants the other view can compute it.
The reason string is stored PER CALL and carries that call's own arithmetic ("at 5.00 it needed +44.7% and had +24.5%"), which means seventeen withdrawals are seventeen distinct strings; the summary blocks show one in full as an example and send the reader to the rows. That was a read-side fix, because the write side is immutable by design — the first time that rule bit, and it bit correctly.
It was 11 calls when the analysis ran and 17 when it executed: the board
kept publishing under the old rule while the work was in flight. Retiring
exactly the original 11 by id would have left six newer calls standing that
fail the identical test, so scripts/retire_edges.py applies a RULE, never
a hand-picked list — "which calls" must be answerable by a predicate, or it
is a curation of the record. The script dry-runs by default.
D-082 — Tackles published for two days against a rule signed in August
Date: 2026-08-13. Found while retiring the calls above.
docs/settlement-sources.md, signed by the operator 2026-08-10, is unambiguous:
"No book/market pair for tackles may publish." Our column counts
ATTEMPTED tackles; every UK book settles on Opta, which counts only
challenges that successfully win the ball. Different quantities, so a
tackles line cannot be priced against our data at all.
That exclusion existed in prose, in the glossary, and in the builder's UI.
It never existed in the publication gate. The engine's only settlement
check was settlement_verified, which is a property of the BOOKMAKER — so
a book with a verified settlement source vouched for every market it
quoted. While no book quoted tackles this was invisible; bet365 started
quoting them and five tackles calls published.
A second defect made it undiagnosable from the code: CANDIDATES_SQL never
selected the market code, so EdgeCandidate.market_code was the empty
string on every candidate the job ever built. No market-level rule COULD
have worked.
Both fixed: settlement_excluded_markets lives in thresholds.yaml (§12 —
thresholds live in config), is enforced ahead of the per-book check because
a verified book cannot rescue a mismatched definition, and carries its own
reject reason — "this book is unchecked" and "our column measures a
different quantity" are different stories and §6.4 wants the right one.
Four tests, including one asserting the SHIPPED config still excludes
tackles, because the defect was a rule that existed only in prose.
The four already-published tackles calls that survive the D-081 margin rule
are NOT touched here. Withdrawing them is a record decision, not a code
one, and scripts/retire_edges.py --cause tackles-settlement is written,
dry-runnable and waiting on the operator.
D-082 addendum, same day: The operator chose to withdraw all four surviving
tackles calls (--cause tackles-settlement). The mismatch is directional,
not neutral, which is what settled it: attempted tackles are always >=
tackles won, so pricing an over against our column systematically
overstates P(over) — those four were not a coin flip on a definitional
quibble, they were overs the model was bound to overprice. 21 of 23
published calls are now withdrawn; the two live rows are fouls.
D-083 — Every per-90 rate is pooled, floored at 30 minutes, and the basis travels on the model
Date: 2026-08-13. Trigger: audit 1.5/3.1; the operator authorised the refits.
The defect: every rate in the pipeline was an expanding MEAN of per-match per-90s, so a one-minute cameo with one foul entered as 90-per-90 at full weight. Displayed career rates hit 9.62 fouls/90 against a pooled truth of 1.61; sub-45-minute players ran x1.5 hot panel-wide; and since stratum priors were means over the same per-row rates, every prior LOCATION was inflated too.
The fix, both layers at once: panel career/season/last-N features pooled (sum of events x 90 over sum of minutes across qualifying prior appearances, career_n counting exactly those); stratum priors minutes-weighted (algebraically the pooled rate); projection SQL and sample_n; the builder's career figures. min_rate_minutes: 30 in thresholds.yaml. Lockstep is structural: fitted_models.rate_basis (migration 0016) records each model's construction, stamped from the panel module itself, and the projection job selects matching SQL per active model — an unknown basis refuses to project.
The k re-search vindicated the whole theory. On pooled input the CV collapsed the prior weights an order of magnitude: fouls 130 to 8, fouls won 90 to 8, tackles 130 to 8, shots 200 to 12, saves 65 to 45 — four of five inside the 8-15 range §5.4 originally predicted. The old giant k values were compensating for cameo noise; clean the input and a 68-appearance player gets 85% of his own price instead of 34%. André Trindade's shots rate now projects at 0.44-0.55/90 against his real 0.43, not 1.08.
The re-gate: six of seven markets PASS the full D-062 band standard — the standard the fouls family had failed since D-064 with what that entry called "structural" bias. Fouls band error 0.060 to 0.012, fouls won 0.054 to 0.010, tackles 0.049 to 0.017, all holding at BREACH_Z. The structure was the denominator. Active set: fouls, fouls won, shots, saves, tackles, cards at v20260813-1451 (basis pooled30); every projection regenerated (run 214: population medians moved little, individual tails honestly wide — p10 about 0.6, p90 about 1.35 — fringe players deflating, evidenced players released toward their own records).
SOT is BENCHED. It failed §5.6 by 0.0002 of MAE (0.3468 vs the naive baseline's 0.3466) while beating it on log loss by 49% and calibrating clean — the pooled fix made the baseline itself stronger, and §5.6 demands both conditions. The gate is the gate. Its previous model could not stay active either: its pass was earned riding the averaged shots parent, which no longer exists — D-064's rule cuts both ways, and run_projections.py now deactivates a stale child automatically when its parent moves on without it (this run it was done by hand, plus removal of run 214's 4,879 SOT rows projected on the unmeasured pairing). SOT returns when a child passes riding the pooled parent; whether the MAE half of the baseline test is the right bar for a thinned market at this margin is a question for the operator, noted not decided.
Golden set regenerated at v20260813-1451 in the same commit (this entry is the §13 explanation): 6 markets frozen, SOT recorded in a new benched_markets field — absent loudly, never silently. thresholds.yaml comments do not survive fit-k --write (yaml.dump strips them); this file is the documentation of record.
D-084 — Referee cards-per-foul strictness, on D-059's own trigger
Date: 2026-08-13. The referee card-per-foul multiplier was deliberately omitted in D-059 with its trigger written down: add it only if the gate shows the thinned cards model UNDERconfident. Audit 2.7's measurement met it — cards ran about 2pp cold in its two biggest deciles at |z| around 13, predictions compressed into 0.11-0.19, the board's best calibration score earned by mostly quoting the base rate.
Implementation: per-referee cards-per-foul ratio over the training window, shrunk toward the league ratio by 300 foul events (about 13 matches), applied as a multiplier on q for REFEREE_THINNED_MARKETS (cards only — nobody has shown a referee SOT effect). Fitted from training seasons, applied to held-out rows through their referee ids, so the gate scores the served model and the q-strength search runs under the same scaling. Persisted on the artefact; unknown or unappointed officials multiply by 1.0. Cards passed at v20260813-1451 (+61.2% vs naive, bands hold) and activated with its pooled fouls parent. The golden set freezes q x strictness per row, so §13 now guards the full cards path.
D-085 — The goals model prices what it can identify, and admits what it cannot
Date: 2026-08-13. Audit 1.9/2.10: Inter 3.49 expected goals vs Monza — the only side above 2.8 across 45 fixtures — with Monza unflagged despite changing division.
Root cause was identifiability, not scale, and it took two conditions to close. fit_league now records per-team EFFECTIVE (decay-weighted) evidence and per-team STALENESS (days between the team's last fitted match and the window's edge — relative to the window, not the clock, so holdout evaluation is not poisoned). predict_match treats a team as newly seen — 25th-percentile prior, provisional flag — when either its evidence is under 5 effective matches (noise wearing a badge; the sum-to-zero centring parks such teams at extremes) or its last match sits over 200 days behind the window (the Monza case: a relegation season still carries about 16 effective matches at the 365-day half-life, but a side returning after a year away is a different team wearing the same id, and the D-063 backtest already showed the promoted-side prior beats old form). No lambda cap: capping distorts matches the model CAN price, and with the evidence rule the tail vanished on its own — max side xG is now 2.77, zero above 2.8, Inter v Monza serves 2.77/0.64 provisional (was 3.49/0.54 unflagged) at home 81% (was 90%).
Two selection-rule failures surfaced and were fixed in one afternoon: the half-life grid chose by log loss alone (180d: 0.0002 better loss for a 3pp calibration breach — the gate failed the grid's own choice), and crossval's best-calibration-first chooser then picked 90d (calibration 0.007 by forgetting nearly everything: log loss 1.0825 vs baseline 1.0832 — the audit-3.4 cards pathology). The half-life is now chosen by THE GATE'S OWN RULE: among half-lives calibrating within tolerance, best log loss wins. Active: match-v20260813-1522, 365d, calibration 0.095, +5.1% vs baseline, 49 fixtures re-projected, 18 honestly provisional.
D-086 — The MAE baseline condition gets the significance discipline; SOT returns
Date: 2026-08-13. Decision: The operator ("do SOT return"), on the question D-083 left him.
SOT had failed §5.6 by 0.0002 of MAE against the naive baseline (0.3468 vs 0.3466) while beating that baseline's log loss by 49.1% and calibrating clean. Every other comparison in the gate already requires a breach to be practically large AND BREACH_Z-significant (D-052 for deciles, D-062 for bands); the MAE condition was the one place a measurement could fail a model over pure noise — no tolerance, no significance, a 0.0002 deficit as fatal as a 0.1 one.
The fix is the house discipline, not an exemption: the evaluator now stores the PAIRED standard error of (model MAE − baseline MAE) — the two are computed on the same rows, so their difference has a per-row distribution — and the MAE condition fails only when the deficit is at least BREACH_Z (2.0) standard errors from zero. A real MAE deficit still fails whatever the log loss says. Log loss stays strict: §14 names it the hard stop, and a probability is what the product sells. Old artefacts carry no SE and read strictly, exactly as they always did.
Measured on the SOT refit: deficit 0.00019, paired SE 0.00085, z = 0.22 — a fifth of a standard error from zero. SOT passed (vv20260813-1536-sot: +49.1% log loss, calibration 0.033, bands hold at 0.015) and ACTIVATED riding the pooled shots parent — a measured pairing, closing the D-083 bench. Golden set regenerated: 7 markets frozen, benched_markets empty. SOT projections resume with the worker's next cycle.
Provenance addendum (2026-08-13, on the operator's challenge). The commit order: SOT benched for the MAE failure at ba505ca (16:29); the significance condition written at 884c825 (16:40) — eleven minutes later. No backlog item, audit recommendation, or earlier entry anticipated the condition; it was designed after, and because, SOT failed. This is a POST-HOC gate change in the direction loosen-gate-market-returns, and it is disclosed as such here and on the methodology page. What predates the failure: BREACH_Z = 2.0 and the practically-large-AND-significant pattern (D-052 deciles, D-062 bands) — D-086 extended an existing standard to the one condition without it, it did not invent a bar for SOT. What does not predate the failure: the decision to extend it, made knowing which market it would return. z = 0.22 argues the rule reached the right answer; it cannot argue the rule's timing, which is why the timing is written down. The genuinely out-of-sample test is the same one it has always been: the closing-line record.
D-087 — The runs rail was showing duplicated matches as runs
Date: 2026-08-13. Audit 3.2, and it was REAL, not the DOM quirk the auditor allowed for: getHotTrends joined player history against sides — one row per (team, upcoming fixture) — so any club with two fixtures inside the horizon duplicated every history row before row_number() ran. Munetsi's rendered "run of five" (1,1,2,2,1) was his true 1,2,1 shown pairwise: two and a half matches wearing five chips. The homepage rail never showed it because it scopes to one day; /trends spans a fortnight. History now joins DISTINCT teams, verified by exact arithmetic against his player_fixture_stats rows.
Same commit applies D-083's 30-minute floor to run qualification and to every displayed per-90 on the trends surfaces (audit 3.3, 3.1's cameo case), and gives every run card its missing context: matches outside the fixture's competition and runs whose last match is over 45 days old are flagged on the card, with the competition in each chip's hover. Munetsi's card now reads "includes matches outside the Championship - last match 265 days before this fixture" — the sentence the auditor had to reconstruct by hand. Early-season note: most cards carry the staleness flag in August because last season ended in May; that is the flag being honest, and it quiets as real matches arrive.
D-088 — Email capture, and the legal pages that make its promises binding
Date: 2026-08-13. Audit 4.1/4.2/4.3/section 6. Capture on /value and /track-record in the house voice — states exactly what will be sent (the CLV record when the first calls settle), no urgency theatre. GDPR-shaped at the schema: subscribers stores the address, the VERBATIM purpose sentence, a consent timestamp and an unsubscribe token (migration 0017); re-subscribing is fresh consent; tokenised /unsubscribe works without an account; erasure is a row delete. Verified end to end before commit.
/privacy and /terms drafted against what the code verifiably does: no analytics of any kind installed (checked the dependency tree), bankroll in localStorage only (board.tsx), session cookie only when auth is enabled — hence no PECR consent banner is owed. Both pages carry a visible draft-status notice: careful, not lawyer-reviewed (D-067 keeps external review out of launch scope). Footer links added; /bets and /account nav links now render only when authConfigured() — the audit's interim monetisation fix.
D-089 — The decisions log is published, and the redaction policy is code
Date: 2026-08-13. Decision: The operator, on audit 3.6's question — site copy cites §/D references that resolved nowhere public; publish the log or strip the references. Publish, conditional on an end-to-end read for anything internal, anything naming a person, and anything giving away more of the method than intended.
The read (all 3,490 lines) found three categories, and the policy that
handles them is committed as scripts/publish_decisions.py rather than
applied by hand — what was withheld is itself on the record:
- Supplier and infrastructure names (~70 mentions) are substituted with role names, in prose and file paths alike. The reasoning is the moat; the supplier list is a shopping list for a copycat, and 3.5 already de-vendored the rest of the site. The odds feed's distinctive rate-limit figures are scrubbed for the same reason a name would be.
- Six entries are withheld in full — D-017 (data-plan economics), D-042 (supplier evaluation), D-051/D-077 (infrastructure capacity and cost), D-067 (internal legal posture), D-068 (auth/billing switch-on internals) — each leaving a numbered stub naming its category, so the sequence stays checkable and a withheld entry can never quietly disappear. D-056 keeps its staking content but loses its final OPEN paragraph to the same posture category.
- "the operator" (39 mentions, the only person named) renders as "the operator".
The generator writes docs/DECISIONS-PUBLIC.md and a TS module the
/decisions page imports (an import survives serverless file-tracing; an
fs path may not). The page renders via marked with D-number heading ids,
so site copy can deep-link (/decisions#d-081); footer link, methodology
cross-link and sitemap entry added. Verified in-browser: 88 entries, 6
stubs, zero vendor or person strings in the rendered DOM.
Regeneration is part of the append ritual: any commit touching DECISIONS.md runs the script in the same commit. The internal file stays the unedited record; this entry, describing its own publication, ships in the first public render — which is the property the whole page sells.
Addendum (2026-08-13, evening — the operator's review). Two corrections to the first render. (1) The supplier role-naming is DROPPED as cosmetic: every fixture page hotlinks the stats provider's CDN in its own markup, so redacting ~70 name mentions in the log protected nothing and made the page look more guarded than it is. Suppliers now render verbatim; the six withheld entries — which hold the actually-sensitive commercial detail — stand unchanged, as does the operator naming (accepted for now; personal accountability on a solo product is worth revisiting). (2) The "every reference resolves" claim was audited rather than assumed: every D-number in rendered site copy (D-016, D-079, D-082, D-084, D-086, D-092) resolves to a PUBLIC entry; none of the six withheld numbers is cited anywhere a reader can see. A handful of §-references to the unpublished spec remain in visible copy (e.g. the referee page's sample-threshold banner) — those predate publication and are a separate wording decision.
D-090 — The operator's review of the audit close-out, executed
Date: 2026-08-13. The operator reviewed the Phase 3/4 close-out and ruled; this entry records the rulings and what each one changed.
Provenance first. His challenge on D-086 — was the MAE significance condition designed before or after SOT failed it? — is answered in that entry's addendum: after, by eleven minutes, and now disclosed as post-hoc in the log and on the methodology page beside its strongest provenance claim. His second demand — that the 21 withdrawn calls still be scored against the close — was verified rather than assumed: capture_closing and settle_edges never read retired_at, the record's aggregates scan withdrawn rows, and the only retirement filters in the web layer are the two forward-looking board queries D-081 specified.
The count rule. "23 on the record" does not ship: no surface may lead with the flattering total. The rule is applied to the 4.9 proposal (docs/proposals/2026-08-13-caveat-hierarchy.md, drafted NOT applied) and governs all future copy: lead with live and withdrawn-still-scored, or with no count at all.
Shipped on his go: the decisions log published with its redaction policy in code (D-089); referee pages enriched — strictness percentile within the official's own league, the SERVED cards model's multiplier read from the D-084 payload, next appointments, cards_total on both referee surfaces (3.4's rule, one surface over); the match page rebuilt to one table at every width, ten rows per market with server-side expansion — 3.0MB measured down to 868KB (audit 4.7, both halves in one change at his instruction); the homepage runs rail hides while every run is stale (3.3 — honest-and-useless is worse than absent, and the rail returns by itself as fresh runs arrive); the privacy notice's draft label dropped because an operative notice is what a live capture form relies on, with an env-driven footer Contact link so controller contactability cannot silently regress.
His to do, flagged not fixed: ICO registration (email capture makes him a data controller; registration is cheap and commonly missed), the controller's legal identity for the footer and privacy page (not invented here), NEXT_PUBLIC_CONTACT_EMAIL in the deploy environment, and the media-licence question to the stats provider — the one audit finding that could force a redesign, drafted for him to send from the account email. Monetisation stays unpriced until a CLV sample exists; his framing stands: the asset is the record, not the board.
D-091 — The reading of the first settlements, committed before they exist
Date: 2026-08-13, evening. Decision: The operator. The first closing lines land at tomorrow's kickoffs and the first results tomorrow night. This entry fixes the interpretation of the 21 withdrawn (regime v1) calls' closing-line scores NOW, while no result exists and no line has been captured — because committing the reading before the data arrives is the difference between a record and a narrative.
The three branches, verbatim from the decision:
- If the 21 beat the close materially → that is evidence the D-081 correction was TOO AGGRESSIVE; the corrected margin gets re-examined, not defended. Withdrawal was the right act under the rule as decided — but the rule itself goes back on the table.
- If they do not beat the close → that is the first empirical support for the correction. First, not conclusive.
- If the outcome is mixed, or the sample is too small to say — and at n=21 across one opening matchday it very likely is — we say exactly that and say nothing more. No surface spins a noise-sized result in either direction.
The record page already carries the machinery this entry depends on: the aggregates and the chart are segmented by regime (v1 withdrawn / v2 standing, never blended — one average over both would measure neither), and the chart's caption states that 23 calls across one matchday is noise in either direction. "Materially" deliberately has no number attached at n=21: any threshold chosen tonight would be false precision, and branch 3 exists precisely so that smallness is an answer rather than an embarrassment. When the sample spans enough matchdays for a paired test against zero, the branch-1/branch-2 call is made with the same BREACH_Z discipline the gate uses — and a new entry records it.
D-092 — D-086 fails the generality test and is reverted; SOT re-benched
Date: 2026-08-13, evening. Decision rule: The operator, committed before the test ran: apply the MAE significance condition retroactively to every market's full gate history; if it only ever changes the SOT outcome it is a rule shaped to a single case and is reverted REGARDLESS of z; if it would have changed other decisions too it is a general rule written late, and stays.
The test. All 76 fitted-model artefacts in the project's history carry model_mae and baseline_mae. Across seven markets and every era — pre-D-052, D-059's thinned rebuilds, D-064's re-gate, D-083's pooled refit — exactly ONE fit ever had an MAE deficit: SOT on the pooled basis (deficit +0.00019, the same model stored under v20260813-1451 and re- evaluated as v20260813-1536-sot). Every other artefact beats its baseline MAE by 0.006 to 0.16 — two to three orders of magnitude beyond any significance band, so the condition could never have flipped any other verdict, past or plausible. The rule fired once, for the case it was written for, eleven minutes after that case failed.
The revert, in full:
beats_baseline_maeis strict again.mae_diff_seis still measured and stored — context for the reader, never forgiveness — and the failure message now reports the deficit's z while failing anyway. A regression test pins the exact SOT numbers (deficit 0.0002, SE 0.00085, z=0.22) as a FAIL.- v20260813-1536-sot DEACTIVATED. No SOT edge was published during the ~29 hours it served (verified: zero publications since activation), so there is nothing to withdraw — the record is untouched.
- Golden set regenerated: 6 markets frozen at v20260813-1451, benched_markets = [player_shots_on_target]. 635 tests pass.
- The methodology bullet that disclosed D-086 as post-hoc now records the test and the revert — the reader who was told "discount it as you see fit" is told how it ended.
What this is not. Not a verdict that SOT's deficit was real — z=0.22 says it almost certainly was not, and the log loss and calibration evidence still argue the model is good. It is a verdict about RULES: a gate whose conditions can be reshaped after the fact, for one market at a time, is not a gate, and the only defensible response to "this rule would never have fired for anyone else" is to remove it. SOT returns when it beats the strict baseline — both halves — or when a principled MAE tolerance is designed, argued and committed BEFORE the next failure it would excuse, like every other tolerance in the gate.
D-093 — P(plays): participation is surfaced, floored at publication, and saves becomes a goalkeeper product
Date: 2026-08-13, late evening. Audit 1.6/1.7 (backlog item 10) — the one specified item Phase 2 never built.
The problem, in the audit's words: projections are correctly conditional on the player taking part (books void a no-show, D-053), but the board presented a bet on an ever-present and a bet on a 38%-participation squad player identically. Same EV, wildly different products — one is capital deployed, the other mostly comes back void — and void legs also release the D-056 correlated-exposure caps the stakes were sized under.
What participation IS here: the share of his team's recent finished
matches the player appeared in (any minutes — a cameo takes part, per
the void rule), over participation.window_matches (25). A historical
fact, not a model estimate: P(start | played) already exists on every
row and D-053 deliberately conditions the no-show branch away, so the
displayed number is the observable void-risk fact, never a probability
wearing more confidence than its construction earns.
Three changes:
- Publication floor —
participation.min_rate: 0.5in thresholds.yaml (the operator's dial, like every threshold). The candidates query computes each projected player's rate once per run; the engine rejects below-floor candidates with their own reason (participation_below_floor). A player with NO measurable window — a new signing — is below the floor by construction: absence of participation evidence must not read as "probably plays". Match-market candidates skip the rule. Measured on the live run before arming: of 25,007 candidates, 5,150 (21%) sat below the floor — the exact squad- player tail the audit described. Zero published calls are affected (both live calls are ever-presents). - The board shows the fact — a Played column ("24/25") on every value-board row, amber under 75%, with the void-risk explanation on hover. Distinct from minutes confidence on purpose: confidence is about HOW LONG if he plays, participation is WHETHER.
- Saves is a goalkeeper product on every surface (audit 1.7: four
keepers at ~90 minutes each read as broken, and an outfielder's saves
price is a claim the model never meant to make). The engine rejects
non-GK saves candidates (
saves_non_keeper, unknown position counts as not a keeper); the match page table, the builder's leg options and the trends "next up" panel all apply the same GK filter. The pipeline still projects everyone — the rows exist, no surface shows them, the gate refuses them. 9 new engine tests; 644 total pass.
Found while verifying, and worth its own line: the board's stale-odds
suppression banner was ON — the worker had died at 18:14 mid-
project_publish (run row stuck at running, process gone) and nothing
had polled for 185 minutes, the night before the first closing lines.
Restarted 21:18 with all of tonight's code; §10's suppression state did
its job, but a dead worker the evening before settlement day argues for
the alerting item §10 already owes.
D-094 — The night the site drowned in its own crawl surface
Date: 2026-08-13, ~23:30. The operator: "the site is really slow, sometimes dont load and isnt responsive at all." Confirmed and worse: every dynamic page on production timed out at 30 seconds with no response at all; only /decisions answered (0.98s), because it is the one page that never touches the database.
Diagnosis, in the order the evidence arrived. The database was healthy (the worker's jobs and local pages ran normally). pg_stat_activity showed the transaction pooler's few server connections either wedged in ClientRead — Vercel clients that died mid-protocol, holding slots with no server-side timeout to reap them — or continuously cycling PLAYER PROFILE queries. That second observation was the cause: the sitemap advertised ~5,200 player pages plus referees, teams and fixtures; the site serves every page per-request (D-073 removed build-time rendering); Vercel's functions ran in the default US region against a Frankfurt database (~100ms per query round trip, several queries per page); and a crawler sweep of thousands of first-hit URLs kept every pooler slot permanently busy. Real visitors queued at the pooler behind the bots until their requests timed out. ISR on the profile pages could not help: a sweep of distinct URLs is all first hits.
The fixes, all shipped in one deploy:
- Functions moved to the database's region — vercel.json pins fra1. The round trip that was ~100ms becomes ~2ms; each render holds its pool slot for a tenth of the time, which is an order-of-magnitude throughput change and should always have been the configuration.
- The sitemap stops soliciting thin pages — player URLs now carry the model's own 10-completed-matches floor (5,208 → 1,083). Pages below it still resolve; they are no longer advertised.
- The SEO-spider fleet is disallowed (Ahrefs, Semrush, MJ12, PetalBot, Bytespider and friends). Google and Bing stay welcome — they are what §8.5's acquisition argument needs; the rest is cost without a reader.
- The heaviest read is cached — getRefereeTrends (an aggregate over every player_fixture_stats row, run by all 131 referee profiles) goes through unstable_cache at 5 minutes. Safe because its rows carry no Date fields; the comment on it warns the next reader that unstable_cache JSON-round-trips Dates into strings.
Still owed, not shipped tonight: a server-side idle timeout so a wedged ClientRead connection can never hold a pooler slot for minutes (Supabase dashboard setting — the operator's console); and §10's alerting item, which tonight argued for itself twice — the worker died silently at 18:14 and the site drowned silently by 23:30, and both were found by looking rather than by being told.
D-095 — The lineup feed never said "confirmed"; we did, and it set the gate's lowest bar
Date: 2026-08-14. Phase 5 B1.
Sportmonks' lineup type_id is the STARTER/BENCH axis (11 = starting XI,
12 = bench). The adapter read it as confirmed-vs-expected —
"expected" if type_id == 12 else "confirmed" — so every starter became
a confirmed teamsheet entry, every bench player an "expected" one, and
any unrecognised type defaulted to CONFIRMED: absence of knowledge
reading as maximum certainty, D-016's rule inverted.
Verified from the stored payload rather than inferred: fixtures
19732742/19732734, 22 entries each, all type_id 11, all carrying
formation_position, captured 52 hours before kickoff. A confirmed
lineup cannot exist that early.
It reached the gate. 210 projections took high confidence and its
4% base bar instead of 10%; two edges published on Sevilla v Rayo against
a 10.99% bar where the honest tier demanded ~17%.
Three fixes. The adapter asserts only what it can see (is_starter from
type_id) and never claims confirmed. sync_lineups makes that call,
because it is the layer that knows the kickoff: confirmed means
captured inside the window where teamsheets are actually published, a
rule that is stated and checkable rather than inherited from a field that
never meant it. And the re-poll exclusion now looks for a CONFIRMED
lineup — it excluded any fixture holding any lineup row, so a fixture
that received an early projected XI was never polled again outside T-3h
and could never upgrade to the real thing. That alone would have starved
a T-60m board.
Lineup responses are now recorded under their own raw_payloads entity;
they were labelled fixtures and swept by D-077's 72-hour retention,
which is why the type_id semantics nearly had to be re-derived by a live
probe — the re-fetch §4.3 exists to prevent. 44 rows retyped; both
fixtures drop off the 4% bar. 6 new tests.
D-096 — A call the current model no longer supports comes off the board
Date: 2026-08-14. Decision: The operator, on the Phase 5 B3 forensic.
The D-081 sweep re-tested the margin RULE against each row's own stored model version. Nothing re-tested a standing call against the model serving today. So two calls published 05:09 on 13 August — the morning before D-083's pooled-rate correction — were still on the board advertising +57.0% and +120.8%.
Repriced under the corrected model at the same prices: −0.3% and +0.0%. Both collapse to exactly Bet365's price. The edges were the old averaged-rate denominator, which D-083 proved inflated every per-90 by roughly twofold; they were never a disagreement with the market. This is the market anchor arriving three days early, and it says D-083 was right.
The cause is a RULE, as retirement always is: clear your own corrected bar using the CURRENT model's probability, or come off the board. Deliberately not "any positive edge" — that split two nested lines on one player at a rounding difference from zero, keeping a fair-value bet because the arithmetic landed a hair above it. Both rows stay in the record, keep their prices, and are still scored against their closing lines. Board: 0 live, 23 withdrawn, 23 still scored.
Also on the record from this forensic: 13 of the 23 calls came from
fixtures containing a team with ZERO matches in the current division, and
Racing Santander — 8 of those 13 — has no conceded-fouls evidence at all,
so the opponent covariate silently fell back to the league mean with no
flag. OPPONENT_CONCEDED_SQL has no minimum-match floor and no division
filter. That is the strongest argument for Phase 5 C's opponent-
sufficiency guard, and it is not yet built.
D-097 — The shadow book
Date: 2026-08-14. Decision: The operator, Phase 5 D.
The product publishes ~2 calls a fortnight at the honest bar, so the CLV record that decides whether the model works cannot answer the question this year. Every cycle already evaluates ~17,000 priced selections against a full projection and discards all but the survivors.
shadow_candidates (migration 0018) keeps them: every candidate clearing
a 2% edge, logged ONCE per selection with its price, timestamp, complete
gate arithmetic, would_publish and its full reject-reason list — then
scored against the same closing lines and graded by the same function as
a published call. The sharing is deliberate: two scoring paths that
drifted apart would make the comparison worthless while still producing
numbers. First cycle logged 293 observations.
Nothing renders and nothing is advice. published_edges keeps meaning
exactly what it says — calls the product stood behind at the time — and
the shadow book answers the questions the record cannot: does CLV rise
with edge size or collapse above 40%, is it positive for fouls at all, do
promoted-opponent fixtures score worse, is it concentrated in one book,
and where does the bar actually belong.
D-098 — The participation CTE took down publication for six hours
Date: 2026-08-14. Found by trying to run the shadow book.
D-093's participation subquery was written as a CTE Postgres inlines by
default. The planner estimated one row against an actual ~2,000, so it
nested-looped the entire participation subplan once per candidate —
25,000 times — and project_publish died on the 2-minute statement
timeout every cycle from 21:40 on 13 August.
The visible state was "yesterday's edges, still there". Nothing on the
site said anything was wrong; only ingest_runs.status knew. That is
D-075's failure mode precisely, and D-075's lesson repeated: a
diagnostic-shaped addition took down the thing it measures.
Both CTEs are now MATERIALIZED, computed once. The cycle runs in 36
seconds. The keyword is load-bearing and the query says so.
Third worker death in two days recorded here too: the process was gone again this morning, last job 22:05. Sleep-on-AC is disabled, so the cause is not the obvious one and is not yet known. Two silent failures (worker, publish) inside twelve hours is the whole argument for §10's alerting item, and the standing recommendation is to put the worker back on the always-on host it was moved off for cost.