Investigation for #46 and #47, before touching either. A scored history is linear-Gaussian, so its true joint posterior has a closed form and the crate can be checked against ground truth. Measured on five competitors: means marginal sd (crate / exact) tree (star) exact 1.000 loopy (robin) exact 0.502 On a tree the crate is exact in both. With cycles the means stay exact — the standard Gaussian-BP result, and the property ratings rely on — while marginal variances come out about half the true width. That is the opposite direction from what #47 reports, so whatever is happening in that consumer's model, the crate being conservative is not it. It also means #46 cannot be implemented as an added covariance accessor. The exact correlation between two nodes here is +0.857, so ignoring it overstates the width of a difference — but the too-narrow marginals partially cancel that, leaving 1.327x rather than 2.646x. Adding true correlations to these marginals without correcting them would give 0.765 against a true 1.524: overconfident, which is the direction the reporter specifically called unsafe. Pins the two real invariants (exactness on a tree, exact means with cycles) and deliberately only records the variance gap, since closing it is what #46 proposes. Also records the working rules this project has converged on: investigate before implementing, fix the root issue, and scout crates.io on measured accuracy rather than adoption. Refs #46, #47 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
135 lines
6.9 KiB
Markdown
135 lines
6.9 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
just test # Full suite across every feature combination CI checks
|
|
just check # Fast inner loop: cargo test --features approx
|
|
just lint # clippy, warnings denied
|
|
just fmt # ALWAYS nightly — rustfmt.toml uses nightly-only options
|
|
just determinism # Bit-identical posteriors at RAYON_NUM_THREADS 1/2/4/8
|
|
just ci # Everything CI runs
|
|
cargo test --lib <test_name> # A single test by name
|
|
cargo bench # Criterion benchmarks
|
|
```
|
|
|
|
**Run tests in release too.** `debug_assert!` is compiled out there, and that
|
|
is where several defects have hidden — a debug-only run is not evidence.
|
|
`just test` includes a release job.
|
|
|
|
### Feature flags
|
|
|
|
- `approx` — `approx::AbsDiffEq` etc. for `Gaussian`. Most numerical goldens need it.
|
|
- `rayon` — opt-in parallel within-slice sweep and per-slice query passes.
|
|
|
|
## Working rules
|
|
|
|
- **Investigate before implementing.** Measure the actual behaviour first —
|
|
against an analytic reference where one exists. Several "obvious" fixes in
|
|
this repo turned out to be wrong in sign or unnecessary, and the measurement
|
|
is what caught them.
|
|
- **Fix the root issue, not the symptom.** A clamp that hides an underflow, or
|
|
a tolerance loosened to make a test pass, is a defect deferred.
|
|
- **Scout crates.io before hand-rolling numerics.** Check accuracy against an
|
|
independent reference rather than trusting downloads: `puruspe` has 1.4M
|
|
downloads and is 346 ULP off in the tail, where `libm` is 1. Fewer
|
|
dependencies is preferable, not mandatory — take the dependency when it is
|
|
measurably better.
|
|
|
|
## Architecture
|
|
|
|
A Rust port of [TrueSkillThroughTime.py](https://github.com/glandfried/TrueSkillThroughTime.py):
|
|
Bayesian skill rating that infers skill at every point in time, propagating
|
|
evidence both forward and backward across a history.
|
|
|
|
### Data flow
|
|
|
|
Ingestion (public types, `event.rs`):
|
|
|
|
```
|
|
Event<T, K> → Team<K>[] → Member<K>[]
|
|
```
|
|
|
|
`History::add_events` flattens that into indices; teams survive only as
|
|
grouping, not as a value. Inference then runs on the internal shapes:
|
|
|
|
```
|
|
History → TimeSlice[] → Event[] → Item[]
|
|
↓
|
|
Game (factor graph) → Schedule → BuiltinFactor[]
|
|
```
|
|
|
|
- **`History`** (`history.rs`) — top level. Interns keys, groups events into
|
|
`TimeSlice`s by time, runs the forward/backward sweep in `converge()`, and
|
|
answers `learning_curves()`, `current_skill()`, `log_evidence()`,
|
|
`predict_quality()`, `predict_outcome()`. Built via `HistoryBuilder`.
|
|
- **`TimeSlice`** (`time_slice.rs`) — all events at one time. Owns a
|
|
`SkillStore` and a `ScratchArena`; `iteration()` sweeps its events, using
|
|
`ColorGroups` to partition independent ones.
|
|
- **`Event`** — two distinct types, do not confuse them. The *public* ingestion
|
|
`Event<T, K>` is in `event.rs` (with `Team`/`Member`); the *internal*
|
|
`pub(crate) Event` in `time_slice.rs` is one match during inference, where
|
|
`compute()` runs inference reading skills immutably and `apply()` folds the
|
|
result back. That split is what lets a color group run in parallel with no
|
|
`unsafe`.
|
|
- **`Game`** (`game.rs`) — a single match's factor graph. `run_chain` builds the
|
|
diff chain between rank-adjacent teams and drives it to convergence.
|
|
- **`Gaussian`** (`gaussian.rs`) — natural parameters (`pi = 1/sigma²`,
|
|
`tau = mu/sigma²`). `Mul`/`Div` are the EP product/cavity: pure adds and
|
|
subtracts. Variance-space ops (`Add`, `Sub`, `exclude`, `forget`) go through
|
|
`from_mv`/`variance()` and take no square root.
|
|
- **`factor/`** — `TeamSumFactor`, `RankDiffFactor`, `TruncFactor` (ranked),
|
|
`MarginFactor` (scored), over a flat `VarStore`. `BuiltinFactor` dispatches
|
|
by enum rather than `dyn`.
|
|
- **`Schedule`** (`schedule.rs`) — drives factor propagation. `EpsilonOrMax` is
|
|
the only implementation.
|
|
- **`Competitor`** (`competitor.rs`) — per-history temporal state (`message`,
|
|
`last_time`). **`Rating`** (`rating.rs`) — static config (prior, `beta`, drift).
|
|
- **`storage/`** — `SkillStore` (per slice, `pub(crate)`) and `CompetitorStore`
|
|
(per history, public), both indexed by `Index`. The module is `pub`, but only
|
|
`CompetitorStore` is reachable from outside the crate.
|
|
- **`KeyTable`** (`key_table.rs`) — user key ↔ `Index`, both directions O(1).
|
|
- **`Drift`** (`drift.rs`) / **`Time`** (`time.rs`) — traits. `Time` is a *trait*
|
|
(`i64`, `Untimed`), not an enum.
|
|
- **`lib.rs`** — public exports, global defaults (`MU`, `SIGMA`, `BETA`,
|
|
`GAMMA`, `P_DRAW`, `EPSILON`, `ITERATIONS`), and the standalone `quality()`.
|
|
The `cdf()` / `erfc()` helpers live here too but are `pub(crate)` and private
|
|
respectively — not public API.
|
|
|
|
### Invariants worth knowing
|
|
|
|
- **A tie needs `p_draw > 0`.** With `p_draw == 0.0` the truncation margin is
|
|
zero and the two-sided tie update evaluates `0/0`. Ingestion rejects such
|
|
events with `InferenceError::TieWithoutDrawProbability`. This includes
|
|
`Outcome::winner(w, n)` for `n >= 3`, which ties every loser.
|
|
- **NaN is never convergence.** Comparisons against NaN are all false, so
|
|
`tuple_gt` reads NaN as "below epsilon". Use `step_converged` /
|
|
`step_is_finite`, never `!tuple_gt(..)` alone.
|
|
- **Evidence accumulates in log space.** A linear product over a long diff
|
|
chain underflows to zero, and `ln(0)` is `-inf`.
|
|
- **Colors are contiguous.** `recompute_color_groups` reorders events so each
|
|
color occupies one range; `ColorGroups::groups_are_contiguous` asserts it.
|
|
- **Transcendentals go through `libm`, not `std`.** IEEE 754 pins the basic
|
|
operations and `sqrt` but says nothing about `exp`/`log`/`erf`, and `std`
|
|
delegates to the *system* math library — measured, `f64::exp` and `libm::exp`
|
|
disagree on 9.7% of inputs by one ULP. Since inference is an iterative fixed
|
|
point, one ULP can change an iteration count. Use `libm::exp` / `libm::log` in
|
|
inference code; `f64::sqrt` is fine (IEEE specifies it). Tests may use either.
|
|
- **The crate is `#![forbid(unsafe_code)]`.** Keep it that way.
|
|
- **Ingestion order must not change the answer.** Events added one at a time
|
|
must converge to the same fixed point as the same events batched — see
|
|
`tests/ingestion_equivalence.rs`.
|
|
|
|
### Testing notes
|
|
|
|
- Numerical goldens are cross-validated against the Python/Julia reference.
|
|
Some are *convergence residuals*, not exact values; treat a small movement
|
|
as suspicious but check whether the new value is closer to the analytic
|
|
truth (symmetric fixtures converge to their prior mean exactly) before
|
|
assuming a regression.
|
|
- `tests/degenerate_inputs.rs` covers empty/boundary/error paths,
|
|
`tests/ingestion_equivalence.rs` covers batching order, `tests/quality.rs`
|
|
covers N-group quality, `tests/determinism.rs` covers thread counts.
|