Files
trueskill-tt/CLAUDE.md
T
logaritmiskandClaude Opus 5 36eacf5f67 test: calibrate the marginals against the exact posterior
Investigation for #46 and #47, before touching either.

A scored history is linear-Gaussian, so its true joint posterior has a
closed form and the crate can be checked against ground truth. Measured
on five competitors:

                    means      marginal sd (crate / exact)
    tree (star)     exact      1.000
    loopy (robin)   exact      0.502

On a tree the crate is exact in both. With cycles the means stay exact —
the standard Gaussian-BP result, and the property ratings rely on —
while marginal variances come out about half the true width.

That is the opposite direction from what #47 reports, so whatever is
happening in that consumer's model, the crate being conservative is not
it.

It also means #46 cannot be implemented as an added covariance accessor.
The exact correlation between two nodes here is +0.857, so ignoring it
overstates the width of a difference — but the too-narrow marginals
partially cancel that, leaving 1.327x rather than 2.646x. Adding true
correlations to these marginals without correcting them would give 0.765
against a true 1.524: overconfident, which is the direction the reporter
specifically called unsafe.

Pins the two real invariants (exactness on a tree, exact means with
cycles) and deliberately only records the variance gap, since closing it
is what #46 proposes.

Also records the working rules this project has converged on: investigate
before implementing, fix the root issue, and scout crates.io on measured
accuracy rather than adoption.

Refs #46, #47

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
2026-09-08 01:38:46 +02:00

6.9 KiB

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Commands

just test                            # Full suite across every feature combination CI checks
just check                           # Fast inner loop: cargo test --features approx
just lint                            # clippy, warnings denied
just fmt                             # ALWAYS nightly — rustfmt.toml uses nightly-only options
just determinism                     # Bit-identical posteriors at RAYON_NUM_THREADS 1/2/4/8
just ci                              # Everything CI runs
cargo test --lib <test_name>         # A single test by name
cargo bench                          # Criterion benchmarks

Run tests in release too. debug_assert! is compiled out there, and that is where several defects have hidden — a debug-only run is not evidence. just test includes a release job.

Feature flags

  • approxapprox::AbsDiffEq etc. for Gaussian. Most numerical goldens need it.
  • rayon — opt-in parallel within-slice sweep and per-slice query passes.

Working rules

  • Investigate before implementing. Measure the actual behaviour first — against an analytic reference where one exists. Several "obvious" fixes in this repo turned out to be wrong in sign or unnecessary, and the measurement is what caught them.
  • Fix the root issue, not the symptom. A clamp that hides an underflow, or a tolerance loosened to make a test pass, is a defect deferred.
  • Scout crates.io before hand-rolling numerics. Check accuracy against an independent reference rather than trusting downloads: puruspe has 1.4M downloads and is 346 ULP off in the tail, where libm is 1. Fewer dependencies is preferable, not mandatory — take the dependency when it is measurably better.

Architecture

A Rust port of TrueSkillThroughTime.py: Bayesian skill rating that infers skill at every point in time, propagating evidence both forward and backward across a history.

Data flow

Ingestion (public types, event.rs):

Event<T, K>  →  Team<K>[]  →  Member<K>[]

History::add_events flattens that into indices; teams survive only as grouping, not as a value. Inference then runs on the internal shapes:

History  →  TimeSlice[]  →  Event[]  →  Item[]
                                ↓
                   Game (factor graph) → Schedule → BuiltinFactor[]
  • History (history.rs) — top level. Interns keys, groups events into TimeSlices by time, runs the forward/backward sweep in converge(), and answers learning_curves(), current_skill(), log_evidence(), predict_quality(), predict_outcome(). Built via HistoryBuilder.
  • TimeSlice (time_slice.rs) — all events at one time. Owns a SkillStore and a ScratchArena; iteration() sweeps its events, using ColorGroups to partition independent ones.
  • Event — two distinct types, do not confuse them. The public ingestion Event<T, K> is in event.rs (with Team/Member); the internal pub(crate) Event in time_slice.rs is one match during inference, where compute() runs inference reading skills immutably and apply() folds the result back. That split is what lets a color group run in parallel with no unsafe.
  • Game (game.rs) — a single match's factor graph. run_chain builds the diff chain between rank-adjacent teams and drives it to convergence.
  • Gaussian (gaussian.rs) — natural parameters (pi = 1/sigma², tau = mu/sigma²). Mul/Div are the EP product/cavity: pure adds and subtracts. Variance-space ops (Add, Sub, exclude, forget) go through from_mv/variance() and take no square root.
  • factor/TeamSumFactor, RankDiffFactor, TruncFactor (ranked), MarginFactor (scored), over a flat VarStore. BuiltinFactor dispatches by enum rather than dyn.
  • Schedule (schedule.rs) — drives factor propagation. EpsilonOrMax is the only implementation.
  • Competitor (competitor.rs) — per-history temporal state (message, last_time). Rating (rating.rs) — static config (prior, beta, drift).
  • storage/SkillStore (per slice, pub(crate)) and CompetitorStore (per history, public), both indexed by Index. The module is pub, but only CompetitorStore is reachable from outside the crate.
  • KeyTable (key_table.rs) — user key ↔ Index, both directions O(1).
  • Drift (drift.rs) / Time (time.rs) — traits. Time is a trait (i64, Untimed), not an enum.
  • lib.rs — public exports, global defaults (MU, SIGMA, BETA, GAMMA, P_DRAW, EPSILON, ITERATIONS), and the standalone quality(). The cdf() / erfc() helpers live here too but are pub(crate) and private respectively — not public API.

Invariants worth knowing

  • A tie needs p_draw > 0. With p_draw == 0.0 the truncation margin is zero and the two-sided tie update evaluates 0/0. Ingestion rejects such events with InferenceError::TieWithoutDrawProbability. This includes Outcome::winner(w, n) for n >= 3, which ties every loser.
  • NaN is never convergence. Comparisons against NaN are all false, so tuple_gt reads NaN as "below epsilon". Use step_converged / step_is_finite, never !tuple_gt(..) alone.
  • Evidence accumulates in log space. A linear product over a long diff chain underflows to zero, and ln(0) is -inf.
  • Colors are contiguous. recompute_color_groups reorders events so each color occupies one range; ColorGroups::groups_are_contiguous asserts it.
  • Transcendentals go through libm, not std. IEEE 754 pins the basic operations and sqrt but says nothing about exp/log/erf, and std delegates to the system math library — measured, f64::exp and libm::exp disagree on 9.7% of inputs by one ULP. Since inference is an iterative fixed point, one ULP can change an iteration count. Use libm::exp / libm::log in inference code; f64::sqrt is fine (IEEE specifies it). Tests may use either.
  • The crate is #![forbid(unsafe_code)]. Keep it that way.
  • Ingestion order must not change the answer. Events added one at a time must converge to the same fixed point as the same events batched — see tests/ingestion_equivalence.rs.

Testing notes

  • Numerical goldens are cross-validated against the Python/Julia reference. Some are convergence residuals, not exact values; treat a small movement as suspicious but check whether the new value is closer to the analytic truth (symmetric fixtures converge to their prior mean exactly) before assuming a regression.
  • tests/degenerate_inputs.rs covers empty/boundary/error paths, tests/ingestion_equivalence.rs covers batching order, tests/quality.rs covers N-group quality, tests/determinism.rs covers thread counts.