Investigation for #46 and #47, before touching either. A scored history is linear-Gaussian, so its true joint posterior has a closed form and the crate can be checked against ground truth. Measured on five competitors: means marginal sd (crate / exact) tree (star) exact 1.000 loopy (robin) exact 0.502 On a tree the crate is exact in both. With cycles the means stay exact — the standard Gaussian-BP result, and the property ratings rely on — while marginal variances come out about half the true width. That is the opposite direction from what #47 reports, so whatever is happening in that consumer's model, the crate being conservative is not it. It also means #46 cannot be implemented as an added covariance accessor. The exact correlation between two nodes here is +0.857, so ignoring it overstates the width of a difference — but the too-narrow marginals partially cancel that, leaving 1.327x rather than 2.646x. Adding true correlations to these marginals without correcting them would give 0.765 against a true 1.524: overconfident, which is the direction the reporter specifically called unsafe. Pins the two real invariants (exactness on a tree, exact means with cycles) and deliberately only records the variance gap, since closing it is what #46 proposes. Also records the working rules this project has converged on: investigate before implementing, fix the root issue, and scout crates.io on measured accuracy rather than adoption. Refs #46, #47 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
6.9 KiB
6.9 KiB
CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
Commands
just test # Full suite across every feature combination CI checks
just check # Fast inner loop: cargo test --features approx
just lint # clippy, warnings denied
just fmt # ALWAYS nightly — rustfmt.toml uses nightly-only options
just determinism # Bit-identical posteriors at RAYON_NUM_THREADS 1/2/4/8
just ci # Everything CI runs
cargo test --lib <test_name> # A single test by name
cargo bench # Criterion benchmarks
Run tests in release too. debug_assert! is compiled out there, and that
is where several defects have hidden — a debug-only run is not evidence.
just test includes a release job.
Feature flags
approx—approx::AbsDiffEqetc. forGaussian. Most numerical goldens need it.rayon— opt-in parallel within-slice sweep and per-slice query passes.
Working rules
- Investigate before implementing. Measure the actual behaviour first — against an analytic reference where one exists. Several "obvious" fixes in this repo turned out to be wrong in sign or unnecessary, and the measurement is what caught them.
- Fix the root issue, not the symptom. A clamp that hides an underflow, or a tolerance loosened to make a test pass, is a defect deferred.
- Scout crates.io before hand-rolling numerics. Check accuracy against an
independent reference rather than trusting downloads:
puruspehas 1.4M downloads and is 346 ULP off in the tail, wherelibmis 1. Fewer dependencies is preferable, not mandatory — take the dependency when it is measurably better.
Architecture
A Rust port of TrueSkillThroughTime.py: Bayesian skill rating that infers skill at every point in time, propagating evidence both forward and backward across a history.
Data flow
Ingestion (public types, event.rs):
Event<T, K> → Team<K>[] → Member<K>[]
History::add_events flattens that into indices; teams survive only as
grouping, not as a value. Inference then runs on the internal shapes:
History → TimeSlice[] → Event[] → Item[]
↓
Game (factor graph) → Schedule → BuiltinFactor[]
History(history.rs) — top level. Interns keys, groups events intoTimeSlices by time, runs the forward/backward sweep inconverge(), and answerslearning_curves(),current_skill(),log_evidence(),predict_quality(),predict_outcome(). Built viaHistoryBuilder.TimeSlice(time_slice.rs) — all events at one time. Owns aSkillStoreand aScratchArena;iteration()sweeps its events, usingColorGroupsto partition independent ones.Event— two distinct types, do not confuse them. The public ingestionEvent<T, K>is inevent.rs(withTeam/Member); the internalpub(crate) Eventintime_slice.rsis one match during inference, wherecompute()runs inference reading skills immutably andapply()folds the result back. That split is what lets a color group run in parallel with nounsafe.Game(game.rs) — a single match's factor graph.run_chainbuilds the diff chain between rank-adjacent teams and drives it to convergence.Gaussian(gaussian.rs) — natural parameters (pi = 1/sigma²,tau = mu/sigma²).Mul/Divare the EP product/cavity: pure adds and subtracts. Variance-space ops (Add,Sub,exclude,forget) go throughfrom_mv/variance()and take no square root.factor/—TeamSumFactor,RankDiffFactor,TruncFactor(ranked),MarginFactor(scored), over a flatVarStore.BuiltinFactordispatches by enum rather thandyn.Schedule(schedule.rs) — drives factor propagation.EpsilonOrMaxis the only implementation.Competitor(competitor.rs) — per-history temporal state (message,last_time).Rating(rating.rs) — static config (prior,beta, drift).storage/—SkillStore(per slice,pub(crate)) andCompetitorStore(per history, public), both indexed byIndex. The module ispub, but onlyCompetitorStoreis reachable from outside the crate.KeyTable(key_table.rs) — user key ↔Index, both directions O(1).Drift(drift.rs) /Time(time.rs) — traits.Timeis a trait (i64,Untimed), not an enum.lib.rs— public exports, global defaults (MU,SIGMA,BETA,GAMMA,P_DRAW,EPSILON,ITERATIONS), and the standalonequality(). Thecdf()/erfc()helpers live here too but arepub(crate)and private respectively — not public API.
Invariants worth knowing
- A tie needs
p_draw > 0. Withp_draw == 0.0the truncation margin is zero and the two-sided tie update evaluates0/0. Ingestion rejects such events withInferenceError::TieWithoutDrawProbability. This includesOutcome::winner(w, n)forn >= 3, which ties every loser. - NaN is never convergence. Comparisons against NaN are all false, so
tuple_gtreads NaN as "below epsilon". Usestep_converged/step_is_finite, never!tuple_gt(..)alone. - Evidence accumulates in log space. A linear product over a long diff
chain underflows to zero, and
ln(0)is-inf. - Colors are contiguous.
recompute_color_groupsreorders events so each color occupies one range;ColorGroups::groups_are_contiguousasserts it. - Transcendentals go through
libm, notstd. IEEE 754 pins the basic operations andsqrtbut says nothing aboutexp/log/erf, andstddelegates to the system math library — measured,f64::expandlibm::expdisagree on 9.7% of inputs by one ULP. Since inference is an iterative fixed point, one ULP can change an iteration count. Uselibm::exp/libm::login inference code;f64::sqrtis fine (IEEE specifies it). Tests may use either. - The crate is
#![forbid(unsafe_code)]. Keep it that way. - Ingestion order must not change the answer. Events added one at a time
must converge to the same fixed point as the same events batched — see
tests/ingestion_equivalence.rs.
Testing notes
- Numerical goldens are cross-validated against the Python/Julia reference. Some are convergence residuals, not exact values; treat a small movement as suspicious but check whether the new value is closer to the analytic truth (symmetric fixtures converge to their prior mean exactly) before assuming a regression.
tests/degenerate_inputs.rscovers empty/boundary/error paths,tests/ingestion_equivalence.rscovers batching order,tests/quality.rscovers N-group quality,tests/determinism.rscovers thread counts.