logaritmiskandClaude Opus 5
c12bc830a5
Three issues from two downstream consumers, all small, all sharing a theme: the crate had the information and would not hand it over. #44 — `UnknownKey { team: 0, member: 0 }` did not say which key. A consumer upgrading 0.1.2 -> 0.4.1 had every one of 5591 predictions return this error, fell back to a neutral 0.5, and lost its entire metadata model for a day. Nothing crashed and nothing logged; it was found by sweeping an unrelated parameter and noticing the output did not move. The 0.4.0 change that made unknown keys an error was right — the error was just too anonymous to act on. It now carries the key's `Debug` rendering, and its `Display` says what to do about it. The precondition is documented on every prediction entry point, which the reporter said would alone have saved the day. #43 — `cdf` was `pub(crate)`, so a consumer asking "is this competitor below the cutoff" approximated it with a `mu + z * sigma` band and had no way to say what confidence any `z` bought. Adds `Gaussian::probability_below` / `probability_above`. The second is separate on purpose: `1 - cdf` collapses to exactly zero past ~8.3 sigma, and a stopping rule is evaluated precisely there. Both route through the survival function added in 0.4.1, so this is visibility rather than new numerics. #50 — `ConvergenceReport` was not `#[must_use]`, so the one signal that a fit stopped short was trivially discarded. It now is, and that immediately found 78 sites doing exactly that — including this crate's own ATP example, which was capped at 10 sweeps when the history needs 30. The example now reads the report and says so. `ITERATIONS = 30` is documented as the floor it is, with the three measurements to hand: 400 events over 100 competitors already stops there at ~7e-3 against a 1e-6 tolerance, the ATP example needs 30 at a much looser one, and a consumer's 2000-node model needs 76 to 161. BREAKING CHANGE: `InferenceError::UnknownKey` gains a `key` field, and the prediction methods now require `K: Debug` in order to fill it. Closes #43, #50. Refs #44 — its third ask, an opt-in `UnknownKeys::Skip` mode, is a live API question and deliberately not answered here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
102 lines
3.3 KiB
Rust
102 lines
3.3 KiB
Rust
//! Determinism tests: identical posteriors across RAYON_NUM_THREADS
|
|
//! values. Only compiled with the `rayon` feature.
|
|
|
|
#![cfg(feature = "rayon")]
|
|
|
|
use smallvec::smallvec;
|
|
use trueskill_tt::{ConstantDrift, ConvergenceOptions, Event, History, Member, Outcome, Team};
|
|
|
|
/// Build a deterministic workload using a simple LCG (no external rand crate).
|
|
fn build_and_converge(seed: u64) -> Vec<(i64, trueskill_tt::Gaussian)> {
|
|
let mut h = History::<i64, _, _, String>::builder_with_key()
|
|
.mu(25.0)
|
|
.sigma(25.0 / 3.0)
|
|
.beta(25.0 / 6.0)
|
|
.drift(ConstantDrift(25.0 / 300.0))
|
|
.convergence(ConvergenceOptions {
|
|
max_iter: 30,
|
|
epsilon: 1e-6,
|
|
alpha: 1.0,
|
|
})
|
|
.build();
|
|
|
|
// LCG for deterministic pseudo-random ints.
|
|
let mut rng = seed;
|
|
let mut next = || {
|
|
rng = rng
|
|
.wrapping_mul(6364136223846793005)
|
|
.wrapping_add(1442695040888963407);
|
|
rng
|
|
};
|
|
|
|
let mut events: Vec<Event<i64, String>> = Vec::with_capacity(200);
|
|
for ev_i in 0..200 {
|
|
let a = (next() % 40) as usize;
|
|
let mut b = (next() % 40) as usize;
|
|
while b == a {
|
|
b = (next() % 40) as usize;
|
|
}
|
|
// ~10 events per slice so color groups have material parallelism.
|
|
events.push(Event {
|
|
time: (ev_i as i64 / 10) + 1,
|
|
teams: smallvec![
|
|
Team::with_members([Member::new(format!("p{a}"))]),
|
|
Team::with_members([Member::new(format!("p{b}"))]),
|
|
],
|
|
outcome: Outcome::winner((next() % 2) as u32, 2),
|
|
});
|
|
}
|
|
h.add_events(events).unwrap();
|
|
let _ = h.converge().unwrap();
|
|
// Sample one competitor's curve for the comparison.
|
|
h.learning_curve("p0")
|
|
}
|
|
|
|
#[test]
|
|
fn posteriors_identical_across_thread_counts() {
|
|
let sizes = [1usize, 2, 4, 8];
|
|
let mut results: Vec<Vec<(i64, trueskill_tt::Gaussian)>> = Vec::new();
|
|
for &n in &sizes {
|
|
let pool = rayon::ThreadPoolBuilder::new()
|
|
.num_threads(n)
|
|
.build()
|
|
.expect("rayon pool build");
|
|
let curve = pool.install(|| build_and_converge(42));
|
|
results.push(curve);
|
|
}
|
|
|
|
let reference = &results[0];
|
|
for (i, curve) in results.iter().enumerate().skip(1) {
|
|
assert_eq!(
|
|
curve.len(),
|
|
reference.len(),
|
|
"curve length differs at {n} threads",
|
|
n = sizes[i],
|
|
);
|
|
for (j, (&(t_ref, g_ref), &(t, g))) in reference.iter().zip(curve.iter()).enumerate() {
|
|
assert_eq!(
|
|
t_ref,
|
|
t,
|
|
"time point {j} differs at {n} threads: ref={t_ref} vs got={t}",
|
|
n = sizes[i],
|
|
);
|
|
assert_eq!(
|
|
g_ref.mu().to_bits(),
|
|
g.mu().to_bits(),
|
|
"mu bits differ at {n} threads, time {t}: ref={ref_mu} got={got_mu}",
|
|
n = sizes[i],
|
|
ref_mu = g_ref.mu(),
|
|
got_mu = g.mu(),
|
|
);
|
|
assert_eq!(
|
|
g_ref.sigma().to_bits(),
|
|
g.sigma().to_bits(),
|
|
"sigma bits differ at {n} threads, time {t}: ref={ref_sigma} got={got_sigma}",
|
|
n = sizes[i],
|
|
ref_sigma = g_ref.sigma(),
|
|
got_sigma = g.sigma(),
|
|
);
|
|
}
|
|
}
|
|
}
|