feat!: name the unknown key, expose tail probabilities, flag short fits
Three issues from two downstream consumers, all small, all sharing a theme: the crate had the information and would not hand it over. #44 — `UnknownKey { team: 0, member: 0 }` did not say which key. A consumer upgrading 0.1.2 -> 0.4.1 had every one of 5591 predictions return this error, fell back to a neutral 0.5, and lost its entire metadata model for a day. Nothing crashed and nothing logged; it was found by sweeping an unrelated parameter and noticing the output did not move. The 0.4.0 change that made unknown keys an error was right — the error was just too anonymous to act on. It now carries the key's `Debug` rendering, and its `Display` says what to do about it. The precondition is documented on every prediction entry point, which the reporter said would alone have saved the day. #43 — `cdf` was `pub(crate)`, so a consumer asking "is this competitor below the cutoff" approximated it with a `mu + z * sigma` band and had no way to say what confidence any `z` bought. Adds `Gaussian::probability_below` / `probability_above`. The second is separate on purpose: `1 - cdf` collapses to exactly zero past ~8.3 sigma, and a stopping rule is evaluated precisely there. Both route through the survival function added in 0.4.1, so this is visibility rather than new numerics. #50 — `ConvergenceReport` was not `#[must_use]`, so the one signal that a fit stopped short was trivially discarded. It now is, and that immediately found 78 sites doing exactly that — including this crate's own ATP example, which was capped at 10 sweeps when the history needs 30. The example now reads the report and says so. `ITERATIONS = 30` is documented as the floor it is, with the three measurements to hand: 400 events over 100 competitors already stops there at ~7e-3 against a 1e-6 tolerance, the ATP example needs 30 at a much looser one, and a consumer's 2000-node model needs 76 to 161. BREAKING CHANGE: `InferenceError::UnknownKey` gains a `key` field, and the prediction methods now require `K: Debug` in order to fill it. Closes #43, #50. Refs #44 — its third ask, an opt-in `UnknownKeys::Skip` mode, is a live API question and deliberately not answered here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
+104
@@ -145,6 +145,45 @@ impl Gaussian {
|
||||
Self::from_mv(self.mu(), self.variance() + variance_delta)
|
||||
}
|
||||
|
||||
/// `P(X < x)` under this Gaussian.
|
||||
///
|
||||
/// The question a stopping rule asks: *how sure am I that this competitor's
|
||||
/// true skill is below the cutoff?* Expressing that as a probability keeps
|
||||
/// its meaning as sigma changes, where a `mu + z * sigma` band silently
|
||||
/// means different confidence at different uncertainties — which is exactly
|
||||
/// the regime a stopping rule operates in.
|
||||
///
|
||||
/// Accurate in the *lower* tail. For the upper tail use
|
||||
/// [`Gaussian::probability_above`] rather than `1.0 - probability_below(x)`,
|
||||
/// which cancels away every significant digit once the result is small.
|
||||
///
|
||||
/// An improper Gaussian (non-positive precision) has no defined mean, so
|
||||
/// this returns `0.5` — the same convention `mu()` and `sigma()` follow.
|
||||
#[must_use]
|
||||
pub fn probability_below(&self, x: f64) -> f64 {
|
||||
if self.pi <= 0.0 {
|
||||
return 0.5;
|
||||
}
|
||||
crate::cdf(x, self.mu(), self.sigma())
|
||||
}
|
||||
|
||||
/// `P(X > x)` under this Gaussian.
|
||||
///
|
||||
/// Computed as a survival function rather than `1 - cdf`, so it keeps full
|
||||
/// relative precision in the upper tail: `1 - cdf` returns exactly zero
|
||||
/// past about 8.3 sigma, where the true value is still 1e-19 and perfectly
|
||||
/// representable. A stopping rule is evaluated precisely there — the
|
||||
/// interesting cases are the ones near certainty.
|
||||
///
|
||||
/// An improper Gaussian returns `0.5`, as [`Gaussian::probability_below`].
|
||||
#[must_use]
|
||||
pub fn probability_above(&self, x: f64) -> f64 {
|
||||
if self.pi <= 0.0 {
|
||||
return 0.5;
|
||||
}
|
||||
crate::sf(x, self.mu(), self.sigma())
|
||||
}
|
||||
|
||||
/// EP damping in natural-parameter space: `α·new + (1−α)·self`.
|
||||
///
|
||||
/// Used by within-game inference to stabilise oscillating fixed-point
|
||||
@@ -340,3 +379,68 @@ mod tests {
|
||||
assert!((damped.tau() - expected_tau).abs() < 1e-12);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tail_probability_tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn probability_below_matches_published_quantiles() {
|
||||
let g = Gaussian::from_ms(0.0, 1.0);
|
||||
for (x, expected) in [
|
||||
(-1.959_963_984_540_054, 0.025),
|
||||
(0.0, 0.5),
|
||||
(1.281_551_565_544_6, 0.9),
|
||||
(1.959_963_984_540_054, 0.975),
|
||||
] {
|
||||
let got = g.probability_below(x);
|
||||
assert!(
|
||||
(got - expected).abs() < 1e-12,
|
||||
"P(X < {x}) = {got}, expected {expected}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_two_tails_partition_the_mass() {
|
||||
let g = Gaussian::from_ms(3.0, 2.0);
|
||||
for x in [-4.0f64, 0.0, 3.0, 7.5] {
|
||||
let total = g.probability_below(x) + g.probability_above(x);
|
||||
assert!((total - 1.0).abs() < 1e-15, "at {x}: {total}");
|
||||
}
|
||||
}
|
||||
|
||||
/// The reason `probability_above` exists rather than `1 - probability_below`.
|
||||
#[test]
|
||||
fn probability_above_keeps_precision_where_the_complement_collapses() {
|
||||
let g = Gaussian::from_ms(0.0, 1.0);
|
||||
for (x, expected) in [(9.0f64, 1.128_588e-19), (20.0, 2.753_624e-89)] {
|
||||
let got = g.probability_above(x);
|
||||
assert!(
|
||||
(got - expected).abs() / expected < 1e-6,
|
||||
"P(X > {x}) = {got}, expected ~{expected}"
|
||||
);
|
||||
assert_eq!(
|
||||
1.0 - g.probability_below(x),
|
||||
0.0,
|
||||
"the complement should still collapse at {x}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_scaled_gaussian_shifts_and_stretches() {
|
||||
let g = Gaussian::from_ms(25.0, 6.0);
|
||||
assert!((g.probability_below(25.0) - 0.5).abs() < 1e-15);
|
||||
// One sigma either side of the mean.
|
||||
assert!((g.probability_below(31.0) - 0.841_344_746_068_543).abs() < 1e-12);
|
||||
assert!((g.probability_above(19.0) - 0.841_344_746_068_543).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_improper_gaussian_is_uninformative_rather_than_nan() {
|
||||
let improper = Gaussian::from_ms(0.0, f64::INFINITY);
|
||||
assert_eq!(improper.probability_below(5.0), 0.5);
|
||||
assert_eq!(improper.probability_above(5.0), 0.5);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user