fix!: report an unresolvable prediction grid instead of clamping
`grid_shape` asked for 12 nodes across the narrowest feature and then clamped to MAX_GRID_POINTS with no detection that the request was not met. Past `step/sigma ~ 1.7` the trapezoid rule stops resolving the density, and the result is unbounded: sigma_a step/sig_a P(a first) exact total 2.0e-3 0.86 0.515953 0.515953 1.000000 1.0e-3 1.72 0.517185 0.515953 1.002388 1.0e-4 17.17 2.791336 0.515953 5.410065 A probability of 2.79. Reachable through `predict_outcome` with a pinned reference competitor — a documented pattern — where `predict_outcome` and `predict_win_probabilities` disagreed 44x and `predict_outcome` was the wrong one. There is no useful answer on the far side of that cliff, so this reports `GridTooCoarse` rather than guessing, and the message points at `predict_win_probabilities`, which answers the same matchup through adaptive quadrature and is accurate there to 1e-13. The floor is 4 nodes per feature rather than the 12 requested, because the request carries margin: measured accurate to 2.2e-12 at 1.4 nodes per sigma and wrong by 1.2e-3 at 0.7. This also fixes the `ln k` ceiling violation. `expected_information_gain` weights `probability * divergence`, so probabilities of 3.97 and 2.62 made it return 3.237828 nats against `ln 2 = 0.693147` — 4.67x over. The crate's docs call that ceiling its sharpest test and record a prototype once returning 4.77 nats; it was live again by a different route. The new sweep then caught a second, independent defect: `kl_divergence` returned NEGATIVE values, worst -5.55e-17, exactly one ULP of its `- 1.0`. Rewritten as `0.5*(u - ln1p(u)) + gap^2/(2*var_p)` with `u = var_q/var_p - 1`, so both terms are non-negative by construction. It is also more accurate where it matters: at `u = 1e-9` the old form returned 0.0 where the true value is 2.5e-19, and well-conditioned cases are unchanged. tests/prediction_bounds.rs sweeps rather than spot-checks, because a single fixture cannot defend a bound like this — the previous check passed throughout. It asserts the sweep still reaches the coarse-grid regime, so it cannot quietly stop testing the case it was written for. BREAKING CHANGE: `predict_outcome`, `predict_ranking` and `expected_information_gain` return `GridTooCoarse` for matchups whose performance sigmas are too far apart to integrate on one grid. They previously returned wrong answers, including probabilities above 1. Closes #55, closes #56 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
+33
-2
@@ -47,7 +47,38 @@ fn kl_divergence(q: Gaussian, p: Gaussian) -> f64 {
|
||||
}
|
||||
|
||||
let mean_gap = q.mu() - p.mu();
|
||||
0.5 * (libm::log(var_p / var_q) + (var_q + mean_gap * mean_gap) / var_p - 1.0)
|
||||
|
||||
// Algebraically `0.5 * (ln(var_p/var_q) + (var_q + gap^2)/var_p - 1)`, but
|
||||
// written so that neither term can go negative.
|
||||
//
|
||||
// The direct form cancels against its `- 1.0` for two near-identical
|
||||
// distributions and returns a *negative* divergence — measured, 762 082 of
|
||||
// 3 000 000 near-identical pairs, worst `-5.55e-17`, which is exactly one
|
||||
// ULP of the 1.0. It also loses the answer entirely where it is small:
|
||||
// at `var_q/var_p - 1 = 1e-9` the direct form gives `0.0` where the true
|
||||
// value is `2.5e-19`.
|
||||
//
|
||||
// With `u = var_q/var_p - 1` the variance part is `0.5 * (u - ln(1+u))`,
|
||||
// which is non-negative for every `u > -1`, and the mean part is a square
|
||||
// over a positive variance. Non-negativity is then structural rather than
|
||||
// incidental.
|
||||
let u = var_q / var_p - 1.0;
|
||||
0.5 * u_minus_ln1p(u) + mean_gap * mean_gap / (2.0 * var_p)
|
||||
}
|
||||
|
||||
/// `u - ln(1 + u)`, without the cancellation that spelling invites.
|
||||
///
|
||||
/// Both terms are approximately `u` for small `u`, so the subtraction loses
|
||||
/// everything just where the result matters. The Taylor series
|
||||
/// `u^2/2 - u^3/3 + u^4/4 - ...` is exact in that regime and manifestly
|
||||
/// non-negative, since `u^2/2` dominates.
|
||||
fn u_minus_ln1p(u: f64) -> f64 {
|
||||
if u.abs() < 1e-4 {
|
||||
let u2 = u * u;
|
||||
u2 * (0.5 - u / 3.0 + u2 / 4.0)
|
||||
} else {
|
||||
u - libm::log1p(u)
|
||||
}
|
||||
}
|
||||
|
||||
/// Expected information gain of a hypothetical matchup, in nats.
|
||||
@@ -146,7 +177,7 @@ pub fn expected_information_gain<T: Time, D: Drift<T>>(
|
||||
|
||||
let mut gain = 0.0;
|
||||
|
||||
for (ranks, probability) in predict::outcome_distribution(&performances, &margins) {
|
||||
for (ranks, probability) in predict::outcome_distribution(&performances, &margins)? {
|
||||
if probability <= NEGLIGIBLE {
|
||||
continue;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user