fix: stop destroying tail precision in evidence and truncation

`erfc` is sound — it holds ~1e-7 *relative* accuracy down to 1e-296 with
no tail degradation. Three expressions built on it threw that away by
subtracting quantities that both approach the same value.

1. `cavity_evidence` computed `1.0 - cdf(margin, ..)`, which is
   algebraically `sf(margin, ..)` and numerically a catastrophe: 7%
   error by eight sigma, and exactly zero past ~8.3, where the true
   probability is 1e-19 and perfectly representable. Clamped, that
   reached `log_evidence` as ln(f64::MIN_POSITIVE) = -708 whatever the
   truth was — off by 665 nats at nine sigma.

   `1 - cdf` is smallest precisely when the result contradicts the
   prior, so the model-comparison number was worst for upsets: the
   observation it exists to notice. Adds `sf`, the survival function,
   computed without the subtraction. The tie branch picks whichever tail
   keeps both of its terms small, for the same reason.

2. `v_w` computed the inverse Mills ratio as `pdf(-a) / cdf(-a)`. Both
   underflow together past about 39 sigma, giving `0 / 0` and putting
   NaN straight into the posterior. Adds `erfcx`, so the shared
   `exp(-alpha^2 / 2)` cancels analytically instead of being evaluated
   twice and divided.

3. With that fixed, `w = v * (v - alpha)` became the next casualty: `v`
   tends to `alpha`, so the gap lost every digit and drove `w` above 1,
   making `sqrt(1 - w)` NaN at alpha = 1e6. The gap now comes from its
   asymptotic series, which forms no difference at all. The tie branch
   had the same defect one expression over — `v * v - u` with both terms
   at 1e18 returned w = -128 — and a far-tail window is
   indistinguishable from a half-line, so it shares the asymptotic.

No public signature changes, and no existing golden moved: every one of
these only alters regions the old code got wrong. The two identity tests
are asserted at 1e-6 rather than tighter because `erfc` is not exactly
antisymmetric — `erfc(z) + erfc(-z)` differs from 2 by ~3e-8, and
`erfc(0)` returns 1.00000003. That floor is tracked separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
2026-09-07 15:28:35 +02:00
co-authored by Claude Opus 5
parent 2fff745c3b
commit 7341669d1a
2 changed files with 354 additions and 15 deletions
+67 -7
View File
@@ -2,6 +2,7 @@ use crate::{
N_INF, approx, cdf,
factor::{Factor, VarId, VarStore},
gaussian::Gaussian,
sf,
};
/// EP truncation factor on a diff variable.
@@ -74,16 +75,29 @@ impl Factor for TruncFactor {
/// P(diff > margin) for non-tie, P(|diff| < margin) for tie.
///
/// Clamped to a positive floor: for a near-certain outcome the tail rounds to
/// exactly 0.0, and the `erfc` approximation used by `cdf` carries ~1e-7 error
/// so it can even return slightly more than 1.0, making the difference
/// negative. Either would send `log_evidence` to `-inf` or NaN and poison the
/// sum across the whole history.
/// Both branches pick whichever tail keeps their terms *small*, because the
/// alternative is subtracting two numbers that both approach 1. That
/// subtraction is not a rounding detail: it loses every digit of an unlikely
/// outcome's evidence, and an unlikely outcome is precisely the one worth
/// scoring. `1 - cdf` returned exactly zero past ~8.3 sigma, where the true
/// probability is 1e-19; clamped, that reached `log_evidence` as -708 instead
/// of -43.
///
/// The clamp remains as a guard rather than a workaround: `erfc` carries ~1e-7
/// relative error, so a probability of exactly 1 can still come back a hair
/// above it, and `ln` of a negative would poison the sum for the whole history.
fn cavity_evidence(diff: Gaussian, margin: f64, tie: bool) -> f64 {
let (mu, sigma) = (diff.mu(), diff.sigma());
let raw = if tie {
cdf(margin, diff.mu(), diff.sigma()) - cdf(-margin, diff.mu(), diff.sigma())
if mu < -margin {
// Both CDFs sit against 1 here; both survival terms are small.
sf(-margin, mu, sigma) - sf(margin, mu, sigma)
} else {
cdf(margin, mu, sigma) - cdf(-margin, mu, sigma)
}
} else {
1.0 - cdf(margin, diff.mu(), diff.sigma())
sf(margin, mu, sigma)
};
raw.clamp(f64::MIN_POSITIVE, 1.0)
@@ -132,6 +146,52 @@ mod tests {
assert_eq!(f.evidence_cached.unwrap(), first);
}
/// The defect this guards: `1 - cdf` collapsed to zero for a surprising
/// result, the clamp turned that into `f64::MIN_POSITIVE`, and
/// `log_evidence` reported ln of *that* — about -708 whatever the truth
/// was. An upset is the observation a model-comparison score exists to
/// notice, so it was wrong exactly where it mattered.
#[test]
fn evidence_of_an_upset_is_not_flattened_to_the_clamp_floor() {
// diff ~ N(-9, 1) with margin 0: the favoured side lost by nine sigma.
let evidence = cavity_evidence(Gaussian::from_ms(-9.0, 1.0), 0.0, false);
assert!(
evidence > f64::MIN_POSITIVE,
"evidence collapsed onto the clamp floor: {evidence}"
);
// P(X > 0) for X ~ N(-9, 1) is the standard normal tail at 9 sigma.
assert!(
(evidence - 1.128_588e-19).abs() / 1.128_588e-19 < 1e-6,
"expected ~1.13e-19, got {evidence}"
);
assert!(
(evidence.ln() + 43.628).abs() < 1e-2,
"log evidence {} should be about -43.6, not -708",
evidence.ln()
);
}
/// Evidence must stay finite and positive however extreme the mismatch,
/// since `log_evidence` sums across the whole history and one `-inf` or
/// `NaN` poisons all of it.
#[test]
fn evidence_stays_positive_and_finite_at_any_separation() {
for mu in [-300.0f64, -50.0, -9.0, 0.0, 9.0, 50.0, 300.0] {
for tie in [false, true] {
let e = cavity_evidence(Gaussian::from_ms(mu, 1.0), 1.0, tie);
assert!(
e.is_finite() && e > 0.0 && e <= 1.0,
"mu={mu} tie={tie}: evidence {e} is not a probability"
);
assert!(
e.ln().is_finite(),
"mu={mu} tie={tie}: ln evidence is not finite"
);
}
}
}
#[test]
fn tie_evidence_uses_two_sided() {
let mut vars = VarStore::new();