fix: stop destroying tail precision in evidence and truncation
`erfc` is sound — it holds ~1e-7 *relative* accuracy down to 1e-296 with no tail degradation. Three expressions built on it threw that away by subtracting quantities that both approach the same value. 1. `cavity_evidence` computed `1.0 - cdf(margin, ..)`, which is algebraically `sf(margin, ..)` and numerically a catastrophe: 7% error by eight sigma, and exactly zero past ~8.3, where the true probability is 1e-19 and perfectly representable. Clamped, that reached `log_evidence` as ln(f64::MIN_POSITIVE) = -708 whatever the truth was — off by 665 nats at nine sigma. `1 - cdf` is smallest precisely when the result contradicts the prior, so the model-comparison number was worst for upsets: the observation it exists to notice. Adds `sf`, the survival function, computed without the subtraction. The tie branch picks whichever tail keeps both of its terms small, for the same reason. 2. `v_w` computed the inverse Mills ratio as `pdf(-a) / cdf(-a)`. Both underflow together past about 39 sigma, giving `0 / 0` and putting NaN straight into the posterior. Adds `erfcx`, so the shared `exp(-alpha^2 / 2)` cancels analytically instead of being evaluated twice and divided. 3. With that fixed, `w = v * (v - alpha)` became the next casualty: `v` tends to `alpha`, so the gap lost every digit and drove `w` above 1, making `sqrt(1 - w)` NaN at alpha = 1e6. The gap now comes from its asymptotic series, which forms no difference at all. The tie branch had the same defect one expression over — `v * v - u` with both terms at 1e18 returned w = -128 — and a far-tail window is indistinguishable from a half-line, so it shares the asymptotic. No public signature changes, and no existing golden moved: every one of these only alters regions the old code got wrong. The two identity tests are asserted at 1e-6 rather than tighter because `erfc` is not exactly antisymmetric — `erfc(z) + erfc(-z)` differs from 2 by ~3e-8, and `erfc(0)` returns 1.00000003. That floor is tracked separately. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
+67
-7
@@ -2,6 +2,7 @@ use crate::{
|
||||
N_INF, approx, cdf,
|
||||
factor::{Factor, VarId, VarStore},
|
||||
gaussian::Gaussian,
|
||||
sf,
|
||||
};
|
||||
|
||||
/// EP truncation factor on a diff variable.
|
||||
@@ -74,16 +75,29 @@ impl Factor for TruncFactor {
|
||||
|
||||
/// P(diff > margin) for non-tie, P(|diff| < margin) for tie.
|
||||
///
|
||||
/// Clamped to a positive floor: for a near-certain outcome the tail rounds to
|
||||
/// exactly 0.0, and the `erfc` approximation used by `cdf` carries ~1e-7 error
|
||||
/// so it can even return slightly more than 1.0, making the difference
|
||||
/// negative. Either would send `log_evidence` to `-inf` or NaN and poison the
|
||||
/// sum across the whole history.
|
||||
/// Both branches pick whichever tail keeps their terms *small*, because the
|
||||
/// alternative is subtracting two numbers that both approach 1. That
|
||||
/// subtraction is not a rounding detail: it loses every digit of an unlikely
|
||||
/// outcome's evidence, and an unlikely outcome is precisely the one worth
|
||||
/// scoring. `1 - cdf` returned exactly zero past ~8.3 sigma, where the true
|
||||
/// probability is 1e-19; clamped, that reached `log_evidence` as -708 instead
|
||||
/// of -43.
|
||||
///
|
||||
/// The clamp remains as a guard rather than a workaround: `erfc` carries ~1e-7
|
||||
/// relative error, so a probability of exactly 1 can still come back a hair
|
||||
/// above it, and `ln` of a negative would poison the sum for the whole history.
|
||||
fn cavity_evidence(diff: Gaussian, margin: f64, tie: bool) -> f64 {
|
||||
let (mu, sigma) = (diff.mu(), diff.sigma());
|
||||
|
||||
let raw = if tie {
|
||||
cdf(margin, diff.mu(), diff.sigma()) - cdf(-margin, diff.mu(), diff.sigma())
|
||||
if mu < -margin {
|
||||
// Both CDFs sit against 1 here; both survival terms are small.
|
||||
sf(-margin, mu, sigma) - sf(margin, mu, sigma)
|
||||
} else {
|
||||
cdf(margin, mu, sigma) - cdf(-margin, mu, sigma)
|
||||
}
|
||||
} else {
|
||||
1.0 - cdf(margin, diff.mu(), diff.sigma())
|
||||
sf(margin, mu, sigma)
|
||||
};
|
||||
|
||||
raw.clamp(f64::MIN_POSITIVE, 1.0)
|
||||
@@ -132,6 +146,52 @@ mod tests {
|
||||
assert_eq!(f.evidence_cached.unwrap(), first);
|
||||
}
|
||||
|
||||
/// The defect this guards: `1 - cdf` collapsed to zero for a surprising
|
||||
/// result, the clamp turned that into `f64::MIN_POSITIVE`, and
|
||||
/// `log_evidence` reported ln of *that* — about -708 whatever the truth
|
||||
/// was. An upset is the observation a model-comparison score exists to
|
||||
/// notice, so it was wrong exactly where it mattered.
|
||||
#[test]
|
||||
fn evidence_of_an_upset_is_not_flattened_to_the_clamp_floor() {
|
||||
// diff ~ N(-9, 1) with margin 0: the favoured side lost by nine sigma.
|
||||
let evidence = cavity_evidence(Gaussian::from_ms(-9.0, 1.0), 0.0, false);
|
||||
|
||||
assert!(
|
||||
evidence > f64::MIN_POSITIVE,
|
||||
"evidence collapsed onto the clamp floor: {evidence}"
|
||||
);
|
||||
// P(X > 0) for X ~ N(-9, 1) is the standard normal tail at 9 sigma.
|
||||
assert!(
|
||||
(evidence - 1.128_588e-19).abs() / 1.128_588e-19 < 1e-6,
|
||||
"expected ~1.13e-19, got {evidence}"
|
||||
);
|
||||
assert!(
|
||||
(evidence.ln() + 43.628).abs() < 1e-2,
|
||||
"log evidence {} should be about -43.6, not -708",
|
||||
evidence.ln()
|
||||
);
|
||||
}
|
||||
|
||||
/// Evidence must stay finite and positive however extreme the mismatch,
|
||||
/// since `log_evidence` sums across the whole history and one `-inf` or
|
||||
/// `NaN` poisons all of it.
|
||||
#[test]
|
||||
fn evidence_stays_positive_and_finite_at_any_separation() {
|
||||
for mu in [-300.0f64, -50.0, -9.0, 0.0, 9.0, 50.0, 300.0] {
|
||||
for tie in [false, true] {
|
||||
let e = cavity_evidence(Gaussian::from_ms(mu, 1.0), 1.0, tie);
|
||||
assert!(
|
||||
e.is_finite() && e > 0.0 && e <= 1.0,
|
||||
"mu={mu} tie={tie}: evidence {e} is not a probability"
|
||||
);
|
||||
assert!(
|
||||
e.ln().is_finite(),
|
||||
"mu={mu} tie={tie}: ln evidence is not finite"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tie_evidence_uses_two_sided() {
|
||||
let mut vars = VarStore::new();
|
||||
|
||||
Reference in New Issue
Block a user