Truncation 1 - w cancellation gives NaN skills from plausible parameters #60

New Issue

2026-09-09T14:52:04Z

logaritmisk commented

2026-09-09 14:52:04 +00:00

src/lib.rs:551 (tie branch) and :561. w is formed by subtraction and 1 - w then loses roughly log10(alpha^2) digits. The escape hatch at :526 keys on alpha * (beta - alpha) >= 10 — distance from the mean in window widths — rather than on the actual precision loss, so a narrow window far in the tail falls straight through it.

Measured, public API

Reproduced independently. Parameters are ordinary for a precise-scoring domain — no extreme constants:

beta=1e-2  p_draw=1e-8  sd=1e-2  gap=10  draw  ->  current_skill = (NaN, NaN)
beta=1e-3  p_draw=1e-9  sd=1e-3  gap=1   draw  ->  current_skill = (NaN, NaN)
beta=1e-3  p_draw=1e-6  sd=1e-3  gap=1   draw  ->  converges fine
defaults   (beta=25/6, p_draw=0.1)             ->  converges fine

So it is a genuine cliff rather than "absurd inputs break": widening the draw window by three orders of magnitude fixes it, and the neighbouring configuration is healthy.

Isolated intermediates for v_w(mu=-1e6, sigma=1, margin=1e-6, tie=true):

erfcx(a) - scale*erfcx(b)   keeps 15.6 digits    <- fine
1 - scale                   keeps 15.9 digits    <- fine
v^2 - u = 9.99999999999374146e11 - 9.99999999998373901e11
                            keeps  4.0 digits    <- the killer
=> 1 - w    = -2.441406e-4   (reference: +2.75941791e-13)
=> trunc(.) = (mu_t = -3.13e-7, sigma_t = NaN)

One step earlier it is silently wrong rather than NaN, which is worse: at mu=-1e8, margin=1e-8 it returns 1 - w = 1.0 exactly — the truncation reported as a no-op — where the truth is 4.99e-17. Sixteen orders of magnitude out, finite, and it passes every existing assertion.

The non-tie branch collapses too

sigma * (1.0 - w).sqrt() at sigma=1, margin=0:

alpha	`1-w` computed	reference	rel err	`sigma_trunc`
1e3	9.999940e-7	9.999940e-7	1.1e-10	ok
1e5	9.999990e-11	1.000000e-10	1.0e-6	ok
1e7	9.992007e-15	1.000000e-14	8.0e-4	ok
1e8	0.0	1.000000e-16	1.0	0.0

sigma_trunc == 0 makes approx() return from_ms(mu, 0.0), i.e. {pi: inf, tau: inf}, whose mu() is inf/inf = NaN.

Public: History::builder().beta(1e-8).sigma(1e-8) with priors 1000 apart gives Err and current_skill = (NaN, NaN).

Fix direction, measured

Return 1 - w directly from half_line_truncation instead of letting the caller subtract — the asymptotic form inv^2 * [(1 - inv^2(10 - 74 inv^2)) + 2 inv^2 B - inv^4 B^2] with B = 2 - inv^2(10 - 74 inv^2):

alpha	current rel err	direct rel err
1e4	1.6e-9	1.7e-16
1e6	8.9e-5	8.1e-16
1e8	1.0 (returns 0)	2.5e-16
1e15	1.0 (returns 0)	5.3e-16

The alpha = 100 crossover point itself is not misplaced — both sides agree to 7e-10 at 99/100/101, so this is about what the function returns, not where it switches.

The escape-hatch condition is the wrong invariant. The loss is governed by alpha^2 * eps against the true 1 - w, not by how many window-widths from the mean the window sits.

Why the existing guard misses it

truncation_stays_finite_arbitrarily_far_into_the_tail (lib.rs:820) fixes margin = 1.0 for the tie case, so alpha * (beta - alpha) = 2*alpha >= 10 always holds and the escape hatch always fires. That test never visits this branch. A test that sweeps margin as well as mean would have caught it.

Also in this branch, v.powi(2) overflows outright: v_w(-1e150, 1.0, 1e-150, true) returns (NaN, NaN).

Found by a floating-point audit, 2026-09-09. Public reproduction verified independently.

`src/lib.rs:551` (tie branch) and `:561`. `w` is formed by subtraction and `1 - w` then loses roughly `log10(alpha^2)` digits. The escape hatch at `:526` keys on `alpha * (beta - alpha) >= 10` — distance from the mean in window widths — rather than on the actual precision loss, so a **narrow window far in the tail** falls straight through it. ## Measured, public API Reproduced independently. Parameters are ordinary for a precise-scoring domain — no extreme constants: ``` beta=1e-2 p_draw=1e-8 sd=1e-2 gap=10 draw -> current_skill = (NaN, NaN) beta=1e-3 p_draw=1e-9 sd=1e-3 gap=1 draw -> current_skill = (NaN, NaN) beta=1e-3 p_draw=1e-6 sd=1e-3 gap=1 draw -> converges fine defaults (beta=25/6, p_draw=0.1) -> converges fine ``` So it is a genuine cliff rather than "absurd inputs break": widening the draw window by three orders of magnitude fixes it, and the neighbouring configuration is healthy. Isolated intermediates for `v_w(mu=-1e6, sigma=1, margin=1e-6, tie=true)`: ``` erfcx(a) - scale*erfcx(b) keeps 15.6 digits <- fine 1 - scale keeps 15.9 digits <- fine v^2 - u = 9.99999999999374146e11 - 9.99999999998373901e11 keeps 4.0 digits <- the killer => 1 - w = -2.441406e-4 (reference: +2.75941791e-13) => trunc(.) = (mu_t = -3.13e-7, sigma_t = NaN) ``` **One step earlier it is silently wrong rather than NaN**, which is worse: at `mu=-1e8, margin=1e-8` it returns `1 - w = 1.0` exactly — the truncation reported as a no-op — where the truth is `4.99e-17`. Sixteen orders of magnitude out, finite, and it passes every existing assertion. ## The non-tie branch collapses too `sigma * (1.0 - w).sqrt()` at `sigma=1`, `margin=0`: | alpha | `1-w` computed | reference | rel err | `sigma_trunc` | |---|---|---|---|---| | 1e3 | 9.999940e-7 | 9.999940e-7 | 1.1e-10 | ok | | 1e5 | 9.999990e-11 | 1.000000e-10 | 1.0e-6 | ok | | 1e7 | 9.992007e-15 | 1.000000e-14 | 8.0e-4 | ok | | 1e8 | **0.0** | 1.000000e-16 | **1.0** | **0.0** | `sigma_trunc == 0` makes `approx()` return `from_ms(mu, 0.0)`, i.e. `{pi: inf, tau: inf}`, whose `mu()` is `inf/inf = NaN`. Public: `History::builder().beta(1e-8).sigma(1e-8)` with priors 1000 apart gives `Err` and `current_skill = (NaN, NaN)`. ## Fix direction, measured Return `1 - w` directly from `half_line_truncation` instead of letting the caller subtract — the asymptotic form `inv^2 * [(1 - inv^2(10 - 74 inv^2)) + 2 inv^2 B - inv^4 B^2]` with `B = 2 - inv^2(10 - 74 inv^2)`: | alpha | current rel err | direct rel err | |---|---|---| | 1e4 | 1.6e-9 | **1.7e-16** | | 1e6 | 8.9e-5 | **8.1e-16** | | 1e8 | 1.0 (returns 0) | **2.5e-16** | | 1e15 | 1.0 (returns 0) | **5.3e-16** | The `alpha = 100` crossover point itself is **not** misplaced — both sides agree to 7e-10 at 99/100/101, so this is about what the function returns, not where it switches. The escape-hatch condition is the wrong invariant. The loss is governed by `alpha^2 * eps` against the true `1 - w`, not by how many window-widths from the mean the window sits. ## Why the existing guard misses it `truncation_stays_finite_arbitrarily_far_into_the_tail` (`lib.rs:820`) fixes `margin = 1.0` for the tie case, so `alpha * (beta - alpha) = 2*alpha >= 10` always holds and the escape hatch always fires. **That test never visits this branch.** A test that sweeps margin as well as mean would have caught it. Also in this branch, `v.powi(2)` overflows outright: `v_w(-1e150, 1.0, 1e-150, true)` returns `(NaN, NaN)`. Found by a floating-point audit, 2026-09-09. Public reproduction verified independently.

logaritmisk added the bug numerics labels 2026-09-09 14:53:59 +00:00

logaritmisk referenced this issue from a commit

2026-09-09 16:10:51 +00:00

fix: keep the truncated variance representable in the far tail

logaritmisk closed this issue

2026-09-09 16:10:51 +00:00

logaritmisk referenced this issue from a commit

2026-09-09 16:10:52 +00:00

Merge branch 'fix/numerics-critical'

Sign in to join this conversation.