tests/determinism.rs never exercises the parallel sweep it exists to prove #64

New Issue

2026-09-09T14:53:27Z

logaritmisk commented

2026-09-09 14:53:27 +00:00

The determinism test is the crate's guarantee that rayon changes nothing. It does not run the code it is guarding.

The parallel branch is unreachable in that fixture

sweep_color_groups (src/time_slice.rs:454-500) takes the par_iter branch only when group_len >= RAYON_THRESHOLD (64). The test builds 200 events at time = ev_i / 10 + 1 — 20 slices of 10 events. A color group is a subset of one slice's events, so group_len <= 10 < 64. The branch cannot be taken.

Confirmed by CPU-vs-wall time rather than by reading alone:

10 ev/slice (the test shape), RAYON_NUM_THREADS=8 : real 0.82  user 0.64  sys 0.24   <- ~1 core
500 ev/slice,                 RAYON_NUM_THREADS=1 : real 0.48  user 0.47  sys 0.02
500 ev/slice,                 RAYON_NUM_THREADS=8 : real 0.49  user 0.88  sys 1.15   <- >1 core

The residual thread activity at the test shape is log_evidence_internal's par_iter over slices, not the sweep.

Three further coverage gaps

One competitor of forty. It compares learning_curve("p0") only. A nondeterminism affecting any other competitor passes.
log_evidence is never compared — the quantity most exposed to accumulation order. Nor are ConvergenceReport::final_step or iterations.
All four thread counts run in one process, so the HashMap hasher seed is identical across them. Any nondeterminism from map iteration order is invisible to it by construction — which is exactly the defect in #62, and exactly why that one went unnoticed.

The good news

With a workload that does cross the threshold (4000 events, 200/slice, color group ~180), everything checked is bit-identical at 1/2/4/8 threads, and identical between the rayon and non-rayon builds:

ranked  digest=4ab5d3baa751487f le=c09062f06e1ec287
        step0=3e8256d868000000 step1=3e64fb25d0000000 iters=8
scored  digest=59f6c6501eca16cc le=c0caa1c6cff4f1be
        filt_le=c0cb42eeaa0852c2 iters=21
lcurves digest=e302633757831a43

All three rayon sites were also verified order-preserving by inspection: time_slice.rs:480 collects a Vec<EventUpdate> and applies it by zip over the range; history.rs:468 and :707 par_iter().map().collect() then reduce in slice order.

So the parallel path is correct — this is a test-coverage defect, not a numerical one. But the test currently proves nothing about the thing it names.

Fix

Raise the events-per-slice so color groups exceed RAYON_THRESHOLD, or lower the threshold under cfg(test). Better still, assert that the parallel branch was actually taken, so the test cannot silently stop covering it again if the threshold or the fixture changes.
Compare all competitors' curves, plus log_evidence, final_step and iterations.
Add a cross-process determinism check (spawn the same binary N times and compare f64::to_bits), since the in-process form cannot see hasher-order effects at all.

Depends on / relates to #62.

Found by a floating-point audit, 2026-09-09.

The determinism test is the crate's guarantee that `rayon` changes nothing. It does not run the code it is guarding. ## The parallel branch is unreachable in that fixture `sweep_color_groups` (`src/time_slice.rs:454-500`) takes the `par_iter` branch only when `group_len >= RAYON_THRESHOLD` (64). The test builds 200 events at `time = ev_i / 10 + 1` — 20 slices of 10 events. A color group is a subset of **one slice's** events, so `group_len <= 10 < 64`. The branch cannot be taken. Confirmed by CPU-vs-wall time rather than by reading alone: ``` 10 ev/slice (the test shape), RAYON_NUM_THREADS=8 : real 0.82 user 0.64 sys 0.24 <- ~1 core 500 ev/slice, RAYON_NUM_THREADS=1 : real 0.48 user 0.47 sys 0.02 500 ev/slice, RAYON_NUM_THREADS=8 : real 0.49 user 0.88 sys 1.15 <- >1 core ``` The residual thread activity at the test shape is `log_evidence_internal`'s `par_iter` over slices, not the sweep. ## Three further coverage gaps 1. **One competitor of forty.** It compares `learning_curve("p0")` only. A nondeterminism affecting any other competitor passes. 2. **`log_evidence` is never compared** — the quantity most exposed to accumulation order. Nor are `ConvergenceReport::final_step` or `iterations`. 3. **All four thread counts run in one process**, so the `HashMap` hasher seed is identical across them. Any nondeterminism from map iteration order is invisible to it *by construction* — which is exactly the defect in #62, and exactly why that one went unnoticed. ## The good news With a workload that **does** cross the threshold (4000 events, 200/slice, color group ~180), everything checked is bit-identical at 1/2/4/8 threads, and identical between the `rayon` and non-`rayon` builds: ``` ranked digest=4ab5d3baa751487f le=c09062f06e1ec287 step0=3e8256d868000000 step1=3e64fb25d0000000 iters=8 scored digest=59f6c6501eca16cc le=c0caa1c6cff4f1be filt_le=c0cb42eeaa0852c2 iters=21 lcurves digest=e302633757831a43 ``` All three `rayon` sites were also verified order-preserving by inspection: `time_slice.rs:480` collects a `Vec<EventUpdate>` and applies it by `zip` over the range; `history.rs:468` and `:707` `par_iter().map().collect()` then reduce in slice order. So **the parallel path is correct** — this is a test-coverage defect, not a numerical one. But the test currently proves nothing about the thing it names. ## Fix - Raise the events-per-slice so color groups exceed `RAYON_THRESHOLD`, or lower the threshold under `cfg(test)`. Better still, assert that the parallel branch was actually taken, so the test cannot silently stop covering it again if the threshold or the fixture changes. - Compare **all** competitors' curves, plus `log_evidence`, `final_step` and `iterations`. - Add a cross-**process** determinism check (spawn the same binary N times and compare `f64::to_bits`), since the in-process form cannot see hasher-order effects at all. Depends on / relates to #62. Found by a floating-point audit, 2026-09-09.

logaritmisk added the testing label 2026-09-09 14:54:12 +00:00

logaritmisk referenced this issue from a commit

2026-09-09 16:10:52 +00:00

test: make the determinism test exercise the parallel sweep

logaritmisk closed this issue

2026-09-09 16:10:52 +00:00

logaritmisk referenced this issue from a commit

2026-09-09 16:10:52 +00:00

Merge branch 'fix/numerics-critical'

Sign in to join this conversation.