fix: make posterior_of reproducible across processes

`ResolvedTerms::unseen` was a `HashMap<String, f64>` and three float
reductions iterated it. Addition is not associative and Rust seeds its
default hasher per process, so `posterior_of` returned different bits run
to run on identical input: measured over 40 processes, two distinct sigma
bit patterns, and five distinct values from `expected_variance_reduction`
spanning about 7 ULP.

A `BTreeMap` fixes it by construction. 40/40 identical after, 24/16
before.

The cross-batch conflict scan had the same cause with a different
symptom. It returns on the FIRST conflict, so hash order decided WHICH
competitor the error blamed — 15 different competitors named across 40
runs on identical input. The error fired every time; only its content was
a lottery, which sends a reader after the wrong key. Now scanned in
sorted order.

Magnitude was 1-7 ULP throughout, so no decision changes. The cost was
reproducibility: a golden test over these would flake at a low rate,
which is the worst kind of CI failure to diagnose.

tests/cross_process_determinism.rs re-executes the test binary and
compares bits, because an in-process test CANNOT see this — every sample
in one process shares one hasher seed. That is not hypothetical:
tests/determinism.rs compares four thread counts inside one process and
passed throughout while this was live.

Tuning that fixture took a measurement. Coefficients spread over nine
decades detected the bug in roughly one run in forty, because the small
terms fall below the running total's ULP and are absorbed whatever the
order. Comparable magnitudes keep every term able to change the last
bits: 5 of 5 attempts detected it, with 3 to 38 of 40 runs differing.
Verified non-vacuous by reverting the BTreeMap and watching it fail.

Closes #62

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
2026-09-09 18:02:42 +02:00
co-authored by Claude Opus 5
parent 305f822964
commit 7aa7fb62dd
2 changed files with 183 additions and 4 deletions
+26 -4
View File
@@ -1,4 +1,9 @@
use std::{borrow::Borrow, collections::HashMap, hash::Hash, marker::PhantomData};
use std::{
borrow::Borrow,
collections::{BTreeMap, HashMap},
hash::Hash,
marker::PhantomData,
};
use crate::{
BETA, GAMMA, Index, MU, P_DRAW, SIGMA,
@@ -261,7 +266,15 @@ struct ResolvedTerms {
contrast: Vec<f64>,
/// Coefficients of competitors the slice has never seen, keyed by their
/// rendering. Independent of everything in the slice by construction.
unseen: HashMap<String, f64>,
///
/// A `BTreeMap` rather than a `HashMap`, and that is load-bearing. These
/// coefficients are summed, addition is not associative, and Rust seeds its
/// default hasher per process — so iterating a `HashMap` here made
/// `posterior_of` return different bits run to run on identical input.
/// Measured over 40 processes: two distinct sigma bit patterns, and five
/// distinct values from `expected_variance_reduction` spanning ~7 ULP.
/// Ordered iteration makes the sum reproducible.
unseen: BTreeMap<String, f64>,
mean: f64,
}
@@ -1067,7 +1080,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
K: std::fmt::Debug,
{
let mut contrast = vec![0.0; width];
let mut unseen: HashMap<String, f64> = HashMap::new();
let mut unseen: BTreeMap<String, f64> = BTreeMap::new();
let mut mean = 0.0;
for (member, (key, coefficient)) in terms.iter().enumerate() {
@@ -1858,7 +1871,16 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
// batched and succeeded, order-dependently, when fed one at a time.
// Checked before anything mutates, so a rejected batch leaves the
// history untouched.
for (agent, batch) in &priors {
// Sorted, not `HashMap` order. This loop returns on the FIRST conflict
// it finds, so hash order decided *which* competitor the error blamed:
// measured, 15 different competitors named across 40 runs on identical
// input. The error fired every time — only its content was a lottery,
// which makes it unreproducible and sends a reader after the wrong key.
let mut conflict_scan: Vec<Index> = priors.keys().copied().collect();
conflict_scan.sort_unstable();
for agent in &conflict_scan {
let batch = priors[agent];
let held = self.declared.get(agent).copied().unwrap_or_default();
if let (Some(existing), Some(new)) = (held.prior, batch.prior) {