fix: make posterior_of reproducible across processes
`ResolvedTerms::unseen` was a `HashMap<String, f64>` and three float reductions iterated it. Addition is not associative and Rust seeds its default hasher per process, so `posterior_of` returned different bits run to run on identical input: measured over 40 processes, two distinct sigma bit patterns, and five distinct values from `expected_variance_reduction` spanning about 7 ULP. A `BTreeMap` fixes it by construction. 40/40 identical after, 24/16 before. The cross-batch conflict scan had the same cause with a different symptom. It returns on the FIRST conflict, so hash order decided WHICH competitor the error blamed — 15 different competitors named across 40 runs on identical input. The error fired every time; only its content was a lottery, which sends a reader after the wrong key. Now scanned in sorted order. Magnitude was 1-7 ULP throughout, so no decision changes. The cost was reproducibility: a golden test over these would flake at a low rate, which is the worst kind of CI failure to diagnose. tests/cross_process_determinism.rs re-executes the test binary and compares bits, because an in-process test CANNOT see this — every sample in one process shares one hasher seed. That is not hypothetical: tests/determinism.rs compares four thread counts inside one process and passed throughout while this was live. Tuning that fixture took a measurement. Coefficients spread over nine decades detected the bug in roughly one run in forty, because the small terms fall below the running total's ULP and are absorbed whatever the order. Comparable magnitudes keep every term able to change the last bits: 5 of 5 attempts detected it, with 3 to 38 of 40 runs differing. Verified non-vacuous by reverting the BTreeMap and watching it fail. Closes #62 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
+26
-4
@@ -1,4 +1,9 @@
|
||||
use std::{borrow::Borrow, collections::HashMap, hash::Hash, marker::PhantomData};
|
||||
use std::{
|
||||
borrow::Borrow,
|
||||
collections::{BTreeMap, HashMap},
|
||||
hash::Hash,
|
||||
marker::PhantomData,
|
||||
};
|
||||
|
||||
use crate::{
|
||||
BETA, GAMMA, Index, MU, P_DRAW, SIGMA,
|
||||
@@ -261,7 +266,15 @@ struct ResolvedTerms {
|
||||
contrast: Vec<f64>,
|
||||
/// Coefficients of competitors the slice has never seen, keyed by their
|
||||
/// rendering. Independent of everything in the slice by construction.
|
||||
unseen: HashMap<String, f64>,
|
||||
///
|
||||
/// A `BTreeMap` rather than a `HashMap`, and that is load-bearing. These
|
||||
/// coefficients are summed, addition is not associative, and Rust seeds its
|
||||
/// default hasher per process — so iterating a `HashMap` here made
|
||||
/// `posterior_of` return different bits run to run on identical input.
|
||||
/// Measured over 40 processes: two distinct sigma bit patterns, and five
|
||||
/// distinct values from `expected_variance_reduction` spanning ~7 ULP.
|
||||
/// Ordered iteration makes the sum reproducible.
|
||||
unseen: BTreeMap<String, f64>,
|
||||
mean: f64,
|
||||
}
|
||||
|
||||
@@ -1067,7 +1080,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
|
||||
K: std::fmt::Debug,
|
||||
{
|
||||
let mut contrast = vec![0.0; width];
|
||||
let mut unseen: HashMap<String, f64> = HashMap::new();
|
||||
let mut unseen: BTreeMap<String, f64> = BTreeMap::new();
|
||||
let mut mean = 0.0;
|
||||
|
||||
for (member, (key, coefficient)) in terms.iter().enumerate() {
|
||||
@@ -1858,7 +1871,16 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
|
||||
// batched and succeeded, order-dependently, when fed one at a time.
|
||||
// Checked before anything mutates, so a rejected batch leaves the
|
||||
// history untouched.
|
||||
for (agent, batch) in &priors {
|
||||
// Sorted, not `HashMap` order. This loop returns on the FIRST conflict
|
||||
// it finds, so hash order decided *which* competitor the error blamed:
|
||||
// measured, 15 different competitors named across 40 runs on identical
|
||||
// input. The error fired every time — only its content was a lottery,
|
||||
// which makes it unreproducible and sends a reader after the wrong key.
|
||||
let mut conflict_scan: Vec<Index> = priors.keys().copied().collect();
|
||||
conflict_scan.sort_unstable();
|
||||
|
||||
for agent in &conflict_scan {
|
||||
let batch = priors[agent];
|
||||
let held = self.declared.get(agent).copied().unwrap_or_default();
|
||||
|
||||
if let (Some(existing), Some(new)) = (held.prior, batch.prior) {
|
||||
|
||||
Reference in New Issue
Block a user