fix(history): stop reprocessing the slice that was just appended to

`add_events_with_prior` advanced `k` past the slice it had written when it
created a new one, but not when it appended to an existing one. The trailing
forward-refresh loop therefore started *on* the slice just modified and ran
`new_forward_info` over it again.

That is not merely redundant work. The loop immediately above it sets each
agent's message to `forward * likelihood` for that slice, and
`new_forward_info` then assigns `skill.forward = message.forget(drift)` —
folding the slice's own likelihood back into its own forward prior. The
skills it produced depended on how events had been batched.

Ingesting one event at a time now converges to the same fixed point as
ingesting the same events in a single call, which it previously did not:
for five events sharing a timestamp, competitor `a` converged to
mu=7.44 sigma=3.90 batched versus mu=7.99 sigma=3.10 incrementally. Both
runs had converged; the gap was not a convergence residual.

The numerical goldens never caught this because they all ingest in one call
with a distinct timestamp per event, so the append-to-existing-slice branch
is never taken. `tests/ingestion_equivalence.rs` covers it directly, and
asserts convergence before comparing so that a residual cannot be mistaken
for agreement.

Removing the redundant re-inference also removes the dominant cost of
incremental ingestion, which was quadratic in the number of events already
in the slice:

    events   before     after    speedup
       500   45.8ms     1.1ms       42x
      1000  179.5ms     2.8ms       64x
      2000  721.8ms     9.9ms       73x
      4000    2.9s     35.4ms       82x

Ingesting one at a time is now 1.8x a single batched call, down from 148x.

Two supporting changes are included:

- Color groups are rebuilt lazily rather than on every append. Nothing
  reads the partition between an append and the next full sweep, so the
  per-append rebuild was pure waste.
- `ColorGroups::groups_are_contiguous` is asserted after each rebuild and
  in `color_range`. The parallel sweep derives one `&mut` sub-slice per
  color from those ranges and relies on them being disjoint; that invariant
  was established by construction but never checked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DnsaJg74eNSva3PJjK2eej
This commit is contained in:
2026-08-04 21:51:02 +02:00
co-authored by Claude Opus 5
parent 0f1a1b8911
commit c088214fed
5 changed files with 233 additions and 17 deletions
+147
View File
@@ -0,0 +1,147 @@
//! Ingesting the same events must give the same answer however they were
//! batched.
//!
//! The numerical goldens all ingest in a single call with one slice per
//! timestamp, so they never exercise the "append to an existing slice" path.
//! These do.
use smallvec::smallvec;
use trueskill_tt::{ConvergenceOptions, Event, Gaussian, History, Member, Outcome, Team};
/// Converge tightly: the default cap of 30 iterations leaves a residual around
/// 1e-6, which would swamp the comparison. Both paths must reach the same
/// fixed point, so drive both well past it.
fn tight() -> ConvergenceOptions {
ConvergenceOptions {
max_iter: 2_000,
epsilon: 1e-12,
..ConvergenceOptions::default()
}
}
fn event(a: &str, b: &str, time: i64) -> Event<i64, String> {
Event {
time,
teams: smallvec![
Team::with_members([Member::new(a.to_string())]),
Team::with_members([Member::new(b.to_string())]),
],
outcome: Outcome::winner(0, 2),
}
}
fn converged_skills(events: Vec<Event<i64, String>>, batched: bool) -> Vec<(String, Gaussian)> {
let mut h: History<i64, _, _, String> =
History::builder_with_key().convergence(tight()).build();
if batched {
h.add_events(events).unwrap();
} else {
for ev in events {
h.add_events(std::iter::once(ev)).unwrap();
}
}
let report = h.converge().unwrap();
assert!(
report.converged,
"fixture must converge before results can be compared; final step {:?}",
report.final_step
);
let mut skills: Vec<(String, Gaussian)> = h
.learning_curves()
.into_iter()
.map(|(key, curve)| (key, curve.last().unwrap().1))
.collect();
skills.sort_by(|a, b| a.0.cmp(&b.0));
skills
}
fn assert_same(batched: &[(String, Gaussian)], incremental: &[(String, Gaussian)], what: &str) {
assert_eq!(
batched.len(),
incremental.len(),
"{what}: competitor count differs"
);
for ((kb, gb), (ki, gi)) in batched.iter().zip(incremental.iter()) {
assert_eq!(kb, ki, "{what}: key order differs");
assert!(
(gb.mu() - gi.mu()).abs() < 1e-8 && (gb.sigma() - gi.sigma()).abs() < 1e-8,
"{what}: {kb} differs — batched mu={} sigma={}, incremental mu={} sigma={}",
gb.mu(),
gb.sigma(),
gi.mu(),
gi.sigma()
);
}
}
/// All events share one timestamp, so incremental ingestion repeatedly appends
/// to an existing slice.
#[test]
fn same_slice_incremental_matches_batched() {
let events = vec![
event("a", "b", 1),
event("c", "d", 1),
event("e", "f", 1),
event("a", "c", 1),
event("b", "e", 1),
];
let batched = converged_skills(events.clone(), true);
let incremental = converged_skills(events, false);
assert_same(&batched, &incremental, "single shared slice");
}
/// Distinct timestamps, so each append lands in a fresh slice appended after
/// the existing ones.
#[test]
fn distinct_slices_incremental_matches_batched() {
let events = vec![
event("a", "b", 1),
event("b", "c", 2),
event("c", "a", 3),
event("a", "c", 4),
];
let batched = converged_skills(events.clone(), true);
let incremental = converged_skills(events, false);
assert_same(&batched, &incremental, "distinct slices");
}
/// Several events per timestamp across several timestamps — appends to
/// existing slices interleaved with new ones.
#[test]
fn mixed_slices_incremental_matches_batched() {
let events = vec![
event("a", "b", 1),
event("c", "d", 1),
event("a", "c", 2),
event("b", "d", 2),
event("a", "d", 3),
event("b", "c", 3),
];
let batched = converged_skills(events.clone(), true);
let incremental = converged_skills(events, false);
assert_same(&batched, &incremental, "mixed slices");
}
/// Appending an event to a slice that is *not* the most recent one exercises
/// the forward refresh of every later slice.
#[test]
fn back_dated_event_matches_batched() {
let events = vec![
event("a", "b", 1),
event("b", "c", 5),
event("c", "a", 9),
// arrives last, but belongs to the middle slice
event("a", "c", 5),
];
let batched = converged_skills(events.clone(), true);
let incremental = converged_skills(events, false);
assert_same(&batched, &incremental, "back-dated event");
}