fix(history): stop reprocessing the slice that was just appended to
`add_events_with_prior` advanced `k` past the slice it had written when it
created a new one, but not when it appended to an existing one. The trailing
forward-refresh loop therefore started *on* the slice just modified and ran
`new_forward_info` over it again.
That is not merely redundant work. The loop immediately above it sets each
agent's message to `forward * likelihood` for that slice, and
`new_forward_info` then assigns `skill.forward = message.forget(drift)` —
folding the slice's own likelihood back into its own forward prior. The
skills it produced depended on how events had been batched.
Ingesting one event at a time now converges to the same fixed point as
ingesting the same events in a single call, which it previously did not:
for five events sharing a timestamp, competitor `a` converged to
mu=7.44 sigma=3.90 batched versus mu=7.99 sigma=3.10 incrementally. Both
runs had converged; the gap was not a convergence residual.
The numerical goldens never caught this because they all ingest in one call
with a distinct timestamp per event, so the append-to-existing-slice branch
is never taken. `tests/ingestion_equivalence.rs` covers it directly, and
asserts convergence before comparing so that a residual cannot be mistaken
for agreement.
Removing the redundant re-inference also removes the dominant cost of
incremental ingestion, which was quadratic in the number of events already
in the slice:
events before after speedup
500 45.8ms 1.1ms 42x
1000 179.5ms 2.8ms 64x
2000 721.8ms 9.9ms 73x
4000 2.9s 35.4ms 82x
Ingesting one at a time is now 1.8x a single batched call, down from 148x.
Two supporting changes are included:
- Color groups are rebuilt lazily rather than on every append. Nothing
reads the partition between an append and the next full sweep, so the
per-append rebuild was pure waste.
- `ColorGroups::groups_are_contiguous` is asserted after each rebuild and
in `color_range`. The parallel sweep derives one `&mut` sub-slice per
color from those ranges and relies on them being disjoint; that invariant
was established by construction but never checked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DnsaJg74eNSva3PJjK2eej
This commit is contained in:
+38
-1
@@ -49,16 +49,53 @@ impl ColorGroups {
|
||||
|
||||
/// Contiguous index range for one color after events have been reordered
|
||||
/// into color-contiguous positions by `TimeSlice::recompute_color_groups`.
|
||||
#[allow(dead_code)]
|
||||
pub(crate) fn color_range(&self, color_idx: usize) -> std::ops::Range<usize> {
|
||||
let group = &self.groups[color_idx];
|
||||
if group.is_empty() {
|
||||
return 0..0;
|
||||
}
|
||||
|
||||
let start = *group.first().unwrap();
|
||||
let end = *group.last().unwrap() + 1;
|
||||
|
||||
debug_assert_eq!(
|
||||
end - start,
|
||||
group.len(),
|
||||
"color {color_idx} is not contiguous; its range would overlap other colors"
|
||||
);
|
||||
|
||||
start..end
|
||||
}
|
||||
|
||||
/// Whether every color occupies a contiguous, ascending range of event
|
||||
/// indices, and no two colors overlap.
|
||||
///
|
||||
/// The parallel sweep derives one `&mut` sub-slice per color from these
|
||||
/// ranges and relies on them being disjoint. That disjointness is what
|
||||
/// makes concurrent writes to distinct skills sound, so it is checked
|
||||
/// rather than assumed.
|
||||
pub(crate) fn groups_are_contiguous(&self) -> bool {
|
||||
let mut expected_start = 0;
|
||||
|
||||
for group in &self.groups {
|
||||
if group.is_empty() {
|
||||
continue;
|
||||
}
|
||||
|
||||
let ascending_run = group
|
||||
.iter()
|
||||
.enumerate()
|
||||
.all(|(offset, &idx)| idx == group[0] + offset);
|
||||
|
||||
if !ascending_run || group[0] != expected_start {
|
||||
return false;
|
||||
}
|
||||
|
||||
expected_start += group.len();
|
||||
}
|
||||
|
||||
true
|
||||
}
|
||||
}
|
||||
|
||||
/// Compute color groups greedily.
|
||||
|
||||
Reference in New Issue
Block a user