logaritmiskandClaude Opus 5
c088214fed
`add_events_with_prior` advanced `k` past the slice it had written when it
created a new one, but not when it appended to an existing one. The trailing
forward-refresh loop therefore started *on* the slice just modified and ran
`new_forward_info` over it again.
That is not merely redundant work. The loop immediately above it sets each
agent's message to `forward * likelihood` for that slice, and
`new_forward_info` then assigns `skill.forward = message.forget(drift)` —
folding the slice's own likelihood back into its own forward prior. The
skills it produced depended on how events had been batched.
Ingesting one event at a time now converges to the same fixed point as
ingesting the same events in a single call, which it previously did not:
for five events sharing a timestamp, competitor `a` converged to
mu=7.44 sigma=3.90 batched versus mu=7.99 sigma=3.10 incrementally. Both
runs had converged; the gap was not a convergence residual.
The numerical goldens never caught this because they all ingest in one call
with a distinct timestamp per event, so the append-to-existing-slice branch
is never taken. `tests/ingestion_equivalence.rs` covers it directly, and
asserts convergence before comparing so that a residual cannot be mistaken
for agreement.
Removing the redundant re-inference also removes the dominant cost of
incremental ingestion, which was quadratic in the number of events already
in the slice:
events before after speedup
500 45.8ms 1.1ms 42x
1000 179.5ms 2.8ms 64x
2000 721.8ms 9.9ms 73x
4000 2.9s 35.4ms 82x
Ingesting one at a time is now 1.8x a single batched call, down from 148x.
Two supporting changes are included:
- Color groups are rebuilt lazily rather than on every append. Nothing
reads the partition between an append and the next full sweep, so the
per-append rebuild was pure waste.
- `ColorGroups::groups_are_contiguous` is asserted after each rebuild and
in `color_range`. The parallel sweep derives one `&mut` sub-slice per
color from those ranges and relies on them being disjoint; that invariant
was established by construction but never checked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DnsaJg74eNSva3PJjK2eej
82 lines
1.8 KiB
Rust
82 lines
1.8 KiB
Rust
use std::{
|
|
borrow::{Borrow, ToOwned},
|
|
collections::HashMap,
|
|
hash::Hash,
|
|
};
|
|
|
|
use crate::Index;
|
|
|
|
/// Maps user keys to internal `Index` handles.
|
|
///
|
|
/// Renamed from the former `IndexMap` to avoid colliding with the `indexmap`
|
|
/// crate. Power users can promote `&K` to `Index` via `get_or_create` and
|
|
/// skip the lookup on subsequent hot-path calls.
|
|
#[derive(Debug)]
|
|
pub struct KeyTable<K> {
|
|
forward: HashMap<K, Index>,
|
|
/// Reverse mapping, indexed by `Index.0`.
|
|
///
|
|
/// Indices are handed out densely and sequentially, so position *is* the
|
|
/// index and `key()` is a lookup rather than a scan over every entry.
|
|
reverse: Vec<K>,
|
|
}
|
|
|
|
impl<K> KeyTable<K>
|
|
where
|
|
K: Eq + Hash + Clone,
|
|
{
|
|
pub fn new() -> Self {
|
|
Self {
|
|
forward: HashMap::new(),
|
|
reverse: Vec::new(),
|
|
}
|
|
}
|
|
|
|
pub fn get<Q: ?Sized + Hash + Eq>(&self, k: &Q) -> Option<Index>
|
|
where
|
|
K: Borrow<Q>,
|
|
{
|
|
self.forward.get(k).cloned()
|
|
}
|
|
|
|
pub fn get_or_create<Q: ?Sized + Hash + Eq + ToOwned<Owned = K>>(&mut self, k: &Q) -> Index
|
|
where
|
|
K: Borrow<Q>,
|
|
{
|
|
if let Some(idx) = self.forward.get(k) {
|
|
*idx
|
|
} else {
|
|
let idx = Index::from(self.reverse.len());
|
|
let owned = k.to_owned();
|
|
self.reverse.push(owned.clone());
|
|
self.forward.insert(owned, idx);
|
|
idx
|
|
}
|
|
}
|
|
|
|
pub fn key(&self, idx: Index) -> Option<&K> {
|
|
self.reverse.get(idx.0)
|
|
}
|
|
|
|
pub fn keys(&self) -> impl Iterator<Item = &K> {
|
|
self.forward.keys()
|
|
}
|
|
|
|
pub fn len(&self) -> usize {
|
|
self.reverse.len()
|
|
}
|
|
|
|
pub fn is_empty(&self) -> bool {
|
|
self.reverse.is_empty()
|
|
}
|
|
}
|
|
|
|
impl<K> Default for KeyTable<K>
|
|
where
|
|
K: Eq + Hash + Clone,
|
|
{
|
|
fn default() -> Self {
|
|
KeyTable::new()
|
|
}
|
|
}
|