Ingestion discards all convergence state, so appending one event costs a full re-converge #32

New Issue

2026-08-27T21:50:42Z

logaritmisk commented

2026-08-27 21:50:42 +00:00

Split out of #4, which measured this while looking for something else.

Appending a single event to a fully converged history costs a complete re-convergence — no cheaper than converging the history from cold.

Measurement

2000 events / 200 competitors / 20 per slice, release, counting slice-sweeps performed by converge:

	slice-sweeps
cold converge	2772
converged, append one event, re-converge	2800

Both take 14 iterations. The append buys nothing.

Why

add_events_with_prior (src/history.rs) begins by discarding every competitor's accumulated state:

competitor::clean(self.agents.values_mut(), true);   // src/history.rs:714

clean with last_time = true sets message = None and last_time = None on every competitor in the history — not just those in the appended event. It then walks the slices, re-running new_forward_info on each (src/history.rs:775, :849), which reinstalls forward and runs a full internal EP sweep per slice.

So by the time converge is called, every slice has been re-primed from the prior and every posterior has moved. There is nothing left of the previous convergence to reuse.

Why it matters

This is the real cost behind the online-add scenario the T3 spec projected at 5-50× (docs/superpowers/specs/2026-04-23-trueskill-engine-redesign-design.md §5). #4 attributed that cost to redundant slice sweeps within an iteration and proposed dirty bits to skip them; measurement showed only ~6% of sweeps are skippable even in principle, because the bottleneck is here instead.

It also shapes how the crate is usable. ustat ingests rounds incrementally; every append currently re-converges the entire history, so cost per append scales with total history size rather than with what changed.

Sketch

The state that gets discarded is per-competitor message / last_time, and it is discarded because ingestion cannot easily tell which competitors an append actually disturbs. Two directions, neither costed yet:

Scope the reset. Clear only competitors reachable from the appended events — the participants, plus whoever shares a slice with them from the earliest touched slice onward. Everyone earlier keeps their forward state.
Re-prime lazily. Leave the state alone at ingestion and let converge re-derive what it needs, so the cost is paid once during convergence rather than eagerly for every slice on every append.

Both need the correctness argument done properly first: the current unconditional reset is sound, and anything narrower has to prove it does not leave a stale message feeding a slice that should have seen a fresh one. #4's post-mortem is the cautionary tale — the agents map is a relay across loop steps, not a per-step feed, and reasoning about it locally produces plausible-looking wrong answers.

Acceptance

Appending one event to a converged history performs materially fewer slice-sweeps than a cold converge of the same history, measured as above.
Converged posteriors after an incremental append match a cold converge of the same event set to within the existing test epsilons — tests/ingestion_equivalence.rs already pins exactly this and must stay green.
A benchmark covers the online-add path so the improvement cannot silently regress. benches/ingest.rs covers incremental ingestion; this needs the re-converge.

Split out of #4, which measured this while looking for something else. Appending a single event to a fully converged history costs a **complete** re-convergence — no cheaper than converging the history from cold. ## Measurement 2000 events / 200 competitors / 20 per slice, release, counting slice-sweeps performed by `converge`: | | slice-sweeps | |---|---:| | cold converge | 2772 | | converged, append one event, re-converge | 2800 | Both take 14 iterations. The append buys nothing. ## Why `add_events_with_prior` (`src/history.rs`) begins by discarding every competitor's accumulated state: ```rust competitor::clean(self.agents.values_mut(), true); // src/history.rs:714 ``` `clean` with `last_time = true` sets `message = None` **and** `last_time = None` on every competitor in the history — not just those in the appended event. It then walks the slices, re-running `new_forward_info` on each (`src/history.rs:775`, `:849`), which reinstalls `forward` and runs a full internal EP sweep per slice. So by the time `converge` is called, every slice has been re-primed from the prior and every posterior has moved. There is nothing left of the previous convergence to reuse. ## Why it matters This is the real cost behind the online-add scenario the T3 spec projected at 5-50× (`docs/superpowers/specs/2026-04-23-trueskill-engine-redesign-design.md` §5). #4 attributed that cost to redundant slice sweeps *within* an iteration and proposed dirty bits to skip them; measurement showed only ~6% of sweeps are skippable even in principle, because the bottleneck is here instead. It also shapes how the crate is usable. [ustat](https://git.aceofba.se/logaritmisk/ustat) ingests rounds incrementally; every append currently re-converges the entire history, so cost per append scales with total history size rather than with what changed. ## Sketch The state that gets discarded is per-competitor `message` / `last_time`, and it is discarded because ingestion cannot easily tell which competitors an append actually disturbs. Two directions, neither costed yet: - **Scope the reset.** Clear only competitors reachable from the appended events — the participants, plus whoever shares a slice with them from the earliest touched slice onward. Everyone earlier keeps their forward state. - **Re-prime lazily.** Leave the state alone at ingestion and let `converge` re-derive what it needs, so the cost is paid once during convergence rather than eagerly for every slice on every append. Both need the correctness argument done properly first: the current unconditional reset is *sound*, and anything narrower has to prove it does not leave a stale `message` feeding a slice that should have seen a fresh one. #4's post-mortem is the cautionary tale — the `agents` map is a relay across loop steps, not a per-step feed, and reasoning about it locally produces plausible-looking wrong answers. ## Acceptance - Appending one event to a converged history performs materially fewer slice-sweeps than a cold converge of the same history, measured as above. - Converged posteriors after an incremental append match a cold converge of the same event set to within the existing test epsilons — `tests/ingestion_equivalence.rs` already pins exactly this and must stay green. - A benchmark covers the online-add path so the improvement cannot silently regress. `benches/ingest.rs` covers incremental *ingestion*; this needs the re-*converge*.

logaritmisk commented

2026-08-27 22:18:21 +00:00

Spiked. The premise of this issue is wrong — the re-convergence is necessary work, not waste. Recommending we close it.

I filed this off the observation that appending one event costs a full re-converge. That observation is correct; the inference that it is avoidable is not.

Measurements

2000 events / 200 competitors / 20 per slice, release.

1. Does the machinery exploit a warm start at all?

NO-OP RE-CONVERGE (already converged, nothing appended)
  iterations 1   final_step (1.18e-7, 6.19e-8)

Yes, perfectly. Converging an already-converged history costs one iteration. There is no "it always grinds through 14 iterations regardless" defect. The loop terminates immediately when nothing moved.

2. Does preserving slice state across an append help?

Patched ingestion to skip the two new_forward_info re-primes, so every slice keeps its converged forward/likelihood. Note competitor::clean wipes agents[].message but not the slices' own state — the re-primes are what destroy it, so this is the minimal experiment.

                        re-prime on    re-prime skipped
cold converge                    14                  13
re-converge after append         14                  13

~7%. Preserving state buys one iteration, and the re-converge still costs the same as a cold converge either way.

3. Why — how far does one appended event actually reach?

Shift in each competitor's first time point after appending one event at the end:

top 5 movers: p0=5.58, p1=5.54, p108=0.78, p117=0.77, p109=0.72
mean shift 0.19    mean first-point sigma 4.77

The two competitors in the appended event move ~1.2σ across their entire history, back to their very first appearance. Everyone else moves through the coupling. I checked the concentration specifically because a 5.58 shift looked implausible at first — it lands exactly on the two involved competitors, which is what makes it credible rather than an artifact.

Conclusion

That is not a bug, it is the defining behaviour of Through Time: new evidence updates beliefs about the past. The previous solution is genuinely far from the new one, so re-convergence is real work, not repeated work. The bottleneck is the model's semantics, not ingestion's bookkeeping.

The cost per append is inherently O(history), and no amount of state preservation changes that. An implementation that made appends cheap would be computing something other than TrueSkill Through Time — a filtering estimate, which is what filtered_learning_curve already provides (#19) for callers who want "what we knew at the time" without the backward pass.

What survives

The ~7% from skipping the ingestion re-prime is real and roughly free. Worth its own small issue if you want it, but two cautions: it changes cold-converge iteration counts as well as incremental ones, and this is the same treacherous code where #4's design went wrong. Not worth taking on casually for 7%.

ConvergenceReport.slices_skipped is still hardcoded to 0 with nothing left that would populate it. Now that both #4 and this are closed, it should be removed or documented as reserved rather than left looking implemented.

**Spiked. The premise of this issue is wrong — the re-convergence is necessary work, not waste.** Recommending we close it. I filed this off the observation that appending one event costs a full re-converge. That observation is correct; the inference that it is *avoidable* is not. ## Measurements 2000 events / 200 competitors / 20 per slice, release. **1. Does the machinery exploit a warm start at all?** ``` NO-OP RE-CONVERGE (already converged, nothing appended) iterations 1 final_step (1.18e-7, 6.19e-8) ``` **Yes, perfectly.** Converging an already-converged history costs one iteration. There is no "it always grinds through 14 iterations regardless" defect. The loop terminates immediately when nothing moved. **2. Does preserving slice state across an append help?** Patched ingestion to skip the two `new_forward_info` re-primes, so every slice keeps its converged `forward`/`likelihood`. Note `competitor::clean` wipes `agents[].message` but *not* the slices' own state — the re-primes are what destroy it, so this is the minimal experiment. ``` re-prime on re-prime skipped cold converge 14 13 re-converge after append 14 13 ``` **~7%.** Preserving state buys one iteration, and the re-converge still costs the same as a cold converge either way. **3. Why — how far does one appended event actually reach?** Shift in each competitor's **first** time point after appending one event at the end: ``` top 5 movers: p0=5.58, p1=5.54, p108=0.78, p117=0.77, p109=0.72 mean shift 0.19 mean first-point sigma 4.77 ``` The two competitors in the appended event move **~1.2σ across their entire history**, back to their very first appearance. Everyone else moves through the coupling. I checked the concentration specifically because a 5.58 shift looked implausible at first — it lands exactly on the two involved competitors, which is what makes it credible rather than an artifact. ## Conclusion That is not a bug, it is the defining behaviour of Through Time: new evidence updates beliefs about the past. The previous solution is genuinely far from the new one, so re-convergence is real work, not repeated work. The bottleneck is the model's semantics, not ingestion's bookkeeping. **The cost per append is inherently O(history), and no amount of state preservation changes that.** An implementation that made appends cheap would be computing something other than TrueSkill Through Time — a filtering estimate, which is what `filtered_learning_curve` already provides (#19) for callers who want "what we knew at the time" without the backward pass. ## What survives The ~7% from skipping the ingestion re-prime is real and roughly free. Worth its own small issue if you want it, but two cautions: it changes cold-converge iteration counts as well as incremental ones, and this is the same treacherous code where #4's design went wrong. Not worth taking on casually for 7%. `ConvergenceReport.slices_skipped` is still hardcoded to `0` with nothing left that would populate it. Now that both #4 and this are closed, it should be removed or documented as reserved rather than left looking implemented.

logaritmisk closed this issue

2026-08-28 06:01:55 +00:00

logaritmisk referenced this issue

2026-08-28 06:02:10 +00:00

`ConvergenceReport.slices_skipped` is public, always zero, and nothing will ever populate it #33

logaritmisk referenced this issue from a commit

2026-08-28 06:27:25 +00:00

refactor!: remove ConvergenceReport::slices_skipped

logaritmisk added the performance label 2026-09-07 13:53:36 +00:00

logaritmisk referenced this issue

2026-09-07 23:21:36 +00:00

Design question: can a fitted `History` be persisted, or must every consumer replay its whole event log? #45