ITERATIONS = 30 is far too low for a dense history, and overrunning it fails quietly #50

New Issue

2026-09-07T21:27:19Z

logaritmisk commented

2026-09-07 21:27:19 +00:00

ConvergenceOptions::default() takes max_iter from crate::ITERATIONS, which is 30. That is not close to enough for a history of even modest size, and the failure mode is the quiet kind.

Measured

ustat's joint hole-level fit is 1,991 nodes over 3,292 events. It needs:

76 sweeps to converge as built
161 unpooled

So the default stops at roughly a fifth to two fifths of the way there. It runs max_iter = 500 for that reason, and boot is tens of milliseconds either way — the default is not buying speed on anything but a toy.

Why it is worth more than a constant change

A fit that stops short is wrong by a little, which is the worst available failure: it converges enough to look plausible, the ratings are all finite and ordered sensibly, and nothing about the output says the numbers moved after the last sweep. The ConvergenceReport does carry the fact, so a careful consumer can detect it — but detection is opt-in and the default is the trap.

Three possible responses, roughly increasing in effort:

Document it. State on ITERATIONS and on ConvergenceOptions roughly what size of history 30 is adequate for, so the number reads as a floor rather than as a recommendation.
Scale the default with the graph — node or event count — rather than shipping one constant for every workload.
Make a non-converged fit hard to swallow. Whether that is a louder signal from converge() or a Result shape the caller must confront is a call for the crate; I only want it to be difficult to consume a short fit by accident.

(1) alone would be a real improvement, and #22 already tightened the neighbouring inconsistency, so this may be a small job.

Not urgent for ustat — it has set 500 since early on. Filing it because the default is what a new consumer gets, and the reason to change it is not discoverable from the output.

`ConvergenceOptions::default()` takes `max_iter` from `crate::ITERATIONS`, which is **30**. That is not close to enough for a history of even modest size, and the failure mode is the quiet kind. ## Measured `ustat`'s joint hole-level fit is **1,991 nodes over 3,292 events**. It needs: - **76** sweeps to converge as built - **161** unpooled So the default stops at roughly a fifth to two fifths of the way there. It runs `max_iter = 500` for that reason, and boot is tens of milliseconds either way — the default is not buying speed on anything but a toy. ## Why it is worth more than a constant change A fit that stops short is **wrong by a little**, which is the worst available failure: it converges enough to look plausible, the ratings are all finite and ordered sensibly, and nothing about the output says the numbers moved after the last sweep. The `ConvergenceReport` does carry the fact, so a careful consumer can detect it — but detection is opt-in and the default is the trap. Three possible responses, roughly increasing in effort: 1. **Document it.** State on `ITERATIONS` and on `ConvergenceOptions` roughly what size of history 30 is adequate for, so the number reads as a floor rather than as a recommendation. 2. **Scale the default** with the graph — node or event count — rather than shipping one constant for every workload. 3. **Make a non-converged fit hard to swallow.** Whether that is a louder signal from `converge()` or a `Result` shape the caller must confront is a call for the crate; I only want it to be difficult to consume a short fit by accident. (1) alone would be a real improvement, and #22 already tightened the neighbouring inconsistency, so this may be a small job. Not urgent for `ustat` — it has set 500 since early on. Filing it because the default is what a new consumer gets, and the reason to change it is not discoverable from the output.

logaritmisk added the api docs labels 2026-09-07 21:27:19 +00:00

logaritmisk referenced this issue from a commit

2026-09-07 21:41:11 +00:00

feat!: name the unknown key, expose tail probabilities, flag short fits

logaritmisk referenced this issue from a commit

2026-09-07 23:20:51 +00:00

fix(test): the ingestion-order property was comparing two truncated fits

logaritmisk referenced this issue

2026-09-07 23:21:36 +00:00

Design question: can a fitted `History` be persisted, or must every consumer replay its whole event log? #45

logaritmisk referenced this issue from a commit

2026-09-08 19:01:23 +00:00