logaritmisk/trueskill-tt
1
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Activity

No joint posterior: anything derived from two or more competitors has to add marginals and overstate its width #46

New Issue
Closed
opened 2026-09-07 21:26:12 +00:00 by logaritmisk · 2 comments
logaritmisk commented 2026-09-07 21:26:12 +00:00
Owner
Copy Link
Copy Source

Every accessor — current_skill, learning_curve, learning_curves, filtered_* — returns a marginal Gaussian. Nothing exposes a covariance or a joint, so a consumer that wants the posterior of a combination of competitors has no option but to add variances as though they were independent. They are not: competitors are correlated through every event they share, which is the whole mechanism the model exists to exploit.

Why this matters more than it sounds

Almost nothing a consumer displays is one node. In ustat (disc golf, TrueSkill over a joint player/layout model) every published figure is a linear combination:

quantity combination
"can we tell these two players apart" player_a − player_b
what a round was worth score + layout
a player's form on one hole hole − player − hole_offset
has a player improved current − career

Each is computed from marginals, and each is therefore wider than the truth. The most visible consequence: /api/connectivity publishes separation(a,b) = sqrt(σ_a² + σ_b²) and has to be documented as an upper bound (logaritmisk/ustat#113). A UI block built on it reports the odds that a ranking is the right way round, as Φ(gap / separation). Because separation is too large, every figure is pushed toward 5-in-10 — the block systematically understates how sure the model is. Five real pairs currently read 6, 7, 8, 8, 8 in ten; with a true separation some of them would probably resolve and leave the list.

That direction is at least the safe one for a block whose message is "we cannot tell". It would not be safe for anything that quoted a probability the other way round.

What would be enough

Not the full matrix — a realistic history here is ~2,000 nodes and nobody wants 4M entries. Two shapes would each solve it, in increasing order of usefulness:

// pairwise, on demand
fn covariance<Q>(&self, a: &Q, b: &Q, time: T) -> Option<f64>;

// or, better: the posterior of an arbitrary linear functional
fn posterior_of<Q>(&self, terms: &[(&Q, f64)], time: T) -> Option<Gaussian>;

The second covers every row in the table above with one call and one concept, and it keeps the correlation structure inside the crate where it belongs rather than asking each consumer to reconstruct it.

Caveat

I do not know how cheaply EP can give this. Message passing does not maintain a joint covariance, so this may mean a linear solve against the factor graph, or a deliberately approximate answer. An approximate covariance that is documented as approximate would still be a large improvement on assuming zero.

Filed from ustat, which currently works around it by refitting the model on disjoint halves of the data to measure empirically what the marginals will not say — see the companion issue on marginal width, and logaritmisk/ustat#113.

Every accessor — `current_skill`, `learning_curve`, `learning_curves`, `filtered_*` — returns a marginal `Gaussian`. Nothing exposes a covariance or a joint, so a consumer that wants the posterior of *a combination* of competitors has no option but to add variances as though they were independent. They are not: competitors are correlated through every event they share, which is the whole mechanism the model exists to exploit. ## Why this matters more than it sounds Almost nothing a consumer displays is one node. In `ustat` (disc golf, TrueSkill over a joint player/layout model) every published figure is a linear combination: | quantity | combination | |---|---| | "can we tell these two players apart" | `player_a − player_b` | | what a round was worth | `score + layout` | | a player's form on one hole | `hole − player − hole_offset` | | has a player improved | `current − career` | Each is computed from marginals, and each is therefore wider than the truth. The most visible consequence: `/api/connectivity` publishes `separation(a,b) = sqrt(σ_a² + σ_b²)` and has to be documented as an **upper bound** (`logaritmisk/ustat#113`). A UI block built on it reports the odds that a ranking is the right way round, as `Φ(gap / separation)`. Because separation is too large, every figure is pushed toward 5-in-10 — the block systematically *understates* how sure the model is. Five real pairs currently read 6, 7, 8, 8, 8 in ten; with a true separation some of them would probably resolve and leave the list. That direction is at least the safe one for a block whose message is "we cannot tell". It would not be safe for anything that quoted a probability the other way round. ## What would be enough Not the full matrix — a realistic history here is ~2,000 nodes and nobody wants 4M entries. Two shapes would each solve it, in increasing order of usefulness: ```rust // pairwise, on demand fn covariance<Q>(&self, a: &Q, b: &Q, time: T) -> Option<f64>; // or, better: the posterior of an arbitrary linear functional fn posterior_of<Q>(&self, terms: &[(&Q, f64)], time: T) -> Option<Gaussian>; ``` The second covers every row in the table above with one call and one concept, and it keeps the correlation structure inside the crate where it belongs rather than asking each consumer to reconstruct it. ## Caveat I do not know how cheaply EP can give this. Message passing does not maintain a joint covariance, so this may mean a linear solve against the factor graph, or a deliberately approximate answer. An approximate covariance that is documented as approximate would still be a large improvement on assuming zero. Filed from `ustat`, which currently works around it by refitting the model on disjoint halves of the data to measure empirically what the marginals will not say — see the companion issue on marginal width, and `logaritmisk/ustat#113`.
logaritmisk added the apinumericsenhancement labels 2026-09-07 21:26:12 +00:00
logaritmisk commented 2026-09-08 00:24:46 +00:00
Author
Owner
Copy Link
Copy Source

posterior_of in 0.5.0 is exactly the shape asked for here — a linear functional, correlations intact — so the API half of this issue is delivered. Thank you.

It does not yet reach this consumer, and the reason is the latest-slice limitation rather than anything about the design. Measured on the real export against 0.5.0, detail in #47:

  • the round fit is 76 time slices over 100 rounds (slices are per-day; players appear on the days they played)
  • the last slice is a solo round by one player
  • so of the four combinations in this issue's table, none is answerable: 0 of 55 pair differences resolve, and the single node that does is degenerate, since a one-competitor slice has no correlation to account for and returns the marginal unchanged

That is a property of Through-Time rather than of a badly-shaped consumer: every comparison a standings table publishes is between competitors read at their own last rounds, which by construction are different slices.

Two notes for whoever picks this up.

The direction claim in this issue was wrong and is worth correcting in place. I wrote that adding marginals overstates, and that this was "at least the safe direction". #47's investigation shows that holds for differences and reverses for sums — p0 + h0 measured 5x too NARROW. This issue's own table lists score + layout, a sum, alongside three differences, so it was half wrong about its own examples.

The useful follow-up is slice selection, not covariance. Either posterior_of_at(time, terms), or terms resolved at each competitor's own latest slice — the latter matching where current_skill already reads from, so the two calls would agree about which posterior they describe.

Happy for this to be closed as delivered with a fresh issue for the slice question, since the thing this asked for does exist now.

`posterior_of` in 0.5.0 is exactly the shape asked for here — a linear functional, correlations intact — so the API half of this issue is delivered. Thank you. **It does not yet reach this consumer, and the reason is the latest-slice limitation rather than anything about the design.** Measured on the real export against 0.5.0, detail in #47: - the round fit is **76 time slices over 100 rounds** (slices are per-day; players appear on the days they played) - the last slice is a **solo round by one player** - so of the four combinations in this issue's table, **none is answerable**: 0 of 55 pair differences resolve, and the single node that does is degenerate, since a one-competitor slice has no correlation to account for and returns the marginal unchanged That is a property of Through-Time rather than of a badly-shaped consumer: every comparison a standings table publishes is between competitors read at *their own* last rounds, which by construction are different slices. Two notes for whoever picks this up. **The direction claim in this issue was wrong and is worth correcting in place.** I wrote that adding marginals overstates, and that this was "at least the safe direction". #47's investigation shows that holds for *differences* and **reverses for sums** — `p0 + h0` measured 5x too NARROW. This issue's own table lists `score + layout`, a sum, alongside three differences, so it was half wrong about its own examples. **The useful follow-up is slice selection, not covariance.** Either `posterior_of_at(time, terms)`, or terms resolved at each competitor's own latest slice — the latter matching where `current_skill` already reads from, so the two calls would agree about which posterior they describe. Happy for this to be closed as delivered with a fresh issue for the slice question, since the thing this asked for does exist now.
logaritmisk commented 2026-09-08 04:47:20 +00:00
Author
Owner
Copy Link
Copy Source

Closing as delivered — and no follow-up issue is needed, because the slice question is fixed rather than deferred.

0.6.0 makes the joint time-expanded: one variable per appearance, linked by the prior on a first appearance, the drift between consecutive ones, and the within-slice event contrasts. posterior_of reads each competitor at their own latest appearance, which is where current_skill reads them, so the two agree about which posterior they describe. posterior_of_at(time, terms) is there for a reading anchored to a moment. Detail and validation in #47.

You proposed either posterior_of_at or per-competitor-latest resolution, and noted the second was the one that would let you delete a calibration constant. Both are in; the second is the default.

The latest-slice limitation was my mistake, not a shortcoming of this issue's framing. I validated the joint against single-slice fixtures, which is exactly the shape that cannot expose it. Your measurement caught what my tests structurally could not.

The direction correction, recorded

You asked for this to be corrected in place. I would rather not rewrite the original text — the thread reads better with the correction stated than with the record altered — so, for anyone arriving here:

This issue says adding marginals overstates the width of a derived quantity, and that this is "at least the safe one" as directions go. That holds for differences. It reverses for sums: measured, p0 + h0 came out five times too narrow, which is overconfident.

The table in this issue lists score + layout — a sum — alongside three differences, so the framing was half wrong about its own examples. tests/additive_model.rs pins the direction per combination shape so a future change cannot quietly flip either.

Worth adding that the magnitudes are still unsettled: on the round-robin fixture ignoring correlation is 2.6x too wide for a difference, on the additive one it is 0.98x, and your export measures 1.30x. Those bracket your case rather than reproduce it. posterior_of now resolves on your model, so the exact figure is available and the estimate can be replaced with a measurement — see #47, which stays open for that.

Closing as delivered — and **no follow-up issue is needed**, because the slice question is fixed rather than deferred. `0.6.0` makes the joint time-expanded: one variable per *appearance*, linked by the prior on a first appearance, the drift between consecutive ones, and the within-slice event contrasts. `posterior_of` reads each competitor at their own latest appearance, which is where `current_skill` reads them, so the two agree about which posterior they describe. `posterior_of_at(time, terms)` is there for a reading anchored to a moment. Detail and validation in #47. You proposed either `posterior_of_at` or per-competitor-latest resolution, and noted the second was the one that would let you delete a calibration constant. Both are in; the second is the default. The latest-slice limitation was my mistake, not a shortcoming of this issue's framing. I validated the joint against single-slice fixtures, which is exactly the shape that cannot expose it. Your measurement caught what my tests structurally could not. ## The direction correction, recorded You asked for this to be corrected in place. I would rather not rewrite the original text — the thread reads better with the correction stated than with the record altered — so, for anyone arriving here: > This issue says adding marginals **overstates** the width of a derived quantity, and that this is "at least the safe one" as directions go. That holds for **differences**. It **reverses for sums**: measured, `p0 + h0` came out **five times too narrow**, which is overconfident. The table in this issue lists `score + layout` — a sum — alongside three differences, so the framing was half wrong about its own examples. `tests/additive_model.rs` pins the direction per combination shape so a future change cannot quietly flip either. Worth adding that the magnitudes are still unsettled: on the round-robin fixture ignoring correlation is 2.6x too wide for a difference, on the additive one it is 0.98x, and your export measures 1.30x. Those bracket your case rather than reproduce it. `posterior_of` now resolves on your model, so the exact figure is available and the estimate can be replaced with a measurement — see #47, which stays open for that.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
api
Shape or ergonomics of the public surface
 blocked
Waiting on another issue or an external decision
 breaking
Resolving it changes the public API; wants a major bump
 bug
Behaviour is wrong, including silently wrong
 decision
Needs a call to be made, not code to be written
 docs
Documentation, including doc comments and the README
 enhancement
New capability the crate does not have
 infra
CI, release process, packaging
 numerics
Floating-point accuracy, cancellation, convergence maths
 performance
Speed or memory
 refactor
Internal cleanup with no intended behaviour change
 testing
Test coverage and test infrastructure
No labels api enhancement numerics
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
logaritmisk
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: logaritmisk/trueskill-tt#46