logaritmisk/trueskill-tt
1
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Activity

fix!: make the joint span slices, not just the latest one

Browse Source 
`posterior_of` shipped in 0.5.0 reading a single slice. Measured against
a real Through-Time history that answers almost nothing: ustat's round
fit is 76 per-day slices whose last one holds a solo round, so 0 of 55
pair differences resolved and the single node that did was degenerate —
a one-competitor slice has no correlation to account for and returns the
marginal unchanged.

That was my mistake, and the fixture chose it. I validated against
single-slice histories, which is exactly the shape that cannot reveal
the problem. In a library whose premise is skill over time, competitors
are read at *their own* last appearance and those are different slices
by construction.

The joint is now time-expanded: one variable per appearance, linked by
the prior on a first appearance, the drift between consecutive ones, and
the within-slice event contrasts. Consecutive appearances with no drift
between them are the same variable rather than two joined by an infinite
precision, which keeps the matrix positive-definite when a competitor is
pinned with `drift_scale = 0`.

`posterior_of` now reads each competitor at their own latest appearance,
which is where `current_skill` reads them, so the two agree about which
posterior they describe. Adds `posterior_of_at(time, terms)` for a
comparison anchored to a moment, matching `learning_curve`'s reading.

Validated against a hand-written exact posterior for a two-competitor,
two-slice history — the precision matrix is spelled out in the test
rather than obtained from the crate, so it is an independent check
rather than a restatement. Also pinned: competitors last seen in
different slices now compare at all, means still agree with the
marginals, zero drift makes slice layout irrelevant, and more drift
widens a comparison across time.

BREAKING CHANGE: `posterior_of` and `expected_variance_reduction` now
consider the whole history rather than its latest slice, so results
change for any multi-slice history. `JointUnavailable` is now returned
when *any* slice holds ranked events, not just the last.

Refs #46, #47

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
logaritmisk
2026-09-08 06:04:41 +02:00
co-authored by Claude Opus 5
parent d9e85cda1d
commit f345e7690e
3 changed files with 553 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/history.rs
+230 -55
View File
@@ -202,6 +202,19 @@ pub(crate) struct CompetitorConfig {
drift_scale: Option<f64>,
}
/// The joint precision over a history's appearances, with the maps needed to
/// address a competitor either at their latest appearance or at a given slice.
struct TimeExpanded {
/// Row-major precision matrix over appearances.
lambda: Vec<f64>,
/// `(row, slice)` of each competitor's latest appearance.
latest: HashMap<Index, (usize, usize)>,
/// Row of each `(competitor, slice)` appearance.
at_slice: HashMap<(Index, usize), usize>,
/// Side length of `lambda`.
width: usize,
}
/// A linear functional resolved against one time slice.
struct ResolvedTerms {
/// Coefficients over the slice's own competitors, in its ordering.
@@ -765,19 +778,104 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
Ok(crate::quality(&group_refs, self.beta))
}
/// Resolve `terms` into a contrast over the slice's competitor order, the
/// coefficients of any competitors the slice has never seen, and the mean.
/// The joint posterior precision over the whole history, time-expanded.
///
/// An unseen competitor shares no event with the slice, so it is
/// independent of everything in it by construction; keeping those
/// coefficients separate is what lets their variance be added rather than
/// solved for.
/// A competitor's skill is not one variable but one per appearance, linked
/// by drift. That is the point of Through Time, and it is why a joint over
/// a single slice answers almost nothing: competitors are each read at
/// *their own* last appearance, and in a history with per-day or per-event
/// slices those are different slices. A 76-slice history whose last slice
/// holds one competitor can answer no pairwise question at all.
///
/// Variables are `(competitor, appearance)`. Factors are the prior on a
/// first appearance, the drift between consecutive appearances, and the
/// within-slice event contrasts. Consecutive appearances with zero drift
/// variance are the same variable rather than two joined by an infinite
/// precision, which keeps the matrix positive-definite when a competitor is
/// pinned with `drift_scale = 0`.
///
/// Returns the matrix, each competitor's row at its latest appearance, and
/// the row at each `(competitor, slice)` for time-addressed queries.
fn time_expanded_joint(&self) -> TimeExpanded {
let mut latest: HashMap<Index, (usize, usize)> = HashMap::new();
let mut at_slice: HashMap<(Index, usize), usize> = HashMap::new();
// Row of a competitor's previous appearance, and the drift variance
// separating it from the current one.
let mut previous: HashMap<Index, usize> = HashMap::new();
let mut drift_links: Vec<(usize, usize, f64)> = Vec::new();
let mut first_rows: Vec<(usize, Index)> = Vec::new();
let mut n = 0usize;
for (slice_idx, slice) in self.time_slices.iter().enumerate() {
for (agent, elapsed) in slice.appearances() {
let rating = &self.agents[agent].rating;
let row = match previous.get(&agent) {
None => {
let row = n;
n += 1;
first_rows.push((row, agent));
row
}
Some(&prev) => {
let drift = rating.drift_variance_for_elapsed(elapsed);
if drift <= 0.0 {
// No drift: the same latent skill, not two.
prev
} else {
let row = n;
n += 1;
drift_links.push((prev, row, drift));
row
}
}
};
previous.insert(agent, row);
latest.insert(agent, (row, slice_idx));
at_slice.insert((agent, slice_idx), row);
}
}
let mut lambda = vec![0.0; n * n];
for (row, agent) in first_rows {
lambda[row * n + row] += 1.0 / self.agents[agent].rating.prior.sigma().powi(2);
}
for (a, b, drift) in drift_links {
lambda[a * n + a] += 1.0 / drift;
lambda[b * n + b] += 1.0 / drift;
lambda[a * n + b] -= 1.0 / drift;
lambda[b * n + a] -= 1.0 / drift;
}
for (slice_idx, slice) in self.time_slices.iter().enumerate() {
for (contrast, noise) in slice.scored_contrasts(&self.agents) {
for (ia, ca) in &contrast {
let ra = at_slice[&(*ia, slice_idx)];
for (ib, cb) in &contrast {
let rb = at_slice[&(*ib, slice_idx)];
lambda[ra * n + rb] += ca * cb / noise;
}
}
}
}
TimeExpanded {
lambda,
latest,
at_slice,
width: n,
}
}
/// Resolve `terms` into a contrast over the time-expanded rows, the
/// coefficients of competitors the history has never seen, and the mean.
///
/// `row_for` picks which appearance of a competitor the caller means —
/// their latest, or the one at a given time.
fn resolve_terms(
&self,
terms: &[(&K, f64)],
slice: &TimeSlice<T>,
row_of: &HashMap<Index, usize>,
width: usize,
row_for: impl Fn(Index) -> Option<(usize, usize)>,
) -> Result<ResolvedTerms, InferenceError>
where
K: std::fmt::Debug,
@@ -790,16 +888,16 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
let located = self
.keys
.get(*key)
.and_then(|index| row_of.get(&index).map(|row| (index, *row)));
.and_then(|index| row_for(index).map(|located| (index, located)));
match located {
Some((index, row)) => {
Some((index, (row, slice_idx))) => {
contrast[row] += coefficient;
mean += coefficient
* slice
* self.time_slices[slice_idx]
.skills
.get(index)
.expect("index came from this slice")
.expect("row came from this slice")
.posterior()
.mu();
}
@@ -844,54 +942,131 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
/// The mean is the same combination of the marginal means, which message
/// passing already gets exactly right. Only the variance needs the joint.
///
/// # Which appearance each competitor is read at
///
/// Each competitor is read at *their own* latest appearance, which is where
/// [`History::current_skill`] reads them too, so the two agree about which
/// posterior they describe. That matters in a Through-Time history: with
/// per-day or per-event slices, competitors are rarely all present in any
/// one of them. Use [`History::posterior_of_at`] to pin a time instead.
///
/// # Limitations
///
/// Currently exact only for a slice whose events are all scored, because a
/// scored likelihood is Gaussian and its factor can be rebuilt exactly. A
/// ranked outcome's truncation is approximated by EP, and reconstructing
/// those factors needs the converged messages, which inference does not
/// retain. Ranked slices return `JointUnavailable` rather than a plausible
/// wrong number.
/// Exact only for a history whose events are all scored, because a scored
/// likelihood is Gaussian and its factor can be rebuilt exactly. A ranked
/// outcome's truncation is approximated by EP, and reconstructing those
/// factors needs the converged messages, which inference does not retain —
/// so a history containing ranked events returns `JointUnavailable` rather
/// than a plausible wrong number.
///
/// Cost is a dense solve over the history's *appearances*, not its
/// competitors: a competitor contributes one variable per slice it appears
/// in, minus any consecutive pair with no drift between them.
///
/// # Errors
///
/// `UnknownKey` for a competitor absent from the latest slice, and
/// `JointUnavailable` if that slice contains ranked events or the system is
/// not positive-definite.
/// `UnknownKey` for a competitor the history has never seen, and
/// `JointUnavailable` for ranked events or a system that is not
/// positive-definite.
pub fn posterior_of(&self, terms: &[(&K, f64)]) -> Result<Gaussian, InferenceError>
where
K: std::fmt::Debug,
{
let slice = self
.time_slices
.last()
.ok_or(InferenceError::JointUnavailable {
reason: "the history has no events",
})?;
if !slice.all_scored() {
if self.time_slices.is_empty() {
return Err(InferenceError::JointUnavailable {
reason: "the latest slice contains ranked events, whose EP factors \
are not retained after convergence",
reason: "the history has no events",
});
}
if !self.time_slices.iter().all(TimeSlice::all_scored) {
return Err(InferenceError::JointUnavailable {
reason: "the history contains ranked events, whose EP factors are \
not retained after convergence",
});
}
let (order, lambda) = slice.joint_precision(&self.agents);
let mut row_of = HashMap::with_capacity(order.len());
for (r, idx) in order.iter().enumerate() {
row_of.insert(*idx, r);
let TimeExpanded {
lambda,
latest,
width,
..
} = self.time_expanded_joint();
let ResolvedTerms {
contrast,
unseen,
mean,
} = self.resolve_terms(terms, width, |index| latest.get(&index).copied())?;
let z =
crate::joint::solve_spd(lambda, &contrast).ok_or(InferenceError::JointUnavailable {
reason: "the precision matrix is not positive-definite, which means \
a competitor has neither a proper prior nor any evidence",
})?;
let prior_var = self.sigma * self.sigma;
let variance: f64 = contrast.iter().zip(&z).map(|(c, z)| c * z).sum::<f64>()
+ unseen.values().map(|c| c * c * prior_var).sum::<f64>();
Ok(Gaussian::from_mv(mean, variance))
}
/// Posterior of a linear combination, read as of `time`.
///
/// Each competitor is taken at their latest appearance at or before `time`,
/// which is the same reading [`History::learning_curve`] gives. Use this
/// when a comparison must be anchored to a moment — "how did these two
/// stand at the end of last season" — rather than to wherever each
/// competitor was last seen.
///
/// # Errors
///
/// As [`History::posterior_of`], plus `UnknownKey` for a competitor with no
/// appearance at or before `time`.
pub fn posterior_of_at(&self, time: T, terms: &[(&K, f64)]) -> Result<Gaussian, InferenceError>
where
K: std::fmt::Debug,
{
if self.time_slices.is_empty() {
return Err(InferenceError::JointUnavailable {
reason: "the history has no events",
});
}
if !self.time_slices.iter().all(TimeSlice::all_scored) {
return Err(InferenceError::JointUnavailable {
reason: "the history contains ranked events, whose EP factors are \
not retained after convergence",
});
}
let TimeExpanded {
lambda,
at_slice,
width,
..
} = self.time_expanded_joint();
// Latest appearance at or before `time`, per competitor.
let mut as_of: HashMap<Index, (usize, usize)> = HashMap::new();
for (slice_idx, slice) in self.time_slices.iter().enumerate() {
if slice.time > time {
break;
}
for (agent, _) in slice.appearances() {
if let Some(row) = at_slice.get(&(agent, slice_idx)) {
as_of.insert(agent, (*row, slice_idx));
}
}
}
let ResolvedTerms {
contrast,
unseen,
mean,
} = self.resolve_terms(terms, slice, &row_of, order.len())?;
} = self.resolve_terms(terms, width, |index| as_of.get(&index).copied())?;
let z =
crate::joint::solve_spd(lambda, &contrast).ok_or(InferenceError::JointUnavailable {
reason: "the precision matrix is not positive-definite, which means \
a competitor has neither a proper prior nor any evidence",
reason: "the precision matrix is not positive-definite",
})?;
let prior_var = self.sigma * self.sigma;
@@ -951,16 +1126,15 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
});
}
let slice = self
.time_slices
.last()
.ok_or(InferenceError::JointUnavailable {
reason: "the history has no events",
})?;
if !slice.all_scored() {
if self.time_slices.is_empty() {
return Err(InferenceError::JointUnavailable {
reason: "the latest slice contains ranked events, whose EP factors \
are not retained after convergence",
reason: "the history has no events",
});
}
if !self.time_slices.iter().all(TimeSlice::all_scored) {
return Err(InferenceError::JointUnavailable {
reason: "the history contains ranked events, whose EP factors are \
not retained after convergence",
});
}
@@ -983,14 +1157,15 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
}
}
let (order, lambda) = slice.joint_precision(&self.agents);
let mut row_of = HashMap::with_capacity(order.len());
for (r, idx) in order.iter().enumerate() {
row_of.insert(*idx, r);
}
let TimeExpanded {
lambda,
latest,
width,
..
} = self.time_expanded_joint();
let target = self.resolve_terms(target, slice, &row_of, order.len())?;
let matchup = self.resolve_terms(&matchup, slice, &row_of, order.len())?;
let target = self.resolve_terms(target, width, |i| latest.get(&i).copied())?;
let matchup = self.resolve_terms(&matchup, width, |i| latest.get(&i).copied())?;
let (target_contrast, target_unseen) = (target.contrast, target.unseen);
let (matchup_contrast, matchup_unseen) = (matchup.contrast, matchup.unseen);
@@ -1003,7 +1178,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
)?;
let prior_var = self.sigma * self.sigma;
// Competitors outside the slice are independent, so they contribute
// Competitors outside the history are independent, so they contribute
// only where the same key appears in both functionals.
let cross_unseen: f64 = target_unseen
.iter()