logaritmisk/trueskill-tt
1
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Activity

feat: add expected information gain for active matchup selection

Browse Source 
`quality()` answers "is this matchup fair". Callers picking which
comparison to run next need "is this matchup informative", and the two
coincide only for two evenly matched competitors. Without a principled
alternative, downstream code was reaching for hand-rolled heuristics
like `quality * sigma_a^2 * sigma_b^2`, which double-counts uncertainty:
the two factors are not independent.

Adds `expected_information_gain`, the outcome-weighted divergence
between current beliefs and the beliefs each result would produce:

    EIG = SUM P(outcome) * KL(posterior_after(outcome) || prior)

Available standalone over `Rating`s, and as
`History::expected_information_gain` using current skills and the
history's own beta, drift and p_draw — so the outcomes it weighs are the
ones that would actually be fitted.

This is the mutual information between the outcome and the skills, which
gives an analytic ceiling: gain cannot exceed the entropy of the thing
being observed, so at most `ln k` nats for k outcomes. That bound is the
sharpest test available, because an acquisition function is unusually
exposed to returning finite, plausible, monotone numbers while being
wrong — it would simply select slightly worse matchups forever. A
prototype of this returned 4.77 nats from a sign error while passing
every monotonicity check; `never_exceeds_the_entropy_of_the_outcome`
catches that class unconditionally.

Measured against the ceiling the values are meaningful rather than
vacuous: 0.382 nats for an even matchup between diffuse priors against
an 0.693 ceiling, falling to 0.013 for a lopsided one and 0.000 for a
hopeless one.

`disagrees_with_the_quality_times_variance_heuristic` pins down that
this is not a monotone transform of the heuristic it replaces — the two
rank a lopsided matchup and a confident even one in opposite orders — so
a later "simplification" cannot quietly revert to it.

Cost is one inference pass per possible outcome, documented on the
public API alongside the shortlist-then-score pattern, so callers do not
discover it in production.

Also folds the duplicated key-gathering in `predict_quality` and
`performances` into one validated `member_skills`.

Refs #39

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
logaritmisk
2026-09-07 15:08:12 +02:00
co-authored by Claude Opus 5
parent 507894dae7
commit 3c2f9ac64c
5 changed files with 581 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/history.rs
+80 -45
View File
@@ -538,10 +538,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
.sum()
}
/// Each team's performance Gaussian, and its member count.
///
/// Performance is skill inflated by `beta`: the question a prediction
/// answers is "how will they do today", not "how good are they".
/// Every team's member skills, validated.
///
/// # Errors
///
@@ -549,39 +546,60 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
/// reported rather than dropped — silently skipping them would turn a team
/// of strangers into a confident-looking prediction about nobody, which is
/// the failure this replaced.
fn performances(&self, teams: &[&[&K]]) -> Result<(Vec<Gaussian>, Vec<usize>), InferenceError> {
fn member_skills(&self, teams: &[&[&K]]) -> Result<Vec<Vec<Gaussian>>, InferenceError> {
if teams.len() < 2 {
return Err(InferenceError::NotEnoughTeams { got: teams.len() });
}
let mut performances = Vec::with_capacity(teams.len());
let mut sizes = Vec::with_capacity(teams.len());
let mut gathered = Vec::with_capacity(teams.len());
for (team_idx, team) in teams.iter().enumerate() {
if team.is_empty() {
return Err(InferenceError::EmptyTeam { team: team_idx });
}
let mut total = crate::N00;
let mut members = Vec::with_capacity(team.len());
for (member_idx, key) in team.iter().enumerate() {
let unknown = InferenceError::UnknownKey {
team: team_idx,
member: member_idx,
};
let index = self.keys.get(*key).ok_or(unknown.clone())?;
let skill = self
.time_slices
.iter()
.rev()
.find_map(|ts| ts.skills.get(index).map(|s| s.posterior()))
.ok_or(unknown)?;
total = total + skill.forget(self.beta.powi(2));
members.push(
self.time_slices
.iter()
.rev()
.find_map(|ts| ts.skills.get(index).map(|s| s.posterior()))
.ok_or(unknown)?,
);
}
performances.push(total);
sizes.push(team.len());
gathered.push(members);
}
Ok(gathered)
}
/// Each team's performance Gaussian, and its member count.
///
/// Performance is skill inflated by `beta`: the question a prediction
/// answers is "how will they do today", not "how good are they".
///
/// # Errors
///
/// As [`History::member_skills`].
fn performances(&self, teams: &[&[&K]]) -> Result<(Vec<Gaussian>, Vec<usize>), InferenceError> {
let skills = self.member_skills(teams)?;
let performances = skills
.iter()
.map(|team| {
team.iter()
.fold(crate::N00, |acc, s| acc + s.forget(self.beta.powi(2)))
})
.collect();
let sizes = skills.iter().map(Vec::len).collect();
Ok((performances, sizes))
}
@@ -618,38 +636,55 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
///
/// `NotEnoughTeams`, `EmptyTeam`, or `UnknownKey`.
pub fn predict_quality(&self, teams: &[&[&K]]) -> Result<f64, InferenceError> {
let mut groups: Vec<Vec<Gaussian>> = Vec::with_capacity(teams.len());
for (team_idx, team) in teams.iter().enumerate() {
if team.is_empty() {
return Err(InferenceError::EmptyTeam { team: team_idx });
}
let mut members = Vec::with_capacity(team.len());
for (member_idx, key) in team.iter().enumerate() {
let unknown = InferenceError::UnknownKey {
team: team_idx,
member: member_idx,
};
let index = self.keys.get(*key).ok_or(unknown.clone())?;
members.push(
self.time_slices
.iter()
.rev()
.find_map(|ts| ts.skills.get(index).map(|s| s.posterior()))
.ok_or(unknown)?,
);
}
groups.push(members);
}
if groups.len() < 2 {
return Err(InferenceError::NotEnoughTeams { got: groups.len() });
}
let groups = self.member_skills(teams)?;
let group_refs: Vec<&[Gaussian]> = groups.iter().map(Vec::as_slice).collect();
Ok(crate::quality(&group_refs, self.beta))
}
/// Expected information gain of running this matchup, in nats.
///
/// Answers "which comparison should I run next" rather than "who will
/// win": the outcome-weighted divergence between current beliefs and the
/// beliefs each possible result would produce. Higher means the result
/// would teach you more.
///
/// Uses each competitor's current skill as the prior, and the history's
/// own `beta`, `drift` and `p_draw`, so the outcomes weighted here are the
/// ones that would actually be fitted if the matchup were played and
/// recorded.
///
/// Distinct from [`History::predict_quality`], which measures *fairness*.
/// The two coincide for two evenly matched competitors and diverge
/// elsewhere. See [`expected_information_gain`](crate::expected_information_gain)
/// for the scale, the analytic `ln k` ceiling, and the cost.
///
/// # Errors
///
/// As [`History::member_skills`], plus `TooManyTeams` and anything
/// inference returns for a hypothetical outcome.
pub fn expected_information_gain(&self, teams: &[&[&K]]) -> Result<f64, InferenceError> {
let skills = self.member_skills(teams)?;
let ratings: Vec<Vec<Rating<T, D>>> = skills
.iter()
.map(|team| {
team.iter()
.map(|&skill| Rating::new(skill, self.beta, self.drift))
.collect()
})
.collect();
let team_refs: Vec<&[Rating<T, D>]> = ratings.iter().map(Vec::as_slice).collect();
crate::expected_information_gain(
&team_refs,
&crate::GameOptions {
p_draw: self.p_draw,
score_sigma: self.score_sigma,
convergence: self.convergence,
},
)
}
/// `P(team i finishes strictly first)`, for every team.
///
/// Supports any number of teams. Because performances are independent