Compare commits
6
Commits
507894dae7
...
v0.4.0
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2a48d10aa9 | ||
|
|
d4f91fd221 | ||
|
|
8c087ad015 | ||
|
|
7341669d1a | ||
|
|
2fff745c3b | ||
|
|
3c2f9ac64c |
@@ -2,6 +2,29 @@
|
|||||||
|
|
||||||
All notable changes to this project will be documented in this file.
|
All notable changes to this project will be documented in this file.
|
||||||
|
|
||||||
|
## 0.4.0 - 2026-09-07
|
||||||
|
|
||||||
|
### Breaking Changes
|
||||||
|
|
||||||
|
- feat!: N-team outcome prediction with draw mass, replacing the 2-team panic
|
||||||
|
- refactor!: close the remaining API gaps from #21
|
||||||
|
- fix!: apply competitor configuration whenever it is supplied
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
- fix(release): skip the changelog hook during a dry run
|
||||||
|
- fix: stop destroying tail precision in evidence and truncation
|
||||||
|
- fix: reject convergence options that silently disable inference
|
||||||
|
|
||||||
|
### Documentation
|
||||||
|
|
||||||
|
- docs: correct drifted documentation and compile the README in CI
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
- feat: add expected information gain for active matchup selection
|
||||||
|
- feat: let observers be shared, boxed, or borrowed
|
||||||
|
|
||||||
## 0.3.0 - 2026-09-01
|
## 0.3.0 - 2026-09-01
|
||||||
|
|
||||||
### Breaking Changes
|
### Breaking Changes
|
||||||
@@ -25,6 +48,7 @@ All notable changes to this project will be documented in this file.
|
|||||||
### Miscellaneous Tasks
|
### Miscellaneous Tasks
|
||||||
|
|
||||||
- chore: ignore proptest regression seed files
|
- chore: ignore proptest regression seed files
|
||||||
|
- chore: Release trueskill-tt version 0.3.0
|
||||||
|
|
||||||
### Performance
|
### Performance
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "trueskill-tt"
|
name = "trueskill-tt"
|
||||||
version = "0.3.0"
|
version = "0.4.0"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
rust-version = "1.85"
|
rust-version = "1.85"
|
||||||
description = "TrueSkill Through Time: Bayesian skill rating that tracks how skill evolves over time, via Gaussian message passing"
|
description = "TrueSkill Through Time: Bayesian skill rating that tracks how skill evolves over time, via Gaussian message passing"
|
||||||
|
|||||||
@@ -164,6 +164,75 @@ h.event(1)
|
|||||||
h.converge().unwrap();
|
h.converge().unwrap();
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Prediction
|
||||||
|
|
||||||
|
`predict_outcome` gives the full distribution over finishing orders. Each entry
|
||||||
|
is a rank vector in the same shape `Outcome::ranking` takes — equal ranks mean a
|
||||||
|
tie — so an outcome feeds straight back into inference.
|
||||||
|
|
||||||
|
```rust
|
||||||
|
use trueskill_tt::History;
|
||||||
|
|
||||||
|
let mut h = History::builder().p_draw(0.1).build();
|
||||||
|
h.record_winner(&"alice", &"bob", 1).unwrap();
|
||||||
|
h.converge().unwrap();
|
||||||
|
|
||||||
|
let p = h.predict_outcome(&[&[&"alice"], &[&"bob"]]).unwrap();
|
||||||
|
|
||||||
|
// Probabilities are exhaustive and disjoint, so they sum to one.
|
||||||
|
assert!((p.total() - 1.0).abs() < 1e-6);
|
||||||
|
|
||||||
|
let (best, likelihood) = p.most_likely().unwrap();
|
||||||
|
println!("most likely: {best:?} at {likelihood:.3}");
|
||||||
|
println!("draw: {:.3}", p.probability_of(&[0, 0]));
|
||||||
|
```
|
||||||
|
|
||||||
|
Supports any number of teams. Because the outcome space grows factorially, the
|
||||||
|
full distribution is capped at `MAX_PREDICTED_TEAMS`; two cheaper entry points
|
||||||
|
stay available at any size:
|
||||||
|
|
||||||
|
- `predict_win_probabilities(teams)` — `P(team i finishes strictly first)`,
|
||||||
|
quadratic in team count.
|
||||||
|
- `predict_ranking(teams, ranks)` — one specific finishing order.
|
||||||
|
|
||||||
|
Unknown keys are an error, not a silent omission: a team the history has never
|
||||||
|
seen cannot produce a confident-looking probability.
|
||||||
|
|
||||||
|
## Which match to play next
|
||||||
|
|
||||||
|
`quality()` measures whether a matchup is *fair*. That is not the same as
|
||||||
|
whether it is *informative*, and the two only coincide for two evenly matched
|
||||||
|
competitors. When each observation costs something, ask
|
||||||
|
`expected_information_gain` instead — the outcome-weighted divergence between
|
||||||
|
what you believe now and what you would believe afterwards.
|
||||||
|
|
||||||
|
```rust
|
||||||
|
use trueskill_tt::History;
|
||||||
|
|
||||||
|
let mut h = History::builder().build();
|
||||||
|
for t in 1..=10 {
|
||||||
|
h.record_winner(&"veteran", &"regular", t).unwrap();
|
||||||
|
h.record_winner(&"regular", &"veteran", t + 100).unwrap();
|
||||||
|
}
|
||||||
|
h.record_winner(&"veteran", &"newcomer", 500).unwrap();
|
||||||
|
h.converge().unwrap();
|
||||||
|
|
||||||
|
let settled = h.expected_information_gain(&[&[&"veteran"], &[&"regular"]]).unwrap();
|
||||||
|
let unknown = h.expected_information_gain(&[&[&"veteran"], &[&"newcomer"]]).unwrap();
|
||||||
|
|
||||||
|
// Playing the newcomer teaches you more than replaying a settled rivalry.
|
||||||
|
assert!(unknown > settled);
|
||||||
|
```
|
||||||
|
|
||||||
|
The result is in nats, and is bounded by the entropy of the outcome: at most
|
||||||
|
`ln 2 ≈ 0.693` for a two-way result, `ln 3` once draws are possible, `ln k` for
|
||||||
|
`k` outcomes. A value near zero means you already know how it ends.
|
||||||
|
|
||||||
|
This costs one full inference pass **per possible outcome**, so it is far more
|
||||||
|
expensive than `quality()`. Scoring every pairing among `n` competitors is
|
||||||
|
`O(n² × outcomes)` passes — shortlist with `quality()` or
|
||||||
|
`predict_win_probabilities` first, then score only the shortlist.
|
||||||
|
|
||||||
## Todo
|
## Todo
|
||||||
|
|
||||||
- [x] Implement approx for Gaussian
|
- [x] Implement approx for Gaussian
|
||||||
@@ -172,6 +241,7 @@ h.converge().unwrap();
|
|||||||
- [x] Add examples (`examples/atp.rs`, `examples/scored.rs`)
|
- [x] Add examples (`examples/atp.rs`, `examples/scored.rs`)
|
||||||
- [x] Add Observer (`Observer` / `NullObserver`)
|
- [x] Add Observer (`Observer` / `NullObserver`)
|
||||||
- [x] Benchmark the inference loop (`benches/batch.rs`, `benches/history_converge.rs`, `benches/ingest.rs`)
|
- [x] Benchmark the inference loop (`benches/batch.rs`, `benches/history_converge.rs`, `benches/ingest.rs`)
|
||||||
|
- [x] N-team `predict_outcome` with draw mass, and `expected_information_gain`
|
||||||
- [ ] Cross-check `quality()` against [sublee/trueskill](https://github.com/sublee/trueskill/tree/master) — N-group support works and is covered by invariants, but no reference values are asserted
|
- [ ] Cross-check `quality()` against [sublee/trueskill](https://github.com/sublee/trueskill/tree/master) — N-group support works and is covered by invariants, but no reference values are asserted
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|||||||
@@ -0,0 +1,352 @@
|
|||||||
|
//! Active learning: which comparison teaches you the most.
|
||||||
|
//!
|
||||||
|
//! [`quality`](crate::quality) answers "is this matchup *fair*". That is a
|
||||||
|
//! different question from "is this matchup *informative*", and the two
|
||||||
|
//! coincide only for two evenly matched competitors. When each observation
|
||||||
|
//! costs something — a human click, a scheduled fixture — the question worth
|
||||||
|
//! asking is the second one.
|
||||||
|
//!
|
||||||
|
//! The quantity here is expected information gain: the outcome-weighted
|
||||||
|
//! divergence between what you believe now and what you would believe after
|
||||||
|
//! seeing the result.
|
||||||
|
//!
|
||||||
|
//! ```text
|
||||||
|
//! EIG(matchup) = SUM P(outcome) * KL( posterior_after(outcome) || prior )
|
||||||
|
//! outcome
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! It is the mutual information between the observed outcome and the skills,
|
||||||
|
//! which is worth remembering because it pins the scale: information gain
|
||||||
|
//! cannot exceed the entropy of the thing you are about to observe. A contest
|
||||||
|
//! with `k` distinguishable outcomes can teach you at most `ln k` nats,
|
||||||
|
//! whatever the ratings. That ceiling is the sharpest available test of an
|
||||||
|
//! implementation — see [`expected_information_gain`].
|
||||||
|
|
||||||
|
use crate::{
|
||||||
|
GameOptions, Gaussian, InferenceError, Outcome, Rating, drift::Drift, predict, time::Time,
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Outcomes below this probability contribute nothing measurable and are not
|
||||||
|
/// worth an inference pass.
|
||||||
|
///
|
||||||
|
/// The contribution of an outcome is `P * KL`, and `KL` is bounded in practice
|
||||||
|
/// by tens of nats, so a probability this small moves the total by less than
|
||||||
|
/// the quadrature error already present in `P` itself.
|
||||||
|
const NEGLIGIBLE: f64 = 1e-12;
|
||||||
|
|
||||||
|
/// `KL(q || p)` for two univariate Gaussians, in nats.
|
||||||
|
///
|
||||||
|
/// Both arguments are proper posteriors from inference, so the degenerate
|
||||||
|
/// cases guarded here (zero or infinite variance) indicate that inference has
|
||||||
|
/// broken down rather than anything a caller did.
|
||||||
|
fn kl_divergence(q: Gaussian, p: Gaussian) -> f64 {
|
||||||
|
let (var_q, var_p) = (q.sigma().powi(2), p.sigma().powi(2));
|
||||||
|
|
||||||
|
if !(var_q.is_finite() && var_p.is_finite()) || var_q <= 0.0 || var_p <= 0.0 {
|
||||||
|
return 0.0;
|
||||||
|
}
|
||||||
|
|
||||||
|
let mean_gap = q.mu() - p.mu();
|
||||||
|
0.5 * ((var_p / var_q).ln() + (var_q + mean_gap * mean_gap) / var_p - 1.0)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Expected information gain of a hypothetical matchup, in nats.
|
||||||
|
///
|
||||||
|
/// Enumerates the outcomes this matchup could have, runs inference for each to
|
||||||
|
/// get the belief it would produce, and weights the resulting divergence by
|
||||||
|
/// that outcome's probability. A higher value means the result would teach you
|
||||||
|
/// more.
|
||||||
|
///
|
||||||
|
/// # Interpreting the value
|
||||||
|
///
|
||||||
|
/// Nats. The upper bound is the entropy of the outcome variable: at most
|
||||||
|
/// `ln 2 ≈ 0.693` for a two-way result, `ln 3 ≈ 1.099` once draws are
|
||||||
|
/// possible, `ln k` for `k` outcomes. A value near the ceiling means the
|
||||||
|
/// result is close to a coin flip *and* would move the posteriors a long way;
|
||||||
|
/// a value near zero means you already know what will happen, or that the
|
||||||
|
/// result would barely change your beliefs if you saw it.
|
||||||
|
///
|
||||||
|
/// This is not a monotone transform of [`quality`](crate::quality). A lopsided
|
||||||
|
/// matchup between two uncertain competitors scores well on quality-times-
|
||||||
|
/// variance heuristics and poorly here, because the near-certain outcome
|
||||||
|
/// carries almost no information.
|
||||||
|
///
|
||||||
|
/// # Cost
|
||||||
|
///
|
||||||
|
/// One full inference pass per possible outcome, so this is far more expensive
|
||||||
|
/// than `quality()` — which is one closed-form evaluation. The outcome count
|
||||||
|
/// grows quickly with team count (3 outcomes for two teams that can draw, 13
|
||||||
|
/// for three, 75 for four), and scoring every candidate pairing among `n`
|
||||||
|
/// competitors is `O(n² × outcomes)` inference passes.
|
||||||
|
///
|
||||||
|
/// For a selector over many candidates, shortlist with the cheap
|
||||||
|
/// [`quality`](crate::quality) or
|
||||||
|
/// [`predict_win_probabilities`](crate::History::predict_win_probabilities)
|
||||||
|
/// first and score only the shortlist here. The expected-variance-reduction
|
||||||
|
/// proxy sometimes suggested as a cheaper alternative is *not* cheaper: it
|
||||||
|
/// needs the same hypothetical posteriors, so it shares the dominant cost.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// - `NotEnoughTeams` if fewer than two teams are supplied.
|
||||||
|
/// - `EmptyTeam` if any team has no members.
|
||||||
|
/// - `TooManyTeams` if the outcome space is too large to enumerate; see
|
||||||
|
/// [`MAX_PREDICTED_TEAMS`](crate::MAX_PREDICTED_TEAMS).
|
||||||
|
/// - `InvalidProbability` if `options.p_draw` is outside `[0.0, 1.0)`.
|
||||||
|
/// - Anything [`Game::ranked`](crate::Game::ranked) returns for a hypothetical
|
||||||
|
/// outcome.
|
||||||
|
pub fn expected_information_gain<T: Time, D: Drift<T>>(
|
||||||
|
teams: &[&[Rating<T, D>]],
|
||||||
|
options: &GameOptions,
|
||||||
|
) -> Result<f64, InferenceError> {
|
||||||
|
if teams.len() < 2 {
|
||||||
|
return Err(InferenceError::NotEnoughTeams { got: teams.len() });
|
||||||
|
}
|
||||||
|
if teams.len() > crate::MAX_PREDICTED_TEAMS {
|
||||||
|
return Err(InferenceError::TooManyTeams {
|
||||||
|
got: teams.len(),
|
||||||
|
max: crate::MAX_PREDICTED_TEAMS,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if !(0.0..1.0).contains(&options.p_draw) {
|
||||||
|
return Err(InferenceError::InvalidProbability {
|
||||||
|
value: options.p_draw,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
for (idx, team) in teams.iter().enumerate() {
|
||||||
|
if team.is_empty() {
|
||||||
|
return Err(InferenceError::EmptyTeam { team: idx });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Prediction runs on performances: skill inflated by each member's beta.
|
||||||
|
let performances: Vec<Gaussian> = teams
|
||||||
|
.iter()
|
||||||
|
.map(|team| {
|
||||||
|
team.iter()
|
||||||
|
.fold(crate::N00, |acc, rating| acc + rating.performance())
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
|
||||||
|
// Draw margins per pair, derived from the teams' betas exactly as
|
||||||
|
// inference derives them, so the outcomes weighted here are the outcomes
|
||||||
|
// that would actually be fitted.
|
||||||
|
let beta_sq: Vec<f64> = teams
|
||||||
|
.iter()
|
||||||
|
.map(|team| team.iter().map(|r| r.beta().powi(2)).sum())
|
||||||
|
.collect();
|
||||||
|
let p_draw = options.p_draw;
|
||||||
|
let margins = predict::Margins::new(teams.len(), |i, j| {
|
||||||
|
if p_draw == 0.0 {
|
||||||
|
0.0
|
||||||
|
} else {
|
||||||
|
crate::compute_margin(p_draw, (beta_sq[i] + beta_sq[j]).sqrt())
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
let mut gain = 0.0;
|
||||||
|
|
||||||
|
for (ranks, probability) in predict::outcome_distribution(&performances, &margins) {
|
||||||
|
if probability <= NEGLIGIBLE {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
let game = crate::Game::ranked(teams, Outcome::ranking(ranks), options)?;
|
||||||
|
let posteriors = game.posteriors();
|
||||||
|
|
||||||
|
// Beliefs factorise across competitors, so the joint divergence is the
|
||||||
|
// sum of the per-competitor ones.
|
||||||
|
let divergence: f64 = teams
|
||||||
|
.iter()
|
||||||
|
.zip(&posteriors)
|
||||||
|
.flat_map(|(team, posterior)| team.iter().zip(posterior))
|
||||||
|
.map(|(rating, &after)| kl_divergence(after, rating.prior()))
|
||||||
|
.sum();
|
||||||
|
|
||||||
|
gain += probability * divergence;
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(gain)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::{BETA, ConstantDrift, GAMMA};
|
||||||
|
|
||||||
|
type R = Rating<i64, ConstantDrift>;
|
||||||
|
|
||||||
|
fn rating(mu: f64, sigma: f64) -> R {
|
||||||
|
R::new(Gaussian::from_ms(mu, sigma), BETA, ConstantDrift(GAMMA))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn options(p_draw: f64) -> GameOptions {
|
||||||
|
GameOptions {
|
||||||
|
p_draw,
|
||||||
|
..GameOptions::default()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn eig(teams: &[&[R]], p_draw: f64) -> f64 {
|
||||||
|
expected_information_gain(teams, &options(p_draw)).unwrap()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The analytic ceiling. Information gain is the mutual information between
|
||||||
|
/// the outcome and the skills, so it cannot exceed the entropy of the
|
||||||
|
/// outcome variable — whatever the ratings. This is the check a subtly
|
||||||
|
/// wrong implementation fails while still returning plausible numbers: an
|
||||||
|
/// early prototype of this returned 4.77 nats from a sign error and passed
|
||||||
|
/// every monotonicity test.
|
||||||
|
#[test]
|
||||||
|
fn never_exceeds_the_entropy_of_the_outcome() {
|
||||||
|
let ceiling_two = std::f64::consts::LN_2;
|
||||||
|
|
||||||
|
for (a, b) in [
|
||||||
|
(rating(0.0, 6.0), rating(0.0, 6.0)),
|
||||||
|
(rating(0.0, 0.5), rating(0.0, 0.5)),
|
||||||
|
(rating(12.0, 6.0), rating(-12.0, 6.0)),
|
||||||
|
(rating(40.0, 1.0), rating(-40.0, 1.0)),
|
||||||
|
(rating(3.0, 6.0), rating(-2.0, 0.1)),
|
||||||
|
(rating(0.0, 25.0), rating(0.0, 25.0)),
|
||||||
|
] {
|
||||||
|
let g = eig(&[&[a], &[b]], 0.0);
|
||||||
|
assert!(
|
||||||
|
g >= 0.0 && g <= ceiling_two,
|
||||||
|
"EIG {g} outside [0, ln 2] for mu=({}, {}) sigma=({}, {})",
|
||||||
|
a.prior().mu(),
|
||||||
|
b.prior().mu(),
|
||||||
|
a.prior().sigma(),
|
||||||
|
b.prior().sigma()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// With draws enabled there are three outcomes, so the ceiling rises to
|
||||||
|
/// `ln 3` — and the two-outcome bound no longer applies.
|
||||||
|
#[test]
|
||||||
|
fn the_ceiling_follows_the_outcome_count() {
|
||||||
|
let ceiling_three = 3.0f64.ln();
|
||||||
|
for sigma in [0.5, 3.0, 6.0, 25.0] {
|
||||||
|
let g = eig(&[&[rating(0.0, sigma)], &[rating(0.0, sigma)]], 0.25);
|
||||||
|
assert!(
|
||||||
|
g >= 0.0 && g <= ceiling_three,
|
||||||
|
"EIG {g} outside [0, ln 3] at sigma {sigma}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An even matchup between uncertain competitors is the informative one.
|
||||||
|
/// A hopelessly lopsided matchup teaches you almost nothing, because you
|
||||||
|
/// already know how it ends.
|
||||||
|
#[test]
|
||||||
|
fn an_even_matchup_beats_a_lopsided_one() {
|
||||||
|
let even = eig(&[&[rating(0.0, 6.0)], &[rating(0.0, 6.0)]], 0.0);
|
||||||
|
let lopsided = eig(&[&[rating(12.0, 6.0)], &[rating(-12.0, 6.0)]], 0.0);
|
||||||
|
assert!(
|
||||||
|
even > lopsided,
|
||||||
|
"even {even} should beat lopsided {lopsided}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Certainty is the thing information gain is measuring the absence of:
|
||||||
|
/// the less you know, the more there is to learn.
|
||||||
|
#[test]
|
||||||
|
fn gain_falls_as_certainty_rises() {
|
||||||
|
let mut previous = f64::INFINITY;
|
||||||
|
for sigma in [12.0, 6.0, 3.0, 1.0, 0.5, 0.1] {
|
||||||
|
let g = eig(&[&[rating(0.0, sigma)], &[rating(0.0, sigma)]], 0.0);
|
||||||
|
assert!(
|
||||||
|
g < previous,
|
||||||
|
"sigma {sigma}: {g} did not fall below {previous}"
|
||||||
|
);
|
||||||
|
previous = g;
|
||||||
|
}
|
||||||
|
assert!(previous >= 0.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The heuristic this replaces is `quality * sigma_a^2 * sigma_b^2`. It is
|
||||||
|
/// not a monotone transform of information gain — it ranks a lopsided
|
||||||
|
/// matchup above a confident even one, and EIG ranks them the other way.
|
||||||
|
/// Pinning the disagreement down is what stops a future "simplification"
|
||||||
|
/// from quietly reverting to the heuristic.
|
||||||
|
#[test]
|
||||||
|
fn disagrees_with_the_quality_times_variance_heuristic() {
|
||||||
|
let heuristic = |a: &R, b: &R| {
|
||||||
|
crate::quality(&[&[a.prior()], &[b.prior()]], BETA)
|
||||||
|
* a.prior().sigma().powi(2)
|
||||||
|
* b.prior().sigma().powi(2)
|
||||||
|
};
|
||||||
|
|
||||||
|
let (confident_a, confident_b) = (rating(0.0, 0.5), rating(0.0, 0.5));
|
||||||
|
let (lopsided_a, lopsided_b) = (rating(12.0, 6.0), rating(-12.0, 6.0));
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
heuristic(&lopsided_a, &lopsided_b) > heuristic(&confident_a, &confident_b),
|
||||||
|
"the heuristic should prefer the lopsided matchup"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
eig(&[&[confident_a], &[confident_b]], 0.0) > eig(&[&[lopsided_a], &[lopsided_b]], 0.0),
|
||||||
|
"information gain should prefer the even matchup"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn supports_more_than_two_teams() {
|
||||||
|
let teams: Vec<Vec<R>> = vec![
|
||||||
|
vec![rating(0.0, 6.0)],
|
||||||
|
vec![rating(0.0, 6.0)],
|
||||||
|
vec![rating(0.0, 6.0)],
|
||||||
|
];
|
||||||
|
let refs: Vec<&[R]> = teams.iter().map(Vec::as_slice).collect();
|
||||||
|
let g = expected_information_gain(&refs, &options(0.0)).unwrap();
|
||||||
|
// Six distinguishable orderings with no draws.
|
||||||
|
assert!(
|
||||||
|
g > 0.0 && g <= 6.0f64.ln(),
|
||||||
|
"three-team EIG {g} out of range"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn multi_member_teams_are_supported() {
|
||||||
|
let a = [rating(0.0, 6.0), rating(1.0, 4.0)];
|
||||||
|
let b = [rating(0.0, 6.0)];
|
||||||
|
let g = expected_information_gain(&[&a, &b], &options(0.0)).unwrap();
|
||||||
|
assert!(g > 0.0 && g <= std::f64::consts::LN_2, "{g}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn degenerate_shapes_are_errors() {
|
||||||
|
let a = [rating(0.0, 6.0)];
|
||||||
|
assert!(matches!(
|
||||||
|
expected_information_gain(&[&a], &options(0.0)),
|
||||||
|
Err(InferenceError::NotEnoughTeams { got: 1 })
|
||||||
|
));
|
||||||
|
let empty: [R; 0] = [];
|
||||||
|
assert!(matches!(
|
||||||
|
expected_information_gain(&[&a, &empty], &options(0.0)),
|
||||||
|
Err(InferenceError::EmptyTeam { team: 1 })
|
||||||
|
));
|
||||||
|
assert!(matches!(
|
||||||
|
expected_information_gain(&[&a, &a], &options(1.5)),
|
||||||
|
Err(InferenceError::InvalidProbability { .. })
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn kl_divergence_is_zero_for_identical_beliefs() {
|
||||||
|
let g = Gaussian::from_ms(3.0, 2.0);
|
||||||
|
assert!(kl_divergence(g, g).abs() < 1e-15);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn kl_divergence_is_non_negative_and_grows_with_separation() {
|
||||||
|
let prior = Gaussian::from_ms(0.0, 3.0);
|
||||||
|
let mut previous = 0.0;
|
||||||
|
for mu in [0.0, 0.5, 1.0, 2.0, 4.0] {
|
||||||
|
let d = kl_divergence(Gaussian::from_ms(mu, 3.0), prior);
|
||||||
|
assert!(d >= 0.0, "negative divergence at mu {mu}: {d}");
|
||||||
|
assert!(d >= previous, "not increasing at mu {mu}");
|
||||||
|
previous = d;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -20,6 +20,37 @@ pub struct ConvergenceOptions {
|
|||||||
pub alpha: f64,
|
pub alpha: f64,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
impl ConvergenceOptions {
|
||||||
|
/// Reject values that would make inference silently meaningless.
|
||||||
|
///
|
||||||
|
/// `HistoryBuilder::convergence` asserts these eagerly, but the fields are
|
||||||
|
/// public and `GameOptions` carries a `ConvergenceOptions` — so a caller
|
||||||
|
/// can hand `Game::ranked` a set the builder never saw. In release the
|
||||||
|
/// engine's `debug_assert!`s are gone, and an `alpha` of zero leaves every
|
||||||
|
/// EP update unapplied: inference returns the priors, with every likelihood
|
||||||
|
/// uninformative and nothing to indicate anything went wrong.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// `InvalidParameter` if `alpha` is outside `(0.0, 1.0]` or `epsilon` is
|
||||||
|
/// negative. NaN fails both comparisons and is rejected.
|
||||||
|
pub(crate) fn validate(&self) -> Result<(), crate::InferenceError> {
|
||||||
|
if !(self.alpha > 0.0 && self.alpha <= 1.0) {
|
||||||
|
return Err(crate::InferenceError::InvalidParameter {
|
||||||
|
name: "alpha",
|
||||||
|
value: self.alpha,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if self.epsilon.is_nan() || self.epsilon < 0.0 {
|
||||||
|
return Err(crate::InferenceError::InvalidParameter {
|
||||||
|
name: "epsilon",
|
||||||
|
value: self.epsilon,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
impl Default for ConvergenceOptions {
|
impl Default for ConvergenceOptions {
|
||||||
fn default() -> Self {
|
fn default() -> Self {
|
||||||
Self {
|
Self {
|
||||||
|
|||||||
+15
-15
@@ -25,11 +25,6 @@ pub enum InferenceError {
|
|||||||
/// result has no representable likelihood. Configure a positive `p_draw`
|
/// result has no representable likelihood. Configure a positive `p_draw`
|
||||||
/// (via `HistoryBuilder::p_draw` or `GameOptions::p_draw`) to admit ties.
|
/// (via `HistoryBuilder::p_draw` or `GameOptions::p_draw`) to admit ties.
|
||||||
TieWithoutDrawProbability { teams: (usize, usize) },
|
TieWithoutDrawProbability { teams: (usize, usize) },
|
||||||
/// Convergence exceeded `max_iter` without falling below `epsilon`.
|
|
||||||
ConvergenceFailed {
|
|
||||||
last_step: (f64, f64),
|
|
||||||
iterations: usize,
|
|
||||||
},
|
|
||||||
/// Inference produced a non-finite value (NaN or infinity).
|
/// Inference produced a non-finite value (NaN or infinity).
|
||||||
///
|
///
|
||||||
/// Indicates numerical breakdown; the resulting skills are meaningless
|
/// Indicates numerical breakdown; the resulting skills are meaningless
|
||||||
@@ -38,8 +33,19 @@ pub enum InferenceError {
|
|||||||
context: &'static str,
|
context: &'static str,
|
||||||
step: (f64, f64),
|
step: (f64, f64),
|
||||||
},
|
},
|
||||||
/// Negative precision: a Gaussian with `pi < 0` slipped into an API call.
|
/// One batch declared two different values for the same competitor's
|
||||||
NegativePrecision { pi: f64 },
|
/// configuration.
|
||||||
|
///
|
||||||
|
/// `prior` and `drift_scale` configure a competitor, not an event, so a
|
||||||
|
/// batch that sets one of them twice with different values has no
|
||||||
|
/// well-defined meaning: events within a batch are not ordered, so
|
||||||
|
/// "last one wins" would make the result depend on iteration order.
|
||||||
|
/// Declaring the same value repeatedly is fine and is the expected shape
|
||||||
|
/// when a competitor's configuration is a property of the domain.
|
||||||
|
ConflictingCompetitorConfig {
|
||||||
|
competitor: usize,
|
||||||
|
field: &'static str,
|
||||||
|
},
|
||||||
/// A prediction referenced a key the history has no skill for.
|
/// A prediction referenced a key the history has no skill for.
|
||||||
///
|
///
|
||||||
/// Reported rather than skipped: dropping unknown keys turns a team of
|
/// Reported rather than skipped: dropping unknown keys turns a team of
|
||||||
@@ -96,18 +102,12 @@ impl fmt::Display for InferenceError {
|
|||||||
Self::InvalidParameter { name, value } => {
|
Self::InvalidParameter { name, value } => {
|
||||||
write!(f, "{name} is invalid: {value}")
|
write!(f, "{name} is invalid: {value}")
|
||||||
}
|
}
|
||||||
Self::ConvergenceFailed {
|
Self::ConflictingCompetitorConfig { competitor, field } => {
|
||||||
last_step,
|
|
||||||
iterations,
|
|
||||||
} => {
|
|
||||||
write!(
|
write!(
|
||||||
f,
|
f,
|
||||||
"convergence failed after {iterations} iterations; last step = {last_step:?}"
|
"competitor {competitor}: this batch sets {field} to two different values"
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
Self::NegativePrecision { pi } => {
|
|
||||||
write!(f, "precision must be non-negative; got {pi}")
|
|
||||||
}
|
|
||||||
Self::UnknownKey { team, member } => {
|
Self::UnknownKey { team, member } => {
|
||||||
write!(
|
write!(
|
||||||
f,
|
f,
|
||||||
|
|||||||
+11
-3
@@ -50,9 +50,17 @@ impl<K> Default for Team<K> {
|
|||||||
/// `weight` applies per event and defaults to 1.0.
|
/// `weight` applies per event and defaults to 1.0.
|
||||||
///
|
///
|
||||||
/// `prior` and `drift_scale` are **competitor configuration**, not per-event
|
/// `prior` and `drift_scale` are **competitor configuration**, not per-event
|
||||||
/// values: both are captured when the competitor is first created and ignored
|
/// values. Setting either applies to the competitor for the whole history, not
|
||||||
/// on every later appearance. Setting either on a key the history already knows
|
/// just to this event, and applies whenever it is supplied — including on a key
|
||||||
/// has no effect.
|
/// the history already knows. Because configuration lives on the competitor and
|
||||||
|
/// `converge` refits from competitor state, configuring one late still refits
|
||||||
|
/// the whole history rather than taking effect only from that event onward.
|
||||||
|
///
|
||||||
|
/// Repeating the same value is inert, which is the expected shape when the
|
||||||
|
/// configuration is a property of the domain. Supplying two *different* values
|
||||||
|
/// for one competitor within a single batch is
|
||||||
|
/// `InferenceError::ConflictingCompetitorConfig`: events in a batch have no
|
||||||
|
/// order, so there would be no well-defined winner.
|
||||||
#[derive(Clone, Debug)]
|
#[derive(Clone, Debug)]
|
||||||
pub struct Member<K> {
|
pub struct Member<K> {
|
||||||
pub key: K,
|
pub key: K,
|
||||||
|
|||||||
+67
-7
@@ -2,6 +2,7 @@ use crate::{
|
|||||||
N_INF, approx, cdf,
|
N_INF, approx, cdf,
|
||||||
factor::{Factor, VarId, VarStore},
|
factor::{Factor, VarId, VarStore},
|
||||||
gaussian::Gaussian,
|
gaussian::Gaussian,
|
||||||
|
sf,
|
||||||
};
|
};
|
||||||
|
|
||||||
/// EP truncation factor on a diff variable.
|
/// EP truncation factor on a diff variable.
|
||||||
@@ -74,16 +75,29 @@ impl Factor for TruncFactor {
|
|||||||
|
|
||||||
/// P(diff > margin) for non-tie, P(|diff| < margin) for tie.
|
/// P(diff > margin) for non-tie, P(|diff| < margin) for tie.
|
||||||
///
|
///
|
||||||
/// Clamped to a positive floor: for a near-certain outcome the tail rounds to
|
/// Both branches pick whichever tail keeps their terms *small*, because the
|
||||||
/// exactly 0.0, and the `erfc` approximation used by `cdf` carries ~1e-7 error
|
/// alternative is subtracting two numbers that both approach 1. That
|
||||||
/// so it can even return slightly more than 1.0, making the difference
|
/// subtraction is not a rounding detail: it loses every digit of an unlikely
|
||||||
/// negative. Either would send `log_evidence` to `-inf` or NaN and poison the
|
/// outcome's evidence, and an unlikely outcome is precisely the one worth
|
||||||
/// sum across the whole history.
|
/// scoring. `1 - cdf` returned exactly zero past ~8.3 sigma, where the true
|
||||||
|
/// probability is 1e-19; clamped, that reached `log_evidence` as -708 instead
|
||||||
|
/// of -43.
|
||||||
|
///
|
||||||
|
/// The clamp remains as a guard rather than a workaround: `erfc` carries ~1e-7
|
||||||
|
/// relative error, so a probability of exactly 1 can still come back a hair
|
||||||
|
/// above it, and `ln` of a negative would poison the sum for the whole history.
|
||||||
fn cavity_evidence(diff: Gaussian, margin: f64, tie: bool) -> f64 {
|
fn cavity_evidence(diff: Gaussian, margin: f64, tie: bool) -> f64 {
|
||||||
|
let (mu, sigma) = (diff.mu(), diff.sigma());
|
||||||
|
|
||||||
let raw = if tie {
|
let raw = if tie {
|
||||||
cdf(margin, diff.mu(), diff.sigma()) - cdf(-margin, diff.mu(), diff.sigma())
|
if mu < -margin {
|
||||||
|
// Both CDFs sit against 1 here; both survival terms are small.
|
||||||
|
sf(-margin, mu, sigma) - sf(margin, mu, sigma)
|
||||||
} else {
|
} else {
|
||||||
1.0 - cdf(margin, diff.mu(), diff.sigma())
|
cdf(margin, mu, sigma) - cdf(-margin, mu, sigma)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
sf(margin, mu, sigma)
|
||||||
};
|
};
|
||||||
|
|
||||||
raw.clamp(f64::MIN_POSITIVE, 1.0)
|
raw.clamp(f64::MIN_POSITIVE, 1.0)
|
||||||
@@ -132,6 +146,52 @@ mod tests {
|
|||||||
assert_eq!(f.evidence_cached.unwrap(), first);
|
assert_eq!(f.evidence_cached.unwrap(), first);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The defect this guards: `1 - cdf` collapsed to zero for a surprising
|
||||||
|
/// result, the clamp turned that into `f64::MIN_POSITIVE`, and
|
||||||
|
/// `log_evidence` reported ln of *that* — about -708 whatever the truth
|
||||||
|
/// was. An upset is the observation a model-comparison score exists to
|
||||||
|
/// notice, so it was wrong exactly where it mattered.
|
||||||
|
#[test]
|
||||||
|
fn evidence_of_an_upset_is_not_flattened_to_the_clamp_floor() {
|
||||||
|
// diff ~ N(-9, 1) with margin 0: the favoured side lost by nine sigma.
|
||||||
|
let evidence = cavity_evidence(Gaussian::from_ms(-9.0, 1.0), 0.0, false);
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
evidence > f64::MIN_POSITIVE,
|
||||||
|
"evidence collapsed onto the clamp floor: {evidence}"
|
||||||
|
);
|
||||||
|
// P(X > 0) for X ~ N(-9, 1) is the standard normal tail at 9 sigma.
|
||||||
|
assert!(
|
||||||
|
(evidence - 1.128_588e-19).abs() / 1.128_588e-19 < 1e-6,
|
||||||
|
"expected ~1.13e-19, got {evidence}"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
(evidence.ln() + 43.628).abs() < 1e-2,
|
||||||
|
"log evidence {} should be about -43.6, not -708",
|
||||||
|
evidence.ln()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Evidence must stay finite and positive however extreme the mismatch,
|
||||||
|
/// since `log_evidence` sums across the whole history and one `-inf` or
|
||||||
|
/// `NaN` poisons all of it.
|
||||||
|
#[test]
|
||||||
|
fn evidence_stays_positive_and_finite_at_any_separation() {
|
||||||
|
for mu in [-300.0f64, -50.0, -9.0, 0.0, 9.0, 50.0, 300.0] {
|
||||||
|
for tie in [false, true] {
|
||||||
|
let e = cavity_evidence(Gaussian::from_ms(mu, 1.0), 1.0, tie);
|
||||||
|
assert!(
|
||||||
|
e.is_finite() && e > 0.0 && e <= 1.0,
|
||||||
|
"mu={mu} tie={tie}: evidence {e} is not a probability"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
e.ln().is_finite(),
|
||||||
|
"mu={mu} tie={tie}: ln evidence is not finite"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn tie_evidence_uses_two_sided() {
|
fn tie_evidence_uses_two_sided() {
|
||||||
let mut vars = VarStore::new();
|
let mut vars = VarStore::new();
|
||||||
|
|||||||
+7
-2
@@ -433,6 +433,9 @@ impl<'a, T: Time, D: Drift<T>> Game<'a, T, D> {
|
|||||||
impl<T: Time, D: Drift<T>> Game<'_, T, D> {
|
impl<T: Time, D: Drift<T>> Game<'_, T, D> {
|
||||||
/// # Errors
|
/// # Errors
|
||||||
///
|
///
|
||||||
|
/// - `InvalidParameter` if `options.convergence` is out of range — an
|
||||||
|
/// `alpha` of zero would leave every EP update unapplied and silently
|
||||||
|
/// return the priors.
|
||||||
/// - `InvalidProbability` if `options.p_draw` is outside `[0.0, 1.0)`.
|
/// - `InvalidProbability` if `options.p_draw` is outside `[0.0, 1.0)`.
|
||||||
/// - `MismatchedShape` if the outcome's rank count differs from `teams.len()`.
|
/// - `MismatchedShape` if the outcome's rank count differs from `teams.len()`.
|
||||||
/// - `WrongOutcomeKind` if `outcome` is not `Outcome::Ranked`.
|
/// - `WrongOutcomeKind` if `outcome` is not `Outcome::Ranked`.
|
||||||
@@ -444,6 +447,7 @@ impl<T: Time, D: Drift<T>> Game<'_, T, D> {
|
|||||||
outcome: crate::Outcome,
|
outcome: crate::Outcome,
|
||||||
options: &GameOptions,
|
options: &GameOptions,
|
||||||
) -> Result<OwnedGame<T, D>, crate::InferenceError> {
|
) -> Result<OwnedGame<T, D>, crate::InferenceError> {
|
||||||
|
options.convergence.validate()?;
|
||||||
if !(0.0..1.0).contains(&options.p_draw) {
|
if !(0.0..1.0).contains(&options.p_draw) {
|
||||||
return Err(crate::InferenceError::InvalidProbability {
|
return Err(crate::InferenceError::InvalidProbability {
|
||||||
value: options.p_draw,
|
value: options.p_draw,
|
||||||
@@ -491,8 +495,8 @@ impl<T: Time, D: Drift<T>> Game<'_, T, D> {
|
|||||||
|
|
||||||
/// # Errors
|
/// # Errors
|
||||||
///
|
///
|
||||||
/// - `InvalidParameter` if `options.score_sigma` is not strictly positive,
|
/// - `InvalidParameter` if `options.score_sigma` is not strictly positive
|
||||||
/// or is NaN.
|
/// or is NaN, or if `options.convergence` is out of range.
|
||||||
/// - `MismatchedShape` if the outcome's score count differs from `teams.len()`.
|
/// - `MismatchedShape` if the outcome's score count differs from `teams.len()`.
|
||||||
/// - `WrongOutcomeKind` if `outcome` is not `Outcome::Scored`.
|
/// - `WrongOutcomeKind` if `outcome` is not `Outcome::Scored`.
|
||||||
pub fn scored(
|
pub fn scored(
|
||||||
@@ -500,6 +504,7 @@ impl<T: Time, D: Drift<T>> Game<'_, T, D> {
|
|||||||
outcome: crate::Outcome,
|
outcome: crate::Outcome,
|
||||||
options: &GameOptions,
|
options: &GameOptions,
|
||||||
) -> Result<OwnedGame<T, D>, crate::InferenceError> {
|
) -> Result<OwnedGame<T, D>, crate::InferenceError> {
|
||||||
|
options.convergence.validate()?;
|
||||||
if options.score_sigma <= 0.0 || options.score_sigma.is_nan() {
|
if options.score_sigma <= 0.0 || options.score_sigma.is_nan() {
|
||||||
return Err(crate::InferenceError::InvalidParameter {
|
return Err(crate::InferenceError::InvalidParameter {
|
||||||
name: "score_sigma",
|
name: "score_sigma",
|
||||||
|
|||||||
+192
-63
@@ -172,6 +172,23 @@ impl Default for HistoryBuilder<i64, ConstantDrift, NullObserver, &'static str>
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Configuration a caller attached to a competitor via `Member`.
|
||||||
|
///
|
||||||
|
/// Carries *what was explicitly set* rather than a merged `Rating`, so a member
|
||||||
|
/// that sets only `drift_scale` does not also assert the default prior — which
|
||||||
|
/// would spuriously conflict with a prior seeded on an earlier event.
|
||||||
|
#[derive(Clone, Copy, Default)]
|
||||||
|
pub(crate) struct CompetitorConfig {
|
||||||
|
prior: Option<Gaussian>,
|
||||||
|
drift_scale: Option<f64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl CompetitorConfig {
|
||||||
|
fn is_empty(self) -> bool {
|
||||||
|
self.prior.is_none() && self.drift_scale.is_none()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub struct History<
|
pub struct History<
|
||||||
T: Time = i64,
|
T: Time = i64,
|
||||||
D: Drift<T> = ConstantDrift,
|
D: Drift<T> = ConstantDrift,
|
||||||
@@ -538,10 +555,27 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
|
|||||||
.sum()
|
.sum()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Each team's performance Gaussian, and its member count.
|
/// The configured observer.
|
||||||
///
|
///
|
||||||
/// Performance is skill inflated by `beta`: the question a prediction
|
/// `History` takes its observer by value, so this is how a caller inspects
|
||||||
/// answers is "how will they do today", not "how good are they".
|
/// one it did not keep a handle to. For an observer that accumulates
|
||||||
|
/// state, prefer passing an `Arc` and keeping a clone — see the
|
||||||
|
/// [`Observer`] docs.
|
||||||
|
#[must_use]
|
||||||
|
pub fn observer(&self) -> &O {
|
||||||
|
&self.observer
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Consume the history and return its observer.
|
||||||
|
///
|
||||||
|
/// Useful for reclaiming a non-shared observer's accumulated state after
|
||||||
|
/// `converge` without needing interior mutability.
|
||||||
|
#[must_use]
|
||||||
|
pub fn into_observer(self) -> O {
|
||||||
|
self.observer
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every team's member skills, validated.
|
||||||
///
|
///
|
||||||
/// # Errors
|
/// # Errors
|
||||||
///
|
///
|
||||||
@@ -549,39 +583,60 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
|
|||||||
/// reported rather than dropped — silently skipping them would turn a team
|
/// reported rather than dropped — silently skipping them would turn a team
|
||||||
/// of strangers into a confident-looking prediction about nobody, which is
|
/// of strangers into a confident-looking prediction about nobody, which is
|
||||||
/// the failure this replaced.
|
/// the failure this replaced.
|
||||||
fn performances(&self, teams: &[&[&K]]) -> Result<(Vec<Gaussian>, Vec<usize>), InferenceError> {
|
fn member_skills(&self, teams: &[&[&K]]) -> Result<Vec<Vec<Gaussian>>, InferenceError> {
|
||||||
if teams.len() < 2 {
|
if teams.len() < 2 {
|
||||||
return Err(InferenceError::NotEnoughTeams { got: teams.len() });
|
return Err(InferenceError::NotEnoughTeams { got: teams.len() });
|
||||||
}
|
}
|
||||||
|
|
||||||
let mut performances = Vec::with_capacity(teams.len());
|
let mut gathered = Vec::with_capacity(teams.len());
|
||||||
let mut sizes = Vec::with_capacity(teams.len());
|
|
||||||
|
|
||||||
for (team_idx, team) in teams.iter().enumerate() {
|
for (team_idx, team) in teams.iter().enumerate() {
|
||||||
if team.is_empty() {
|
if team.is_empty() {
|
||||||
return Err(InferenceError::EmptyTeam { team: team_idx });
|
return Err(InferenceError::EmptyTeam { team: team_idx });
|
||||||
}
|
}
|
||||||
|
|
||||||
let mut total = crate::N00;
|
let mut members = Vec::with_capacity(team.len());
|
||||||
for (member_idx, key) in team.iter().enumerate() {
|
for (member_idx, key) in team.iter().enumerate() {
|
||||||
let unknown = InferenceError::UnknownKey {
|
let unknown = InferenceError::UnknownKey {
|
||||||
team: team_idx,
|
team: team_idx,
|
||||||
member: member_idx,
|
member: member_idx,
|
||||||
};
|
};
|
||||||
let index = self.keys.get(*key).ok_or(unknown.clone())?;
|
let index = self.keys.get(*key).ok_or(unknown.clone())?;
|
||||||
let skill = self
|
members.push(
|
||||||
.time_slices
|
self.time_slices
|
||||||
.iter()
|
.iter()
|
||||||
.rev()
|
.rev()
|
||||||
.find_map(|ts| ts.skills.get(index).map(|s| s.posterior()))
|
.find_map(|ts| ts.skills.get(index).map(|s| s.posterior()))
|
||||||
.ok_or(unknown)?;
|
.ok_or(unknown)?,
|
||||||
total = total + skill.forget(self.beta.powi(2));
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
performances.push(total);
|
gathered.push(members);
|
||||||
sizes.push(team.len());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
Ok(gathered)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Each team's performance Gaussian, and its member count.
|
||||||
|
///
|
||||||
|
/// Performance is skill inflated by `beta`: the question a prediction
|
||||||
|
/// answers is "how will they do today", not "how good are they".
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// As [`History::member_skills`].
|
||||||
|
fn performances(&self, teams: &[&[&K]]) -> Result<(Vec<Gaussian>, Vec<usize>), InferenceError> {
|
||||||
|
let skills = self.member_skills(teams)?;
|
||||||
|
|
||||||
|
let performances = skills
|
||||||
|
.iter()
|
||||||
|
.map(|team| {
|
||||||
|
team.iter()
|
||||||
|
.fold(crate::N00, |acc, s| acc + s.forget(self.beta.powi(2)))
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
let sizes = skills.iter().map(Vec::len).collect();
|
||||||
|
|
||||||
Ok((performances, sizes))
|
Ok((performances, sizes))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -618,38 +673,55 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
|
|||||||
///
|
///
|
||||||
/// `NotEnoughTeams`, `EmptyTeam`, or `UnknownKey`.
|
/// `NotEnoughTeams`, `EmptyTeam`, or `UnknownKey`.
|
||||||
pub fn predict_quality(&self, teams: &[&[&K]]) -> Result<f64, InferenceError> {
|
pub fn predict_quality(&self, teams: &[&[&K]]) -> Result<f64, InferenceError> {
|
||||||
let mut groups: Vec<Vec<Gaussian>> = Vec::with_capacity(teams.len());
|
let groups = self.member_skills(teams)?;
|
||||||
|
|
||||||
for (team_idx, team) in teams.iter().enumerate() {
|
|
||||||
if team.is_empty() {
|
|
||||||
return Err(InferenceError::EmptyTeam { team: team_idx });
|
|
||||||
}
|
|
||||||
let mut members = Vec::with_capacity(team.len());
|
|
||||||
for (member_idx, key) in team.iter().enumerate() {
|
|
||||||
let unknown = InferenceError::UnknownKey {
|
|
||||||
team: team_idx,
|
|
||||||
member: member_idx,
|
|
||||||
};
|
|
||||||
let index = self.keys.get(*key).ok_or(unknown.clone())?;
|
|
||||||
members.push(
|
|
||||||
self.time_slices
|
|
||||||
.iter()
|
|
||||||
.rev()
|
|
||||||
.find_map(|ts| ts.skills.get(index).map(|s| s.posterior()))
|
|
||||||
.ok_or(unknown)?,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
groups.push(members);
|
|
||||||
}
|
|
||||||
|
|
||||||
if groups.len() < 2 {
|
|
||||||
return Err(InferenceError::NotEnoughTeams { got: groups.len() });
|
|
||||||
}
|
|
||||||
|
|
||||||
let group_refs: Vec<&[Gaussian]> = groups.iter().map(Vec::as_slice).collect();
|
let group_refs: Vec<&[Gaussian]> = groups.iter().map(Vec::as_slice).collect();
|
||||||
Ok(crate::quality(&group_refs, self.beta))
|
Ok(crate::quality(&group_refs, self.beta))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Expected information gain of running this matchup, in nats.
|
||||||
|
///
|
||||||
|
/// Answers "which comparison should I run next" rather than "who will
|
||||||
|
/// win": the outcome-weighted divergence between current beliefs and the
|
||||||
|
/// beliefs each possible result would produce. Higher means the result
|
||||||
|
/// would teach you more.
|
||||||
|
///
|
||||||
|
/// Uses each competitor's current skill as the prior, and the history's
|
||||||
|
/// own `beta`, `drift` and `p_draw`, so the outcomes weighted here are the
|
||||||
|
/// ones that would actually be fitted if the matchup were played and
|
||||||
|
/// recorded.
|
||||||
|
///
|
||||||
|
/// Distinct from [`History::predict_quality`], which measures *fairness*.
|
||||||
|
/// The two coincide for two evenly matched competitors and diverge
|
||||||
|
/// elsewhere. See [`expected_information_gain`](crate::expected_information_gain)
|
||||||
|
/// for the scale, the analytic `ln k` ceiling, and the cost.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
///
|
||||||
|
/// As [`History::member_skills`], plus `TooManyTeams` and anything
|
||||||
|
/// inference returns for a hypothetical outcome.
|
||||||
|
pub fn expected_information_gain(&self, teams: &[&[&K]]) -> Result<f64, InferenceError> {
|
||||||
|
let skills = self.member_skills(teams)?;
|
||||||
|
|
||||||
|
let ratings: Vec<Vec<Rating<T, D>>> = skills
|
||||||
|
.iter()
|
||||||
|
.map(|team| {
|
||||||
|
team.iter()
|
||||||
|
.map(|&skill| Rating::new(skill, self.beta, self.drift))
|
||||||
|
.collect()
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
let team_refs: Vec<&[Rating<T, D>]> = ratings.iter().map(Vec::as_slice).collect();
|
||||||
|
|
||||||
|
crate::expected_information_gain(
|
||||||
|
&team_refs,
|
||||||
|
&crate::GameOptions {
|
||||||
|
p_draw: self.p_draw,
|
||||||
|
score_sigma: self.score_sigma,
|
||||||
|
convergence: self.convergence,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
/// `P(team i finishes strictly first)`, for every team.
|
/// `P(team i finishes strictly first)`, for every team.
|
||||||
///
|
///
|
||||||
/// Supports any number of teams. Because performances are independent
|
/// Supports any number of teams. Because performances are independent
|
||||||
@@ -821,7 +893,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
|
|||||||
times: Vec<T>,
|
times: Vec<T>,
|
||||||
mut weights: Option<Vec<Vec<Vec<f64>>>>,
|
mut weights: Option<Vec<Vec<Vec<f64>>>>,
|
||||||
kinds: Vec<EventKind>,
|
kinds: Vec<EventKind>,
|
||||||
mut priors: HashMap<Index, Rating<T, D>>,
|
priors: HashMap<Index, CompetitorConfig>,
|
||||||
) -> Result<(), InferenceError> {
|
) -> Result<(), InferenceError> {
|
||||||
if results
|
if results
|
||||||
.as_ref()
|
.as_ref()
|
||||||
@@ -888,17 +960,60 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
|
|||||||
|
|
||||||
this_agent.push(*agent);
|
this_agent.push(*agent);
|
||||||
|
|
||||||
if !self.agents.contains(*agent) {
|
let config = priors.get(agent).copied().unwrap_or_default();
|
||||||
self.agents.insert(
|
|
||||||
*agent,
|
if self.agents.contains(*agent) {
|
||||||
Competitor {
|
// Seeding a competitor the history already knows. This used to
|
||||||
rating: priors.remove(agent).unwrap_or_else(|| {
|
// be dropped on the floor: `remove` was only reached on the
|
||||||
Rating::new(
|
// create path, so a prior applied on a competitor's very first
|
||||||
|
// event and was silently ignored ever after.
|
||||||
|
if config.is_empty() {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
let rating = &mut self.agents.get_mut(*agent).unwrap().rating;
|
||||||
|
if let Some(prior) = config.prior {
|
||||||
|
rating.prior = prior;
|
||||||
|
}
|
||||||
|
if let Some(scale) = config.drift_scale {
|
||||||
|
rating.drift_scale = scale;
|
||||||
|
}
|
||||||
|
let seeded = rating.prior;
|
||||||
|
|
||||||
|
if config.prior.is_some() {
|
||||||
|
// The prior is not re-derived every pass the way drift is.
|
||||||
|
// A competitor's earliest slice has its forward message set
|
||||||
|
// to the prior once, at ingestion, and `iteration` refreshes
|
||||||
|
// only slices after the first — so without this, a late
|
||||||
|
// prior would reach the drift terms and nothing else, which
|
||||||
|
// is a subtler version of the silent drop this replaced.
|
||||||
|
//
|
||||||
|
// `clean` has just nulled every message, so the earliest
|
||||||
|
// slice's forward is exactly the prior.
|
||||||
|
for slice in &mut self.time_slices {
|
||||||
|
if let Some(skill) = slice.skills.get_mut(*agent) {
|
||||||
|
skill.forward = seeded;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
let mut rating = Rating::new(
|
||||||
Gaussian::from_ms(self.mu, self.sigma),
|
Gaussian::from_ms(self.mu, self.sigma),
|
||||||
self.beta,
|
self.beta,
|
||||||
self.drift,
|
self.drift,
|
||||||
)
|
);
|
||||||
}),
|
if let Some(prior) = config.prior {
|
||||||
|
rating.prior = prior;
|
||||||
|
}
|
||||||
|
if let Some(scale) = config.drift_scale {
|
||||||
|
rating.drift_scale = scale;
|
||||||
|
}
|
||||||
|
|
||||||
|
self.agents.insert(
|
||||||
|
*agent,
|
||||||
|
Competitor {
|
||||||
|
rating,
|
||||||
message: None,
|
message: None,
|
||||||
last_time: None,
|
last_time: None,
|
||||||
},
|
},
|
||||||
@@ -1115,7 +1230,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
|
|||||||
let mut times: Vec<T> = Vec::with_capacity(events.len());
|
let mut times: Vec<T> = Vec::with_capacity(events.len());
|
||||||
let mut weights: Vec<Vec<Vec<f64>>> = Vec::with_capacity(events.len());
|
let mut weights: Vec<Vec<Vec<f64>>> = Vec::with_capacity(events.len());
|
||||||
let mut kinds: Vec<EventKind> = Vec::with_capacity(events.len());
|
let mut kinds: Vec<EventKind> = Vec::with_capacity(events.len());
|
||||||
let mut priors: HashMap<Index, Rating<T, D>> = HashMap::new();
|
let mut priors: HashMap<Index, CompetitorConfig> = HashMap::new();
|
||||||
|
|
||||||
for ev in events {
|
for ev in events {
|
||||||
if ev.outcome.team_count() != ev.teams.len() {
|
if ev.outcome.team_count() != ev.teams.len() {
|
||||||
@@ -1149,23 +1264,37 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// `prior` and `drift_scale` are competitor configuration,
|
// `prior` and `drift_scale` configure the competitor, not
|
||||||
// captured here and consumed at competitor creation. Both
|
// the event. Both land in the same entry so a member may
|
||||||
// land in the same entry so a member may set either alone.
|
// set either alone.
|
||||||
|
//
|
||||||
|
// Events within a batch are not ordered, so a batch that
|
||||||
|
// sets one field twice with different values has no
|
||||||
|
// well-defined result — "last one wins" would depend on
|
||||||
|
// iteration order, which `tests/ingestion_equivalence.rs`
|
||||||
|
// exists to rule out. Repeating the *same* value is fine,
|
||||||
|
// and is the expected shape when the configuration is a
|
||||||
|
// property of the domain rather than of one event.
|
||||||
if member.prior.is_some() || member.drift_scale.is_some() {
|
if member.prior.is_some() || member.drift_scale.is_some() {
|
||||||
let rating = priors.entry(idx).or_insert_with(|| {
|
let entry = priors.entry(idx).or_default();
|
||||||
Rating::new(
|
|
||||||
Gaussian::from_ms(self.mu, self.sigma),
|
|
||||||
self.beta,
|
|
||||||
self.drift,
|
|
||||||
)
|
|
||||||
});
|
|
||||||
|
|
||||||
if let Some(prior) = member.prior {
|
if let Some(prior) = member.prior {
|
||||||
rating.prior = prior;
|
if entry.prior.is_some_and(|held| held != prior) {
|
||||||
|
return Err(InferenceError::ConflictingCompetitorConfig {
|
||||||
|
competitor: idx.get(),
|
||||||
|
field: "prior",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
entry.prior = Some(prior);
|
||||||
}
|
}
|
||||||
if let Some(scale) = member.drift_scale {
|
if let Some(scale) = member.drift_scale {
|
||||||
rating.drift_scale = scale;
|
if entry.drift_scale.is_some_and(|held| held != scale) {
|
||||||
|
return Err(InferenceError::ConflictingCompetitorConfig {
|
||||||
|
competitor: idx.get(),
|
||||||
|
field: "drift_scale",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
entry.drift_scale = Some(scale);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+289
-8
@@ -110,6 +110,7 @@ pub(crate) mod arena;
|
|||||||
mod time;
|
mod time;
|
||||||
mod time_slice;
|
mod time_slice;
|
||||||
pub use time_slice::{EventKind, TimeSlice};
|
pub use time_slice::{EventKind, TimeSlice};
|
||||||
|
mod acquisition;
|
||||||
mod color_group;
|
mod color_group;
|
||||||
mod competitor;
|
mod competitor;
|
||||||
mod convergence;
|
mod convergence;
|
||||||
@@ -132,6 +133,7 @@ mod rating;
|
|||||||
pub(crate) mod schedule;
|
pub(crate) mod schedule;
|
||||||
pub mod storage;
|
pub mod storage;
|
||||||
|
|
||||||
|
pub use acquisition::expected_information_gain;
|
||||||
pub use competitor::Competitor;
|
pub use competitor::Competitor;
|
||||||
pub use convergence::{ConvergenceOptions, ConvergenceReport};
|
pub use convergence::{ConvergenceOptions, ConvergenceReport};
|
||||||
pub use drift::{ConstantDrift, Drift};
|
pub use drift::{ConstantDrift, Drift};
|
||||||
@@ -166,6 +168,21 @@ pub const ITERATIONS: usize = 30;
|
|||||||
pub const MAX_PREDICTED_TEAMS: usize = predict::MAX_TEAMS_FOR_DISTRIBUTION;
|
pub const MAX_PREDICTED_TEAMS: usize = predict::MAX_TEAMS_FOR_DISTRIBUTION;
|
||||||
|
|
||||||
const SQRT_TAU: f64 = 2.5066282746310002;
|
const SQRT_TAU: f64 = 2.5066282746310002;
|
||||||
|
/// `1 / sqrt(pi)`, the leading factor of the `erfcx` continued fraction.
|
||||||
|
const FRAC_1_SQRT_PI: f64 = 0.564_189_583_547_756_3;
|
||||||
|
/// `sqrt(2 / pi)`, the numerator of the inverse Mills ratio in scaled form.
|
||||||
|
const SQRT_2_OVER_PI: f64 = 0.797_884_560_802_865_4;
|
||||||
|
/// How many window widths into the tail before a tie window is treated as a
|
||||||
|
/// half-line. Beyond this the truncated mass is concentrated within `1/alpha`
|
||||||
|
/// of the near edge, so the far edge contributes nothing measurable.
|
||||||
|
const HALF_LINE_WINDOW: f64 = 10.0;
|
||||||
|
/// Where `v - alpha` switches from subtraction to its asymptotic series.
|
||||||
|
///
|
||||||
|
/// The subtraction loses roughly `eps * alpha^2` of relative precision, and the
|
||||||
|
/// four-term series is good to ~1e-10 by here, so the two are at their closest
|
||||||
|
/// agreement around this point. Below it the subtraction is exact; above it the
|
||||||
|
/// series is.
|
||||||
|
const ASYMPTOTIC_MILLS_ALPHA: f64 = 100.0;
|
||||||
|
|
||||||
pub const N01: Gaussian = Gaussian::from_ms(0.0, 1.0);
|
pub const N01: Gaussian = Gaussian::from_ms(0.0, 1.0);
|
||||||
pub const N00: Gaussian = Gaussian::from_ms(0.0, 0.0);
|
pub const N00: Gaussian = Gaussian::from_ms(0.0, 0.0);
|
||||||
@@ -257,6 +274,50 @@ pub(crate) fn cdf(x: f64, mu: f64, sigma: f64) -> f64 {
|
|||||||
0.5 * erfc(z)
|
0.5 * erfc(z)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// `P(X > x)` for `X ~ N(mu, sigma^2)`.
|
||||||
|
///
|
||||||
|
/// The survival function, computed directly rather than as `1 - cdf(..)`.
|
||||||
|
///
|
||||||
|
/// The two are algebraically identical and numerically are not. `cdf` returns
|
||||||
|
/// a value approaching 1 for an upper tail, so subtracting it from 1 cancels
|
||||||
|
/// away every significant digit the tail had: measured against this function,
|
||||||
|
/// `1 - cdf` carries 7% error by four sigma past the mean and returns exactly
|
||||||
|
/// zero beyond about 8.3 sigma — where the true value is still 1e-19 and
|
||||||
|
/// perfectly representable. `erfc` itself holds ~1e-7 *relative* accuracy down
|
||||||
|
/// to 1e-296, so the precision is there to keep; only the subtraction threw it
|
||||||
|
/// away.
|
||||||
|
///
|
||||||
|
/// This matters most where evidence is smallest, which is exactly where an
|
||||||
|
/// upset makes it interesting: `ln` of a clamped zero is -708 regardless of
|
||||||
|
/// whether the truth was -43 or -600.
|
||||||
|
pub(crate) fn sf(x: f64, mu: f64, sigma: f64) -> f64 {
|
||||||
|
0.5 * erfc((x - mu) / (sigma * SQRT_2))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `e^(x^2) * erfc(x)`, the scaled complementary error function, for `x >= 0`.
|
||||||
|
///
|
||||||
|
/// Exists so the exponential factor common to a Gaussian density and its tail
|
||||||
|
/// integral can be cancelled *analytically* instead of being computed twice
|
||||||
|
/// and divided. Both underflow to zero past about 26 sigma, and their ratio is
|
||||||
|
/// then `0/0` — finite in the limit, `NaN` in floating point.
|
||||||
|
fn erfcx(x: f64) -> f64 {
|
||||||
|
if x < 2.0 {
|
||||||
|
// Below the crossover neither factor is extreme: erfc is O(1) and
|
||||||
|
// exp(x^2) is at most e^4, so the direct product is exact enough and
|
||||||
|
// cheaper than the continued fraction.
|
||||||
|
(x * x).exp() * erfc(x)
|
||||||
|
} else {
|
||||||
|
// erfcx(x) = 1/sqrt(pi) * 1/(x + (1/2)/(x + 1/(x + (3/2)/(x + ...)))),
|
||||||
|
// evaluated by backward recurrence. Converges quickly for x >= 2 and,
|
||||||
|
// unlike the product form, never touches an exponential.
|
||||||
|
let mut f = 0.0;
|
||||||
|
for n in (1..=60u32).rev() {
|
||||||
|
f = (f64::from(n) * 0.5) / (x + f);
|
||||||
|
}
|
||||||
|
FRAC_1_SQRT_PI / (x + f)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fn pdf(x: f64, mu: f64, sigma: f64) -> f64 {
|
fn pdf(x: f64, mu: f64, sigma: f64) -> f64 {
|
||||||
let normalizer = (SQRT_TAU * sigma).powi(-1);
|
let normalizer = (SQRT_TAU * sigma).powi(-1);
|
||||||
let functional = (-((x - mu).powi(2)) / (2.0 * sigma.powi(2))).exp();
|
let functional = (-((x - mu).powi(2)) / (2.0 * sigma.powi(2))).exp();
|
||||||
@@ -264,25 +325,100 @@ fn pdf(x: f64, mu: f64, sigma: f64) -> f64 {
|
|||||||
normalizer * functional
|
normalizer * functional
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Truncated-Gaussian correction terms `(v, w)`.
|
||||||
|
///
|
||||||
|
/// `v` shifts the mean and `w` shrinks the variance. Both are ratios whose
|
||||||
|
/// numerator and denominator underflow together in the tails, so both are
|
||||||
|
/// computed in scaled form there: the shared `exp(-alpha^2 / 2)` is cancelled
|
||||||
|
/// analytically rather than evaluated and divided out. Without that, a
|
||||||
|
/// truncation point beyond about 39 sigma produced `0 / 0` and put `NaN`
|
||||||
|
/// straight into the posterior.
|
||||||
|
/// Truncation terms for a boundary `alpha` standard deviations into the upper
|
||||||
|
/// tail, from the asymptotic expansion of the inverse Mills ratio.
|
||||||
|
///
|
||||||
|
/// `v` tends to `alpha` out here, so the gap between them cannot be obtained by
|
||||||
|
/// subtracting one from the other — the series computes the gap directly, and
|
||||||
|
/// `w = v * gap` then never forms the difference of two large near-equal
|
||||||
|
/// numbers. A far-tail *window* behaves like a half-line once it is more than a
|
||||||
|
/// few multiples of its own width from the mean, so the tie branch shares this.
|
||||||
|
fn half_line_truncation(alpha: f64) -> (f64, f64) {
|
||||||
|
let inv = alpha.recip();
|
||||||
|
let inv_sq = inv * inv;
|
||||||
|
let gap = inv * (1.0 - inv_sq * (2.0 - inv_sq * (10.0 - 74.0 * inv_sq)));
|
||||||
|
let v = alpha + gap;
|
||||||
|
|
||||||
|
(v, v * gap)
|
||||||
|
}
|
||||||
|
|
||||||
fn v_w(mu: f64, sigma: f64, margin: f64, tie: bool) -> (f64, f64) {
|
fn v_w(mu: f64, sigma: f64, margin: f64, tie: bool) -> (f64, f64) {
|
||||||
if !tie {
|
if !tie {
|
||||||
let alpha = (margin - mu) / sigma;
|
let alpha = (margin - mu) / sigma;
|
||||||
|
|
||||||
let v = pdf(-alpha, 0.0, 1.0) / cdf(-alpha, 0.0, 1.0);
|
// v is the inverse Mills ratio, phi(alpha) / Phi(-alpha), and w needs
|
||||||
let w = v * (v + (-alpha));
|
// the gap `v - alpha` as well as v itself. Far into the tail v tends to
|
||||||
|
// alpha, so that gap is a subtraction of two nearly equal numbers and
|
||||||
|
// loses every digit it has: at alpha = 1e6 it drove w above 1 and made
|
||||||
|
// `sqrt(1 - w)` NaN. Past the crossover the gap comes from its
|
||||||
|
// asymptotic series instead, which has no subtraction in it.
|
||||||
|
if alpha >= ASYMPTOTIC_MILLS_ALPHA {
|
||||||
|
return half_line_truncation(alpha);
|
||||||
|
}
|
||||||
|
|
||||||
(v, w)
|
let (v, gap) = if alpha > 0.0 {
|
||||||
|
// Both terms carry exp(-alpha^2 / 2); in scaled form it cancels
|
||||||
|
// and the result stays exact however far into the tail alpha sits.
|
||||||
|
let v = SQRT_2_OVER_PI / erfcx(alpha / SQRT_2);
|
||||||
|
(v, v - alpha)
|
||||||
} else {
|
} else {
|
||||||
|
// Phi(-alpha) >= 1/2 here, so the direct ratio loses nothing.
|
||||||
|
let v = pdf(-alpha, 0.0, 1.0) / cdf(-alpha, 0.0, 1.0);
|
||||||
|
(v, v - alpha)
|
||||||
|
};
|
||||||
|
|
||||||
|
(v, v * gap)
|
||||||
|
} else {
|
||||||
|
// v is odd in mu and w is even, so fold to mu <= 0. Both truncation
|
||||||
|
// points then sit in the upper tail, where the scaled form applies.
|
||||||
|
let flipped = mu > 0.0;
|
||||||
|
let mu = if flipped { -mu } else { mu };
|
||||||
|
|
||||||
let alpha = (-margin - mu) / sigma;
|
let alpha = (-margin - mu) / sigma;
|
||||||
let beta = (margin - mu) / sigma;
|
let beta = (margin - mu) / sigma;
|
||||||
|
|
||||||
let v = (pdf(alpha, 0.0, 1.0) - pdf(beta, 0.0, 1.0))
|
// `w` comes out of `v * v - u`, and both terms grow as alpha^2 while
|
||||||
/ (cdf(beta, 0.0, 1.0) - cdf(alpha, 0.0, 1.0));
|
// their difference stays O(1) — at alpha = 1e9 that subtraction had no
|
||||||
let u = (alpha * pdf(alpha, 0.0, 1.0) - beta * pdf(beta, 0.0, 1.0))
|
// digits left and returned w = -128, making `sqrt(1 - w)` nonsense.
|
||||||
/ (cdf(beta, 0.0, 1.0) - cdf(alpha, 0.0, 1.0));
|
// Once the window sits many of its own widths into the tail it is
|
||||||
|
// indistinguishable from a half-line, so the asymptotic covers it with
|
||||||
|
// no subtraction at all.
|
||||||
|
if alpha >= ASYMPTOTIC_MILLS_ALPHA && alpha * (beta - alpha) >= HALF_LINE_WINDOW {
|
||||||
|
let (v, w) = half_line_truncation(alpha);
|
||||||
|
return (if flipped { -v } else { v }, w);
|
||||||
|
}
|
||||||
|
|
||||||
|
let (v, u) = if alpha > 0.0 {
|
||||||
|
// beta > alpha > 0, so this ratio of exponentials is at most 1 and
|
||||||
|
// cannot overflow.
|
||||||
|
let scale = (0.5 * (alpha * alpha - beta * beta)).exp();
|
||||||
|
let denominator = 0.5 * (erfcx(alpha / SQRT_2) - scale * erfcx(beta / SQRT_2));
|
||||||
|
|
||||||
|
(
|
||||||
|
(1.0 - scale) / SQRT_TAU / denominator,
|
||||||
|
(alpha - beta * scale) / SQRT_TAU / denominator,
|
||||||
|
)
|
||||||
|
} else {
|
||||||
|
// The interval straddles the mean, so nothing here is small.
|
||||||
|
let denominator = cdf(beta, 0.0, 1.0) - cdf(alpha, 0.0, 1.0);
|
||||||
|
|
||||||
|
(
|
||||||
|
(pdf(alpha, 0.0, 1.0) - pdf(beta, 0.0, 1.0)) / denominator,
|
||||||
|
(alpha * pdf(alpha, 0.0, 1.0) - beta * pdf(beta, 0.0, 1.0)) / denominator,
|
||||||
|
)
|
||||||
|
};
|
||||||
|
|
||||||
let w = -(u - v.powi(2));
|
let w = -(u - v.powi(2));
|
||||||
|
|
||||||
(v, w)
|
(if flipped { -v } else { v }, w)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -465,6 +601,151 @@ mod tests {
|
|||||||
assert_eq!(sort_time(&[0i64, 1, 2, 0], true), vec![2, 1, 0, 3]);
|
assert_eq!(sort_time(&[0i64, 1, 2, 0], true), vec![2, 1, 0, 3]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Upper-tail values of the standard normal, from published tables. The
|
||||||
|
/// point is not the digits — `erfc` only carries ~1e-7 relative — but that
|
||||||
|
/// a number comes back at all: `1 - cdf` returned exactly zero for every
|
||||||
|
/// one of these.
|
||||||
|
#[test]
|
||||||
|
fn survival_function_survives_the_far_tail() {
|
||||||
|
for (z, expected) in [
|
||||||
|
(9.0f64, 1.128_588e-19),
|
||||||
|
(12.0, 1.776_482e-33),
|
||||||
|
(20.0, 2.753_624e-89),
|
||||||
|
(37.0, 5.725_571e-300),
|
||||||
|
] {
|
||||||
|
let got = sf(z, 0.0, 1.0);
|
||||||
|
assert!(got > 0.0, "sf({z}) collapsed to zero");
|
||||||
|
assert!(
|
||||||
|
(got - expected).abs() / expected < 1e-6,
|
||||||
|
"sf({z}) = {got}, expected ~{expected}"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
1.0 - cdf(z, 0.0, 1.0),
|
||||||
|
0.0,
|
||||||
|
"the naive form should still be zero here"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Where no cancellation happens the two forms must agree exactly enough
|
||||||
|
/// that nothing else in the crate shifts.
|
||||||
|
#[test]
|
||||||
|
fn survival_function_matches_the_naive_form_where_that_form_works() {
|
||||||
|
for z in [-4.0f64, -1.0, 0.0, 0.5, 1.0, 2.0, 3.0, 4.0] {
|
||||||
|
let naive = 1.0 - cdf(z, 0.0, 1.0);
|
||||||
|
let direct = sf(z, 0.0, 1.0);
|
||||||
|
// Bounded by `erfc`'s own ~1e-7 relative error, not by the
|
||||||
|
// subtraction: the two forms evaluate `erfc` at different points
|
||||||
|
// and the approximation is not exactly antisymmetric.
|
||||||
|
assert!(
|
||||||
|
(naive - direct).abs() < 1e-6,
|
||||||
|
"z={z}: naive {naive} vs direct {direct}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn survival_and_cdf_partition_the_mass() {
|
||||||
|
for z in [-3.0f64, -0.5, 0.0, 1.0, 2.5] {
|
||||||
|
let total = sf(z, 1.0, 2.0) + cdf(z, 1.0, 2.0);
|
||||||
|
// `erfc(z) + erfc(-z) == 2` only to the accuracy of the
|
||||||
|
// approximation, which is ~1e-7 relative.
|
||||||
|
assert!((total - 1.0).abs() < 1e-6, "z={z}: {total}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `erfcx` switches formulation at x = 2; the two sides must meet.
|
||||||
|
#[test]
|
||||||
|
fn erfcx_is_continuous_across_its_crossover() {
|
||||||
|
for x in [1.90f64, 1.99, 1.999, 2.0, 2.001, 2.01, 2.10] {
|
||||||
|
let direct = (x * x).exp() * erfc(x);
|
||||||
|
let scaled = erfcx(x);
|
||||||
|
assert!(
|
||||||
|
(direct - scaled).abs() / scaled < 1e-6,
|
||||||
|
"x={x}: direct {direct} vs erfcx {scaled}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The whole reason `erfcx` exists: it stays finite and O(1/x) exactly
|
||||||
|
/// where `exp(x^2)` overflows and `erfc(x)` underflows.
|
||||||
|
#[test]
|
||||||
|
fn erfcx_stays_finite_where_its_factors_do_not() {
|
||||||
|
for x in [27.0f64, 50.0, 1.0e3, 1.0e8] {
|
||||||
|
let scaled = erfcx(x);
|
||||||
|
assert!(scaled.is_finite() && scaled > 0.0, "erfcx({x}) = {scaled}");
|
||||||
|
// Asymptotically erfcx(x) -> 1 / (x * sqrt(pi)).
|
||||||
|
let asymptote = 1.0 / (x * std::f64::consts::PI.sqrt());
|
||||||
|
assert!(
|
||||||
|
(scaled - asymptote).abs() / asymptote < 1e-2,
|
||||||
|
"erfcx({x}) = {scaled} strays from its asymptote {asymptote}"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
(x * x).exp().is_infinite(),
|
||||||
|
"x={x} should overflow the direct form"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Truncation must never produce a non-finite posterior. Before the scaled
|
||||||
|
/// formulation these returned NaN from `0 / 0` past about 39 sigma.
|
||||||
|
#[test]
|
||||||
|
fn truncation_stays_finite_arbitrarily_far_into_the_tail() {
|
||||||
|
for alpha in [0.0f64, 8.0, 38.0, 40.0, 100.0, 1.0e3, 1.0e6, 1.0e9, 1.0e15] {
|
||||||
|
for tie in [false, true] {
|
||||||
|
let (v, w) = v_w(-alpha, 1.0, if tie { 1.0 } else { 0.0 }, tie);
|
||||||
|
assert!(v.is_finite(), "alpha={alpha} tie={tie}: v = {v}");
|
||||||
|
assert!(w.is_finite(), "alpha={alpha} tie={tie}: w = {w}");
|
||||||
|
// sigma_trunc = sigma * sqrt(1 - w) must stay real.
|
||||||
|
assert!(
|
||||||
|
(0.0..=1.0).contains(&w),
|
||||||
|
"alpha={alpha} tie={tie}: w = {w} leaves sqrt(1 - w) imaginary"
|
||||||
|
);
|
||||||
|
|
||||||
|
let (mu_t, sigma_t) = trunc(-alpha, 1.0, if tie { 1.0 } else { 0.0 }, tie);
|
||||||
|
assert!(
|
||||||
|
mu_t.is_finite() && sigma_t.is_finite(),
|
||||||
|
"alpha={alpha} tie={tie}: trunc = ({mu_t}, {sigma_t})"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The Mills gap switches from subtraction to series at alpha = 100. Both
|
||||||
|
/// are supposed to be right there; if they disagree, the crossover is in
|
||||||
|
/// the wrong place.
|
||||||
|
#[test]
|
||||||
|
fn the_mills_gap_series_meets_the_scaled_form() {
|
||||||
|
for alpha in [50.0f64, 99.0, 100.0, 101.0, 200.0] {
|
||||||
|
let scaled = SQRT_2_OVER_PI / erfcx(alpha / SQRT_2) - alpha;
|
||||||
|
let inv = alpha.recip();
|
||||||
|
let inv_sq = inv * inv;
|
||||||
|
let series = inv * (1.0 - inv_sq * (2.0 - inv_sq * (10.0 - 74.0 * inv_sq)));
|
||||||
|
assert!(
|
||||||
|
(scaled - series).abs() / series < 1e-9,
|
||||||
|
"alpha={alpha}: scaled {scaled} vs series {series}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Folding the tie branch to `mu <= 0` is only valid if v is odd in mu and
|
||||||
|
/// w is even. Assert the symmetry the implementation relies on.
|
||||||
|
#[test]
|
||||||
|
fn tie_truncation_is_odd_in_v_and_even_in_w() {
|
||||||
|
for mu in [0.5f64, 3.0, 20.0, 40.0, 100.0, 1.0e3] {
|
||||||
|
let (v_pos, w_pos) = v_w(mu, 1.0, 1.0, true);
|
||||||
|
let (v_neg, w_neg) = v_w(-mu, 1.0, 1.0, true);
|
||||||
|
assert!(
|
||||||
|
(v_pos + v_neg).abs() < 1e-9,
|
||||||
|
"mu={mu}: v should be odd, got {v_pos} and {v_neg}"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
(w_pos - w_neg).abs() < 1e-9,
|
||||||
|
"mu={mu}: w should be even, got {w_pos} and {w_neg}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn test_quality() {
|
fn test_quality() {
|
||||||
let a = Gaussian::from_ms(25.0, 3.0);
|
let a = Gaussian::from_ms(25.0, 3.0);
|
||||||
|
|||||||
@@ -26,6 +26,83 @@ pub trait Observer<T: Time>: Send + Sync {
|
|||||||
fn on_converged(&self, _iters: usize, _final_step: (f64, f64), _converged: bool) {}
|
fn on_converged(&self, _iters: usize, _final_step: (f64, f64), _converged: bool) {}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Shared and boxed observers forward to what they point at.
|
||||||
|
///
|
||||||
|
/// `History` takes its observer by value, so a caller who wants to *read* what
|
||||||
|
/// an observer recorded has to keep a handle to it. Without these impls the
|
||||||
|
/// natural spelling does not compile:
|
||||||
|
///
|
||||||
|
/// ```
|
||||||
|
/// # use std::sync::{Arc, Mutex};
|
||||||
|
/// # use trueskill_tt::{History, Observer};
|
||||||
|
/// #[derive(Default)]
|
||||||
|
/// struct Recorder {
|
||||||
|
/// iterations: Mutex<Vec<usize>>,
|
||||||
|
/// }
|
||||||
|
///
|
||||||
|
/// impl Observer<i64> for Recorder {
|
||||||
|
/// fn on_iteration_end(&self, iter: usize, _step: (f64, f64)) {
|
||||||
|
/// self.iterations.lock().unwrap().push(iter);
|
||||||
|
/// }
|
||||||
|
/// }
|
||||||
|
///
|
||||||
|
/// let recorder = Arc::new(Recorder::default());
|
||||||
|
/// let mut h = History::builder().observer(Arc::clone(&recorder)).build();
|
||||||
|
/// h.record_winner(&"a", &"b", 1).unwrap();
|
||||||
|
/// h.converge().unwrap();
|
||||||
|
///
|
||||||
|
/// // The caller's handle sees what the history's copy recorded.
|
||||||
|
/// assert!(!recorder.iterations.lock().unwrap().is_empty());
|
||||||
|
/// ```
|
||||||
|
///
|
||||||
|
/// The alternative was for every observer to wrap each of its own fields in an
|
||||||
|
/// `Arc` and derive `Clone` — one allocation and one lock per field, and a
|
||||||
|
/// pattern each implementor had to rediscover.
|
||||||
|
///
|
||||||
|
/// `?Sized` is deliberate: it makes `Arc<dyn Observer<T>>` and
|
||||||
|
/// `Box<dyn Observer<T>>` work, so observers can be chosen at runtime.
|
||||||
|
impl<T: Time, O: Observer<T> + ?Sized> Observer<T> for std::sync::Arc<O> {
|
||||||
|
fn on_iteration_end(&self, iter: usize, max_step: (f64, f64)) {
|
||||||
|
(**self).on_iteration_end(iter, max_step);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_slice_processed(&self, time: &T, slice_idx: usize, n_events: usize) {
|
||||||
|
(**self).on_slice_processed(time, slice_idx, n_events);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_converged(&self, iters: usize, final_step: (f64, f64), converged: bool) {
|
||||||
|
(**self).on_converged(iters, final_step, converged);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T: Time, O: Observer<T> + ?Sized> Observer<T> for Box<O> {
|
||||||
|
fn on_iteration_end(&self, iter: usize, max_step: (f64, f64)) {
|
||||||
|
(**self).on_iteration_end(iter, max_step);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_slice_processed(&self, time: &T, slice_idx: usize, n_events: usize) {
|
||||||
|
(**self).on_slice_processed(time, slice_idx, n_events);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_converged(&self, iters: usize, final_step: (f64, f64), converged: bool) {
|
||||||
|
(**self).on_converged(iters, final_step, converged);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T: Time, O: Observer<T> + ?Sized> Observer<T> for &O {
|
||||||
|
fn on_iteration_end(&self, iter: usize, max_step: (f64, f64)) {
|
||||||
|
(**self).on_iteration_end(iter, max_step);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_slice_processed(&self, time: &T, slice_idx: usize, n_events: usize) {
|
||||||
|
(**self).on_slice_processed(time, slice_idx, n_events);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_converged(&self, iters: usize, final_step: (f64, f64), converged: bool) {
|
||||||
|
(**self).on_converged(iters, final_step, converged);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// ZST no-op observer; the default when none is configured.
|
/// ZST no-op observer; the default when none is configured.
|
||||||
#[derive(Copy, Clone, Debug, Default)]
|
#[derive(Copy, Clone, Debug, Default)]
|
||||||
pub struct NullObserver;
|
pub struct NullObserver;
|
||||||
|
|||||||
@@ -0,0 +1,222 @@
|
|||||||
|
//! `Member::with_prior` / `with_drift_scale` — competitor configuration.
|
||||||
|
//!
|
||||||
|
//! Both were previously consumed only on the branch that *creates* a
|
||||||
|
//! competitor, so configuration supplied for a key the history already knew was
|
||||||
|
//! dropped with no error. `with_prior` had no coverage in this directory at
|
||||||
|
//! all, which is how that survived.
|
||||||
|
|
||||||
|
use smallvec::smallvec;
|
||||||
|
use trueskill_tt::{
|
||||||
|
ConvergenceOptions, Event, Gaussian, History, InferenceError, Member, Outcome, Team,
|
||||||
|
};
|
||||||
|
|
||||||
|
const CONVERGENCE: ConvergenceOptions = ConvergenceOptions {
|
||||||
|
max_iter: 2_000,
|
||||||
|
epsilon: 1e-12,
|
||||||
|
alpha: 1.0,
|
||||||
|
};
|
||||||
|
|
||||||
|
fn history() -> History {
|
||||||
|
History::builder()
|
||||||
|
.mu(25.0)
|
||||||
|
.sigma(25.0 / 3.0)
|
||||||
|
.beta(25.0 / 6.0)
|
||||||
|
.p_draw(0.0)
|
||||||
|
.convergence(CONVERGENCE)
|
||||||
|
.build()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One event, optionally configuring `a`.
|
||||||
|
fn bout(
|
||||||
|
a: &'static str,
|
||||||
|
b: &'static str,
|
||||||
|
time: i64,
|
||||||
|
prior: Option<Gaussian>,
|
||||||
|
scale: Option<f64>,
|
||||||
|
) -> Event<i64, &'static str> {
|
||||||
|
let mut member = Member::new(a);
|
||||||
|
if let Some(p) = prior {
|
||||||
|
member = member.with_prior(p);
|
||||||
|
}
|
||||||
|
if let Some(s) = scale {
|
||||||
|
member = member.with_drift_scale(s);
|
||||||
|
}
|
||||||
|
|
||||||
|
Event {
|
||||||
|
time,
|
||||||
|
teams: smallvec![
|
||||||
|
Team::with_members([member]),
|
||||||
|
Team::with_members([Member::new(b)]),
|
||||||
|
],
|
||||||
|
outcome: Outcome::winner(0, 2),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn skill_of(h: &History, key: &str) -> Gaussian {
|
||||||
|
h.current_skill(&key).expect("key in history")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Baseline: the mechanism works at all on a competitor's first appearance.
|
||||||
|
#[test]
|
||||||
|
fn a_prior_applies_to_a_new_competitor() {
|
||||||
|
let seeded = Gaussian::from_ms(40.0, 1.0);
|
||||||
|
|
||||||
|
let mut with = history();
|
||||||
|
with.add_events(vec![bout("a", "b", 0, Some(seeded), None)])
|
||||||
|
.unwrap();
|
||||||
|
with.converge().unwrap();
|
||||||
|
|
||||||
|
let mut without = history();
|
||||||
|
without
|
||||||
|
.add_events(vec![bout("a", "b", 0, None, None)])
|
||||||
|
.unwrap();
|
||||||
|
without.converge().unwrap();
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
(skill_of(&with, "a").mu() - skill_of(&without, "a").mu()).abs() > 1.0,
|
||||||
|
"a seeded prior should move the fit"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The defect in #10: a prior supplied for a competitor the history already
|
||||||
|
/// knows was silently discarded, and the caller got output computed from the
|
||||||
|
/// default prior with no indication anything had been dropped.
|
||||||
|
#[test]
|
||||||
|
fn a_prior_applies_to_a_competitor_the_history_already_knows() {
|
||||||
|
let seeded = Gaussian::from_ms(40.0, 1.0);
|
||||||
|
|
||||||
|
let mut late = history();
|
||||||
|
late.add_events(vec![bout("a", "b", 0, None, None)])
|
||||||
|
.unwrap();
|
||||||
|
// "a" now exists. Configuring it here used to do nothing whatsoever.
|
||||||
|
late.add_events(vec![bout("a", "b", 1, Some(seeded), None)])
|
||||||
|
.unwrap();
|
||||||
|
late.converge().unwrap();
|
||||||
|
|
||||||
|
let mut never = history();
|
||||||
|
never
|
||||||
|
.add_events(vec![
|
||||||
|
bout("a", "b", 0, None, None),
|
||||||
|
bout("a", "b", 1, None, None),
|
||||||
|
])
|
||||||
|
.unwrap();
|
||||||
|
never.converge().unwrap();
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
(skill_of(&late, "a").mu() - skill_of(&never, "a").mu()).abs() > 1.0,
|
||||||
|
"a late prior must not be silently dropped: {} vs {}",
|
||||||
|
skill_of(&late, "a").mu(),
|
||||||
|
skill_of(&never, "a").mu()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Configuration is competitor-scoped, not event-scoped, and `converge` refits
|
||||||
|
/// from competitor state — so seeding late reaches the same fit as seeding from
|
||||||
|
/// the start. This is the documented scope, asserted rather than assumed.
|
||||||
|
#[test]
|
||||||
|
fn a_prior_is_whole_history_scoped_not_per_event() {
|
||||||
|
let seeded = Gaussian::from_ms(40.0, 1.0);
|
||||||
|
|
||||||
|
let mut late = history();
|
||||||
|
late.add_events(vec![bout("a", "b", 0, None, None)])
|
||||||
|
.unwrap();
|
||||||
|
late.add_events(vec![bout("a", "b", 1, Some(seeded), None)])
|
||||||
|
.unwrap();
|
||||||
|
late.converge().unwrap();
|
||||||
|
|
||||||
|
let mut early = history();
|
||||||
|
early
|
||||||
|
.add_events(vec![
|
||||||
|
bout("a", "b", 0, Some(seeded), None),
|
||||||
|
bout("a", "b", 1, Some(seeded), None),
|
||||||
|
])
|
||||||
|
.unwrap();
|
||||||
|
early.converge().unwrap();
|
||||||
|
|
||||||
|
let (l, e) = (skill_of(&late, "a"), skill_of(&early, "a"));
|
||||||
|
assert!(
|
||||||
|
(l.mu() - e.mu()).abs() < 1e-9 && (l.sigma() - e.sigma()).abs() < 1e-9,
|
||||||
|
"late seeding should refit the whole history: {l:?} vs {e:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn repeating_the_same_prior_is_inert() {
|
||||||
|
let seeded = Gaussian::from_ms(40.0, 1.0);
|
||||||
|
|
||||||
|
let mut once = history();
|
||||||
|
once.add_events(vec![
|
||||||
|
bout("a", "b", 0, Some(seeded), None),
|
||||||
|
bout("a", "b", 1, None, None),
|
||||||
|
])
|
||||||
|
.unwrap();
|
||||||
|
once.converge().unwrap();
|
||||||
|
|
||||||
|
let mut every_time = history();
|
||||||
|
every_time
|
||||||
|
.add_events(vec![
|
||||||
|
bout("a", "b", 0, Some(seeded), None),
|
||||||
|
bout("a", "b", 1, Some(seeded), None),
|
||||||
|
])
|
||||||
|
.unwrap();
|
||||||
|
every_time.converge().unwrap();
|
||||||
|
|
||||||
|
let (o, e) = (skill_of(&once, "a"), skill_of(&every_time, "a"));
|
||||||
|
assert!(
|
||||||
|
(o.mu() - e.mu()).abs() < 1e-12 && (o.sigma() - e.sigma()).abs() < 1e-12,
|
||||||
|
"declaring the same prior repeatedly changed the fit: {o:?} vs {e:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Events within a batch have no order, so two different values for one
|
||||||
|
/// competitor have no well-defined winner. Rejecting is what keeps the answer
|
||||||
|
/// independent of iteration order.
|
||||||
|
#[test]
|
||||||
|
fn a_batch_declaring_two_different_priors_is_rejected() {
|
||||||
|
let mut h = history();
|
||||||
|
let err = h
|
||||||
|
.add_events(vec![
|
||||||
|
bout("a", "b", 0, Some(Gaussian::from_ms(40.0, 1.0)), None),
|
||||||
|
bout("a", "b", 1, Some(Gaussian::from_ms(10.0, 1.0)), None),
|
||||||
|
])
|
||||||
|
.expect_err("two different priors for one competitor in one batch");
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
matches!(
|
||||||
|
err,
|
||||||
|
InferenceError::ConflictingCompetitorConfig { field: "prior", .. }
|
||||||
|
),
|
||||||
|
"got {err:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A member setting only `drift_scale` must not also assert the default prior,
|
||||||
|
/// or it would silently undo a prior seeded earlier. This is why the collected
|
||||||
|
/// configuration tracks each field separately rather than a merged `Rating`.
|
||||||
|
#[test]
|
||||||
|
fn setting_one_field_late_leaves_the_other_alone() {
|
||||||
|
let seeded = Gaussian::from_ms(40.0, 1.0);
|
||||||
|
|
||||||
|
let mut h = history();
|
||||||
|
h.add_events(vec![bout("a", "b", 0, Some(seeded), None)])
|
||||||
|
.unwrap();
|
||||||
|
// Only the scale this time — the prior above must survive.
|
||||||
|
h.add_events(vec![bout("a", "b", 1, None, Some(0.5))])
|
||||||
|
.unwrap();
|
||||||
|
h.converge().unwrap();
|
||||||
|
|
||||||
|
let mut both_upfront = history();
|
||||||
|
both_upfront
|
||||||
|
.add_events(vec![
|
||||||
|
bout("a", "b", 0, Some(seeded), Some(0.5)),
|
||||||
|
bout("a", "b", 1, None, None),
|
||||||
|
])
|
||||||
|
.unwrap();
|
||||||
|
both_upfront.converge().unwrap();
|
||||||
|
|
||||||
|
let (a, b) = (skill_of(&h, "a"), skill_of(&both_upfront, "a"));
|
||||||
|
assert!(
|
||||||
|
(a.mu() - b.mu()).abs() < 1e-9 && (a.sigma() - b.sigma()).abs() < 1e-9,
|
||||||
|
"setting drift_scale late clobbered the earlier prior: {a:?} vs {b:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
+112
-15
@@ -341,13 +341,20 @@ fn zero_scale_pins_a_competitor_in_the_filtered_pass() {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `drift_scale` is competitor configuration captured at first appearance, the
|
/// `drift_scale` is competitor configuration, and configuration supplied for a
|
||||||
/// same as `prior` — a later `with_drift_scale` on a key the history already
|
/// competitor the history already knows is now *applied* rather than dropped.
|
||||||
/// knows is ignored. This guards that decision rather than driving it: the
|
///
|
||||||
/// behaviour falls out of where the capture happens, and the point of the test
|
/// This test previously asserted the opposite. It was written as a deliberate
|
||||||
/// is that moving the capture would be a visible break, not a silent one.
|
/// change-detector — "moving the capture would be a visible break, not a silent
|
||||||
|
/// one" — and that is exactly what happened: the capture moved, and the
|
||||||
|
/// assertion inverted rather than being deleted.
|
||||||
|
///
|
||||||
|
/// Because configuration lives on the competitor and `converge` refits from
|
||||||
|
/// competitor state, a late pin applies to the *whole* history, not just to
|
||||||
|
/// events after it. So a scale set on the second batch must reach the same fit
|
||||||
|
/// as one set from the very first event.
|
||||||
#[test]
|
#[test]
|
||||||
fn drift_scale_is_ignored_after_first_appearance() {
|
fn drift_scale_applies_when_set_after_first_appearance() {
|
||||||
let mut late = History::builder()
|
let mut late = History::builder()
|
||||||
.mu(25.0)
|
.mu(25.0)
|
||||||
.sigma(25.0 / 3.0)
|
.sigma(25.0 / 3.0)
|
||||||
@@ -368,7 +375,7 @@ fn drift_scale_is_ignored_after_first_appearance() {
|
|||||||
}])
|
}])
|
||||||
.unwrap();
|
.unwrap();
|
||||||
|
|
||||||
// Second batch asks for a pin. Too late: the competitor already exists.
|
// Second batch asks for a pin. No longer too late.
|
||||||
late.add_events(vec![Event {
|
late.add_events(vec![Event {
|
||||||
time: 1000,
|
time: 1000,
|
||||||
teams: smallvec![
|
teams: smallvec![
|
||||||
@@ -380,23 +387,113 @@ fn drift_scale_is_ignored_after_first_appearance() {
|
|||||||
.unwrap();
|
.unwrap();
|
||||||
late.converge().unwrap();
|
late.converge().unwrap();
|
||||||
|
|
||||||
let ignored = curve(&late, "anchor");
|
let applied = curve(&late, "anchor");
|
||||||
let drifting = curve(&fit(distant_pair(None), 25.0 / 300.0), "anchor");
|
let pinned_from_the_start = curve(&fit(distant_pair(Some(0.0)), 25.0 / 300.0), "anchor");
|
||||||
|
let never_pinned = curve(&fit(distant_pair(None), 25.0 / 300.0), "anchor");
|
||||||
|
|
||||||
for ((t_l, g_l), (t_r, g_r)) in ignored.iter().zip(drifting.iter()) {
|
for ((t_l, g_l), (t_r, g_r)) in applied.iter().zip(pinned_from_the_start.iter()) {
|
||||||
assert_eq!(t_l, t_r);
|
assert_eq!(t_l, t_r);
|
||||||
assert!(
|
assert!(
|
||||||
(g_l.sigma() - g_r.sigma()).abs() < 1e-9,
|
(g_l.sigma() - g_r.sigma()).abs() < 1e-9,
|
||||||
"a scale set after first appearance must be ignored, leaving the fit \
|
"a late pin should refit the whole history: t={t_l}, {} vs {}",
|
||||||
identical to one that never set it: t={t_l}, {} vs {}",
|
|
||||||
g_l.sigma(),
|
g_l.sigma(),
|
||||||
g_r.sigma()
|
g_r.sigma()
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
let pinned = curve(&fit(distant_pair(Some(0.0)), 25.0 / 300.0), "anchor");
|
// And it must actually have done something.
|
||||||
assert!(
|
assert!(
|
||||||
(ignored[1].1.sigma() - pinned[1].1.sigma()).abs() > 1e-6,
|
applied
|
||||||
"sanity: the pinned fit must actually differ, or the assertion above is vacuous"
|
.iter()
|
||||||
|
.zip(never_pinned.iter())
|
||||||
|
.any(|((_, a), (_, b))| (a.sigma() - b.sigma()).abs() > 1e-9),
|
||||||
|
"the pin had no effect at all — the silent drop is back"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Re-declaring the same configuration must be inert. This is the shape a
|
||||||
|
/// caller gets when the configuration is a property of the domain — "layouts
|
||||||
|
/// are static" — so every ingestion path repeats it on every event.
|
||||||
|
///
|
||||||
|
/// Both histories see exactly the same events; only how many times the scale
|
||||||
|
/// is declared differs.
|
||||||
|
#[test]
|
||||||
|
fn repeating_the_same_configuration_changes_nothing() {
|
||||||
|
let events = |declare_every_time: bool| {
|
||||||
|
let anchor = |first: bool| {
|
||||||
|
if first || declare_every_time {
|
||||||
|
Member::new("anchor").with_drift_scale(0.0)
|
||||||
|
} else {
|
||||||
|
Member::new("anchor")
|
||||||
|
}
|
||||||
|
};
|
||||||
|
vec![
|
||||||
|
Event {
|
||||||
|
time: 0,
|
||||||
|
teams: smallvec![
|
||||||
|
Team::with_members([anchor(true)]),
|
||||||
|
Team::with_members([Member::new("player")]),
|
||||||
|
],
|
||||||
|
outcome: Outcome::winner(0, 2),
|
||||||
|
},
|
||||||
|
Event {
|
||||||
|
time: 1000,
|
||||||
|
teams: smallvec![
|
||||||
|
Team::with_members([anchor(false)]),
|
||||||
|
Team::with_members([Member::new("player")]),
|
||||||
|
],
|
||||||
|
outcome: Outcome::winner(1, 2),
|
||||||
|
},
|
||||||
|
]
|
||||||
|
};
|
||||||
|
|
||||||
|
let once = curve(&fit(events(false), 25.0 / 300.0), "anchor");
|
||||||
|
let every_time = curve(&fit(events(true), 25.0 / 300.0), "anchor");
|
||||||
|
|
||||||
|
for ((t_l, a), (t_r, b)) in once.iter().zip(every_time.iter()) {
|
||||||
|
assert_eq!(t_l, t_r);
|
||||||
|
assert!(
|
||||||
|
(a.sigma() - b.sigma()).abs() < 1e-12,
|
||||||
|
"t={t_l}: declaring the same scale repeatedly changed the fit, {} vs {}",
|
||||||
|
a.sigma(),
|
||||||
|
b.sigma()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_batch_that_contradicts_itself_is_rejected() {
|
||||||
|
let mut h = History::builder().convergence(CONVERGENCE).build();
|
||||||
|
|
||||||
|
let err = h
|
||||||
|
.add_events(vec![
|
||||||
|
Event {
|
||||||
|
time: 0,
|
||||||
|
teams: smallvec![
|
||||||
|
Team::with_members([Member::new("anchor").with_drift_scale(0.0)]),
|
||||||
|
Team::with_members([Member::new("player")]),
|
||||||
|
],
|
||||||
|
outcome: Outcome::winner(0, 2),
|
||||||
|
},
|
||||||
|
Event {
|
||||||
|
time: 1,
|
||||||
|
teams: smallvec![
|
||||||
|
Team::with_members([Member::new("anchor").with_drift_scale(1.0)]),
|
||||||
|
Team::with_members([Member::new("player")]),
|
||||||
|
],
|
||||||
|
outcome: Outcome::winner(0, 2),
|
||||||
|
},
|
||||||
|
])
|
||||||
|
.expect_err("two different scales for one competitor in one batch");
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
matches!(
|
||||||
|
err,
|
||||||
|
InferenceError::ConflictingCompetitorConfig {
|
||||||
|
field: "drift_scale",
|
||||||
|
..
|
||||||
|
}
|
||||||
|
),
|
||||||
|
"got {err:?}"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -30,6 +30,22 @@ fn event(a: &str, b: &str, time: i64) -> Event<i64, String> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Like [`event`], but `a` carries competitor configuration.
|
||||||
|
///
|
||||||
|
/// `prior` and `drift_scale` configure the competitor rather than the event, so
|
||||||
|
/// they are the part of ingestion most exposed to order: they are consumed once,
|
||||||
|
/// where the competitor's state is written.
|
||||||
|
fn configured_event(a: &str, b: &str, time: i64, scale: f64) -> Event<i64, String> {
|
||||||
|
Event {
|
||||||
|
time,
|
||||||
|
teams: smallvec![
|
||||||
|
Team::with_members([Member::new(a.to_string()).with_drift_scale(scale)]),
|
||||||
|
Team::with_members([Member::new(b.to_string())]),
|
||||||
|
],
|
||||||
|
outcome: Outcome::winner(0, 2),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fn converged_skills(events: Vec<Event<i64, String>>, batched: bool) -> Vec<(String, Gaussian)> {
|
fn converged_skills(events: Vec<Event<i64, String>>, batched: bool) -> Vec<(String, Gaussian)> {
|
||||||
let mut h: History<i64, _, _, String> =
|
let mut h: History<i64, _, _, String> =
|
||||||
History::builder_with_key().convergence(tight()).build();
|
History::builder_with_key().convergence(tight()).build();
|
||||||
@@ -145,3 +161,65 @@ fn back_dated_event_matches_batched() {
|
|||||||
let incremental = converged_skills(events, false);
|
let incremental = converged_skills(events, false);
|
||||||
assert_same(&batched, &incremental, "back-dated event");
|
assert_same(&batched, &incremental, "back-dated event");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The invariant this file protects was only ever checked for *unconfigured*
|
||||||
|
/// competitors — every helper above built members with `Member::new`.
|
||||||
|
///
|
||||||
|
/// Configuration is the part most exposed to ordering, because it is consumed
|
||||||
|
/// once at the point the competitor's state is written rather than replayed per
|
||||||
|
/// event. These cover it.
|
||||||
|
#[test]
|
||||||
|
fn configured_competitors_are_order_independent() {
|
||||||
|
let events = vec![
|
||||||
|
configured_event("a", "b", 0, 0.0),
|
||||||
|
configured_event("a", "c", 1, 0.0),
|
||||||
|
configured_event("a", "b", 2, 0.0),
|
||||||
|
event("b", "c", 3),
|
||||||
|
];
|
||||||
|
|
||||||
|
assert_same(
|
||||||
|
&converged_skills(events.clone(), true),
|
||||||
|
&converged_skills(events, false),
|
||||||
|
"configuration repeated on every appearance",
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Configuration supplied only on a *later* event is the case that used to be
|
||||||
|
/// silently dropped. It must now reach the same fit either way it is ingested.
|
||||||
|
#[test]
|
||||||
|
fn late_configuration_is_order_independent() {
|
||||||
|
let events = vec![
|
||||||
|
event("a", "b", 0),
|
||||||
|
configured_event("a", "c", 1, 0.0),
|
||||||
|
event("a", "b", 2),
|
||||||
|
];
|
||||||
|
|
||||||
|
assert_same(
|
||||||
|
&converged_skills(events.clone(), true),
|
||||||
|
&converged_skills(events, false),
|
||||||
|
"configuration supplied after first appearance",
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// And it must actually be doing something — an implementation that dropped
|
||||||
|
/// configuration entirely would pass both tests above.
|
||||||
|
#[test]
|
||||||
|
fn configuration_changes_the_fit_however_it_is_ingested() {
|
||||||
|
let configured = vec![
|
||||||
|
event("a", "b", 0),
|
||||||
|
configured_event("a", "c", 1, 0.0),
|
||||||
|
event("a", "b", 2),
|
||||||
|
];
|
||||||
|
let plain = vec![event("a", "b", 0), event("a", "c", 1), event("a", "b", 2)];
|
||||||
|
|
||||||
|
for batched in [true, false] {
|
||||||
|
let with = converged_skills(configured.clone(), batched);
|
||||||
|
let without = converged_skills(plain.clone(), batched);
|
||||||
|
assert!(
|
||||||
|
with.iter()
|
||||||
|
.zip(&without)
|
||||||
|
.any(|((_, x), (_, y))| (x.sigma() - y.sigma()).abs() > 1e-9),
|
||||||
|
"batched={batched}: configuration had no effect, so the order tests are vacuous"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+69
-13
@@ -8,14 +8,13 @@ use std::sync::{Arc, Mutex};
|
|||||||
|
|
||||||
use trueskill_tt::{History, Observer};
|
use trueskill_tt::{History, Observer};
|
||||||
|
|
||||||
/// `History` takes its observer by value and never hands it back, so a test
|
/// Plain fields. `Arc<O>` implements `Observer`, so the caller shares the
|
||||||
/// that wants to read what was recorded shares the storage rather than the
|
/// observer itself rather than wrapping each field in its own `Arc`.
|
||||||
/// observer: the handles are cloned, the buffers are not.
|
#[derive(Default)]
|
||||||
#[derive(Clone, Default)]
|
|
||||||
struct Recorder {
|
struct Recorder {
|
||||||
iterations: Arc<Mutex<Vec<usize>>>,
|
iterations: Mutex<Vec<usize>>,
|
||||||
slices: Arc<Mutex<Vec<(i64, usize, usize)>>>,
|
slices: Mutex<Vec<(i64, usize, usize)>>,
|
||||||
converged: Arc<Mutex<Vec<(usize, bool)>>>,
|
converged: Mutex<Vec<(usize, bool)>>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Observer<i64> for Recorder {
|
impl Observer<i64> for Recorder {
|
||||||
@@ -37,8 +36,8 @@ impl Observer<i64> for Recorder {
|
|||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn every_observer_callback_fires() {
|
fn every_observer_callback_fires() {
|
||||||
let recorder = Recorder::default();
|
let recorder = Arc::new(Recorder::default());
|
||||||
let mut h = History::builder().observer(recorder.clone()).build();
|
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
|
||||||
|
|
||||||
h.record_winner(&"a", &"b", 1).unwrap();
|
h.record_winner(&"a", &"b", 1).unwrap();
|
||||||
h.record_winner(&"b", &"c", 2).unwrap();
|
h.record_winner(&"b", &"c", 2).unwrap();
|
||||||
@@ -61,8 +60,8 @@ fn every_observer_callback_fires() {
|
|||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn slice_callbacks_report_the_slice_they_swept() {
|
fn slice_callbacks_report_the_slice_they_swept() {
|
||||||
let recorder = Recorder::default();
|
let recorder = Arc::new(Recorder::default());
|
||||||
let mut h = History::builder().observer(recorder.clone()).build();
|
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
|
||||||
|
|
||||||
h.record_winner(&"a", &"b", 10).unwrap();
|
h.record_winner(&"a", &"b", 10).unwrap();
|
||||||
h.record_winner(&"a", &"b", 20).unwrap();
|
h.record_winner(&"a", &"b", 20).unwrap();
|
||||||
@@ -90,8 +89,8 @@ fn slice_callbacks_report_the_slice_they_swept() {
|
|||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_single_slice_history_still_reports_its_sweep() {
|
fn a_single_slice_history_still_reports_its_sweep() {
|
||||||
let recorder = Recorder::default();
|
let recorder = Arc::new(Recorder::default());
|
||||||
let mut h = History::builder().observer(recorder.clone()).build();
|
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
|
||||||
|
|
||||||
h.record_winner(&"a", &"b", 1).unwrap();
|
h.record_winner(&"a", &"b", 1).unwrap();
|
||||||
h.converge().unwrap();
|
h.converge().unwrap();
|
||||||
@@ -103,3 +102,60 @@ fn a_single_slice_history_still_reports_its_sweep() {
|
|||||||
);
|
);
|
||||||
assert!(slices.iter().all(|&(t, idx, _)| t == 1 && idx == 0));
|
assert!(slices.iter().all(|&(t, idx, _)| t == 1 && idx == 0));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The gap #40 closed: without `impl Observer for Arc<O>`, an observer that
|
||||||
|
/// accumulates anything had to wrap every field in its own `Arc` and derive
|
||||||
|
/// `Clone`, because `History` consumes the observer and never hands it back.
|
||||||
|
#[test]
|
||||||
|
fn a_shared_observer_reaches_the_callers_handle() {
|
||||||
|
let recorder = Arc::new(Recorder::default());
|
||||||
|
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
|
||||||
|
|
||||||
|
h.record_winner(&"a", &"b", 1).unwrap();
|
||||||
|
h.converge().unwrap();
|
||||||
|
|
||||||
|
assert!(!recorder.iterations.lock().unwrap().is_empty());
|
||||||
|
assert!(!recorder.slices.lock().unwrap().is_empty());
|
||||||
|
assert!(!recorder.converged.lock().unwrap().is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `?Sized` on the blanket impls means the observer can be chosen at runtime.
|
||||||
|
#[test]
|
||||||
|
fn a_trait_object_observer_works() {
|
||||||
|
let boxed: Box<dyn Observer<i64>> = Box::new(Recorder::default());
|
||||||
|
let mut h = History::builder().observer(boxed).build();
|
||||||
|
h.record_winner(&"a", &"b", 1).unwrap();
|
||||||
|
h.converge().unwrap();
|
||||||
|
|
||||||
|
let shared: Arc<dyn Observer<i64>> = Arc::new(Recorder::default());
|
||||||
|
let mut h = History::builder().observer(Arc::clone(&shared)).build();
|
||||||
|
h.record_winner(&"a", &"b", 1).unwrap();
|
||||||
|
h.converge().unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A non-shared observer can be reclaimed after convergence instead.
|
||||||
|
#[test]
|
||||||
|
fn into_observer_returns_the_accumulated_state() {
|
||||||
|
let mut h = History::builder().observer(Recorder::default()).build();
|
||||||
|
h.record_winner(&"a", &"b", 1).unwrap();
|
||||||
|
h.converge().unwrap();
|
||||||
|
|
||||||
|
// Readable in place...
|
||||||
|
assert!(!h.observer().iterations.lock().unwrap().is_empty());
|
||||||
|
|
||||||
|
// ...and reclaimable by value.
|
||||||
|
let recorder = h.into_observer();
|
||||||
|
assert!(!recorder.slices.lock().unwrap().is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Borrowing works too, for an observer that outlives the history.
|
||||||
|
#[test]
|
||||||
|
fn a_borrowed_observer_works() {
|
||||||
|
let recorder = Recorder::default();
|
||||||
|
{
|
||||||
|
let mut h = History::builder().observer(&recorder).build();
|
||||||
|
h.record_winner(&"a", &"b", 1).unwrap();
|
||||||
|
h.converge().unwrap();
|
||||||
|
}
|
||||||
|
assert!(!recorder.iterations.lock().unwrap().is_empty());
|
||||||
|
}
|
||||||
|
|||||||
@@ -212,3 +212,80 @@ fn team_size_affects_the_prediction() {
|
|||||||
assert!((p.total() - 1.0).abs() < 1e-6, "total = {}", p.total());
|
assert!((p.total() - 1.0).abs() < 1e-6, "total = {}", p.total());
|
||||||
assert!(p.probability_of(&[0, 0]) > 0.0);
|
assert!(p.probability_of(&[0, 0]) > 0.0);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Expected information gain
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/// The whole point of #39: "which comparison should I run next?" is a
|
||||||
|
/// different question from "who will win?" or "is this fair?".
|
||||||
|
#[test]
|
||||||
|
fn information_gain_prefers_the_uncertain_pairing() {
|
||||||
|
let mut h = History::builder().build();
|
||||||
|
|
||||||
|
// "known" and "rival" have played a lot; "newcomer" has played once.
|
||||||
|
for t in 1..=15 {
|
||||||
|
h.record_winner(&"known", &"rival", t).unwrap();
|
||||||
|
h.record_winner(&"rival", &"known", t + 100).unwrap();
|
||||||
|
}
|
||||||
|
h.record_winner(&"known", &"newcomer", 500).unwrap();
|
||||||
|
h.converge().unwrap();
|
||||||
|
|
||||||
|
let settled = h
|
||||||
|
.expected_information_gain(&[&[&"known"], &[&"rival"]])
|
||||||
|
.unwrap();
|
||||||
|
let unknown = h
|
||||||
|
.expected_information_gain(&[&[&"known"], &[&"newcomer"]])
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
unknown > settled,
|
||||||
|
"pairing against the newcomer should teach more: {unknown} vs {settled}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The analytic ceiling, through the `History` entry point rather than the
|
||||||
|
/// standalone one.
|
||||||
|
#[test]
|
||||||
|
fn information_gain_respects_the_entropy_ceiling() {
|
||||||
|
let h = history_with(&["a", "b", "c"], 0.0);
|
||||||
|
|
||||||
|
let two = h.expected_information_gain(&[&[&"a"], &[&"b"]]).unwrap();
|
||||||
|
assert!(
|
||||||
|
(0.0..=std::f64::consts::LN_2).contains(&two),
|
||||||
|
"two-team EIG {two} outside [0, ln 2]"
|
||||||
|
);
|
||||||
|
|
||||||
|
let three = h
|
||||||
|
.expected_information_gain(&[&[&"a"], &[&"b"], &[&"c"]])
|
||||||
|
.unwrap();
|
||||||
|
assert!(
|
||||||
|
(0.0..=6.0f64.ln()).contains(&three),
|
||||||
|
"three-team EIG {three} outside [0, ln 6]"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn information_gain_reports_unknown_keys() {
|
||||||
|
let h = history_with(&["a", "b"], 0.0);
|
||||||
|
assert_eq!(
|
||||||
|
h.expected_information_gain(&[&[&"a"], &[&"ghost"]])
|
||||||
|
.unwrap_err(),
|
||||||
|
InferenceError::UnknownKey { team: 1, member: 0 }
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A draw-enabled history has three outcomes to weigh rather than two, so the
|
||||||
|
/// draw branch must actually be reachable through this path.
|
||||||
|
#[test]
|
||||||
|
fn information_gain_accounts_for_draws() {
|
||||||
|
let with_draws = history_with(&["a", "b"], 0.25);
|
||||||
|
let g = with_draws
|
||||||
|
.expected_information_gain(&[&[&"a"], &[&"b"]])
|
||||||
|
.unwrap();
|
||||||
|
assert!(g > 0.0 && g <= 3.0f64.ln(), "{g}");
|
||||||
|
|
||||||
|
// The draw outcome carries mass, so it is genuinely being weighed.
|
||||||
|
let dist = with_draws.predict_outcome(&[&[&"a"], &[&"b"]]).unwrap();
|
||||||
|
assert!(dist.probability_of(&[0, 0]) > 0.0);
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,186 @@
|
|||||||
|
//! Input validation must hold in **release**, where `debug_assert!` is gone.
|
||||||
|
//!
|
||||||
|
//! The engine guards itself with `debug_assert!`, which documents invariants
|
||||||
|
//! but vanishes in the profile users actually ship. Anything reachable from the
|
||||||
|
//! public API has to be rejected with an `InferenceError` instead, at the
|
||||||
|
//! boundary, rather than becoming NaN or an out-of-bounds panic deep inside
|
||||||
|
//! `run_chain`.
|
||||||
|
//!
|
||||||
|
//! `GameOptions` and `ConvergenceOptions` both have public fields, so the
|
||||||
|
//! eager asserts on `HistoryBuilder` do not cover the `Game` constructors —
|
||||||
|
//! a caller can build the options struct directly.
|
||||||
|
|
||||||
|
use smallvec::smallvec;
|
||||||
|
use trueskill_tt::{
|
||||||
|
ConstantDrift, ConvergenceOptions, Event, Game, GameOptions, Gaussian, History, InferenceError,
|
||||||
|
Member, Outcome, Rating, Team,
|
||||||
|
};
|
||||||
|
|
||||||
|
type R = Rating<i64, ConstantDrift>;
|
||||||
|
|
||||||
|
fn rating() -> R {
|
||||||
|
R::new(
|
||||||
|
Gaussian::from_ms(25.0, 25.0 / 3.0),
|
||||||
|
25.0 / 6.0,
|
||||||
|
ConstantDrift(0.0),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn options_with_alpha(alpha: f64) -> GameOptions {
|
||||||
|
GameOptions {
|
||||||
|
convergence: ConvergenceOptions {
|
||||||
|
alpha,
|
||||||
|
..ConvergenceOptions::default()
|
||||||
|
},
|
||||||
|
..GameOptions::default()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `alpha == 0.0` leaves every EP update unapplied, so inference silently
|
||||||
|
/// returns the priors — the worst possible failure, since the output looks
|
||||||
|
/// entirely reasonable.
|
||||||
|
#[test]
|
||||||
|
fn ranked_rejects_a_zero_damping_factor() {
|
||||||
|
let (a, b) = (rating(), rating());
|
||||||
|
let err = Game::<i64, _>::ranked(
|
||||||
|
&[&[a], &[b]],
|
||||||
|
Outcome::winner(0, 2),
|
||||||
|
&options_with_alpha(0.0),
|
||||||
|
)
|
||||||
|
.expect_err("alpha = 0 must be rejected");
|
||||||
|
assert!(
|
||||||
|
matches!(err, InferenceError::InvalidParameter { name: "alpha", .. }),
|
||||||
|
"got {err:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn ranked_rejects_an_out_of_range_damping_factor() {
|
||||||
|
let (a, b) = (rating(), rating());
|
||||||
|
for alpha in [-0.5, 1.5, f64::NAN] {
|
||||||
|
let err = Game::<i64, _>::ranked(
|
||||||
|
&[&[a], &[b]],
|
||||||
|
Outcome::winner(0, 2),
|
||||||
|
&options_with_alpha(alpha),
|
||||||
|
)
|
||||||
|
.expect_err("alpha out of (0, 1] must be rejected");
|
||||||
|
assert!(
|
||||||
|
matches!(err, InferenceError::InvalidParameter { name: "alpha", .. }),
|
||||||
|
"alpha={alpha}: got {err:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn scored_rejects_a_bad_damping_factor() {
|
||||||
|
let (a, b) = (rating(), rating());
|
||||||
|
let err = Game::<i64, _>::scored(
|
||||||
|
&[&[a], &[b]],
|
||||||
|
Outcome::scores([21.0, 9.0]),
|
||||||
|
&options_with_alpha(0.0),
|
||||||
|
)
|
||||||
|
.expect_err("alpha = 0 must be rejected");
|
||||||
|
assert!(
|
||||||
|
matches!(err, InferenceError::InvalidParameter { name: "alpha", .. }),
|
||||||
|
"got {err:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Already covered by `Game::ranked`, asserted here so the release-mode
|
||||||
|
/// guarantee is stated in one place.
|
||||||
|
#[test]
|
||||||
|
fn ranked_rejects_an_out_of_range_draw_probability() {
|
||||||
|
let (a, b) = (rating(), rating());
|
||||||
|
for p_draw in [-0.5, 1.0, 1.5] {
|
||||||
|
let options = GameOptions {
|
||||||
|
p_draw,
|
||||||
|
..GameOptions::default()
|
||||||
|
};
|
||||||
|
assert!(
|
||||||
|
Game::<i64, _>::ranked(&[&[a], &[b]], Outcome::winner(0, 2), &options).is_err(),
|
||||||
|
"p_draw={p_draw} must be rejected"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn scored_rejects_a_non_positive_noise() {
|
||||||
|
let (a, b) = (rating(), rating());
|
||||||
|
for score_sigma in [0.0, -1.0, f64::NAN] {
|
||||||
|
let options = GameOptions {
|
||||||
|
score_sigma,
|
||||||
|
..GameOptions::default()
|
||||||
|
};
|
||||||
|
assert!(
|
||||||
|
Game::<i64, _>::scored(&[&[a], &[b]], Outcome::scores([21.0, 9.0]), &options).is_err(),
|
||||||
|
"score_sigma={score_sigma} must be rejected"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A tie with no draw probability makes the truncation margin zero and the
|
||||||
|
/// two-sided update evaluate 0/0. Ingestion must refuse it.
|
||||||
|
#[test]
|
||||||
|
fn ingestion_rejects_a_tie_without_a_draw_probability() {
|
||||||
|
let mut h = History::builder().p_draw(0.0).build();
|
||||||
|
let err = h
|
||||||
|
.add_events(vec![Event {
|
||||||
|
time: 0,
|
||||||
|
teams: smallvec![
|
||||||
|
Team::with_members([Member::new("a")]),
|
||||||
|
Team::with_members([Member::new("b")]),
|
||||||
|
],
|
||||||
|
outcome: Outcome::draw(2),
|
||||||
|
}])
|
||||||
|
.expect_err("a tie with p_draw = 0 must be rejected");
|
||||||
|
assert!(
|
||||||
|
matches!(err, InferenceError::TieWithoutDrawProbability { .. }),
|
||||||
|
"got {err:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `Outcome::scores_with_sigma` documents that a non-positive sigma is
|
||||||
|
/// accepted at construction and rejected at ingestion.
|
||||||
|
#[test]
|
||||||
|
fn ingestion_rejects_a_non_positive_per_event_score_sigma() {
|
||||||
|
for sigma in [0.0, -1.0, f64::NAN] {
|
||||||
|
let mut h = History::builder().build();
|
||||||
|
let err = h
|
||||||
|
.add_events(vec![Event {
|
||||||
|
time: 0,
|
||||||
|
teams: smallvec![
|
||||||
|
Team::with_members([Member::new("a")]),
|
||||||
|
Team::with_members([Member::new("b")]),
|
||||||
|
],
|
||||||
|
outcome: Outcome::scores_with_sigma([21.0, 9.0], sigma),
|
||||||
|
}])
|
||||||
|
.expect_err("a non-positive per-event sigma must be rejected");
|
||||||
|
assert!(
|
||||||
|
matches!(err, InferenceError::InvalidParameter { .. }),
|
||||||
|
"sigma={sigma}: got {err:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Per-team weights must match that team's membership. The top-level length
|
||||||
|
/// checks in ingestion do not cover the inner dimension.
|
||||||
|
#[test]
|
||||||
|
fn ingestion_rejects_weights_that_do_not_match_their_team() {
|
||||||
|
let mut h = History::builder().build();
|
||||||
|
let mut team = Team::with_members([Member::new("a"), Member::new("b")]);
|
||||||
|
team.members[0].weight = 1.0;
|
||||||
|
|
||||||
|
let err = h
|
||||||
|
.event(0)
|
||||||
|
.team(["a", "b"])
|
||||||
|
.team(["c"])
|
||||||
|
// Three weights for a two-member team.
|
||||||
|
.weights([1.0, 1.0, 1.0])
|
||||||
|
.winner(0)
|
||||||
|
.commit()
|
||||||
|
.expect_err("a weight/member length mismatch must be rejected");
|
||||||
|
assert!(
|
||||||
|
matches!(err, InferenceError::MismatchedShape { .. }),
|
||||||
|
"got {err:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user