10 Commits
Author SHA1 Message Date
logaritmisk 2a48d10aa9 chore: Release trueskill-tt version 0.4.0 2026-09-07 15:49:12 +02:00
logaritmiskandClaude Opus 5 d4f91fd221 fix: reject convergence options that silently disable inference
`Game::ranked` and `Game::scored` validated `p_draw` and `score_sigma`
but never `convergence`. `ConvergenceOptions` has public fields and
`GameOptions` carries one, so a caller could hand the engine a set that
`HistoryBuilder`'s eager asserts never saw. Past that, the only guard
was a `debug_assert!`, which is gone in the profile users ship.

An `alpha` of zero is the bad case, and it fails silently rather than
loudly. Measured in release before the fix:

    likelihoods: [[Gaussian { pi: 0.0, tau: 0.0 }],
                  [Gaussian { pi: 0.0, tau: 0.0 }]]

Every EP update unapplied, every likelihood uninformative, inference
returning the priors it was given — and an `OwnedGame` that looks
entirely ordinary to the caller. `HistoryBuilder::convergence` already
documents exactly this hazard; the `Game` constructors just did not
share the check.

Adds `ConvergenceOptions::validate`, called by both constructors.
Rejects `alpha` outside `(0.0, 1.0]` and negative `epsilon`; NaN fails
both comparisons and is rejected too.

`tests/validation.rs` states the release-mode guarantee for the whole
public surface, not just this hole, and CI already runs the suite in
release. Probing the other conditions #18 lists found five of eight
already enforced — ties without a draw probability, per-event score
sigma, weight/team dimensions, draw-probability range, score-sigma
range — so this closes the remaining gap rather than the whole issue.
The engine keeps its `debug_assert!`s as invariant documentation.

Refs #18

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
2026-09-07 15:48:31 +02:00
logaritmiskandClaude Opus 5 8c087ad015 fix!: apply competitor configuration whenever it is supplied
`Member::with_prior` and `with_drift_scale` were consumed only on the
branch that *creates* a competitor — `priors.remove` sat inside
`if !self.agents.contains(..)`. Supplying either for a key the history
already knew did nothing at all: no error, no warning, and output
computed from the default prior. A prior applied on a competitor's very
first event and was silently discarded ever after.

Configuration now applies whenever supplied. Two details this forced:

Configuration is tracked per *field* rather than as a merged `Rating`.
A member setting only `drift_scale` must not also assert the default
prior, or it would silently undo a prior seeded on an earlier event.

Slice state has to be refreshed. `drift_scale` is re-derived on every
forward pass, but a prior is written into the competitor's earliest
slice once, at ingestion, and `iteration` refreshes only slices after
the first. Without the refresh a late prior would reach the drift terms
and nothing else — a subtler version of the drop being fixed. This was
caught by a test, not by reading the code.

Conflicting values for one competitor within a single batch are now
`ConflictingCompetitorConfig` rather than resolved by iteration order.
Events in a batch are unordered, so "last one wins" would make the
result depend on traversal — and `tests/ingestion_equivalence.rs` exists
to rule exactly that out. Repeating the same value stays inert, which is
the shape callers get when configuration is a property of the domain.

That invariant turned out to be tested only for *unconfigured*
competitors: every helper in that file built members with `Member::new`.
Extended to cover configured ones, including a check that configuration
changes the fit at all, so the order tests cannot pass vacuously.

`with_prior` had no coverage under `tests/` whatsoever, which is how
this survived. Adds `tests/competitor_config.rs`.

`drift_scale_is_ignored_after_first_appearance` asserted the old
behaviour and now asserts the new one. It was written as a deliberate
change-detector — "moving the capture would be a visible break, not a
silent one" — so it inverted rather than being deleted.

Also removes `InferenceError::ConvergenceFailed` and `NegativePrecision`,
which no code path ever constructed: public variants advertising failure
modes no caller could observe. Partial #20 — its other items were
already resolved, except `Outcome::winner` still panicking.

BREAKING CHANGE: `prior` and `drift_scale` now take effect for
competitors the history already knows, where they were previously
ignored; a batch supplying conflicting values for one competitor is now
an error. `InferenceError::ConvergenceFailed` and
`InferenceError::NegativePrecision` are removed.

Closes #10. Refs #20.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
2026-09-07 15:42:31 +02:00
logaritmiskandClaude Opus 5 7341669d1a fix: stop destroying tail precision in evidence and truncation
`erfc` is sound — it holds ~1e-7 *relative* accuracy down to 1e-296 with
no tail degradation. Three expressions built on it threw that away by
subtracting quantities that both approach the same value.

1. `cavity_evidence` computed `1.0 - cdf(margin, ..)`, which is
   algebraically `sf(margin, ..)` and numerically a catastrophe: 7%
   error by eight sigma, and exactly zero past ~8.3, where the true
   probability is 1e-19 and perfectly representable. Clamped, that
   reached `log_evidence` as ln(f64::MIN_POSITIVE) = -708 whatever the
   truth was — off by 665 nats at nine sigma.

   `1 - cdf` is smallest precisely when the result contradicts the
   prior, so the model-comparison number was worst for upsets: the
   observation it exists to notice. Adds `sf`, the survival function,
   computed without the subtraction. The tie branch picks whichever tail
   keeps both of its terms small, for the same reason.

2. `v_w` computed the inverse Mills ratio as `pdf(-a) / cdf(-a)`. Both
   underflow together past about 39 sigma, giving `0 / 0` and putting
   NaN straight into the posterior. Adds `erfcx`, so the shared
   `exp(-alpha^2 / 2)` cancels analytically instead of being evaluated
   twice and divided.

3. With that fixed, `w = v * (v - alpha)` became the next casualty: `v`
   tends to `alpha`, so the gap lost every digit and drove `w` above 1,
   making `sqrt(1 - w)` NaN at alpha = 1e6. The gap now comes from its
   asymptotic series, which forms no difference at all. The tie branch
   had the same defect one expression over — `v * v - u` with both terms
   at 1e18 returned w = -128 — and a far-tail window is
   indistinguishable from a half-line, so it shares the asymptotic.

No public signature changes, and no existing golden moved: every one of
these only alters regions the old code got wrong. The two identity tests
are asserted at 1e-6 rather than tighter because `erfc` is not exactly
antisymmetric — `erfc(z) + erfc(-z)` differs from 2 by ~3e-8, and
`erfc(0)` returns 1.00000003. That floor is tracked separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
2026-09-07 15:28:35 +02:00
logaritmiskandClaude Opus 5 2fff745c3b feat: let observers be shared, boxed, or borrowed
`History` takes its observer by value and never hands it back, so a
caller who wanted to read what an observer recorded had no way to keep a
handle to it. The natural spelling did not compile:

    let recorder = Arc::new(Recorder::default());
    History::builder().observer(Arc::clone(&recorder))
    // error[E0277]: `Arc<Recorder>: Observer<i64>` is not satisfied

The workaround was for every observer to wrap each of its own fields in
an `Arc` and derive `Clone` — one allocation and one lock per field, a
pattern each implementor had to rediscover, and nothing documenting it.

Adds blanket `Observer` impls for `Arc<O>`, `Box<O>` and `&O`. All are
`?Sized`, so `Arc<dyn Observer<T>>` and `Box<dyn Observer<T>>` work too
and an observer can be chosen at runtime. Also adds
`History::observer()` and `into_observer()`, so a non-shared observer's
state can be inspected in place or reclaimed after `converge` without
needing interior mutability at all.

`tests/observer.rs` is simplified to the shared spelling, so the
recommended pattern is the one demonstrated rather than the workaround.

Closes #40

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
2026-09-07 15:13:28 +02:00
logaritmiskandClaude Opus 5 3c2f9ac64c feat: add expected information gain for active matchup selection
`quality()` answers "is this matchup fair". Callers picking which
comparison to run next need "is this matchup informative", and the two
coincide only for two evenly matched competitors. Without a principled
alternative, downstream code was reaching for hand-rolled heuristics
like `quality * sigma_a^2 * sigma_b^2`, which double-counts uncertainty:
the two factors are not independent.

Adds `expected_information_gain`, the outcome-weighted divergence
between current beliefs and the beliefs each result would produce:

    EIG = SUM P(outcome) * KL(posterior_after(outcome) || prior)

Available standalone over `Rating`s, and as
`History::expected_information_gain` using current skills and the
history's own beta, drift and p_draw — so the outcomes it weighs are the
ones that would actually be fitted.

This is the mutual information between the outcome and the skills, which
gives an analytic ceiling: gain cannot exceed the entropy of the thing
being observed, so at most `ln k` nats for k outcomes. That bound is the
sharpest test available, because an acquisition function is unusually
exposed to returning finite, plausible, monotone numbers while being
wrong — it would simply select slightly worse matchups forever. A
prototype of this returned 4.77 nats from a sign error while passing
every monotonicity check; `never_exceeds_the_entropy_of_the_outcome`
catches that class unconditionally.

Measured against the ceiling the values are meaningful rather than
vacuous: 0.382 nats for an even matchup between diffuse priors against
an 0.693 ceiling, falling to 0.013 for a lopsided one and 0.000 for a
hopeless one.

`disagrees_with_the_quality_times_variance_heuristic` pins down that
this is not a monotone transform of the heuristic it replaces — the two
rank a lopsided matchup and a confident even one in opposite orders — so
a later "simplification" cannot quietly revert to it.

Cost is one inference pass per possible outcome, documented on the
public API alongside the shortlist-then-score pattern, so callers do not
discover it in production.

Also folds the duplicated key-gathering in `predict_quality` and
`performances` into one validated `member_skills`.

Refs #39

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
2026-09-07 15:08:12 +02:00
logaritmiskandClaude Opus 5 507894dae7 refactor!: close the remaining API gaps from #21
Three unrelated small defects, all requiring signature changes:

- `Game::one_v_one` hardcoded `GameOptions::default()`, so a 1v1 could
  never set `p_draw` or convergence options — and a drawn 1v1 was
  therefore unreachable through it, since the default `p_draw` is zero.
  It now takes `&GameOptions` like every other constructor.

- `Observer::on_batch_processed` was declared on the trait and never
  called from anywhere: implementors wired up a callback that could not
  fire. It is now called after each slice sweep, and renamed
  `on_slice_processed` to match the vocabulary the codebase adopted in
  T2 — the unit of work is a `TimeSlice`, not a batch. A slice is swept
  once travelling backward and once forward, so a multi-slice history
  fires it twice per slice per iteration; the doc comment says so.

- `pub mod factors` sat beside `pub(crate) mod factor`, two module paths
  differing by one character with only one of them importable. The
  public facade is now `graph`.

Tests cover each as a behaviour rather than a compile check: a drawn 1v1
succeeds only when p_draw is supplied, and the observer tests fail if
any callback stops firing.

BREAKING CHANGE: `Game::one_v_one` takes a fourth `&GameOptions`
argument; `Observer::on_batch_processed` is renamed
`on_slice_processed`; the `factors` module is renamed `graph`.

Closes #21

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
2026-09-07 14:57:39 +02:00
logaritmiskandClaude Opus 5 bb2a845882 feat!: N-team outcome prediction with draw mass, replacing the 2-team panic
`predict_outcome` asserted `teams.len() == 2` and returned `[p, 1 - p]`,
allocating no probability to a draw even with `p_draw > 0`. For a
draw-enabled model the numbers were simply wrong, at any team count.

It now returns `Result<Prediction, InferenceError>` and supports N teams.

Two algorithms, both deterministic:

- Who finishes first. Performances are independent Gaussians, so this
  separates into a one-dimensional integral per team rather than a
  multivariate orthant probability. Adaptive Gauss-Kronrod evaluates it
  to ~1e-15, matching the exact two-team closed form.
- A specific finishing order. The factor graph only constrains
  rank-adjacent teams, so a full order is a chain of local constraints,
  not a general orthant integral. That chain collapses into a sequential
  recursion over cumulative integrals: O(teams * grid) per order.

Fixed-node Gauss-Hermite is the obvious tool for the first and is a trap:
when a rival's sigma is small the CDF product becomes a step narrower
than the node spacing, and the nodes step over it. Measured 4.4e-4 off
the closed form on a mildly skewed matchup and 1.7e-2 on a small-sigma
one, while still returning something that looks like a probability.
Adaptive refinement is what makes that case safe, and
`win_probabilities_survive_a_rival_with_a_tiny_sigma` pins it down.

The acceptance test is an identity rather than a golden: the outcome
space is exhaustive and disjoint, so the probabilities sum to one. Any
drift is integration error and nothing else. Gauss-Hermite failed it at
4.4e-4; this holds to ~1e-9.

Also from #21: unknown keys are now reported rather than dropped, so a
team of strangers can no longer produce a confident-looking prediction.
`predict_quality` returns `Result` for the same reason.

BREAKING CHANGE: `predict_outcome` returns `Result<Prediction, _>`
instead of `Vec<f64>`; `predict_quality` returns `Result<f64, _>`.

Refs #21, #39

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
2026-09-07 14:55:11 +02:00
logaritmiskandClaude Opus 5 87fca8dcca docs: correct drifted documentation and compile the README in CI
Four README code blocks no longer compiled: `Player` was renamed `Rating`
in T2, the `Drift` trait gained a `T: Time` parameter and a second method,
and two blocks were missing imports outright. The `Rating` example needed
more than a rename — with the binding unused, `T` is ambiguous because
`ConstantDrift` implements `Drift<T>` for every `T`, so it now carries an
explicit annotation.

Nothing compiled those blocks. `src/lib.rs` gains a `cfg(doctest)` struct
carrying `#[doc = include_str!("../README.md")]`, which turns every `rust`
block into a doctest without displacing the curated crate docs as the
front page. Verified it bites: reintroducing `Player` fails the build with
E0432 rather than shipping. Illustrative blocks are fenced `text` — note
that a bare fence defaults to `rust` under rustdoc, which is how the
`variance_delta = elapsed * γ²` formula became a compile error.

Prose fixes: README claimed `Gaussian::forget` takes a square root (it
works in variance space) and pointed at a `.gamma()` builder method that
does not exist. CLAUDE.md's data-flow diagram spliced the public ingestion
shape into the internal one — `Team` is not in that chain — listed
`cdf()`/`erfc()` as public when they are `pub(crate)` and private, and
called `SkillStore` public when only `CompetitorStore` escapes the crate.

Rustdoc fixes: `EventBuilder::scores_with_sigma` claimed a debug-assert
that `Outcome::scores_with_sigma` never had and whose own docs contradict;
rejection happens at ingestion as `InvalidParameter`. `event.rs` described
`add_events_with_prior` as replaced when it is still the ingestion
chokepoint. `factors.rs` advertised `Game::custom` without noting it is
`#[doc(hidden)]`. Internal T2/T4 milestone labels are dropped from public
items; the ones in the private `time_slice` module are left alone.

Closes #35

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014b6wy2q8rnFK8U8GPJVQNU
2026-09-01 19:30:32 +02:00
logaritmiskandClaude Opus 5 ef62b57a08 fix(release): skip the changelog hook during a dry run
`just release-plan` is documented as a preview that writes nothing, but
cargo-release runs pre-release hooks during a dry run too. The hook wrote
CHANGELOG.md and `git add`ed it, so the clean-tree check in `just release`
then refused to run — the repo's own two-step release workflow could not
be followed as written.

Guard the hook on DRY_RUN, which cargo-release 1.1.5 exports to the hook
environment (verified by dumping `env` from a throwaway hook; it also sets
CRATE_NAME, PREV_VERSION and NEW_VERSION). The clean-tree check itself is
left alone: it is load-bearing, because publishing is irreversible.

Closes #36

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014b6wy2q8rnFK8U8GPJVQNU
2026-09-01 19:26:22 +02:00
31 changed files with 3677 additions and 200 deletions
+24
View File
@@ -2,6 +2,29 @@
All notable changes to this project will be documented in this file. All notable changes to this project will be documented in this file.
## 0.4.0 - 2026-09-07
### Breaking Changes
- feat!: N-team outcome prediction with draw mass, replacing the 2-team panic
- refactor!: close the remaining API gaps from #21
- fix!: apply competitor configuration whenever it is supplied
### Bug Fixes
- fix(release): skip the changelog hook during a dry run
- fix: stop destroying tail precision in evidence and truncation
- fix: reject convergence options that silently disable inference
### Documentation
- docs: correct drifted documentation and compile the README in CI
### Features
- feat: add expected information gain for active matchup selection
- feat: let observers be shared, boxed, or borrowed
## 0.3.0 - 2026-09-01 ## 0.3.0 - 2026-09-01
### Breaking Changes ### Breaking Changes
@@ -25,6 +48,7 @@ All notable changes to this project will be documented in this file.
### Miscellaneous Tasks ### Miscellaneous Tasks
- chore: ignore proptest regression seed files - chore: ignore proptest regression seed files
- chore: Release trueskill-tt version 0.3.0
### Performance ### Performance
+24 -10
View File
@@ -32,10 +32,19 @@ evidence both forward and backward across a history.
### Data flow ### Data flow
Ingestion (public types, `event.rs`):
``` ```
History → TimeSlice[] → Event[] → Team[] → Item[] Event<T, K> → Team<K>[] Member<K>[]
```
Game (factor graph) → Schedule → BuiltinFactor[]
`History::add_events` flattens that into indices; teams survive only as
grouping, not as a value. Inference then runs on the internal shapes:
```
History → TimeSlice[] → Event[] → Item[]
Game (factor graph) → Schedule → BuiltinFactor[]
``` ```
- **`History`** (`history.rs`) — top level. Interns keys, groups events into - **`History`** (`history.rs`) — top level. Interns keys, groups events into
@@ -45,9 +54,12 @@ History → TimeSlice[] → Event[] → Team[] → Item[]
- **`TimeSlice`** (`time_slice.rs`) — all events at one time. Owns a - **`TimeSlice`** (`time_slice.rs`) — all events at one time. Owns a
`SkillStore` and a `ScratchArena`; `iteration()` sweeps its events, using `SkillStore` and a `ScratchArena`; `iteration()` sweeps its events, using
`ColorGroups` to partition independent ones. `ColorGroups` to partition independent ones.
- **`Event`** (`time_slice.rs`) — one match. `compute()` runs inference reading - **`Event`** — two distinct types, do not confuse them. The *public* ingestion
skills immutably; `apply()` folds the result back. The split is what lets a `Event<T, K>` is in `event.rs` (with `Team`/`Member`); the *internal*
color group run in parallel with no `unsafe`. `pub(crate) Event` in `time_slice.rs` is one match during inference, where
`compute()` runs inference reading skills immutably and `apply()` folds the
result back. That split is what lets a color group run in parallel with no
`unsafe`.
- **`Game`** (`game.rs`) — a single match's factor graph. `run_chain` builds the - **`Game`** (`game.rs`) — a single match's factor graph. `run_chain` builds the
diff chain between rank-adjacent teams and drives it to convergence. diff chain between rank-adjacent teams and drives it to convergence.
- **`Gaussian`** (`gaussian.rs`) — natural parameters (`pi = 1/sigma²`, - **`Gaussian`** (`gaussian.rs`) — natural parameters (`pi = 1/sigma²`,
@@ -61,14 +73,16 @@ History → TimeSlice[] → Event[] → Team[] → Item[]
the only implementation. the only implementation.
- **`Competitor`** (`competitor.rs`) — per-history temporal state (`message`, - **`Competitor`** (`competitor.rs`) — per-history temporal state (`message`,
`last_time`). **`Rating`** (`rating.rs`) — static config (prior, `beta`, drift). `last_time`). **`Rating`** (`rating.rs`) — static config (prior, `beta`, drift).
- **`storage/`** — `SkillStore` (per slice) and `CompetitorStore` (per history), - **`storage/`** — `SkillStore` (per slice, `pub(crate)`) and `CompetitorStore`
both dense `Vec`s indexed by `Index`. (per history, public), both indexed by `Index`. The module is `pub`, but only
`CompetitorStore` is reachable from outside the crate.
- **`KeyTable`** (`key_table.rs`) — user key ↔ `Index`, both directions O(1). - **`KeyTable`** (`key_table.rs`) — user key ↔ `Index`, both directions O(1).
- **`Drift`** (`drift.rs`) / **`Time`** (`time.rs`) — traits. `Time` is a *trait* - **`Drift`** (`drift.rs`) / **`Time`** (`time.rs`) — traits. `Time` is a *trait*
(`i64`, `Untimed`), not an enum. (`i64`, `Untimed`), not an enum.
- **`lib.rs`** — public exports, global defaults (`MU`, `SIGMA`, `BETA`, - **`lib.rs`** — public exports, global defaults (`MU`, `SIGMA`, `BETA`,
`GAMMA`, `P_DRAW`, `EPSILON`, `ITERATIONS`), and the standalone `quality()`, `GAMMA`, `P_DRAW`, `EPSILON`, `ITERATIONS`), and the standalone `quality()`.
`cdf()`, `erfc()`. The `cdf()` / `erfc()` helpers live here too but are `pub(crate)` and private
respectively — not public API.
### Invariants worth knowing ### Invariants worth knowing
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "trueskill-tt" name = "trueskill-tt"
version = "0.3.0" version = "0.4.0"
edition = "2024" edition = "2024"
rust-version = "1.85" rust-version = "1.85"
description = "TrueSkill Through Time: Bayesian skill rating that tracks how skill evolves over time, via Gaussian message passing" description = "TrueSkill Through Time: Bayesian skill rating that tracks how skill evolves over time, via Gaussian message passing"
+142 -30
View File
@@ -13,63 +13,92 @@ Rust port of [TrueSkillThroughTime.py](https://github.com/glandfried/TrueSkillTh
## Drift ## Drift
Skill drift models how a player's true skill can change between appearances. Each time a player reappears after a gap, their skill uncertainty is widened by the drift model before the new evidence is incorporated. Skill drift models how a competitor's true skill can change between appearances.
Each time they reappear after a gap, their skill uncertainty is widened by the
drift model before the new evidence is incorporated.
Drift is represented by the `Drift` trait: Drift is represented by the `Drift` trait (`src/drift.rs`), generic over the
history's time type:
```rust ```text
pub trait Drift: Copy + Debug { pub trait Drift<T: Time>: Copy + Debug + Send + Sync {
fn variance_delta(&self, elapsed: i64) -> f64; fn variance_delta(&self, from: &T, to: &T) -> f64;
fn variance_for_elapsed(&self, elapsed: i64) -> f64;
} }
``` ```
`variance_delta` returns the amount to add to `σ²` given the elapsed time since the player last played. Internally, `Gaussian::forget` uses this to compute the new sigma: `σ_new = sqrt(σ² + variance_delta)`. Both methods return the amount to add to `σ²`, not to `σ`. `variance_delta`
works from two timestamps; `variance_for_elapsed` takes an already-computed
elapsed count, and is used on the paths that cache it. `Gaussian::forget`
applies the result entirely in variance space — `from_mv(mu, variance() +
variance_delta)` — taking no square root.
That block is a quotation rather than a doctest. The custom-drift example below
is compiled by CI, so it is what actually pins the signature.
### ConstantDrift ### ConstantDrift
The built-in `ConstantDrift` implements a linear random walk — skill uncertainty grows proportionally to time: The built-in `ConstantDrift` implements a linear random walk — skill uncertainty
grows proportionally to time:
``` ```text
variance_delta = elapsed * γ² variance_delta = elapsed * γ²
``` ```
This is the standard TrueSkill Through Time model. Use it by passing a `ConstantDrift(gamma)` when constructing a `Player`: This is the standard TrueSkill Through Time model. Pass a `ConstantDrift(gamma)`
when constructing a `Rating`:
```rust ```rust
use trueskill_tt::{Player, Gaussian, drift::ConstantDrift}; use trueskill_tt::{ConstantDrift, Gaussian, Rating};
// gamma = 0.1 means skill can shift ~0.1 per time unit // gamma = 0.1 means skill can shift ~0.1 per time unit.
let player = Player::new(Gaussian::from_ms(0.0, 6.0), 1.0, ConstantDrift(0.1)); let rating: Rating<i64, ConstantDrift> =
Rating::new(Gaussian::from_ms(0.0, 6.0), 1.0, ConstantDrift(0.1));
assert_eq!(rating.drift().0, 0.1);
``` ```
The type annotation is load-bearing: `ConstantDrift` implements `Drift<T>` for
every `T: Time`, so without it `T` is ambiguous.
### Custom drift ### Custom drift
Implement `Drift` to express any other model. For example, a drift that saturates after a long absence (uncertainty grows with the square root of elapsed time instead of linearly): Implement `Drift<T>` to express any other model. For example, a drift that
saturates after a long absence, with uncertainty growing as the square root of
elapsed time instead of linearly:
```rust ```rust
use trueskill_tt::drift::Drift; use trueskill_tt::{Drift, Gaussian, History, Rating, Time};
#[derive(Clone, Copy, Debug)] #[derive(Clone, Copy, Debug)]
struct SqrtDrift { struct SqrtDrift {
gamma: f64, gamma: f64,
} }
impl Drift for SqrtDrift { impl<T: Time> Drift<T> for SqrtDrift {
fn variance_delta(&self, elapsed: i64) -> f64 { fn variance_delta(&self, from: &T, to: &T) -> f64 {
(elapsed as f64).sqrt() * self.gamma * self.gamma let elapsed = from.elapsed_to(to).max(0) as f64;
elapsed.sqrt() * self.gamma * self.gamma
}
fn variance_for_elapsed(&self, elapsed: i64) -> f64 {
(elapsed.max(0) as f64).sqrt() * self.gamma * self.gamma
} }
} }
let player = Player::new(Gaussian::from_ms(0.0, 6.0), 1.0, SqrtDrift { gamma: 0.5 }); // On a single Rating:
let rating: Rating<i64, SqrtDrift> =
Rating::new(Gaussian::from_ms(0.0, 6.0), 1.0, SqrtDrift { gamma: 0.5 });
// Or for a whole History, via the builder:
let history = History::builder().drift(SqrtDrift { gamma: 0.5 }).build();
assert_eq!(rating.beta(), 1.0);
assert_eq!(history.log_evidence(), 0.0);
``` ```
To use a custom drift type with `History`, use the `.drift()` builder method instead of `.gamma()`: `HistoryBuilder::drift` is the only way to set a history's drift model; there is
no `gamma()` shorthand. The default is `ConstantDrift(GAMMA)`.
```rust
let h = History::builder()
.drift(SqrtDrift { gamma: 0.5 })
.build();
```
### Per-competitor drift ### Per-competitor drift
@@ -84,16 +113,25 @@ expressible in the same graph as moving competitors — a bot at a known
strength, a rating floor, a course difficulty: strength, a rating floor, a course difficulty:
```rust ```rust
let events = vec![Event { use trueskill_tt::{ConstantDrift, Event, History, Member, Outcome, Team};
let mut h = History::builder().drift(ConstantDrift(0.1)).build();
h.add_events(vec![Event {
time: 0, time: 0,
teams: smallvec![ teams: [
Team::with_members([Member::new("player")]), Team::with_members([Member::new("player")]),
// A course does not improve. Pin it, and the round's evidence // A course does not improve. Pin it, and the round's evidence
// lands on the player instead of being split between the two. // lands on the player instead of being split between the two.
Team::with_members([Member::new("layout_7").with_drift_scale(0.0)]), Team::with_members([Member::new("layout_7").with_drift_scale(0.0)]),
], ]
.into_iter()
.collect(),
outcome: Outcome::winner(0, 2), outcome: Outcome::winner(0, 2),
}]; }])
.unwrap();
h.converge().unwrap();
``` ```
Like `with_prior`, the scale is **competitor configuration captured at first Like `with_prior`, the scale is **competitor configuration captured at first
@@ -101,6 +139,10 @@ appearance** — setting it on a key the history already knows has no effect. It
must be finite and non-negative; ingestion otherwise fails with must be finite and non-negative; ingestion otherwise fails with
`InferenceError::InvalidParameter`. `InferenceError::InvalidParameter`.
Note that the fluent `EventBuilder` (`h.event(t).team([...])`) sets weights but
not `drift_scale` or `prior`; those need the typed `Event` / `Team` / `Member`
shape shown above.
## Scored outcomes ## Scored outcomes
Use `Outcome::scores([...])` when you have continuous per-team scores rather Use `Outcome::scores([...])` when you have continuous per-team scores rather
@@ -110,7 +152,7 @@ soft Gaussian evidence about the latent performance diff. Configure
(smaller σ = more trust). (smaller σ = more trust).
```rust ```rust
use trueskill_tt::{History, Outcome}; use trueskill_tt::History;
let mut h = History::builder().score_sigma(2.0).build(); let mut h = History::builder().score_sigma(2.0).build();
h.event(1) h.event(1)
@@ -122,6 +164,75 @@ h.event(1)
h.converge().unwrap(); h.converge().unwrap();
``` ```
## Prediction
`predict_outcome` gives the full distribution over finishing orders. Each entry
is a rank vector in the same shape `Outcome::ranking` takes — equal ranks mean a
tie — so an outcome feeds straight back into inference.
```rust
use trueskill_tt::History;
let mut h = History::builder().p_draw(0.1).build();
h.record_winner(&"alice", &"bob", 1).unwrap();
h.converge().unwrap();
let p = h.predict_outcome(&[&[&"alice"], &[&"bob"]]).unwrap();
// Probabilities are exhaustive and disjoint, so they sum to one.
assert!((p.total() - 1.0).abs() < 1e-6);
let (best, likelihood) = p.most_likely().unwrap();
println!("most likely: {best:?} at {likelihood:.3}");
println!("draw: {:.3}", p.probability_of(&[0, 0]));
```
Supports any number of teams. Because the outcome space grows factorially, the
full distribution is capped at `MAX_PREDICTED_TEAMS`; two cheaper entry points
stay available at any size:
- `predict_win_probabilities(teams)``P(team i finishes strictly first)`,
quadratic in team count.
- `predict_ranking(teams, ranks)` — one specific finishing order.
Unknown keys are an error, not a silent omission: a team the history has never
seen cannot produce a confident-looking probability.
## Which match to play next
`quality()` measures whether a matchup is *fair*. That is not the same as
whether it is *informative*, and the two only coincide for two evenly matched
competitors. When each observation costs something, ask
`expected_information_gain` instead — the outcome-weighted divergence between
what you believe now and what you would believe afterwards.
```rust
use trueskill_tt::History;
let mut h = History::builder().build();
for t in 1..=10 {
h.record_winner(&"veteran", &"regular", t).unwrap();
h.record_winner(&"regular", &"veteran", t + 100).unwrap();
}
h.record_winner(&"veteran", &"newcomer", 500).unwrap();
h.converge().unwrap();
let settled = h.expected_information_gain(&[&[&"veteran"], &[&"regular"]]).unwrap();
let unknown = h.expected_information_gain(&[&[&"veteran"], &[&"newcomer"]]).unwrap();
// Playing the newcomer teaches you more than replaying a settled rivalry.
assert!(unknown > settled);
```
The result is in nats, and is bounded by the entropy of the outcome: at most
`ln 2 ≈ 0.693` for a two-way result, `ln 3` once draws are possible, `ln k` for
`k` outcomes. A value near zero means you already know how it ends.
This costs one full inference pass **per possible outcome**, so it is far more
expensive than `quality()`. Scoring every pairing among `n` competitors is
`O(n² × outcomes)` passes — shortlist with `quality()` or
`predict_win_probabilities` first, then score only the shortlist.
## Todo ## Todo
- [x] Implement approx for Gaussian - [x] Implement approx for Gaussian
@@ -130,6 +241,7 @@ h.converge().unwrap();
- [x] Add examples (`examples/atp.rs`, `examples/scored.rs`) - [x] Add examples (`examples/atp.rs`, `examples/scored.rs`)
- [x] Add Observer (`Observer` / `NullObserver`) - [x] Add Observer (`Observer` / `NullObserver`)
- [x] Benchmark the inference loop (`benches/batch.rs`, `benches/history_converge.rs`, `benches/ingest.rs`) - [x] Benchmark the inference loop (`benches/batch.rs`, `benches/history_converge.rs`, `benches/ingest.rs`)
- [x] N-team `predict_outcome` with draw mass, and `expected_information_gain`
- [ ] Cross-check `quality()` against [sublee/trueskill](https://github.com/sublee/trueskill/tree/master) — N-group support works and is covered by invariants, but no reference values are asserted - [ ] Cross-check `quality()` against [sublee/trueskill](https://github.com/sublee/trueskill/tree/master) — N-group support works and is covered by invariants, but no reference values are asserted
## License ## License
+9 -1
View File
@@ -3,4 +3,12 @@ publish = true
# Hold off pushing until tags and publish have both succeeded; `just release` # Hold off pushing until tags and publish have both succeeded; `just release`
# pushes last. # pushes last.
push = false push = false
pre-release-hook = ["sh", "-c", "git cliff -o CHANGELOG.md --tag {{version}} && git add CHANGELOG.md"] # Regenerate the changelog and stage it so it lands in the release commit.
#
# Guarded on DRY_RUN because cargo-release runs pre-release hooks during a dry
# run too (verified against cargo-release 1.1.5, which exports DRY_RUN=true,
# CRATE_NAME, PREV_VERSION and NEW_VERSION to the hook). Without the guard,
# `just release-plan` — documented as a preview that writes nothing — writes and
# `git add`s CHANGELOG.md, and the clean-tree check in `just release` then
# refuses to run. That check is load-bearing: publishing is irreversible.
pre-release-hook = ["sh", "-c", '[ "$DRY_RUN" = "true" ] || (git cliff -o CHANGELOG.md --tag {{version}} && git add CHANGELOG.md)']
+352
View File
@@ -0,0 +1,352 @@
//! Active learning: which comparison teaches you the most.
//!
//! [`quality`](crate::quality) answers "is this matchup *fair*". That is a
//! different question from "is this matchup *informative*", and the two
//! coincide only for two evenly matched competitors. When each observation
//! costs something — a human click, a scheduled fixture — the question worth
//! asking is the second one.
//!
//! The quantity here is expected information gain: the outcome-weighted
//! divergence between what you believe now and what you would believe after
//! seeing the result.
//!
//! ```text
//! EIG(matchup) = SUM P(outcome) * KL( posterior_after(outcome) || prior )
//! outcome
//! ```
//!
//! It is the mutual information between the observed outcome and the skills,
//! which is worth remembering because it pins the scale: information gain
//! cannot exceed the entropy of the thing you are about to observe. A contest
//! with `k` distinguishable outcomes can teach you at most `ln k` nats,
//! whatever the ratings. That ceiling is the sharpest available test of an
//! implementation — see [`expected_information_gain`].
use crate::{
GameOptions, Gaussian, InferenceError, Outcome, Rating, drift::Drift, predict, time::Time,
};
/// Outcomes below this probability contribute nothing measurable and are not
/// worth an inference pass.
///
/// The contribution of an outcome is `P * KL`, and `KL` is bounded in practice
/// by tens of nats, so a probability this small moves the total by less than
/// the quadrature error already present in `P` itself.
const NEGLIGIBLE: f64 = 1e-12;
/// `KL(q || p)` for two univariate Gaussians, in nats.
///
/// Both arguments are proper posteriors from inference, so the degenerate
/// cases guarded here (zero or infinite variance) indicate that inference has
/// broken down rather than anything a caller did.
fn kl_divergence(q: Gaussian, p: Gaussian) -> f64 {
let (var_q, var_p) = (q.sigma().powi(2), p.sigma().powi(2));
if !(var_q.is_finite() && var_p.is_finite()) || var_q <= 0.0 || var_p <= 0.0 {
return 0.0;
}
let mean_gap = q.mu() - p.mu();
0.5 * ((var_p / var_q).ln() + (var_q + mean_gap * mean_gap) / var_p - 1.0)
}
/// Expected information gain of a hypothetical matchup, in nats.
///
/// Enumerates the outcomes this matchup could have, runs inference for each to
/// get the belief it would produce, and weights the resulting divergence by
/// that outcome's probability. A higher value means the result would teach you
/// more.
///
/// # Interpreting the value
///
/// Nats. The upper bound is the entropy of the outcome variable: at most
/// `ln 2 ≈ 0.693` for a two-way result, `ln 3 ≈ 1.099` once draws are
/// possible, `ln k` for `k` outcomes. A value near the ceiling means the
/// result is close to a coin flip *and* would move the posteriors a long way;
/// a value near zero means you already know what will happen, or that the
/// result would barely change your beliefs if you saw it.
///
/// This is not a monotone transform of [`quality`](crate::quality). A lopsided
/// matchup between two uncertain competitors scores well on quality-times-
/// variance heuristics and poorly here, because the near-certain outcome
/// carries almost no information.
///
/// # Cost
///
/// One full inference pass per possible outcome, so this is far more expensive
/// than `quality()` — which is one closed-form evaluation. The outcome count
/// grows quickly with team count (3 outcomes for two teams that can draw, 13
/// for three, 75 for four), and scoring every candidate pairing among `n`
/// competitors is `O(n² × outcomes)` inference passes.
///
/// For a selector over many candidates, shortlist with the cheap
/// [`quality`](crate::quality) or
/// [`predict_win_probabilities`](crate::History::predict_win_probabilities)
/// first and score only the shortlist here. The expected-variance-reduction
/// proxy sometimes suggested as a cheaper alternative is *not* cheaper: it
/// needs the same hypothetical posteriors, so it shares the dominant cost.
///
/// # Errors
///
/// - `NotEnoughTeams` if fewer than two teams are supplied.
/// - `EmptyTeam` if any team has no members.
/// - `TooManyTeams` if the outcome space is too large to enumerate; see
/// [`MAX_PREDICTED_TEAMS`](crate::MAX_PREDICTED_TEAMS).
/// - `InvalidProbability` if `options.p_draw` is outside `[0.0, 1.0)`.
/// - Anything [`Game::ranked`](crate::Game::ranked) returns for a hypothetical
/// outcome.
pub fn expected_information_gain<T: Time, D: Drift<T>>(
teams: &[&[Rating<T, D>]],
options: &GameOptions,
) -> Result<f64, InferenceError> {
if teams.len() < 2 {
return Err(InferenceError::NotEnoughTeams { got: teams.len() });
}
if teams.len() > crate::MAX_PREDICTED_TEAMS {
return Err(InferenceError::TooManyTeams {
got: teams.len(),
max: crate::MAX_PREDICTED_TEAMS,
});
}
if !(0.0..1.0).contains(&options.p_draw) {
return Err(InferenceError::InvalidProbability {
value: options.p_draw,
});
}
for (idx, team) in teams.iter().enumerate() {
if team.is_empty() {
return Err(InferenceError::EmptyTeam { team: idx });
}
}
// Prediction runs on performances: skill inflated by each member's beta.
let performances: Vec<Gaussian> = teams
.iter()
.map(|team| {
team.iter()
.fold(crate::N00, |acc, rating| acc + rating.performance())
})
.collect();
// Draw margins per pair, derived from the teams' betas exactly as
// inference derives them, so the outcomes weighted here are the outcomes
// that would actually be fitted.
let beta_sq: Vec<f64> = teams
.iter()
.map(|team| team.iter().map(|r| r.beta().powi(2)).sum())
.collect();
let p_draw = options.p_draw;
let margins = predict::Margins::new(teams.len(), |i, j| {
if p_draw == 0.0 {
0.0
} else {
crate::compute_margin(p_draw, (beta_sq[i] + beta_sq[j]).sqrt())
}
});
let mut gain = 0.0;
for (ranks, probability) in predict::outcome_distribution(&performances, &margins) {
if probability <= NEGLIGIBLE {
continue;
}
let game = crate::Game::ranked(teams, Outcome::ranking(ranks), options)?;
let posteriors = game.posteriors();
// Beliefs factorise across competitors, so the joint divergence is the
// sum of the per-competitor ones.
let divergence: f64 = teams
.iter()
.zip(&posteriors)
.flat_map(|(team, posterior)| team.iter().zip(posterior))
.map(|(rating, &after)| kl_divergence(after, rating.prior()))
.sum();
gain += probability * divergence;
}
Ok(gain)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{BETA, ConstantDrift, GAMMA};
type R = Rating<i64, ConstantDrift>;
fn rating(mu: f64, sigma: f64) -> R {
R::new(Gaussian::from_ms(mu, sigma), BETA, ConstantDrift(GAMMA))
}
fn options(p_draw: f64) -> GameOptions {
GameOptions {
p_draw,
..GameOptions::default()
}
}
fn eig(teams: &[&[R]], p_draw: f64) -> f64 {
expected_information_gain(teams, &options(p_draw)).unwrap()
}
/// The analytic ceiling. Information gain is the mutual information between
/// the outcome and the skills, so it cannot exceed the entropy of the
/// outcome variable — whatever the ratings. This is the check a subtly
/// wrong implementation fails while still returning plausible numbers: an
/// early prototype of this returned 4.77 nats from a sign error and passed
/// every monotonicity test.
#[test]
fn never_exceeds_the_entropy_of_the_outcome() {
let ceiling_two = std::f64::consts::LN_2;
for (a, b) in [
(rating(0.0, 6.0), rating(0.0, 6.0)),
(rating(0.0, 0.5), rating(0.0, 0.5)),
(rating(12.0, 6.0), rating(-12.0, 6.0)),
(rating(40.0, 1.0), rating(-40.0, 1.0)),
(rating(3.0, 6.0), rating(-2.0, 0.1)),
(rating(0.0, 25.0), rating(0.0, 25.0)),
] {
let g = eig(&[&[a], &[b]], 0.0);
assert!(
g >= 0.0 && g <= ceiling_two,
"EIG {g} outside [0, ln 2] for mu=({}, {}) sigma=({}, {})",
a.prior().mu(),
b.prior().mu(),
a.prior().sigma(),
b.prior().sigma()
);
}
}
/// With draws enabled there are three outcomes, so the ceiling rises to
/// `ln 3` — and the two-outcome bound no longer applies.
#[test]
fn the_ceiling_follows_the_outcome_count() {
let ceiling_three = 3.0f64.ln();
for sigma in [0.5, 3.0, 6.0, 25.0] {
let g = eig(&[&[rating(0.0, sigma)], &[rating(0.0, sigma)]], 0.25);
assert!(
g >= 0.0 && g <= ceiling_three,
"EIG {g} outside [0, ln 3] at sigma {sigma}"
);
}
}
/// An even matchup between uncertain competitors is the informative one.
/// A hopelessly lopsided matchup teaches you almost nothing, because you
/// already know how it ends.
#[test]
fn an_even_matchup_beats_a_lopsided_one() {
let even = eig(&[&[rating(0.0, 6.0)], &[rating(0.0, 6.0)]], 0.0);
let lopsided = eig(&[&[rating(12.0, 6.0)], &[rating(-12.0, 6.0)]], 0.0);
assert!(
even > lopsided,
"even {even} should beat lopsided {lopsided}"
);
}
/// Certainty is the thing information gain is measuring the absence of:
/// the less you know, the more there is to learn.
#[test]
fn gain_falls_as_certainty_rises() {
let mut previous = f64::INFINITY;
for sigma in [12.0, 6.0, 3.0, 1.0, 0.5, 0.1] {
let g = eig(&[&[rating(0.0, sigma)], &[rating(0.0, sigma)]], 0.0);
assert!(
g < previous,
"sigma {sigma}: {g} did not fall below {previous}"
);
previous = g;
}
assert!(previous >= 0.0);
}
/// The heuristic this replaces is `quality * sigma_a^2 * sigma_b^2`. It is
/// not a monotone transform of information gain — it ranks a lopsided
/// matchup above a confident even one, and EIG ranks them the other way.
/// Pinning the disagreement down is what stops a future "simplification"
/// from quietly reverting to the heuristic.
#[test]
fn disagrees_with_the_quality_times_variance_heuristic() {
let heuristic = |a: &R, b: &R| {
crate::quality(&[&[a.prior()], &[b.prior()]], BETA)
* a.prior().sigma().powi(2)
* b.prior().sigma().powi(2)
};
let (confident_a, confident_b) = (rating(0.0, 0.5), rating(0.0, 0.5));
let (lopsided_a, lopsided_b) = (rating(12.0, 6.0), rating(-12.0, 6.0));
assert!(
heuristic(&lopsided_a, &lopsided_b) > heuristic(&confident_a, &confident_b),
"the heuristic should prefer the lopsided matchup"
);
assert!(
eig(&[&[confident_a], &[confident_b]], 0.0) > eig(&[&[lopsided_a], &[lopsided_b]], 0.0),
"information gain should prefer the even matchup"
);
}
#[test]
fn supports_more_than_two_teams() {
let teams: Vec<Vec<R>> = vec![
vec![rating(0.0, 6.0)],
vec![rating(0.0, 6.0)],
vec![rating(0.0, 6.0)],
];
let refs: Vec<&[R]> = teams.iter().map(Vec::as_slice).collect();
let g = expected_information_gain(&refs, &options(0.0)).unwrap();
// Six distinguishable orderings with no draws.
assert!(
g > 0.0 && g <= 6.0f64.ln(),
"three-team EIG {g} out of range"
);
}
#[test]
fn multi_member_teams_are_supported() {
let a = [rating(0.0, 6.0), rating(1.0, 4.0)];
let b = [rating(0.0, 6.0)];
let g = expected_information_gain(&[&a, &b], &options(0.0)).unwrap();
assert!(g > 0.0 && g <= std::f64::consts::LN_2, "{g}");
}
#[test]
fn degenerate_shapes_are_errors() {
let a = [rating(0.0, 6.0)];
assert!(matches!(
expected_information_gain(&[&a], &options(0.0)),
Err(InferenceError::NotEnoughTeams { got: 1 })
));
let empty: [R; 0] = [];
assert!(matches!(
expected_information_gain(&[&a, &empty], &options(0.0)),
Err(InferenceError::EmptyTeam { team: 1 })
));
assert!(matches!(
expected_information_gain(&[&a, &a], &options(1.5)),
Err(InferenceError::InvalidProbability { .. })
));
}
#[test]
fn kl_divergence_is_zero_for_identical_beliefs() {
let g = Gaussian::from_ms(3.0, 2.0);
assert!(kl_divergence(g, g).abs() < 1e-15);
}
#[test]
fn kl_divergence_is_non_negative_and_grows_with_separation() {
let prior = Gaussian::from_ms(0.0, 3.0);
let mut previous = 0.0;
for mu in [0.0, 0.5, 1.0, 2.0, 4.0] {
let d = kl_divergence(Gaussian::from_ms(mu, 3.0), prior);
assert!(d >= 0.0, "negative divergence at mu {mu}: {d}");
assert!(d >= previous, "not increasing at mu {mu}");
previous = d;
}
}
}
+2 -2
View File
@@ -7,8 +7,8 @@ use crate::{
/// Per-history, temporal state for someone competing. /// Per-history, temporal state for someone competing.
/// ///
/// Renamed from `Agent` in T2; the former `.player` field is now /// The mutable half of a competitor: `Rating` holds their static
/// `.rating` to match the `Player → Rating` rename. /// configuration, this holds what inference learns as it sweeps.
#[derive(Debug)] #[derive(Debug)]
pub struct Competitor<T: Time = i64, D: Drift<T> = ConstantDrift> { pub struct Competitor<T: Time = i64, D: Drift<T> = ConstantDrift> {
pub rating: Rating<T, D>, pub rating: Rating<T, D>,
+31
View File
@@ -20,6 +20,37 @@ pub struct ConvergenceOptions {
pub alpha: f64, pub alpha: f64,
} }
impl ConvergenceOptions {
/// Reject values that would make inference silently meaningless.
///
/// `HistoryBuilder::convergence` asserts these eagerly, but the fields are
/// public and `GameOptions` carries a `ConvergenceOptions` — so a caller
/// can hand `Game::ranked` a set the builder never saw. In release the
/// engine's `debug_assert!`s are gone, and an `alpha` of zero leaves every
/// EP update unapplied: inference returns the priors, with every likelihood
/// uninformative and nothing to indicate anything went wrong.
///
/// # Errors
///
/// `InvalidParameter` if `alpha` is outside `(0.0, 1.0]` or `epsilon` is
/// negative. NaN fails both comparisons and is rejected.
pub(crate) fn validate(&self) -> Result<(), crate::InferenceError> {
if !(self.alpha > 0.0 && self.alpha <= 1.0) {
return Err(crate::InferenceError::InvalidParameter {
name: "alpha",
value: self.alpha,
});
}
if self.epsilon.is_nan() || self.epsilon < 0.0 {
return Err(crate::InferenceError::InvalidParameter {
name: "epsilon",
value: self.epsilon,
});
}
Ok(())
}
}
impl Default for ConvergenceOptions { impl Default for ConvergenceOptions {
fn default() -> Self { fn default() -> Self {
Self { Self {
+51 -14
View File
@@ -25,11 +25,6 @@ pub enum InferenceError {
/// result has no representable likelihood. Configure a positive `p_draw` /// result has no representable likelihood. Configure a positive `p_draw`
/// (via `HistoryBuilder::p_draw` or `GameOptions::p_draw`) to admit ties. /// (via `HistoryBuilder::p_draw` or `GameOptions::p_draw`) to admit ties.
TieWithoutDrawProbability { teams: (usize, usize) }, TieWithoutDrawProbability { teams: (usize, usize) },
/// Convergence exceeded `max_iter` without falling below `epsilon`.
ConvergenceFailed {
last_step: (f64, f64),
iterations: usize,
},
/// Inference produced a non-finite value (NaN or infinity). /// Inference produced a non-finite value (NaN or infinity).
/// ///
/// Indicates numerical breakdown; the resulting skills are meaningless /// Indicates numerical breakdown; the resulting skills are meaningless
@@ -38,8 +33,37 @@ pub enum InferenceError {
context: &'static str, context: &'static str,
step: (f64, f64), step: (f64, f64),
}, },
/// Negative precision: a Gaussian with `pi < 0` slipped into an API call. /// One batch declared two different values for the same competitor's
NegativePrecision { pi: f64 }, /// configuration.
///
/// `prior` and `drift_scale` configure a competitor, not an event, so a
/// batch that sets one of them twice with different values has no
/// well-defined meaning: events within a batch are not ordered, so
/// "last one wins" would make the result depend on iteration order.
/// Declaring the same value repeatedly is fine and is the expected shape
/// when a competitor's configuration is a property of the domain.
ConflictingCompetitorConfig {
competitor: usize,
field: &'static str,
},
/// A prediction referenced a key the history has no skill for.
///
/// Reported rather than skipped: dropping unknown keys turns a team of
/// strangers into a confident-looking probability about nobody.
UnknownKey { team: usize, member: usize },
/// A prediction was given a team with no members.
EmptyTeam { team: usize },
/// Fewer than two teams were supplied to a prediction.
NotEnoughTeams { got: usize },
/// The full outcome distribution was requested for too many teams.
///
/// Each realisation sorts into exactly one (order, tie-pattern) event, so
/// the space holds `n! * 2^(n-1)` members — 1_920 at five teams, 23_040 at
/// six, 322_560 at seven. Past `max` this stops being something to
/// enumerate on a caller's behalf; ask for individual rankings with
/// `predict_ranking`, or for `predict_win_probabilities`, both of which
/// stay cheap at any team count.
TooManyTeams { got: usize, max: usize },
} }
impl fmt::Display for InferenceError { impl fmt::Display for InferenceError {
@@ -78,17 +102,30 @@ impl fmt::Display for InferenceError {
Self::InvalidParameter { name, value } => { Self::InvalidParameter { name, value } => {
write!(f, "{name} is invalid: {value}") write!(f, "{name} is invalid: {value}")
} }
Self::ConvergenceFailed { Self::ConflictingCompetitorConfig { competitor, field } => {
last_step,
iterations,
} => {
write!( write!(
f, f,
"convergence failed after {iterations} iterations; last step = {last_step:?}" "competitor {competitor}: this batch sets {field} to two different values"
) )
} }
Self::NegativePrecision { pi } => { Self::UnknownKey { team, member } => {
write!(f, "precision must be non-negative; got {pi}") write!(
f,
"team {team}, member {member}: no skill recorded for this key"
)
}
Self::EmptyTeam { team } => {
write!(f, "team {team} has no members")
}
Self::NotEnoughTeams { got } => {
write!(f, "prediction needs at least 2 teams, got {got}")
}
Self::TooManyTeams { got, max } => {
write!(
f,
"the outcome distribution over {got} teams is too large to enumerate (limit {max}); \
use predict_ranking or predict_win_probabilities instead"
)
} }
} }
} }
+16 -6
View File
@@ -1,8 +1,10 @@
//! Typed event description for bulk ingestion. //! Typed event description for bulk ingestion.
//! //!
//! `Event<T, K>` is the new public event shape (spec Section 4). Replaces //! `Event<T, K>` is the public event shape taken by `History::add_events`. It
//! the nested `Vec<Vec<Vec<Index>>>`, `Vec<Vec<f64>>`, `Vec<Vec<Vec<f64>>>` //! is a typed front end, not a replacement: `add_events` flattens it into the
//! that the old `add_events_with_prior` took. //! nested `Vec<Vec<Vec<Index>>>` / `Vec<Vec<f64>>` / `Vec<Vec<Vec<f64>>>` that
//! the internal `add_events_with_prior` chokepoint still takes, and which
//! `record_winner` and `record_draw` also route through.
use smallvec::SmallVec; use smallvec::SmallVec;
@@ -48,9 +50,17 @@ impl<K> Default for Team<K> {
/// `weight` applies per event and defaults to 1.0. /// `weight` applies per event and defaults to 1.0.
/// ///
/// `prior` and `drift_scale` are **competitor configuration**, not per-event /// `prior` and `drift_scale` are **competitor configuration**, not per-event
/// values: both are captured when the competitor is first created and ignored /// values. Setting either applies to the competitor for the whole history, not
/// on every later appearance. Setting either on a key the history already knows /// just to this event, and applies whenever it is supplied — including on a key
/// has no effect. /// the history already knows. Because configuration lives on the competitor and
/// `converge` refits from competitor state, configuring one late still refits
/// the whole history rather than taking effect only from that event onward.
///
/// Repeating the same value is inert, which is the expected shape when the
/// configuration is a property of the domain. Supplying two *different* values
/// for one competitor within a single batch is
/// `InferenceError::ConflictingCompetitorConfig`: events in a batch have no
/// order, so there would be no well-defined winner.
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct Member<K> { pub struct Member<K> {
pub key: K, pub key: K,
+4 -1
View File
@@ -107,7 +107,10 @@ where
/// Set explicit per-team continuous scores with a per-event noise override. /// Set explicit per-team continuous scores with a per-event noise override.
/// ///
/// `sigma` overrides `HistoryBuilder::score_sigma` for this event only. /// `sigma` overrides `HistoryBuilder::score_sigma` for this event only.
/// Must be `> 0.0`; debug-asserts otherwise via `Outcome::scores_with_sigma`. /// Must be `> 0.0`. Constructing the outcome with a non-positive or NaN
/// sigma is allowed; the value is rejected with
/// `InferenceError::InvalidParameter` when the event is ingested, so
/// callers get an error from `commit` rather than a panic.
pub fn scores_with_sigma<I: IntoIterator<Item = f64>>(mut self, scores: I, sigma: f64) -> Self { pub fn scores_with_sigma<I: IntoIterator<Item = f64>>(mut self, scores: I, sigma: f64) -> Self {
self.event.outcome = crate::Outcome::scores_with_sigma(scores, sigma); self.event.outcome = crate::Outcome::scores_with_sigma(scores, sigma);
self self
+67 -7
View File
@@ -2,6 +2,7 @@ use crate::{
N_INF, approx, cdf, N_INF, approx, cdf,
factor::{Factor, VarId, VarStore}, factor::{Factor, VarId, VarStore},
gaussian::Gaussian, gaussian::Gaussian,
sf,
}; };
/// EP truncation factor on a diff variable. /// EP truncation factor on a diff variable.
@@ -74,16 +75,29 @@ impl Factor for TruncFactor {
/// P(diff > margin) for non-tie, P(|diff| < margin) for tie. /// P(diff > margin) for non-tie, P(|diff| < margin) for tie.
/// ///
/// Clamped to a positive floor: for a near-certain outcome the tail rounds to /// Both branches pick whichever tail keeps their terms *small*, because the
/// exactly 0.0, and the `erfc` approximation used by `cdf` carries ~1e-7 error /// alternative is subtracting two numbers that both approach 1. That
/// so it can even return slightly more than 1.0, making the difference /// subtraction is not a rounding detail: it loses every digit of an unlikely
/// negative. Either would send `log_evidence` to `-inf` or NaN and poison the /// outcome's evidence, and an unlikely outcome is precisely the one worth
/// sum across the whole history. /// scoring. `1 - cdf` returned exactly zero past ~8.3 sigma, where the true
/// probability is 1e-19; clamped, that reached `log_evidence` as -708 instead
/// of -43.
///
/// The clamp remains as a guard rather than a workaround: `erfc` carries ~1e-7
/// relative error, so a probability of exactly 1 can still come back a hair
/// above it, and `ln` of a negative would poison the sum for the whole history.
fn cavity_evidence(diff: Gaussian, margin: f64, tie: bool) -> f64 { fn cavity_evidence(diff: Gaussian, margin: f64, tie: bool) -> f64 {
let (mu, sigma) = (diff.mu(), diff.sigma());
let raw = if tie { let raw = if tie {
cdf(margin, diff.mu(), diff.sigma()) - cdf(-margin, diff.mu(), diff.sigma()) if mu < -margin {
// Both CDFs sit against 1 here; both survival terms are small.
sf(-margin, mu, sigma) - sf(margin, mu, sigma)
} else {
cdf(margin, mu, sigma) - cdf(-margin, mu, sigma)
}
} else { } else {
1.0 - cdf(margin, diff.mu(), diff.sigma()) sf(margin, mu, sigma)
}; };
raw.clamp(f64::MIN_POSITIVE, 1.0) raw.clamp(f64::MIN_POSITIVE, 1.0)
@@ -132,6 +146,52 @@ mod tests {
assert_eq!(f.evidence_cached.unwrap(), first); assert_eq!(f.evidence_cached.unwrap(), first);
} }
/// The defect this guards: `1 - cdf` collapsed to zero for a surprising
/// result, the clamp turned that into `f64::MIN_POSITIVE`, and
/// `log_evidence` reported ln of *that* — about -708 whatever the truth
/// was. An upset is the observation a model-comparison score exists to
/// notice, so it was wrong exactly where it mattered.
#[test]
fn evidence_of_an_upset_is_not_flattened_to_the_clamp_floor() {
// diff ~ N(-9, 1) with margin 0: the favoured side lost by nine sigma.
let evidence = cavity_evidence(Gaussian::from_ms(-9.0, 1.0), 0.0, false);
assert!(
evidence > f64::MIN_POSITIVE,
"evidence collapsed onto the clamp floor: {evidence}"
);
// P(X > 0) for X ~ N(-9, 1) is the standard normal tail at 9 sigma.
assert!(
(evidence - 1.128_588e-19).abs() / 1.128_588e-19 < 1e-6,
"expected ~1.13e-19, got {evidence}"
);
assert!(
(evidence.ln() + 43.628).abs() < 1e-2,
"log evidence {} should be about -43.6, not -708",
evidence.ln()
);
}
/// Evidence must stay finite and positive however extreme the mismatch,
/// since `log_evidence` sums across the whole history and one `-inf` or
/// `NaN` poisons all of it.
#[test]
fn evidence_stays_positive_and_finite_at_any_separation() {
for mu in [-300.0f64, -50.0, -9.0, 0.0, 9.0, 50.0, 300.0] {
for tie in [false, true] {
let e = cavity_evidence(Gaussian::from_ms(mu, 1.0), 1.0, tie);
assert!(
e.is_finite() && e > 0.0 && e <= 1.0,
"mu={mu} tie={tie}: evidence {e} is not a probability"
);
assert!(
e.ln().is_finite(),
"mu={mu} tie={tie}: ln evidence is not finite"
);
}
}
}
#[test] #[test]
fn tie_evidence_uses_two_sided() { fn tie_evidence_uses_two_sided() {
let mut vars = VarStore::new(); let mut vars = VarStore::new();
-13
View File
@@ -1,13 +0,0 @@
//! Factor-graph public API.
//!
//! Power users can construct custom factor graphs via `Game::custom` (T2
//! minimal; full ergonomics in T4) and drive them with custom `Schedule`
//! implementations.
pub use crate::{
factor::{
BuiltinFactor, Factor, VarId, VarStore, margin::MarginFactor, rank_diff::RankDiffFactor,
team_sum::TeamSumFactor, trunc::TruncFactor,
},
schedule::{EpsilonOrMax, Schedule, ScheduleReport},
};
+18 -11
View File
@@ -433,6 +433,9 @@ impl<'a, T: Time, D: Drift<T>> Game<'a, T, D> {
impl<T: Time, D: Drift<T>> Game<'_, T, D> { impl<T: Time, D: Drift<T>> Game<'_, T, D> {
/// # Errors /// # Errors
/// ///
/// - `InvalidParameter` if `options.convergence` is out of range — an
/// `alpha` of zero would leave every EP update unapplied and silently
/// return the priors.
/// - `InvalidProbability` if `options.p_draw` is outside `[0.0, 1.0)`. /// - `InvalidProbability` if `options.p_draw` is outside `[0.0, 1.0)`.
/// - `MismatchedShape` if the outcome's rank count differs from `teams.len()`. /// - `MismatchedShape` if the outcome's rank count differs from `teams.len()`.
/// - `WrongOutcomeKind` if `outcome` is not `Outcome::Ranked`. /// - `WrongOutcomeKind` if `outcome` is not `Outcome::Ranked`.
@@ -444,6 +447,7 @@ impl<T: Time, D: Drift<T>> Game<'_, T, D> {
outcome: crate::Outcome, outcome: crate::Outcome,
options: &GameOptions, options: &GameOptions,
) -> Result<OwnedGame<T, D>, crate::InferenceError> { ) -> Result<OwnedGame<T, D>, crate::InferenceError> {
options.convergence.validate()?;
if !(0.0..1.0).contains(&options.p_draw) { if !(0.0..1.0).contains(&options.p_draw) {
return Err(crate::InferenceError::InvalidProbability { return Err(crate::InferenceError::InvalidProbability {
value: options.p_draw, value: options.p_draw,
@@ -491,8 +495,8 @@ impl<T: Time, D: Drift<T>> Game<'_, T, D> {
/// # Errors /// # Errors
/// ///
/// - `InvalidParameter` if `options.score_sigma` is not strictly positive, /// - `InvalidParameter` if `options.score_sigma` is not strictly positive
/// or is NaN. /// or is NaN, or if `options.convergence` is out of range.
/// - `MismatchedShape` if the outcome's score count differs from `teams.len()`. /// - `MismatchedShape` if the outcome's score count differs from `teams.len()`.
/// - `WrongOutcomeKind` if `outcome` is not `Outcome::Scored`. /// - `WrongOutcomeKind` if `outcome` is not `Outcome::Scored`.
pub fn scored( pub fn scored(
@@ -500,6 +504,7 @@ impl<T: Time, D: Drift<T>> Game<'_, T, D> {
outcome: crate::Outcome, outcome: crate::Outcome,
options: &GameOptions, options: &GameOptions,
) -> Result<OwnedGame<T, D>, crate::InferenceError> { ) -> Result<OwnedGame<T, D>, crate::InferenceError> {
options.convergence.validate()?;
if options.score_sigma <= 0.0 || options.score_sigma.is_nan() { if options.score_sigma <= 0.0 || options.score_sigma.is_nan() {
return Err(crate::InferenceError::InvalidParameter { return Err(crate::InferenceError::InvalidParameter {
name: "score_sigma", name: "score_sigma",
@@ -532,18 +537,20 @@ impl<T: Time, D: Drift<T>> Game<'_, T, D> {
)) ))
} }
/// Convenience wrapper over [`Game::ranked`] for two single-player teams.
///
/// # Errors /// # Errors
/// ///
/// Delegates to [`Game::ranked`] with default options, so it returns the /// Delegates to [`Game::ranked`], so it returns the same errors — in
/// same errors — in practice `WrongOutcomeKind` for a non-ranked outcome, /// practice `WrongOutcomeKind` for a non-ranked outcome, or
/// or `TieWithoutDrawProbability` for a draw, since the default `p_draw` /// `TieWithoutDrawProbability` for a draw when `options.p_draw` is zero.
/// applies rather than one you chose.
pub fn one_v_one( pub fn one_v_one(
a: &Rating<T, D>, a: &Rating<T, D>,
b: &Rating<T, D>, b: &Rating<T, D>,
outcome: crate::Outcome, outcome: crate::Outcome,
options: &GameOptions,
) -> Result<(Gaussian, Gaussian), crate::InferenceError> { ) -> Result<(Gaussian, Gaussian), crate::InferenceError> {
let game = Self::ranked(&[&[*a], &[*b]], outcome, &GameOptions::default())?; let game = Self::ranked(&[&[*a], &[*b]], outcome, options)?;
let post = game.posteriors(); let post = game.posteriors();
Ok((post[0][0], post[1][0])) Ok((post[0][0], post[1][0]))
} }
@@ -563,11 +570,11 @@ impl<T: Time, D: Drift<T>> Game<'_, T, D> {
} }
#[doc(hidden)] #[doc(hidden)]
pub fn custom<S: crate::factors::Schedule>( pub fn custom<S: crate::graph::Schedule>(
factors: &mut [crate::factors::BuiltinFactor], factors: &mut [crate::graph::BuiltinFactor],
vars: &mut crate::factors::VarStore, vars: &mut crate::graph::VarStore,
schedule: &S, schedule: &S,
) -> crate::factors::ScheduleReport { ) -> crate::graph::ScheduleReport {
schedule.run(factors, vars) schedule.run(factors, vars)
} }
} }
+20
View File
@@ -0,0 +1,20 @@
//! Factor-graph public API.
//!
//! Named `graph` rather than `factors` because the private implementation
//! module beside it is `factor`: two module paths differing by one character,
//! one public and one not, was a standing invitation to import the wrong one.
//!
//! The factor types, `VarStore` and the `Schedule` trait are public so custom
//! schedules can be written against them.
//!
//! Building a factor graph by hand goes through `Game::custom`, which is
//! deliberately `#[doc(hidden)]`: it works, but its signature is not yet
//! considered stable API and so is not listed in these docs.
pub use crate::{
factor::{
BuiltinFactor, Factor, VarId, VarStore, margin::MarginFactor, rank_diff::RankDiffFactor,
team_sum::TeamSumFactor, trunc::TruncFactor,
},
schedule::{EpsilonOrMax, Schedule, ScheduleReport},
};
+365 -68
View File
@@ -9,6 +9,7 @@ use crate::{
gaussian::Gaussian, gaussian::Gaussian,
key_table::KeyTable, key_table::KeyTable,
observer::{NullObserver, Observer}, observer::{NullObserver, Observer},
predict::Prediction,
rating::Rating, rating::Rating,
sort_time, sort_time,
storage::CompetitorStore, storage::CompetitorStore,
@@ -171,6 +172,23 @@ impl Default for HistoryBuilder<i64, ConstantDrift, NullObserver, &'static str>
} }
} }
/// Configuration a caller attached to a competitor via `Member`.
///
/// Carries *what was explicitly set* rather than a merged `Rating`, so a member
/// that sets only `drift_scale` does not also assert the default prior — which
/// would spuriously conflict with a prior seeded on an earlier event.
#[derive(Clone, Copy, Default)]
pub(crate) struct CompetitorConfig {
prior: Option<Gaussian>,
drift_scale: Option<f64>,
}
impl CompetitorConfig {
fn is_empty(self) -> bool {
self.prior.is_none() && self.drift_scale.is_none()
}
}
pub struct History< pub struct History<
T: Time = i64, T: Time = i64,
D: Drift<T> = ConstantDrift, D: Drift<T> = ConstantDrift,
@@ -260,6 +278,11 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
let old = self.time_slices[j].posteriors(); let old = self.time_slices[j].posteriors();
self.time_slices[j].new_backward_info(&self.agents); self.time_slices[j].new_backward_info(&self.agents);
self.observer.on_slice_processed(
&self.time_slices[j].time,
j,
self.time_slices[j].events.len(),
);
let new = self.time_slices[j].posteriors(); let new = self.time_slices[j].posteriors();
@@ -279,6 +302,11 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
let old = self.time_slices[j].posteriors(); let old = self.time_slices[j].posteriors();
self.time_slices[j].new_forward_info(&self.agents); self.time_slices[j].new_forward_info(&self.agents);
self.observer.on_slice_processed(
&self.time_slices[j].time,
j,
self.time_slices[j].events.len(),
);
let new = self.time_slices[j].posteriors(); let new = self.time_slices[j].posteriors();
@@ -291,6 +319,11 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
let old = self.time_slices[0].posteriors(); let old = self.time_slices[0].posteriors();
self.time_slices[0].iteration(0, &self.agents); self.time_slices[0].iteration(0, &self.agents);
self.observer.on_slice_processed(
&self.time_slices[0].time,
0,
self.time_slices[0].events.len(),
);
let new = self.time_slices[0].posteriors(); let new = self.time_slices[0].posteriors();
@@ -522,61 +555,268 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
.sum() .sum()
} }
/// Draw-probability quality metric for the given teams (key slices). /// The configured observer.
/// ///
/// Values range roughly [0, 1]; 1 == perfectly matched. Supports any /// `History` takes its observer by value, so this is how a caller inspects
/// number of teams. /// one it did not keep a handle to. For an observer that accumulates
/// /// state, prefer passing an `Arc` and keeping a clone — see the
/// # Panics /// [`Observer`] docs.
/// #[must_use]
/// Panics if fewer than two teams are supplied, or if a team resolves to pub fn observer(&self) -> &O {
/// no known competitors — keys absent from the history, or competitors &self.observer
/// with no recorded skill, are dropped, so a team of entirely-unknown
/// keys becomes empty. Use `lookup` to check keys first.
pub fn predict_quality(&self, teams: &[&[&K]]) -> f64 {
let groups: Vec<Vec<Gaussian>> = teams
.iter()
.map(|team| {
team.iter()
.filter_map(|k| self.keys.get(*k))
.filter_map(|idx| {
self.time_slices
.iter()
.rev()
.find_map(|ts| ts.skills.get(idx).map(|s| s.posterior()))
})
.collect()
})
.collect();
let group_refs: Vec<&[Gaussian]> = groups.iter().map(|g| g.as_slice()).collect();
crate::quality(&group_refs, self.beta)
} }
/// 2-team win probability: returns `[P(team0 wins), P(team1 wins)]`. /// Consume the history and return its observer.
/// ///
/// N-team support lands in T4. /// Useful for reclaiming a non-shared observer's accumulated state after
/// `converge` without needing interior mutability.
#[must_use]
pub fn into_observer(self) -> O {
self.observer
}
/// Every team's member skills, validated.
/// ///
/// # Panics /// # Errors
/// ///
/// Panics if `teams.len() != 2`. /// `NotEnoughTeams`, `EmptyTeam`, or `UnknownKey`. Unknown keys are
pub fn predict_outcome(&self, teams: &[&[&K]]) -> Vec<f64> { /// reported rather than dropped — silently skipping them would turn a team
assert_eq!(teams.len(), 2, "predict_outcome T2: 2 teams only"); /// of strangers into a confident-looking prediction about nobody, which is
let gather = |team: &[&K]| -> Gaussian { /// the failure this replaced.
team.iter() fn member_skills(&self, teams: &[&[&K]]) -> Result<Vec<Vec<Gaussian>>, InferenceError> {
.filter_map(|k| self.keys.get(*k)) if teams.len() < 2 {
.filter_map(|idx| { return Err(InferenceError::NotEnoughTeams { got: teams.len() });
}
let mut gathered = Vec::with_capacity(teams.len());
for (team_idx, team) in teams.iter().enumerate() {
if team.is_empty() {
return Err(InferenceError::EmptyTeam { team: team_idx });
}
let mut members = Vec::with_capacity(team.len());
for (member_idx, key) in team.iter().enumerate() {
let unknown = InferenceError::UnknownKey {
team: team_idx,
member: member_idx,
};
let index = self.keys.get(*key).ok_or(unknown.clone())?;
members.push(
self.time_slices self.time_slices
.iter() .iter()
.rev() .rev()
.find_map(|ts| ts.skills.get(idx).map(|s| s.posterior())) .find_map(|ts| ts.skills.get(index).map(|s| s.posterior()))
}) .ok_or(unknown)?,
.fold(crate::N00, |acc, g| acc + g.forget(self.beta.powi(2))) );
}; }
let a = gather(teams[0]);
let b = gather(teams[1]); gathered.push(members);
let diff = a - b; }
let p_a = 1.0 - crate::cdf(0.0, diff.mu(), diff.sigma());
vec![p_a, 1.0 - p_a] Ok(gathered)
}
/// Each team's performance Gaussian, and its member count.
///
/// Performance is skill inflated by `beta`: the question a prediction
/// answers is "how will they do today", not "how good are they".
///
/// # Errors
///
/// As [`History::member_skills`].
fn performances(&self, teams: &[&[&K]]) -> Result<(Vec<Gaussian>, Vec<usize>), InferenceError> {
let skills = self.member_skills(teams)?;
let performances = skills
.iter()
.map(|team| {
team.iter()
.fold(crate::N00, |acc, s| acc + s.forget(self.beta.powi(2)))
})
.collect();
let sizes = skills.iter().map(Vec::len).collect();
Ok((performances, sizes))
}
/// Draw margins per team pair.
///
/// Inference derives the margin per rank-adjacent pair from those two
/// teams' betas (`Game::likelihoods`), so prediction must too — a single
/// game-wide margin would describe a different model than the one that
/// will actually be fitted.
fn margins(&self, sizes: &[usize]) -> crate::predict::Margins {
let beta_sq = self.beta.powi(2);
let p_draw = self.p_draw;
crate::predict::Margins::new(sizes.len(), |i, j| {
if p_draw == 0.0 {
0.0
} else {
let sd = ((sizes[i] + sizes[j]) as f64 * beta_sq).sqrt();
crate::compute_margin(p_draw, sd)
}
})
}
/// Draw-probability quality metric for the given teams (key slices).
///
/// Values range roughly `[0, 1]`; 1 == perfectly matched. Supports any
/// number of teams.
///
/// Note this answers "is this matchup *fair*", which is not the same as
/// "is this matchup *informative*" — the two coincide for two evenly
/// matched teams and diverge elsewhere.
///
/// # Errors
///
/// `NotEnoughTeams`, `EmptyTeam`, or `UnknownKey`.
pub fn predict_quality(&self, teams: &[&[&K]]) -> Result<f64, InferenceError> {
let groups = self.member_skills(teams)?;
let group_refs: Vec<&[Gaussian]> = groups.iter().map(Vec::as_slice).collect();
Ok(crate::quality(&group_refs, self.beta))
}
/// Expected information gain of running this matchup, in nats.
///
/// Answers "which comparison should I run next" rather than "who will
/// win": the outcome-weighted divergence between current beliefs and the
/// beliefs each possible result would produce. Higher means the result
/// would teach you more.
///
/// Uses each competitor's current skill as the prior, and the history's
/// own `beta`, `drift` and `p_draw`, so the outcomes weighted here are the
/// ones that would actually be fitted if the matchup were played and
/// recorded.
///
/// Distinct from [`History::predict_quality`], which measures *fairness*.
/// The two coincide for two evenly matched competitors and diverge
/// elsewhere. See [`expected_information_gain`](crate::expected_information_gain)
/// for the scale, the analytic `ln k` ceiling, and the cost.
///
/// # Errors
///
/// As [`History::member_skills`], plus `TooManyTeams` and anything
/// inference returns for a hypothetical outcome.
pub fn expected_information_gain(&self, teams: &[&[&K]]) -> Result<f64, InferenceError> {
let skills = self.member_skills(teams)?;
let ratings: Vec<Vec<Rating<T, D>>> = skills
.iter()
.map(|team| {
team.iter()
.map(|&skill| Rating::new(skill, self.beta, self.drift))
.collect()
})
.collect();
let team_refs: Vec<&[Rating<T, D>]> = ratings.iter().map(Vec::as_slice).collect();
crate::expected_information_gain(
&team_refs,
&crate::GameOptions {
p_draw: self.p_draw,
score_sigma: self.score_sigma,
convergence: self.convergence,
},
)
}
/// `P(team i finishes strictly first)`, for every team.
///
/// Supports any number of teams. Because performances are independent
/// Gaussians, this separates into a one-dimensional integral per team —
/// no multivariate orthant probability is involved — and is evaluated by
/// adaptive quadrature to within the precision of the underlying normal
/// CDF (~1e-8).
///
/// With a zero `p_draw` these sum to one. With a positive `p_draw` the
/// shortfall is the probability that the top place is shared.
///
/// Cheap at any team count: cost grows as the square of the team count,
/// not factorially. Prefer this to [`History::predict_outcome`] when you
/// only need to know who wins.
///
/// # Errors
///
/// `NotEnoughTeams`, `EmptyTeam`, or `UnknownKey`.
pub fn predict_win_probabilities(&self, teams: &[&[&K]]) -> Result<Vec<f64>, InferenceError> {
let (performances, sizes) = self.performances(teams)?;
Ok(crate::predict::win_probabilities(
&performances,
&self.margins(&sizes),
))
}
/// The full distribution over finishing orders.
///
/// Every entry is a rank vector — the shape [`crate::Outcome::ranking`]
/// takes, equal ranks meaning a tie — paired with its probability. The
/// entries are exhaustive and disjoint, so they sum to one; that identity
/// is the strongest available check on the numerics and is worth asserting
/// in tests via [`Prediction::total`].
///
/// Accounts for `p_draw`: with a positive draw probability, tied outcomes
/// carry real mass rather than being silently omitted.
///
/// # Cost
///
/// This enumerates the outcome space, which holds `n! * 2^(n-1)` events —
/// 24 at three teams, 192 at four, 1_920 at five, 23_040 at six. Each
/// costs one `O(teams * grid)` pass, so this is milliseconds at three or
/// four teams and seconds at six. Above
/// [`MAX_TEAMS_FOR_DISTRIBUTION`](crate::MAX_PREDICTED_TEAMS) it returns
/// `TooManyTeams` rather than hanging. When you need one specific ordering
/// use [`History::predict_ranking`], and when you only need the winner use
/// [`History::predict_win_probabilities`]; both stay cheap at any size.
///
/// # Errors
///
/// `NotEnoughTeams`, `EmptyTeam`, `UnknownKey`, or `TooManyTeams`.
pub fn predict_outcome(&self, teams: &[&[&K]]) -> Result<Prediction, InferenceError> {
if teams.len() > crate::MAX_PREDICTED_TEAMS {
return Err(InferenceError::TooManyTeams {
got: teams.len(),
max: crate::MAX_PREDICTED_TEAMS,
});
}
let (performances, sizes) = self.performances(teams)?;
Ok(Prediction::new(crate::predict::outcome_distribution(
&performances,
&self.margins(&sizes),
)))
}
/// Probability of one specific finishing order.
///
/// `ranks` follows [`crate::Outcome::ranking`]: lower is better, and equal
/// values mean those teams tied. Teams sharing a rank may finish in any
/// internal order, so this sums over those orders rather than picking one.
///
/// Unlike [`History::predict_outcome`] this does not enumerate the outcome
/// space, so it stays cheap at any team count — use it when you know which
/// orderings you care about.
///
/// # Errors
///
/// `NotEnoughTeams`, `EmptyTeam`, `UnknownKey`, or `MismatchedShape` if
/// `ranks` does not have one entry per team.
pub fn predict_ranking(&self, teams: &[&[&K]], ranks: &[u32]) -> Result<f64, InferenceError> {
if ranks.len() != teams.len() {
return Err(InferenceError::MismatchedShape {
kind: "ranks vs teams",
expected: teams.len(),
got: ranks.len(),
});
}
let (performances, sizes) = self.performances(teams)?;
Ok(crate::predict::ranking_probability(
&performances,
&self.margins(&sizes),
ranks,
))
} }
/// Run the full forward+backward convergence loop and return a summary. /// Run the full forward+backward convergence loop and return a summary.
@@ -653,7 +893,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
times: Vec<T>, times: Vec<T>,
mut weights: Option<Vec<Vec<Vec<f64>>>>, mut weights: Option<Vec<Vec<Vec<f64>>>>,
kinds: Vec<EventKind>, kinds: Vec<EventKind>,
mut priors: HashMap<Index, Rating<T, D>>, priors: HashMap<Index, CompetitorConfig>,
) -> Result<(), InferenceError> { ) -> Result<(), InferenceError> {
if results if results
.as_ref() .as_ref()
@@ -720,17 +960,60 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
this_agent.push(*agent); this_agent.push(*agent);
if !self.agents.contains(*agent) { let config = priors.get(agent).copied().unwrap_or_default();
if self.agents.contains(*agent) {
// Seeding a competitor the history already knows. This used to
// be dropped on the floor: `remove` was only reached on the
// create path, so a prior applied on a competitor's very first
// event and was silently ignored ever after.
if config.is_empty() {
continue;
}
let rating = &mut self.agents.get_mut(*agent).unwrap().rating;
if let Some(prior) = config.prior {
rating.prior = prior;
}
if let Some(scale) = config.drift_scale {
rating.drift_scale = scale;
}
let seeded = rating.prior;
if config.prior.is_some() {
// The prior is not re-derived every pass the way drift is.
// A competitor's earliest slice has its forward message set
// to the prior once, at ingestion, and `iteration` refreshes
// only slices after the first — so without this, a late
// prior would reach the drift terms and nothing else, which
// is a subtler version of the silent drop this replaced.
//
// `clean` has just nulled every message, so the earliest
// slice's forward is exactly the prior.
for slice in &mut self.time_slices {
if let Some(skill) = slice.skills.get_mut(*agent) {
skill.forward = seeded;
break;
}
}
}
} else {
let mut rating = Rating::new(
Gaussian::from_ms(self.mu, self.sigma),
self.beta,
self.drift,
);
if let Some(prior) = config.prior {
rating.prior = prior;
}
if let Some(scale) = config.drift_scale {
rating.drift_scale = scale;
}
self.agents.insert( self.agents.insert(
*agent, *agent,
Competitor { Competitor {
rating: priors.remove(agent).unwrap_or_else(|| { rating,
Rating::new(
Gaussian::from_ms(self.mu, self.sigma),
self.beta,
self.drift,
)
}),
message: None, message: None,
last_time: None, last_time: None,
}, },
@@ -947,7 +1230,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
let mut times: Vec<T> = Vec::with_capacity(events.len()); let mut times: Vec<T> = Vec::with_capacity(events.len());
let mut weights: Vec<Vec<Vec<f64>>> = Vec::with_capacity(events.len()); let mut weights: Vec<Vec<Vec<f64>>> = Vec::with_capacity(events.len());
let mut kinds: Vec<EventKind> = Vec::with_capacity(events.len()); let mut kinds: Vec<EventKind> = Vec::with_capacity(events.len());
let mut priors: HashMap<Index, Rating<T, D>> = HashMap::new(); let mut priors: HashMap<Index, CompetitorConfig> = HashMap::new();
for ev in events { for ev in events {
if ev.outcome.team_count() != ev.teams.len() { if ev.outcome.team_count() != ev.teams.len() {
@@ -981,23 +1264,37 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
} }
} }
// `prior` and `drift_scale` are competitor configuration, // `prior` and `drift_scale` configure the competitor, not
// captured here and consumed at competitor creation. Both // the event. Both land in the same entry so a member may
// land in the same entry so a member may set either alone. // set either alone.
//
// Events within a batch are not ordered, so a batch that
// sets one field twice with different values has no
// well-defined result — "last one wins" would depend on
// iteration order, which `tests/ingestion_equivalence.rs`
// exists to rule out. Repeating the *same* value is fine,
// and is the expected shape when the configuration is a
// property of the domain rather than of one event.
if member.prior.is_some() || member.drift_scale.is_some() { if member.prior.is_some() || member.drift_scale.is_some() {
let rating = priors.entry(idx).or_insert_with(|| { let entry = priors.entry(idx).or_default();
Rating::new(
Gaussian::from_ms(self.mu, self.sigma),
self.beta,
self.drift,
)
});
if let Some(prior) = member.prior { if let Some(prior) = member.prior {
rating.prior = prior; if entry.prior.is_some_and(|held| held != prior) {
return Err(InferenceError::ConflictingCompetitorConfig {
competitor: idx.get(),
field: "prior",
});
}
entry.prior = Some(prior);
} }
if let Some(scale) = member.drift_scale { if let Some(scale) = member.drift_scale {
rating.drift_scale = scale; if entry.drift_scale.is_some_and(|held| held != scale) {
return Err(InferenceError::ConflictingCompetitorConfig {
competitor: idx.get(),
field: "drift_scale",
});
}
entry.drift_scale = Some(scale);
} }
} }
} }
+313 -9
View File
@@ -86,6 +86,19 @@
#![forbid(unsafe_code)] #![forbid(unsafe_code)]
/// Compiles every `rust` block in `README.md` as a doctest.
///
/// The README is not the crate's front page — the module docs above are — so it
/// is pulled in here rather than via a crate-level `#![doc = ...]`, purely so
/// its examples are type-checked. Without this nothing compiled them, and they
/// had drifted far enough that four blocks no longer built (#35). `cfg(doctest)`
/// means this type exists only while collecting doctests.
///
/// Blocks that are illustrative rather than runnable are fenced as `text`.
#[cfg(doctest)]
#[doc = include_str!("../README.md")]
pub struct ReadmeDoctests;
use std::{ use std::{
cmp::Reverse, cmp::Reverse,
f64::consts::{FRAC_1_SQRT_2, FRAC_2_SQRT_PI, SQRT_2}, f64::consts::{FRAC_1_SQRT_2, FRAC_2_SQRT_PI, SQRT_2},
@@ -97,6 +110,7 @@ pub(crate) mod arena;
mod time; mod time;
mod time_slice; mod time_slice;
pub use time_slice::{EventKind, TimeSlice}; pub use time_slice::{EventKind, TimeSlice};
mod acquisition;
mod color_group; mod color_group;
mod competitor; mod competitor;
mod convergence; mod convergence;
@@ -105,18 +119,21 @@ mod error;
mod event; mod event;
mod event_builder; mod event_builder;
pub(crate) mod factor; pub(crate) mod factor;
pub mod factors;
mod game; mod game;
pub mod gaussian; pub mod gaussian;
pub mod graph;
mod history; mod history;
mod key_table; mod key_table;
mod matrix; mod matrix;
mod observer; mod observer;
mod outcome; mod outcome;
mod predict;
pub(crate) mod quadrature;
mod rating; mod rating;
pub(crate) mod schedule; pub(crate) mod schedule;
pub mod storage; pub mod storage;
pub use acquisition::expected_information_gain;
pub use competitor::Competitor; pub use competitor::Competitor;
pub use convergence::{ConvergenceOptions, ConvergenceReport}; pub use convergence::{ConvergenceOptions, ConvergenceReport};
pub use drift::{ConstantDrift, Drift}; pub use drift::{ConstantDrift, Drift};
@@ -130,6 +147,7 @@ pub use key_table::KeyTable;
use matrix::Matrix; use matrix::Matrix;
pub use observer::{NullObserver, Observer}; pub use observer::{NullObserver, Observer};
pub use outcome::Outcome; pub use outcome::Outcome;
pub use predict::Prediction;
pub use rating::Rating; pub use rating::Rating;
pub use schedule::ScheduleReport; pub use schedule::ScheduleReport;
pub use time::{Time, Untimed}; pub use time::{Time, Untimed};
@@ -142,7 +160,29 @@ pub const P_DRAW: f64 = 0.0;
pub const EPSILON: f64 = 1e-6; pub const EPSILON: f64 = 1e-6;
pub const ITERATIONS: usize = 30; pub const ITERATIONS: usize = 30;
/// Largest team count `History::predict_outcome` will enumerate.
///
/// The outcome space holds `n! * 2^(n-1)` events, so it grows factorially:
/// 1_920 at five teams, 23_040 at six, 322_560 at seven. Six is where
/// enumerating on a caller's behalf stops being reasonable.
pub const MAX_PREDICTED_TEAMS: usize = predict::MAX_TEAMS_FOR_DISTRIBUTION;
const SQRT_TAU: f64 = 2.5066282746310002; const SQRT_TAU: f64 = 2.5066282746310002;
/// `1 / sqrt(pi)`, the leading factor of the `erfcx` continued fraction.
const FRAC_1_SQRT_PI: f64 = 0.564_189_583_547_756_3;
/// `sqrt(2 / pi)`, the numerator of the inverse Mills ratio in scaled form.
const SQRT_2_OVER_PI: f64 = 0.797_884_560_802_865_4;
/// How many window widths into the tail before a tie window is treated as a
/// half-line. Beyond this the truncated mass is concentrated within `1/alpha`
/// of the near edge, so the far edge contributes nothing measurable.
const HALF_LINE_WINDOW: f64 = 10.0;
/// Where `v - alpha` switches from subtraction to its asymptotic series.
///
/// The subtraction loses roughly `eps * alpha^2` of relative precision, and the
/// four-term series is good to ~1e-10 by here, so the two are at their closest
/// agreement around this point. Below it the subtraction is exact; above it the
/// series is.
const ASYMPTOTIC_MILLS_ALPHA: f64 = 100.0;
pub const N01: Gaussian = Gaussian::from_ms(0.0, 1.0); pub const N01: Gaussian = Gaussian::from_ms(0.0, 1.0);
pub const N00: Gaussian = Gaussian::from_ms(0.0, 0.0); pub const N00: Gaussian = Gaussian::from_ms(0.0, 0.0);
@@ -234,6 +274,50 @@ pub(crate) fn cdf(x: f64, mu: f64, sigma: f64) -> f64 {
0.5 * erfc(z) 0.5 * erfc(z)
} }
/// `P(X > x)` for `X ~ N(mu, sigma^2)`.
///
/// The survival function, computed directly rather than as `1 - cdf(..)`.
///
/// The two are algebraically identical and numerically are not. `cdf` returns
/// a value approaching 1 for an upper tail, so subtracting it from 1 cancels
/// away every significant digit the tail had: measured against this function,
/// `1 - cdf` carries 7% error by four sigma past the mean and returns exactly
/// zero beyond about 8.3 sigma — where the true value is still 1e-19 and
/// perfectly representable. `erfc` itself holds ~1e-7 *relative* accuracy down
/// to 1e-296, so the precision is there to keep; only the subtraction threw it
/// away.
///
/// This matters most where evidence is smallest, which is exactly where an
/// upset makes it interesting: `ln` of a clamped zero is -708 regardless of
/// whether the truth was -43 or -600.
pub(crate) fn sf(x: f64, mu: f64, sigma: f64) -> f64 {
0.5 * erfc((x - mu) / (sigma * SQRT_2))
}
/// `e^(x^2) * erfc(x)`, the scaled complementary error function, for `x >= 0`.
///
/// Exists so the exponential factor common to a Gaussian density and its tail
/// integral can be cancelled *analytically* instead of being computed twice
/// and divided. Both underflow to zero past about 26 sigma, and their ratio is
/// then `0/0` — finite in the limit, `NaN` in floating point.
fn erfcx(x: f64) -> f64 {
if x < 2.0 {
// Below the crossover neither factor is extreme: erfc is O(1) and
// exp(x^2) is at most e^4, so the direct product is exact enough and
// cheaper than the continued fraction.
(x * x).exp() * erfc(x)
} else {
// erfcx(x) = 1/sqrt(pi) * 1/(x + (1/2)/(x + 1/(x + (3/2)/(x + ...)))),
// evaluated by backward recurrence. Converges quickly for x >= 2 and,
// unlike the product form, never touches an exponential.
let mut f = 0.0;
for n in (1..=60u32).rev() {
f = (f64::from(n) * 0.5) / (x + f);
}
FRAC_1_SQRT_PI / (x + f)
}
}
fn pdf(x: f64, mu: f64, sigma: f64) -> f64 { fn pdf(x: f64, mu: f64, sigma: f64) -> f64 {
let normalizer = (SQRT_TAU * sigma).powi(-1); let normalizer = (SQRT_TAU * sigma).powi(-1);
let functional = (-((x - mu).powi(2)) / (2.0 * sigma.powi(2))).exp(); let functional = (-((x - mu).powi(2)) / (2.0 * sigma.powi(2))).exp();
@@ -241,25 +325,100 @@ fn pdf(x: f64, mu: f64, sigma: f64) -> f64 {
normalizer * functional normalizer * functional
} }
/// Truncated-Gaussian correction terms `(v, w)`.
///
/// `v` shifts the mean and `w` shrinks the variance. Both are ratios whose
/// numerator and denominator underflow together in the tails, so both are
/// computed in scaled form there: the shared `exp(-alpha^2 / 2)` is cancelled
/// analytically rather than evaluated and divided out. Without that, a
/// truncation point beyond about 39 sigma produced `0 / 0` and put `NaN`
/// straight into the posterior.
/// Truncation terms for a boundary `alpha` standard deviations into the upper
/// tail, from the asymptotic expansion of the inverse Mills ratio.
///
/// `v` tends to `alpha` out here, so the gap between them cannot be obtained by
/// subtracting one from the other — the series computes the gap directly, and
/// `w = v * gap` then never forms the difference of two large near-equal
/// numbers. A far-tail *window* behaves like a half-line once it is more than a
/// few multiples of its own width from the mean, so the tie branch shares this.
fn half_line_truncation(alpha: f64) -> (f64, f64) {
let inv = alpha.recip();
let inv_sq = inv * inv;
let gap = inv * (1.0 - inv_sq * (2.0 - inv_sq * (10.0 - 74.0 * inv_sq)));
let v = alpha + gap;
(v, v * gap)
}
fn v_w(mu: f64, sigma: f64, margin: f64, tie: bool) -> (f64, f64) { fn v_w(mu: f64, sigma: f64, margin: f64, tie: bool) -> (f64, f64) {
if !tie { if !tie {
let alpha = (margin - mu) / sigma; let alpha = (margin - mu) / sigma;
let v = pdf(-alpha, 0.0, 1.0) / cdf(-alpha, 0.0, 1.0); // v is the inverse Mills ratio, phi(alpha) / Phi(-alpha), and w needs
let w = v * (v + (-alpha)); // the gap `v - alpha` as well as v itself. Far into the tail v tends to
// alpha, so that gap is a subtraction of two nearly equal numbers and
// loses every digit it has: at alpha = 1e6 it drove w above 1 and made
// `sqrt(1 - w)` NaN. Past the crossover the gap comes from its
// asymptotic series instead, which has no subtraction in it.
if alpha >= ASYMPTOTIC_MILLS_ALPHA {
return half_line_truncation(alpha);
}
(v, w) let (v, gap) = if alpha > 0.0 {
// Both terms carry exp(-alpha^2 / 2); in scaled form it cancels
// and the result stays exact however far into the tail alpha sits.
let v = SQRT_2_OVER_PI / erfcx(alpha / SQRT_2);
(v, v - alpha)
} else {
// Phi(-alpha) >= 1/2 here, so the direct ratio loses nothing.
let v = pdf(-alpha, 0.0, 1.0) / cdf(-alpha, 0.0, 1.0);
(v, v - alpha)
};
(v, v * gap)
} else { } else {
// v is odd in mu and w is even, so fold to mu <= 0. Both truncation
// points then sit in the upper tail, where the scaled form applies.
let flipped = mu > 0.0;
let mu = if flipped { -mu } else { mu };
let alpha = (-margin - mu) / sigma; let alpha = (-margin - mu) / sigma;
let beta = (margin - mu) / sigma; let beta = (margin - mu) / sigma;
let v = (pdf(alpha, 0.0, 1.0) - pdf(beta, 0.0, 1.0)) // `w` comes out of `v * v - u`, and both terms grow as alpha^2 while
/ (cdf(beta, 0.0, 1.0) - cdf(alpha, 0.0, 1.0)); // their difference stays O(1) — at alpha = 1e9 that subtraction had no
let u = (alpha * pdf(alpha, 0.0, 1.0) - beta * pdf(beta, 0.0, 1.0)) // digits left and returned w = -128, making `sqrt(1 - w)` nonsense.
/ (cdf(beta, 0.0, 1.0) - cdf(alpha, 0.0, 1.0)); // Once the window sits many of its own widths into the tail it is
// indistinguishable from a half-line, so the asymptotic covers it with
// no subtraction at all.
if alpha >= ASYMPTOTIC_MILLS_ALPHA && alpha * (beta - alpha) >= HALF_LINE_WINDOW {
let (v, w) = half_line_truncation(alpha);
return (if flipped { -v } else { v }, w);
}
let (v, u) = if alpha > 0.0 {
// beta > alpha > 0, so this ratio of exponentials is at most 1 and
// cannot overflow.
let scale = (0.5 * (alpha * alpha - beta * beta)).exp();
let denominator = 0.5 * (erfcx(alpha / SQRT_2) - scale * erfcx(beta / SQRT_2));
(
(1.0 - scale) / SQRT_TAU / denominator,
(alpha - beta * scale) / SQRT_TAU / denominator,
)
} else {
// The interval straddles the mean, so nothing here is small.
let denominator = cdf(beta, 0.0, 1.0) - cdf(alpha, 0.0, 1.0);
(
(pdf(alpha, 0.0, 1.0) - pdf(beta, 0.0, 1.0)) / denominator,
(alpha * pdf(alpha, 0.0, 1.0) - beta * pdf(beta, 0.0, 1.0)) / denominator,
)
};
let w = -(u - v.powi(2)); let w = -(u - v.powi(2));
(v, w) (if flipped { -v } else { v }, w)
} }
} }
@@ -442,6 +601,151 @@ mod tests {
assert_eq!(sort_time(&[0i64, 1, 2, 0], true), vec![2, 1, 0, 3]); assert_eq!(sort_time(&[0i64, 1, 2, 0], true), vec![2, 1, 0, 3]);
} }
/// Upper-tail values of the standard normal, from published tables. The
/// point is not the digits — `erfc` only carries ~1e-7 relative — but that
/// a number comes back at all: `1 - cdf` returned exactly zero for every
/// one of these.
#[test]
fn survival_function_survives_the_far_tail() {
for (z, expected) in [
(9.0f64, 1.128_588e-19),
(12.0, 1.776_482e-33),
(20.0, 2.753_624e-89),
(37.0, 5.725_571e-300),
] {
let got = sf(z, 0.0, 1.0);
assert!(got > 0.0, "sf({z}) collapsed to zero");
assert!(
(got - expected).abs() / expected < 1e-6,
"sf({z}) = {got}, expected ~{expected}"
);
assert_eq!(
1.0 - cdf(z, 0.0, 1.0),
0.0,
"the naive form should still be zero here"
);
}
}
/// Where no cancellation happens the two forms must agree exactly enough
/// that nothing else in the crate shifts.
#[test]
fn survival_function_matches_the_naive_form_where_that_form_works() {
for z in [-4.0f64, -1.0, 0.0, 0.5, 1.0, 2.0, 3.0, 4.0] {
let naive = 1.0 - cdf(z, 0.0, 1.0);
let direct = sf(z, 0.0, 1.0);
// Bounded by `erfc`'s own ~1e-7 relative error, not by the
// subtraction: the two forms evaluate `erfc` at different points
// and the approximation is not exactly antisymmetric.
assert!(
(naive - direct).abs() < 1e-6,
"z={z}: naive {naive} vs direct {direct}"
);
}
}
#[test]
fn survival_and_cdf_partition_the_mass() {
for z in [-3.0f64, -0.5, 0.0, 1.0, 2.5] {
let total = sf(z, 1.0, 2.0) + cdf(z, 1.0, 2.0);
// `erfc(z) + erfc(-z) == 2` only to the accuracy of the
// approximation, which is ~1e-7 relative.
assert!((total - 1.0).abs() < 1e-6, "z={z}: {total}");
}
}
/// `erfcx` switches formulation at x = 2; the two sides must meet.
#[test]
fn erfcx_is_continuous_across_its_crossover() {
for x in [1.90f64, 1.99, 1.999, 2.0, 2.001, 2.01, 2.10] {
let direct = (x * x).exp() * erfc(x);
let scaled = erfcx(x);
assert!(
(direct - scaled).abs() / scaled < 1e-6,
"x={x}: direct {direct} vs erfcx {scaled}"
);
}
}
/// The whole reason `erfcx` exists: it stays finite and O(1/x) exactly
/// where `exp(x^2)` overflows and `erfc(x)` underflows.
#[test]
fn erfcx_stays_finite_where_its_factors_do_not() {
for x in [27.0f64, 50.0, 1.0e3, 1.0e8] {
let scaled = erfcx(x);
assert!(scaled.is_finite() && scaled > 0.0, "erfcx({x}) = {scaled}");
// Asymptotically erfcx(x) -> 1 / (x * sqrt(pi)).
let asymptote = 1.0 / (x * std::f64::consts::PI.sqrt());
assert!(
(scaled - asymptote).abs() / asymptote < 1e-2,
"erfcx({x}) = {scaled} strays from its asymptote {asymptote}"
);
assert!(
(x * x).exp().is_infinite(),
"x={x} should overflow the direct form"
);
}
}
/// Truncation must never produce a non-finite posterior. Before the scaled
/// formulation these returned NaN from `0 / 0` past about 39 sigma.
#[test]
fn truncation_stays_finite_arbitrarily_far_into_the_tail() {
for alpha in [0.0f64, 8.0, 38.0, 40.0, 100.0, 1.0e3, 1.0e6, 1.0e9, 1.0e15] {
for tie in [false, true] {
let (v, w) = v_w(-alpha, 1.0, if tie { 1.0 } else { 0.0 }, tie);
assert!(v.is_finite(), "alpha={alpha} tie={tie}: v = {v}");
assert!(w.is_finite(), "alpha={alpha} tie={tie}: w = {w}");
// sigma_trunc = sigma * sqrt(1 - w) must stay real.
assert!(
(0.0..=1.0).contains(&w),
"alpha={alpha} tie={tie}: w = {w} leaves sqrt(1 - w) imaginary"
);
let (mu_t, sigma_t) = trunc(-alpha, 1.0, if tie { 1.0 } else { 0.0 }, tie);
assert!(
mu_t.is_finite() && sigma_t.is_finite(),
"alpha={alpha} tie={tie}: trunc = ({mu_t}, {sigma_t})"
);
}
}
}
/// The Mills gap switches from subtraction to series at alpha = 100. Both
/// are supposed to be right there; if they disagree, the crossover is in
/// the wrong place.
#[test]
fn the_mills_gap_series_meets_the_scaled_form() {
for alpha in [50.0f64, 99.0, 100.0, 101.0, 200.0] {
let scaled = SQRT_2_OVER_PI / erfcx(alpha / SQRT_2) - alpha;
let inv = alpha.recip();
let inv_sq = inv * inv;
let series = inv * (1.0 - inv_sq * (2.0 - inv_sq * (10.0 - 74.0 * inv_sq)));
assert!(
(scaled - series).abs() / series < 1e-9,
"alpha={alpha}: scaled {scaled} vs series {series}"
);
}
}
/// Folding the tie branch to `mu <= 0` is only valid if v is odd in mu and
/// w is even. Assert the symmetry the implementation relies on.
#[test]
fn tie_truncation_is_odd_in_v_and_even_in_w() {
for mu in [0.5f64, 3.0, 20.0, 40.0, 100.0, 1.0e3] {
let (v_pos, w_pos) = v_w(mu, 1.0, 1.0, true);
let (v_neg, w_neg) = v_w(-mu, 1.0, 1.0, true);
assert!(
(v_pos + v_neg).abs() < 1e-9,
"mu={mu}: v should be odd, got {v_pos} and {v_neg}"
);
assert!(
(w_pos - w_neg).abs() < 1e-9,
"mu={mu}: w should be even, got {w_pos} and {w_neg}"
);
}
}
#[test] #[test]
fn test_quality() { fn test_quality() {
let a = Gaussian::from_ms(25.0, 3.0); let a = Gaussian::from_ms(25.0, 3.0);
+85 -2
View File
@@ -14,13 +14,95 @@ pub trait Observer<T: Time>: Send + Sync {
/// Called after each convergence iteration across the whole history. /// Called after each convergence iteration across the whole history.
fn on_iteration_end(&self, _iter: usize, _max_step: (f64, f64)) {} fn on_iteration_end(&self, _iter: usize, _max_step: (f64, f64)) {}
/// Called after each time slice is processed within an iteration. /// Called after each time slice is swept within an iteration.
fn on_batch_processed(&self, _time: &T, _slice_idx: usize, _n_events: usize) {} ///
/// A convergence iteration sweeps every slice twice — once travelling
/// backward through the history and once forward — so a multi-slice
/// history fires this twice per slice per iteration. A single-slice
/// history is swept once and fires once.
fn on_slice_processed(&self, _time: &T, _slice_idx: usize, _n_events: usize) {}
/// Called once when convergence completes (or max iters is reached). /// Called once when convergence completes (or max iters is reached).
fn on_converged(&self, _iters: usize, _final_step: (f64, f64), _converged: bool) {} fn on_converged(&self, _iters: usize, _final_step: (f64, f64), _converged: bool) {}
} }
/// Shared and boxed observers forward to what they point at.
///
/// `History` takes its observer by value, so a caller who wants to *read* what
/// an observer recorded has to keep a handle to it. Without these impls the
/// natural spelling does not compile:
///
/// ```
/// # use std::sync::{Arc, Mutex};
/// # use trueskill_tt::{History, Observer};
/// #[derive(Default)]
/// struct Recorder {
/// iterations: Mutex<Vec<usize>>,
/// }
///
/// impl Observer<i64> for Recorder {
/// fn on_iteration_end(&self, iter: usize, _step: (f64, f64)) {
/// self.iterations.lock().unwrap().push(iter);
/// }
/// }
///
/// let recorder = Arc::new(Recorder::default());
/// let mut h = History::builder().observer(Arc::clone(&recorder)).build();
/// h.record_winner(&"a", &"b", 1).unwrap();
/// h.converge().unwrap();
///
/// // The caller's handle sees what the history's copy recorded.
/// assert!(!recorder.iterations.lock().unwrap().is_empty());
/// ```
///
/// The alternative was for every observer to wrap each of its own fields in an
/// `Arc` and derive `Clone` — one allocation and one lock per field, and a
/// pattern each implementor had to rediscover.
///
/// `?Sized` is deliberate: it makes `Arc<dyn Observer<T>>` and
/// `Box<dyn Observer<T>>` work, so observers can be chosen at runtime.
impl<T: Time, O: Observer<T> + ?Sized> Observer<T> for std::sync::Arc<O> {
fn on_iteration_end(&self, iter: usize, max_step: (f64, f64)) {
(**self).on_iteration_end(iter, max_step);
}
fn on_slice_processed(&self, time: &T, slice_idx: usize, n_events: usize) {
(**self).on_slice_processed(time, slice_idx, n_events);
}
fn on_converged(&self, iters: usize, final_step: (f64, f64), converged: bool) {
(**self).on_converged(iters, final_step, converged);
}
}
impl<T: Time, O: Observer<T> + ?Sized> Observer<T> for Box<O> {
fn on_iteration_end(&self, iter: usize, max_step: (f64, f64)) {
(**self).on_iteration_end(iter, max_step);
}
fn on_slice_processed(&self, time: &T, slice_idx: usize, n_events: usize) {
(**self).on_slice_processed(time, slice_idx, n_events);
}
fn on_converged(&self, iters: usize, final_step: (f64, f64), converged: bool) {
(**self).on_converged(iters, final_step, converged);
}
}
impl<T: Time, O: Observer<T> + ?Sized> Observer<T> for &O {
fn on_iteration_end(&self, iter: usize, max_step: (f64, f64)) {
(**self).on_iteration_end(iter, max_step);
}
fn on_slice_processed(&self, time: &T, slice_idx: usize, n_events: usize) {
(**self).on_slice_processed(time, slice_idx, n_events);
}
fn on_converged(&self, iters: usize, final_step: (f64, f64), converged: bool) {
(**self).on_converged(iters, final_step, converged);
}
}
/// ZST no-op observer; the default when none is configured. /// ZST no-op observer; the default when none is configured.
#[derive(Copy, Clone, Debug, Default)] #[derive(Copy, Clone, Debug, Default)]
pub struct NullObserver; pub struct NullObserver;
@@ -35,6 +117,7 @@ mod tests {
fn null_observer_compiles_for_i64() { fn null_observer_compiles_for_i64() {
let o = NullObserver; let o = NullObserver;
<NullObserver as Observer<i64>>::on_iteration_end(&o, 1, (0.0, 0.0)); <NullObserver as Observer<i64>>::on_iteration_end(&o, 1, (0.0, 0.0));
<NullObserver as Observer<i64>>::on_slice_processed(&o, &7, 0, 3);
<NullObserver as Observer<i64>>::on_converged(&o, 5, (1e-6, 1e-6), true); <NullObserver as Observer<i64>>::on_converged(&o, 5, (1e-6, 1e-6), true);
} }
+723
View File
@@ -0,0 +1,723 @@
//! Outcome prediction: who wins, and how likely is a given finishing order.
//!
//! Prediction runs on *performances*, not skills. A competitor's skill is
//! inflated by their performance noise `beta` before any comparison, which is
//! what separates "how good are they" from "how will they do today".
//!
//! Two questions, two algorithms:
//!
//! - **Who finishes first.** Because performances are independent Gaussians,
//! the probability that team `i` beats every other team separates into a
//! *one-dimensional* integral — no multivariate orthant integral is
//! involved. [`quadrature::integrate`] evaluates it to near machine
//! precision for a few hundred `cdf` calls.
//! - **A specific finishing order.** The factor graph only ever constrains
//! rank-*adjacent* teams (see `Game::run_chain`), so the joint probability
//! of a full order is a chain of local constraints rather than a general
//! orthant probability. That chain collapses into a sequential recursion:
//! one cumulative integral per adjacent pair, `O(teams * grid)` overall.
//!
//! Both are deterministic. A sampler would have been easier to write and
//! would have made every `predict_*` call return a slightly different number,
//! which is not a property a rating library should have.
use crate::{Gaussian, quadrature};
/// Teams beyond this count make the outcome enumeration impractical.
///
/// Each realisation sorts into exactly one (permutation, tie-pattern) event,
/// so the space has `n! * 2^(n-1)` members: 24 at 3 teams, 192 at 4, 1_920 at
/// 5, 23_040 at 6. The jump to 322_560 at 7 is where enumerating stops being
/// a reasonable thing to do on a caller's behalf.
pub(crate) const MAX_TEAMS_FOR_DISTRIBUTION: usize = 6;
/// Relative tolerance for the first-place integrals.
///
/// Tightening past this buys nothing: the underlying `cdf` is a rational
/// approximation with fractional error ~1.2e-7, which contributes ~6e-9 to a
/// finished probability and dominates any further quadrature refinement.
const WIN_TOLERANCE: f64 = 1e-8;
/// Nodes for the ranking grid, and the floor below which a grid is pointless.
///
/// The recursion converges as O(h^2). Measured against the exact two-team
/// closed form, 2_048 nodes leave ~1.2e-6 of discretisation error while 8_192
/// reach ~1e-7 — at which point the residual is the `cdf` rational
/// approximation (~2.4e-8), not the grid, and refining further buys nothing.
const MIN_GRID_POINTS: usize = 8_192;
const MAX_GRID_POINTS: usize = 262_144;
/// How many standard deviations of support the grid and integrals cover.
///
/// The normal density is below 1e-18 of its peak past nine sigma, far under
/// the precision of everything else here.
const SUPPORT_SIGMAS: f64 = 9.0;
/// Standard normal CDF at `z`.
fn phi(z: f64) -> f64 {
crate::cdf(z, 0.0, 1.0)
}
/// Normal density of `x` under `g`.
fn density(g: Gaussian, x: f64) -> f64 {
let sigma = g.sigma();
let z = (x - g.mu()) / sigma;
(-0.5 * z * z).exp() / (sigma * (2.0 * std::f64::consts::PI).sqrt())
}
/// Per-pair draw margins.
///
/// The margin is *not* a single number for the whole game: inference derives
/// it per rank-adjacent pair from those two teams' betas (`Game::likelihoods`).
/// Prediction has to use the same per-pair values or it answers a question
/// about a different model than the one that will actually be fitted.
pub(crate) struct Margins {
n: usize,
values: Vec<f64>,
}
impl Margins {
/// Build from a per-pair margin function.
pub(crate) fn new<F: Fn(usize, usize) -> f64>(n: usize, f: F) -> Self {
let mut values = vec![0.0; n * n];
for i in 0..n {
for j in 0..n {
if i != j {
values[i * n + j] = f(i, j);
}
}
}
Self { n, values }
}
fn get(&self, i: usize, j: usize) -> f64 {
self.values[i * self.n + j]
}
/// True when no pair can draw, so every tie has probability zero.
fn all_zero(&self) -> bool {
self.values.iter().all(|&v| v == 0.0)
}
}
/// `P(team i finishes strictly first)` for every team.
///
/// Strictly means beating each rival by more than that pair's draw margin, so
/// with a non-zero margin these sum to less than one; the shortfall is the
/// probability that the top place is shared.
pub(crate) fn win_probabilities(perf: &[Gaussian], margins: &Margins) -> Vec<f64> {
(0..perf.len())
.map(|i| {
let (mu, sigma) = (perf[i].mu(), perf[i].sigma());
let (lo, hi) = (mu - SUPPORT_SIGMAS * sigma, mu + SUPPORT_SIGMAS * sigma);
// Each rival's CDF turns over near its own mean plus the margin.
// Seeding there is what keeps a rival with a tiny sigma — a step
// function in disguise — from being stepped over.
let mut seeds = Vec::with_capacity(3 * perf.len());
for (j, rival) in perf.iter().enumerate().filter(|&(j, _)| j != i) {
let centre = rival.mu() + margins.get(i, j);
seeds.extend_from_slice(&[centre - rival.sigma(), centre, centre + rival.sigma()]);
}
quadrature::integrate(
|x| {
let d = density(perf[i], x);
if d == 0.0 {
return 0.0;
}
let beaten: f64 = (0..perf.len())
.filter(|&j| j != i)
.map(|j| phi((x - margins.get(i, j) - perf[j].mu()) / perf[j].sigma()))
.product();
d * beaten
},
lo,
hi,
&seeds,
WIN_TOLERANCE,
)
})
.collect()
}
/// Grid bounds and resolution covering every team's support.
///
/// Resolution is set by the *smallest* feature in play — the narrowest sigma,
/// or a draw margin narrower still — because that is what the recursion has to
/// resolve. A grid sized off the widest team would step over the narrow one.
fn grid_shape(perf: &[Gaussian], margins: &Margins) -> (f64, f64, usize) {
let lo = perf
.iter()
.map(|g| g.mu() - SUPPORT_SIGMAS * g.sigma())
.fold(f64::INFINITY, f64::min);
let hi = perf
.iter()
.map(|g| g.mu() + SUPPORT_SIGMAS * g.sigma())
.fold(f64::NEG_INFINITY, f64::max);
let narrowest = perf
.iter()
.map(Gaussian::sigma)
.fold(f64::INFINITY, f64::min);
let smallest_margin = margins
.values
.iter()
.copied()
.filter(|&m| m > 0.0)
.fold(f64::INFINITY, f64::min);
let feature = narrowest.min(smallest_margin);
let wanted = if feature.is_finite() && feature > 0.0 {
((hi - lo) / (feature / 12.0)).ceil()
} else {
MIN_GRID_POINTS as f64
};
let points = if wanted.is_finite() {
(wanted as usize).clamp(MIN_GRID_POINTS, MAX_GRID_POINTS)
} else {
MIN_GRID_POINTS
};
(lo, hi, points)
}
/// Densities of each team sampled on the shared grid.
struct Sampled {
lo: f64,
step: f64,
points: usize,
density: Vec<Vec<f64>>,
}
impl Sampled {
fn new(perf: &[Gaussian], margins: &Margins) -> Self {
let (lo, hi, points) = grid_shape(perf, margins);
let step = (hi - lo) / (points - 1) as f64;
let density = perf
.iter()
.map(|&g| {
(0..points)
.map(|i| density(g, lo + i as f64 * step))
.collect()
})
.collect();
Self {
lo,
step,
points,
density,
}
}
fn node(&self, i: usize) -> f64 {
self.lo + i as f64 * self.step
}
}
/// `P(order[0] >= order[1] >= ... )` with the given adjacency pattern.
///
/// `tied[k]` says whether `order[k]` and `order[k + 1]` finish within that
/// pair's draw margin. The recursion runs bottom-up: `carry` holds, for each
/// grid node, the probability that everything *below* the current team holds
/// given that team landed on that node. A strict gap reads a cumulative
/// integral; a tie reads a window. Both are O(1) against one prefix array,
/// so each level costs O(grid) and the whole order costs O(teams * grid).
fn order_probability(margins: &Margins, sampled: &Sampled, order: &[usize], tied: &[bool]) -> f64 {
let mut carry = vec![1.0; sampled.points];
for k in (0..order.len() - 1).rev() {
let below = order[k + 1];
let above = order[k];
let margin = margins.get(above, below);
let integrand: Vec<f64> = (0..sampled.points)
.map(|i| sampled.density[below][i] * carry[i])
.collect();
let cumulative = quadrature::Grid::from_values(sampled.lo, sampled.step, integrand);
carry = (0..sampled.points)
.map(|i| {
let x = sampled.node(i);
if tied[k] {
// Sorted order already implies `below <= above`, so the
// tie window is one-sided: [x - margin, x].
cumulative.integral_between(x - margin, x)
} else {
cumulative.integral_to(x - margin)
}
})
.collect();
}
let top = order[0];
let integrand: Vec<f64> = (0..sampled.points)
.map(|i| sampled.density[top][i] * carry[i])
.collect();
quadrature::Grid::from_values(sampled.lo, sampled.step, integrand).total()
}
/// Dense ranks implied by a sorted order and its tie pattern.
fn ranks_of(order: &[usize], tied: &[bool], n: usize) -> Vec<u32> {
let mut ranks = vec![0u32; n];
let mut rank = 0u32;
ranks[order[0]] = 0;
for k in 0..order.len() - 1 {
if !tied[k] {
rank += 1;
}
ranks[order[k + 1]] = rank;
}
ranks
}
/// Every (order, tie-pattern) event, or only the strict ones when no pair can
/// draw — a tie then has probability exactly zero and is not worth integrating.
fn events(n: usize, strict_only: bool) -> Vec<(Vec<usize>, Vec<bool>)> {
fn permute(current: &mut Vec<usize>, k: usize, out: &mut Vec<Vec<usize>>) {
if k == current.len() {
out.push(current.clone());
return;
}
for i in k..current.len() {
current.swap(k, i);
permute(current, k + 1, out);
current.swap(k, i);
}
}
let mut orders = Vec::new();
permute(&mut (0..n).collect(), 0, &mut orders);
let patterns: Vec<Vec<bool>> = if strict_only {
vec![vec![false; n - 1]]
} else {
(0..(1u32 << (n - 1)))
.map(|mask| (0..n - 1).map(|i| mask >> i & 1 == 1).collect())
.collect()
};
let mut out = Vec::with_capacity(orders.len() * patterns.len());
for order in orders {
for pattern in &patterns {
out.push((order.clone(), pattern.clone()));
}
}
out
}
/// The full distribution over finishing orders, aggregated by rank vector.
///
/// Orders that differ only *within* a tied group describe the same finishing
/// order, so their probabilities are summed into one entry.
pub(crate) fn outcome_distribution(perf: &[Gaussian], margins: &Margins) -> Vec<(Vec<u32>, f64)> {
let n = perf.len();
let sampled = Sampled::new(perf, margins);
let mut aggregated: Vec<(Vec<u32>, f64)> = Vec::new();
for (order, tied) in events(n, margins.all_zero()) {
let p = order_probability(margins, &sampled, &order, &tied);
let ranks = ranks_of(&order, &tied, n);
match aggregated.iter_mut().find(|(r, _)| *r == ranks) {
Some((_, acc)) => *acc += p,
None => aggregated.push((ranks, p)),
}
}
aggregated.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal));
aggregated
}
/// All permutations of `items`.
fn permutations(items: &[usize]) -> Vec<Vec<usize>> {
fn go(current: &mut Vec<usize>, k: usize, out: &mut Vec<Vec<usize>>) {
if k == current.len() {
out.push(current.clone());
return;
}
for i in k..current.len() {
current.swap(k, i);
go(current, k + 1, out);
current.swap(k, i);
}
}
let mut out = Vec::new();
go(&mut items.to_vec(), 0, &mut out);
out
}
/// Every (order, tie-pattern) event consistent with a grouping by rank.
///
/// Teams sharing a rank may finish in any internal order, so this is the
/// product of each group's permutations. Adjacencies inside a group are ties;
/// the adjacency joining one group to the next is not.
fn orders_for_groups(groups: &[Vec<usize>]) -> Vec<(Vec<usize>, Vec<bool>)> {
let per_group: Vec<Vec<Vec<usize>>> = groups.iter().map(|g| permutations(g)).collect();
let mut out = Vec::new();
let mut choice = vec![0usize; groups.len()];
loop {
let mut order = Vec::new();
let mut tied = Vec::new();
for (gi, group) in per_group.iter().enumerate() {
for (offset, &member) in group[choice[gi]].iter().enumerate() {
if !order.is_empty() {
tied.push(offset != 0);
}
order.push(member);
}
}
out.push((order, tied));
let mut k = 0;
loop {
if k == choice.len() {
return out;
}
choice[k] += 1;
if choice[k] < per_group[k].len() {
break;
}
choice[k] = 0;
k += 1;
}
}
}
/// Probability of one specific rank vector.
///
/// Ties in `ranks` mean the tied teams may finish in any internal order, so
/// this sums the orders consistent with the requested ranking rather than
/// picking one.
pub(crate) fn ranking_probability(perf: &[Gaussian], margins: &Margins, ranks: &[u32]) -> f64 {
let n = perf.len();
let sampled = Sampled::new(perf, margins);
let mut distinct: Vec<u32> = ranks.to_vec();
distinct.sort_unstable();
distinct.dedup();
let groups: Vec<Vec<usize>> = distinct
.iter()
.map(|&r| (0..n).filter(|&i| ranks[i] == r).collect())
.collect();
orders_for_groups(&groups)
.iter()
.map(|(order, tied)| order_probability(margins, &sampled, order, tied))
.sum()
}
/// A distribution over the ways a contest could finish.
///
/// Each entry pairs a rank vector — the same shape [`crate::Outcome::ranking`]
/// takes, with equal ranks meaning a tie — against its probability. Entries
/// are ordered most likely first, and cover the whole outcome space, so the
/// probabilities sum to one.
///
/// The rank vectors compose directly with inference: feeding one to
/// `Game::ranked` asks "what would we believe if *this* happened", which is
/// what an expected-information-gain calculation needs alongside the weight.
#[derive(Clone, Debug, PartialEq)]
pub struct Prediction {
outcomes: Vec<(Vec<u32>, f64)>,
}
impl Prediction {
pub(crate) fn new(outcomes: Vec<(Vec<u32>, f64)>) -> Self {
Self { outcomes }
}
/// Every possible finishing order and its probability, most likely first.
pub fn outcomes(&self) -> impl ExactSizeIterator<Item = (&[u32], f64)> {
self.outcomes.iter().map(|(r, p)| (r.as_slice(), *p))
}
/// The single most likely finishing order.
#[must_use]
pub fn most_likely(&self) -> Option<(&[u32], f64)> {
self.outcomes.first().map(|(r, p)| (r.as_slice(), *p))
}
/// Probability of one specific finishing order, or zero if it cannot occur.
#[must_use]
pub fn probability_of(&self, ranks: &[u32]) -> f64 {
self.outcomes
.iter()
.find(|(r, _)| r.as_slice() == ranks)
.map_or(0.0, |(_, p)| *p)
}
/// `P(team i finishes strictly first)`, for each team.
///
/// Sums to less than one exactly when the top place can be shared; the
/// shortfall is [`Prediction::shared_first_place`].
#[must_use]
pub fn win_probabilities(&self) -> Vec<f64> {
let n = self.outcomes.first().map_or(0, |(r, _)| r.len());
let mut wins = vec![0.0; n];
for (ranks, p) in &self.outcomes {
let leaders = ranks.iter().filter(|&&r| r == 0).count();
if leaders == 1 {
let winner = ranks.iter().position(|&r| r == 0).expect("a rank-0 team");
wins[winner] += p;
}
}
wins
}
/// Probability that two or more teams share first place.
#[must_use]
pub fn shared_first_place(&self) -> f64 {
self.outcomes
.iter()
.filter(|(r, _)| r.iter().filter(|&&x| x == 0).count() > 1)
.map(|(_, p)| p)
.sum()
}
/// Total probability mass, which should be one.
///
/// Exposed because it is a genuine check on the numerics rather than a
/// formality: the outcome space is exhaustive and disjoint by construction,
/// so any drift from one is integration error and nothing else.
#[must_use]
pub fn total(&self) -> f64 {
self.outcomes.iter().map(|(_, p)| p).sum()
}
}
#[cfg(test)]
mod tests {
use super::*;
fn g(mu: f64, sigma: f64) -> Gaussian {
Gaussian::from_ms(mu, sigma)
}
fn flat(n: usize, eps: f64) -> Margins {
Margins::new(n, |_, _| eps)
}
/// Exact two-team result: `P(a first) = Phi((mu_a - mu_b - eps) / sd)`.
fn closed_form_two(a: Gaussian, b: Gaussian, eps: f64) -> (f64, f64) {
let sd = (a.sigma().powi(2) + b.sigma().powi(2)).sqrt();
(
phi((a.mu() - b.mu() - eps) / sd),
phi((b.mu() - a.mu() - eps) / sd),
)
}
#[test]
fn two_team_win_probabilities_match_the_closed_form() {
for (ma, sa, mb, sb, eps) in [
(0.0, 6.0, 0.0, 6.0, 0.0),
(3.0, 6.0, -2.0, 1.0, 0.0),
(0.0, 6.0, 0.0, 6.0, 2.0),
(3.0, 6.0, -2.0, 1.0, 1.5),
(40.0, 1.0, 0.0, 1.0, 0.0),
] {
let perf = [g(ma, sa), g(mb, sb)];
let got = win_probabilities(&perf, &flat(2, eps));
let (wa, wb) = closed_form_two(perf[0], perf[1], eps);
assert!(
(got[0] - wa).abs() < 1e-7 && (got[1] - wb).abs() < 1e-7,
"mu=({ma},{mb}) sigma=({sa},{sb}) eps={eps}: got {got:?}, want [{wa}, {wb}]"
);
}
}
/// The identity that a wrong-but-plausible implementation cannot fake:
/// with no draw margin, exactly one team finishes first.
#[test]
fn win_probabilities_sum_to_one_without_a_draw_margin() {
for perf in [
vec![g(0.0, 6.0), g(0.0, 6.0)],
vec![g(5.0, 6.0), g(0.0, 3.0), g(-5.0, 1.0)],
vec![
g(8.0, 2.0),
g(3.0, 6.0),
g(0.0, 1.0),
g(-3.0, 4.0),
g(-8.0, 6.0),
],
] {
let sum: f64 = win_probabilities(&perf, &flat(perf.len(), 0.0))
.iter()
.sum();
assert!(
(sum - 1.0).abs() < 1e-7,
"{} teams: sum = {sum}",
perf.len()
);
}
}
/// A rival with a tiny sigma is a step function in disguise. Fixed-node
/// quadrature steps over it and lands ~1e-2 out while still looking like a
/// probability; this is the case that rules that approach out.
#[test]
fn win_probabilities_survive_a_rival_with_a_tiny_sigma() {
let perf = [g(0.0, 0.001), g(0.5, 6.0), g(-0.5, 6.0)];
let got = win_probabilities(&perf, &flat(3, 0.0));
let sum: f64 = got.iter().sum();
assert!((sum - 1.0).abs() < 1e-6, "sum = {sum}, probs = {got:?}");
}
#[test]
fn a_stronger_team_is_more_likely_to_win() {
let perf = [g(10.0, 3.0), g(0.0, 3.0), g(-10.0, 3.0)];
let p = win_probabilities(&perf, &flat(3, 0.0));
assert!(p[0] > p[1] && p[1] > p[2], "not monotone: {p:?}");
}
#[test]
fn identical_teams_are_equally_likely_to_win() {
let perf = [g(1.0, 4.0), g(1.0, 4.0), g(1.0, 4.0)];
let p = win_probabilities(&perf, &flat(3, 0.0));
for probs in p.windows(2) {
assert!((probs[0] - probs[1]).abs() < 1e-9, "asymmetric: {p:?}");
}
}
/// Every realisation sorts into exactly one finishing order, so the whole
/// distribution must sum to one — with or without a draw margin.
#[test]
fn outcome_distribution_sums_to_one() {
for (perf, eps) in [
(vec![g(0.0, 6.0), g(0.0, 6.0)], 0.0),
(vec![g(0.0, 6.0), g(0.0, 6.0)], 2.0),
(vec![g(0.0, 6.0), g(0.0, 6.0), g(0.0, 6.0)], 0.0),
(vec![g(5.0, 6.0), g(0.0, 3.0), g(-5.0, 1.0)], 1.5),
(vec![g(0.0, 0.05), g(0.5, 6.0), g(-0.5, 6.0)], 1.0),
(
vec![g(6.0, 2.0), g(2.0, 6.0), g(-2.0, 1.0), g(-6.0, 4.0)],
1.0,
),
] {
let n = perf.len();
let dist = outcome_distribution(&perf, &flat(n, eps));
let sum: f64 = dist.iter().map(|(_, p)| p).sum();
assert!(
(sum - 1.0).abs() < 1e-6,
"{n} teams, eps={eps}: sum = {sum} over {} outcomes",
dist.len()
);
assert!(dist.iter().all(|(_, p)| *p >= 0.0), "negative probability");
}
}
/// With two teams the distribution is the exact win/draw/loss triple.
#[test]
fn two_team_distribution_matches_the_closed_form() {
let perf = [g(3.0, 6.0), g(-2.0, 1.0)];
let eps = 1.5;
let dist = outcome_distribution(&perf, &flat(2, eps));
let (wa, wb) = closed_form_two(perf[0], perf[1], eps);
let find = |ranks: &[u32]| {
dist.iter()
.find(|(r, _)| r == ranks)
.map_or(0.0, |(_, p)| *p)
};
assert!(
(find(&[0, 1]) - wa).abs() < 1e-6,
"a wins: {}",
find(&[0, 1])
);
assert!(
(find(&[1, 0]) - wb).abs() < 1e-6,
"b wins: {}",
find(&[1, 0])
);
assert!(
(find(&[0, 0]) - (1.0 - wa - wb)).abs() < 1e-6,
"draw: {}",
find(&[0, 0])
);
}
/// Asking for one ranking must agree with that ranking's entry in the
/// full distribution — the two use different code paths to the same value.
#[test]
fn ranking_probability_agrees_with_the_distribution() {
let perf = [g(5.0, 6.0), g(0.0, 3.0), g(-5.0, 1.0)];
let eps = 1.5;
let margins = flat(3, eps);
let dist = outcome_distribution(&perf, &margins);
for (ranks, expected) in &dist {
let direct = ranking_probability(&perf, &margins, ranks);
assert!(
(direct - expected).abs() < 1e-9,
"ranks {ranks:?}: direct {direct} vs distribution {expected}"
);
}
}
/// Tie mass is controlled by the draw margin. Only the *all-tied* outcome
/// is monotone in it: every one of its constraints is a window that widens
/// with the margin. A partially-tied outcome like `[0, 0, 1]` is not, and
/// must not be asserted to be — widening the margin makes its tie easier
/// but its "and the last team is strictly behind by more than the margin"
/// clause harder, so it peaks and then falls.
#[test]
fn all_tied_probability_grows_with_the_draw_margin() {
let perf = [g(0.0, 4.0), g(0.0, 4.0), g(-8.0, 2.0)];
let mut previous = 0.0;
for eps in [0.0, 0.5, 1.0, 2.0, 4.0, 8.0, 24.0] {
let p = ranking_probability(&perf, &flat(3, eps), &[0, 0, 0]);
assert!(p >= previous, "eps={eps}: {p} < {previous}");
if eps == 0.0 {
assert!(p < 1e-12, "a tie needs a margin, got {p}");
}
previous = p;
}
assert!(
previous > 0.9,
"a very wide margin ties everyone: {previous}"
);
}
/// The converse, stated as the non-property it is: a partially-tied
/// outcome is non-monotone in the margin. Pinning this down stops a future
/// change from "fixing" it into monotonicity and quietly breaking the model.
#[test]
fn a_partially_tied_outcome_peaks_in_the_middle() {
let perf = [g(0.0, 4.0), g(0.0, 4.0), g(-8.0, 2.0)];
let sweep: Vec<f64> = [0.5, 2.0, 4.0, 8.0, 16.0]
.iter()
.map(|&eps| ranking_probability(&perf, &flat(3, eps), &[0, 0, 1]))
.collect();
let peak = sweep
.iter()
.enumerate()
.fold(
(0, 0.0),
|(bi, bv), (i, &v)| if v > bv { (i, v) } else { (bi, bv) },
)
.0;
assert!(
peak > 0 && peak < sweep.len() - 1,
"expected an interior peak: {sweep:?}"
);
}
/// With no draw margin a tie has probability exactly zero, and the
/// enumeration must not waste work pretending otherwise.
#[test]
fn ties_are_impossible_without_a_draw_margin() {
let perf = [g(0.0, 4.0), g(0.0, 4.0), g(0.0, 4.0)];
let dist = outcome_distribution(&perf, &flat(3, 0.0));
assert_eq!(dist.len(), 6, "expected only the 6 strict orders: {dist:?}");
assert!(dist.iter().all(|(r, _)| {
let mut seen = r.clone();
seen.sort_unstable();
seen.dedup();
seen.len() == r.len()
}));
}
}
+322
View File
@@ -0,0 +1,322 @@
//! Deterministic numerical integration for the prediction paths.
//!
//! Prediction asks two questions that have no closed form beyond two teams:
//! "who finishes first" and "how likely is this exact finishing order". Both
//! reduce to integrals over a single performance variable, so neither needs a
//! sampler — and that matters, because a Monte Carlo predictor would make
//! `predict_*` non-reproducible and would answer a slightly different question
//! on every call.
//!
//! Two routines live here:
//!
//! - [`integrate`], adaptive Gauss-Kronrod G7-K15, for the first-place
//! marginals. It carries its own error estimate, so it can refine where the
//! integrand actually bends instead of guessing a node count up front.
//! - [`Grid`], a uniform grid with trapezoid prefix sums, for the ranking
//! chain recursion, where each level needs the *running* integral of the
//! level below at arbitrary points rather than one definite integral.
//!
//! Fixed-node Gauss-Hermite is the obvious tool for the first of these and is
//! a trap: the integrand is a product of normal CDFs, and when one team's
//! sigma is much smaller than the integrating team's, that product turns into
//! a near-step function narrower than the node spacing. The nodes step over
//! it and the result is wrong by ~1e-2 while still looking like a probability.
//! Adaptive refinement is what makes the small-sigma case safe.
/// Kronrod 15-point abscissae, non-negative half, descending.
const XGK: [f64; 8] = [
0.991_455_371_120_813,
0.949_107_912_342_759,
0.864_864_423_359_769,
0.741_531_185_599_394,
0.586_087_235_467_691,
0.405_845_151_377_397,
0.207_784_955_007_898,
0.0,
];
/// Kronrod 15-point weights, matching [`XGK`].
const WGK: [f64; 8] = [
0.022_935_322_010_529,
0.063_092_092_629_979,
0.104_790_010_322_250,
0.140_653_259_715_525,
0.169_004_726_639_267,
0.190_350_578_064_785,
0.204_432_940_075_298,
0.209_482_141_084_728,
];
/// Gauss 7-point weights, applying to the odd-indexed [`XGK`] entries.
const WG: [f64; 4] = [
0.129_484_966_168_870,
0.279_705_391_489_277,
0.381_830_050_505_119,
0.417_959_183_673_469,
];
/// Panels are bisected worst-first; this bounds the work on a pathological
/// integrand rather than letting it spin.
const MAX_SUBDIVISIONS: usize = 200;
/// One G7-K15 panel over `[a, b]`: `(integral, absolute error estimate)`.
///
/// The error estimate is the gap between the embedded 7-point Gauss rule and
/// the 15-point Kronrod extension. It is the only reason this is preferable
/// to a fixed rule: it tells the caller *where* the integrand is hard.
fn gk15<F: Fn(f64) -> f64>(f: &F, a: f64, b: f64) -> (f64, f64) {
let centre = 0.5 * (a + b);
let half = 0.5 * (b - a);
let mut kronrod = 0.0;
let mut gauss = 0.0;
for i in 0..8 {
let offset = XGK[i] * half;
// XGK[7] is the centre node and must not be counted twice.
let sum = if i == 7 {
f(centre)
} else {
f(centre - offset) + f(centre + offset)
};
kronrod += WGK[i] * sum;
if i % 2 == 1 {
gauss += WG[i / 2] * sum;
}
}
(kronrod * half, ((kronrod - gauss) * half).abs())
}
/// Adaptively integrate `f` over `[a, b]` to relative tolerance `tol`.
///
/// `seeds` are interior points where the integrand is known to bend sharply —
/// for a product of normal CDFs, each rival's transition centre. Splitting
/// there up front costs nothing and saves the adaptive loop from having to
/// discover a step by bisection.
///
/// Returns the integral. The error estimate is consumed internally rather
/// than returned: callers here integrate probability densities, where the
/// meaningful check is the sum-to-one identity over a whole outcome space,
/// not a per-integral residual.
pub(crate) fn integrate<F: Fn(f64) -> f64>(f: F, a: f64, b: f64, seeds: &[f64], tol: f64) -> f64 {
// Explicit rather than `!(b > a)`: a NaN bound must fall through to zero
// rather than being read as a valid ordering.
if a.partial_cmp(&b) != Some(std::cmp::Ordering::Less) {
return 0.0;
}
let mut edges: Vec<f64> = Vec::with_capacity(seeds.len() + 2);
edges.push(a);
edges.push(b);
for &s in seeds {
if s > a && s < b {
edges.push(s);
}
}
edges.sort_by(|p, q| p.partial_cmp(q).expect("integration bounds are finite"));
edges.dedup();
// (lo, hi, integral, error)
let mut panels: Vec<(f64, f64, f64, f64)> = edges
.windows(2)
.map(|w| {
let (v, e) = gk15(&f, w[0], w[1]);
(w[0], w[1], v, e)
})
.collect();
for _ in 0..MAX_SUBDIVISIONS {
let total: f64 = panels.iter().map(|p| p.2).sum();
let error: f64 = panels.iter().map(|p| p.3).sum();
// Absolute floor as well as relative: these integrands are
// probabilities, so an absolute 1e-15 is already past the useful
// precision of the underlying `cdf`.
if error <= tol * total.abs().max(1e-12) || error < 1e-15 {
break;
}
let worst = panels
.iter()
.enumerate()
.fold((0usize, f64::NEG_INFINITY), |(bi, be), (i, p)| {
if p.3 > be { (i, p.3) } else { (bi, be) }
})
.0;
let (lo, hi, _, _) = panels[worst];
let mid = 0.5 * (lo + hi);
// Bisection has hit the floating-point floor; refining further would
// loop without reducing the error.
if !(mid > lo && mid < hi) {
break;
}
let (v1, e1) = gk15(&f, lo, mid);
let (v2, e2) = gk15(&f, mid, hi);
panels[worst] = (lo, mid, v1, e1);
panels.push((mid, hi, v2, e2));
}
panels.iter().map(|p| p.2).sum()
}
/// A uniform grid carrying trapezoid prefix sums of one integrand.
///
/// The ranking recursion needs, at every level, the running integral of the
/// level below evaluated at arbitrary points — a cumulative integral, not a
/// definite one. Prefix sums give that in O(1) per query after an O(G) build,
/// which is what keeps a full ranking probability linear in the team count.
pub(crate) struct Grid {
lo: f64,
step: f64,
/// Integrand sampled at each node.
values: Vec<f64>,
/// `prefix[i]` is the integral from `lo` to node `i`.
prefix: Vec<f64>,
}
impl Grid {
/// Build directly from already-sampled values.
///
/// The ranking recursion evaluates every level on the same nodes, so the
/// per-team densities are sampled once and reused; re-evaluating `exp`
/// per level would dominate the cost.
pub(crate) fn from_values(lo: f64, step: f64, values: Vec<f64>) -> Self {
let mut prefix = vec![0.0; values.len()];
for i in 1..values.len() {
prefix[i] = prefix[i - 1] + 0.5 * step * (values[i - 1] + values[i]);
}
Self {
lo,
step,
values,
prefix,
}
}
/// Integral from the grid's lower bound up to `x`.
///
/// Clamped at both ends: the caller sizes the grid to cover the whole
/// support, so a query outside it is asking for a tail that is zero (below)
/// or the whole mass (above).
pub(crate) fn integral_to(&self, x: f64) -> f64 {
let last = self.values.len() - 1;
if x <= self.lo {
return 0.0;
}
if x >= self.lo + last as f64 * self.step {
return self.prefix[last];
}
let scaled = (x - self.lo) / self.step;
let i = scaled.floor() as usize;
let frac = scaled - i as f64;
// Whole cells, plus the trapezoid over the partial cell. The integrand
// is linear within a cell under the trapezoid rule, so the partial
// piece is exact with respect to that same approximation.
self.prefix[i]
+ frac
* self.step
* (self.values[i] + 0.5 * frac * (self.values[i + 1] - self.values[i]))
}
/// Integral over `[from, to]`.
pub(crate) fn integral_between(&self, from: f64, to: f64) -> f64 {
(self.integral_to(to) - self.integral_to(from)).max(0.0)
}
/// Total integral over the whole grid.
pub(crate) fn total(&self) -> f64 {
self.prefix[self.values.len() - 1]
}
}
#[cfg(test)]
mod tests {
use super::*;
const TOL: f64 = 1e-10;
/// Sample `f` over `[lo, hi]` at `points` nodes.
fn sample<F: FnMut(f64) -> f64>(lo: f64, hi: f64, points: usize, mut f: F) -> Grid {
let step = (hi - lo) / (points - 1) as f64;
Grid::from_values(
lo,
step,
(0..points).map(|i| f(lo + i as f64 * step)).collect(),
)
}
#[test]
fn integrates_a_polynomial_exactly() {
// G7-K15 is exact for polynomials well past cubic, so a single panel
// should already be at round-off.
let v = integrate(|x| 3.0 * x * x + 2.0 * x + 1.0, 0.0, 2.0, &[], TOL);
assert!((v - 14.0).abs() < 1e-12, "got {v}");
}
#[test]
fn integrates_a_gaussian_density_to_one() {
let f = |x: f64| (-0.5 * x * x).exp() / (2.0 * std::f64::consts::PI).sqrt();
let v = integrate(f, -10.0, 10.0, &[], TOL);
assert!((v - 1.0).abs() < 1e-12, "got {v}");
}
#[test]
fn resolves_a_step_far_narrower_than_the_initial_panel() {
// The failure mode that rules out fixed-node quadrature: a transition
// 1e-4 wide inside a range of 20. A fixed rule steps over it.
let f = |x: f64| if x < 0.5 { 0.0 } else { 1.0 };
let v = integrate(f, -10.0, 10.0, &[0.5], TOL);
assert!((v - 9.5).abs() < 1e-6, "got {v}");
}
#[test]
fn seeds_do_not_change_the_value_of_a_smooth_integrand() {
let f = |x: f64| (-0.5 * x * x).exp();
let plain = integrate(f, -8.0, 8.0, &[], TOL);
let seeded = integrate(f, -8.0, 8.0, &[-3.0, 0.25, 5.5], TOL);
assert!((plain - seeded).abs() < 1e-12, "{plain} vs {seeded}");
}
#[test]
fn empty_or_inverted_range_integrates_to_zero() {
assert_eq!(integrate(|_| 1.0, 1.0, 1.0, &[], TOL), 0.0);
assert_eq!(integrate(|_| 1.0, 2.0, 1.0, &[], TOL), 0.0);
}
#[test]
fn grid_prefix_matches_a_known_cumulative_integral() {
// f(x) = x over [0, 4]; integral to x is x^2/2.
let g = sample(0.0, 4.0, 4001, |x| x);
for probe in [0.0, 0.5, 1.0, 2.5, 3.75, 4.0] {
let want = probe * probe / 2.0;
let got = g.integral_to(probe);
assert!(
(got - want).abs() < 1e-9,
"at {probe}: got {got}, want {want}"
);
}
assert!((g.total() - 8.0).abs() < 1e-9);
}
#[test]
fn grid_between_is_the_difference_of_two_prefixes() {
let g = sample(-5.0, 5.0, 8001, |x| (-0.5 * x * x).exp());
let whole = g.integral_between(-5.0, 5.0);
let split = g.integral_between(-5.0, 0.3) + g.integral_between(0.3, 5.0);
assert!((whole - split).abs() < 1e-12, "{whole} vs {split}");
}
#[test]
fn grid_clamps_queries_outside_its_support() {
let g = sample(0.0, 1.0, 101, |_| 1.0);
assert_eq!(g.integral_to(-3.0), 0.0);
assert!((g.integral_to(9.0) - 1.0).abs() < 1e-12);
// Reversed bounds must not produce negative probability mass.
assert_eq!(g.integral_between(0.8, 0.2), 0.0);
}
}
+2 -2
View File
@@ -9,8 +9,8 @@ use crate::{
/// Static rating configuration: prior skill, performance noise `beta`, drift. /// Static rating configuration: prior skill, performance noise `beta`, drift.
/// ///
/// Renamed from `Player` in T2; `Rating` better describes the data /// A configuration rather than a person: the per-history temporal state
/// (a configuration) vs. a person (who's a `Competitor` with state). /// (messages, last appearance) lives on `Competitor`.
#[derive(Clone, Copy, Debug)] #[derive(Clone, Copy, Debug)]
pub struct Rating<T: Time = i64, D: Drift<T> = ConstantDrift> { pub struct Rating<T: Time = i64, D: Drift<T> = ConstantDrift> {
pub(crate) prior: Gaussian, pub(crate) prior: Gaussian,
+9 -5
View File
@@ -203,7 +203,7 @@ fn predict_quality_two_teams() {
h.record_winner(&"a", &"b", 1).unwrap(); h.record_winner(&"a", &"b", 1).unwrap();
h.converge().unwrap(); h.converge().unwrap();
let q = h.predict_quality(&[&[&"a"], &[&"b"]]); let q = h.predict_quality(&[&[&"a"], &[&"b"]]).unwrap();
assert!(q > 0.0 && q <= 1.0); assert!(q > 0.0 && q <= 1.0);
} }
@@ -219,10 +219,14 @@ fn predict_outcome_two_teams_sums_to_one() {
h.record_winner(&"a", &"b", 1).unwrap(); h.record_winner(&"a", &"b", 1).unwrap();
h.converge().unwrap(); h.converge().unwrap();
let p = h.predict_outcome(&[&[&"a"], &[&"b"]]); let p = h.predict_outcome(&[&[&"a"], &[&"b"]]).unwrap();
assert_eq!(p.len(), 2); let wins = p.win_probabilities();
assert!((p[0] + p[1] - 1.0).abs() < 1e-9); assert_eq!(wins.len(), 2);
assert!(p[0] > p[1]); // With p_draw == 0 there is no draw outcome, so the two win
// probabilities are the whole space.
assert!((p.total() - 1.0).abs() < 1e-9, "total = {}", p.total());
assert!((wins[0] + wins[1] - 1.0).abs() < 1e-9);
assert!(wins[0] > wins[1]);
} }
#[test] #[test]
+222
View File
@@ -0,0 +1,222 @@
//! `Member::with_prior` / `with_drift_scale` — competitor configuration.
//!
//! Both were previously consumed only on the branch that *creates* a
//! competitor, so configuration supplied for a key the history already knew was
//! dropped with no error. `with_prior` had no coverage in this directory at
//! all, which is how that survived.
use smallvec::smallvec;
use trueskill_tt::{
ConvergenceOptions, Event, Gaussian, History, InferenceError, Member, Outcome, Team,
};
const CONVERGENCE: ConvergenceOptions = ConvergenceOptions {
max_iter: 2_000,
epsilon: 1e-12,
alpha: 1.0,
};
fn history() -> History {
History::builder()
.mu(25.0)
.sigma(25.0 / 3.0)
.beta(25.0 / 6.0)
.p_draw(0.0)
.convergence(CONVERGENCE)
.build()
}
/// One event, optionally configuring `a`.
fn bout(
a: &'static str,
b: &'static str,
time: i64,
prior: Option<Gaussian>,
scale: Option<f64>,
) -> Event<i64, &'static str> {
let mut member = Member::new(a);
if let Some(p) = prior {
member = member.with_prior(p);
}
if let Some(s) = scale {
member = member.with_drift_scale(s);
}
Event {
time,
teams: smallvec![
Team::with_members([member]),
Team::with_members([Member::new(b)]),
],
outcome: Outcome::winner(0, 2),
}
}
fn skill_of(h: &History, key: &str) -> Gaussian {
h.current_skill(&key).expect("key in history")
}
/// Baseline: the mechanism works at all on a competitor's first appearance.
#[test]
fn a_prior_applies_to_a_new_competitor() {
let seeded = Gaussian::from_ms(40.0, 1.0);
let mut with = history();
with.add_events(vec![bout("a", "b", 0, Some(seeded), None)])
.unwrap();
with.converge().unwrap();
let mut without = history();
without
.add_events(vec![bout("a", "b", 0, None, None)])
.unwrap();
without.converge().unwrap();
assert!(
(skill_of(&with, "a").mu() - skill_of(&without, "a").mu()).abs() > 1.0,
"a seeded prior should move the fit"
);
}
/// The defect in #10: a prior supplied for a competitor the history already
/// knows was silently discarded, and the caller got output computed from the
/// default prior with no indication anything had been dropped.
#[test]
fn a_prior_applies_to_a_competitor_the_history_already_knows() {
let seeded = Gaussian::from_ms(40.0, 1.0);
let mut late = history();
late.add_events(vec![bout("a", "b", 0, None, None)])
.unwrap();
// "a" now exists. Configuring it here used to do nothing whatsoever.
late.add_events(vec![bout("a", "b", 1, Some(seeded), None)])
.unwrap();
late.converge().unwrap();
let mut never = history();
never
.add_events(vec![
bout("a", "b", 0, None, None),
bout("a", "b", 1, None, None),
])
.unwrap();
never.converge().unwrap();
assert!(
(skill_of(&late, "a").mu() - skill_of(&never, "a").mu()).abs() > 1.0,
"a late prior must not be silently dropped: {} vs {}",
skill_of(&late, "a").mu(),
skill_of(&never, "a").mu()
);
}
/// Configuration is competitor-scoped, not event-scoped, and `converge` refits
/// from competitor state — so seeding late reaches the same fit as seeding from
/// the start. This is the documented scope, asserted rather than assumed.
#[test]
fn a_prior_is_whole_history_scoped_not_per_event() {
let seeded = Gaussian::from_ms(40.0, 1.0);
let mut late = history();
late.add_events(vec![bout("a", "b", 0, None, None)])
.unwrap();
late.add_events(vec![bout("a", "b", 1, Some(seeded), None)])
.unwrap();
late.converge().unwrap();
let mut early = history();
early
.add_events(vec![
bout("a", "b", 0, Some(seeded), None),
bout("a", "b", 1, Some(seeded), None),
])
.unwrap();
early.converge().unwrap();
let (l, e) = (skill_of(&late, "a"), skill_of(&early, "a"));
assert!(
(l.mu() - e.mu()).abs() < 1e-9 && (l.sigma() - e.sigma()).abs() < 1e-9,
"late seeding should refit the whole history: {l:?} vs {e:?}"
);
}
#[test]
fn repeating_the_same_prior_is_inert() {
let seeded = Gaussian::from_ms(40.0, 1.0);
let mut once = history();
once.add_events(vec![
bout("a", "b", 0, Some(seeded), None),
bout("a", "b", 1, None, None),
])
.unwrap();
once.converge().unwrap();
let mut every_time = history();
every_time
.add_events(vec![
bout("a", "b", 0, Some(seeded), None),
bout("a", "b", 1, Some(seeded), None),
])
.unwrap();
every_time.converge().unwrap();
let (o, e) = (skill_of(&once, "a"), skill_of(&every_time, "a"));
assert!(
(o.mu() - e.mu()).abs() < 1e-12 && (o.sigma() - e.sigma()).abs() < 1e-12,
"declaring the same prior repeatedly changed the fit: {o:?} vs {e:?}"
);
}
/// Events within a batch have no order, so two different values for one
/// competitor have no well-defined winner. Rejecting is what keeps the answer
/// independent of iteration order.
#[test]
fn a_batch_declaring_two_different_priors_is_rejected() {
let mut h = history();
let err = h
.add_events(vec![
bout("a", "b", 0, Some(Gaussian::from_ms(40.0, 1.0)), None),
bout("a", "b", 1, Some(Gaussian::from_ms(10.0, 1.0)), None),
])
.expect_err("two different priors for one competitor in one batch");
assert!(
matches!(
err,
InferenceError::ConflictingCompetitorConfig { field: "prior", .. }
),
"got {err:?}"
);
}
/// A member setting only `drift_scale` must not also assert the default prior,
/// or it would silently undo a prior seeded earlier. This is why the collected
/// configuration tracks each field separately rather than a merged `Rating`.
#[test]
fn setting_one_field_late_leaves_the_other_alone() {
let seeded = Gaussian::from_ms(40.0, 1.0);
let mut h = history();
h.add_events(vec![bout("a", "b", 0, Some(seeded), None)])
.unwrap();
// Only the scale this time — the prior above must survive.
h.add_events(vec![bout("a", "b", 1, None, Some(0.5))])
.unwrap();
h.converge().unwrap();
let mut both_upfront = history();
both_upfront
.add_events(vec![
bout("a", "b", 0, Some(seeded), Some(0.5)),
bout("a", "b", 1, None, None),
])
.unwrap();
both_upfront.converge().unwrap();
let (a, b) = (skill_of(&h, "a"), skill_of(&both_upfront, "a"));
assert!(
(a.mu() - b.mu()).abs() < 1e-9 && (a.sigma() - b.sigma()).abs() < 1e-9,
"setting drift_scale late clobbered the earlier prior: {a:?} vs {b:?}"
);
}
+112 -15
View File
@@ -341,13 +341,20 @@ fn zero_scale_pins_a_competitor_in_the_filtered_pass() {
); );
} }
/// `drift_scale` is competitor configuration captured at first appearance, the /// `drift_scale` is competitor configuration, and configuration supplied for a
/// same as `prior` — a later `with_drift_scale` on a key the history already /// competitor the history already knows is now *applied* rather than dropped.
/// knows is ignored. This guards that decision rather than driving it: the ///
/// behaviour falls out of where the capture happens, and the point of the test /// This test previously asserted the opposite. It was written as a deliberate
/// is that moving the capture would be a visible break, not a silent one. /// change-detector — "moving the capture would be a visible break, not a silent
/// one" — and that is exactly what happened: the capture moved, and the
/// assertion inverted rather than being deleted.
///
/// Because configuration lives on the competitor and `converge` refits from
/// competitor state, a late pin applies to the *whole* history, not just to
/// events after it. So a scale set on the second batch must reach the same fit
/// as one set from the very first event.
#[test] #[test]
fn drift_scale_is_ignored_after_first_appearance() { fn drift_scale_applies_when_set_after_first_appearance() {
let mut late = History::builder() let mut late = History::builder()
.mu(25.0) .mu(25.0)
.sigma(25.0 / 3.0) .sigma(25.0 / 3.0)
@@ -368,7 +375,7 @@ fn drift_scale_is_ignored_after_first_appearance() {
}]) }])
.unwrap(); .unwrap();
// Second batch asks for a pin. Too late: the competitor already exists. // Second batch asks for a pin. No longer too late.
late.add_events(vec![Event { late.add_events(vec![Event {
time: 1000, time: 1000,
teams: smallvec![ teams: smallvec![
@@ -380,23 +387,113 @@ fn drift_scale_is_ignored_after_first_appearance() {
.unwrap(); .unwrap();
late.converge().unwrap(); late.converge().unwrap();
let ignored = curve(&late, "anchor"); let applied = curve(&late, "anchor");
let drifting = curve(&fit(distant_pair(None), 25.0 / 300.0), "anchor"); let pinned_from_the_start = curve(&fit(distant_pair(Some(0.0)), 25.0 / 300.0), "anchor");
let never_pinned = curve(&fit(distant_pair(None), 25.0 / 300.0), "anchor");
for ((t_l, g_l), (t_r, g_r)) in ignored.iter().zip(drifting.iter()) { for ((t_l, g_l), (t_r, g_r)) in applied.iter().zip(pinned_from_the_start.iter()) {
assert_eq!(t_l, t_r); assert_eq!(t_l, t_r);
assert!( assert!(
(g_l.sigma() - g_r.sigma()).abs() < 1e-9, (g_l.sigma() - g_r.sigma()).abs() < 1e-9,
"a scale set after first appearance must be ignored, leaving the fit \ "a late pin should refit the whole history: t={t_l}, {} vs {}",
identical to one that never set it: t={t_l}, {} vs {}",
g_l.sigma(), g_l.sigma(),
g_r.sigma() g_r.sigma()
); );
} }
let pinned = curve(&fit(distant_pair(Some(0.0)), 25.0 / 300.0), "anchor"); // And it must actually have done something.
assert!( assert!(
(ignored[1].1.sigma() - pinned[1].1.sigma()).abs() > 1e-6, applied
"sanity: the pinned fit must actually differ, or the assertion above is vacuous" .iter()
.zip(never_pinned.iter())
.any(|((_, a), (_, b))| (a.sigma() - b.sigma()).abs() > 1e-9),
"the pin had no effect at all — the silent drop is back"
);
}
/// Re-declaring the same configuration must be inert. This is the shape a
/// caller gets when the configuration is a property of the domain — "layouts
/// are static" — so every ingestion path repeats it on every event.
///
/// Both histories see exactly the same events; only how many times the scale
/// is declared differs.
#[test]
fn repeating_the_same_configuration_changes_nothing() {
let events = |declare_every_time: bool| {
let anchor = |first: bool| {
if first || declare_every_time {
Member::new("anchor").with_drift_scale(0.0)
} else {
Member::new("anchor")
}
};
vec![
Event {
time: 0,
teams: smallvec![
Team::with_members([anchor(true)]),
Team::with_members([Member::new("player")]),
],
outcome: Outcome::winner(0, 2),
},
Event {
time: 1000,
teams: smallvec![
Team::with_members([anchor(false)]),
Team::with_members([Member::new("player")]),
],
outcome: Outcome::winner(1, 2),
},
]
};
let once = curve(&fit(events(false), 25.0 / 300.0), "anchor");
let every_time = curve(&fit(events(true), 25.0 / 300.0), "anchor");
for ((t_l, a), (t_r, b)) in once.iter().zip(every_time.iter()) {
assert_eq!(t_l, t_r);
assert!(
(a.sigma() - b.sigma()).abs() < 1e-12,
"t={t_l}: declaring the same scale repeatedly changed the fit, {} vs {}",
a.sigma(),
b.sigma()
);
}
}
#[test]
fn a_batch_that_contradicts_itself_is_rejected() {
let mut h = History::builder().convergence(CONVERGENCE).build();
let err = h
.add_events(vec![
Event {
time: 0,
teams: smallvec![
Team::with_members([Member::new("anchor").with_drift_scale(0.0)]),
Team::with_members([Member::new("player")]),
],
outcome: Outcome::winner(0, 2),
},
Event {
time: 1,
teams: smallvec![
Team::with_members([Member::new("anchor").with_drift_scale(1.0)]),
Team::with_members([Member::new("player")]),
],
outcome: Outcome::winner(0, 2),
},
])
.expect_err("two different scales for one competitor in one batch");
assert!(
matches!(
err,
InferenceError::ConflictingCompetitorConfig {
field: "drift_scale",
..
}
),
"got {err:?}"
); );
} }
+2 -1
View File
@@ -19,7 +19,8 @@ fn ts_rating(mu: f64, sigma: f64, beta: f64, gamma: f64) -> R {
fn game_1v1_golden_matches_historical() { fn game_1v1_golden_matches_historical() {
let a = ts_rating(25.0, 25.0 / 3.0, 25.0 / 6.0, 25.0 / 300.0); let a = ts_rating(25.0, 25.0 / 3.0, 25.0 / 6.0, 25.0 / 300.0);
let b = ts_rating(25.0, 25.0 / 3.0, 25.0 / 6.0, 25.0 / 300.0); let b = ts_rating(25.0, 25.0 / 3.0, 25.0 / 6.0, 25.0 / 300.0);
let (a_post, b_post) = Game::<i64, _>::one_v_one(&a, &b, Outcome::winner(0, 2)).unwrap(); let (a_post, b_post) =
Game::<i64, _>::one_v_one(&a, &b, Outcome::winner(0, 2), &GameOptions::default()).unwrap();
// Historical golden from pre-T2 test_1vs1 (team 0 wins): // Historical golden from pre-T2 test_1vs1 (team 0 wins):
assert_ulps_eq!( assert_ulps_eq!(
a_post, a_post,
+44 -1
View File
@@ -32,7 +32,8 @@ fn game_ranked_1v1_golden() {
fn game_one_v_one_shortcut() { fn game_one_v_one_shortcut() {
let a = default_rating(); let a = default_rating();
let b = default_rating(); let b = default_rating();
let (a_post, b_post) = Game::<i64, _>::one_v_one(&a, &b, Outcome::winner(0, 2)).unwrap(); let (a_post, b_post) =
Game::<i64, _>::one_v_one(&a, &b, Outcome::winner(0, 2), &GameOptions::default()).unwrap();
assert!(a_post.mu() > 25.0); assert!(a_post.mu() > 25.0);
assert!(b_post.mu() < 25.0); assert!(b_post.mu() < 25.0);
} }
@@ -95,3 +96,45 @@ fn game_log_evidence_is_finite() {
assert!(g.log_evidence().is_finite()); assert!(g.log_evidence().is_finite());
assert!(g.log_evidence() < 0.0); assert!(g.log_evidence() < 0.0);
} }
/// `one_v_one` used to hardcode `GameOptions::default()`, so a 1v1 could
/// never set `p_draw` and a drawn 1v1 was unreachable through it.
#[test]
fn one_v_one_honours_the_draw_probability_it_is_given() {
let a = default_rating();
let b = default_rating();
// Default options still reject a draw, because the default p_draw is zero.
let err = Game::<i64, _>::one_v_one(&a, &b, Outcome::draw(2), &GameOptions::default())
.expect_err("a draw needs a positive p_draw");
assert!(matches!(
err,
InferenceError::TieWithoutDrawProbability { .. }
));
// With a draw probability supplied it succeeds — which was impossible
// before the signature took options.
let options = GameOptions {
p_draw: 0.25,
..GameOptions::default()
};
let (a_post, b_post) = Game::<i64, _>::one_v_one(&a, &b, Outcome::draw(2), &options)
.expect("a draw is representable once p_draw is positive");
// A symmetric draw leaves the means alone and sharpens both sides.
assert!((a_post.mu() - b_post.mu()).abs() < 1e-9);
assert!(a_post.sigma() < 25.0 / 3.0);
}
/// Convergence options reach the 1v1 path too, not just `p_draw`.
#[test]
fn one_v_one_honours_convergence_options() {
let a = default_rating();
let b = default_rating();
let options = GameOptions {
convergence: ConvergenceOptions::default(),
..GameOptions::default()
};
let (a_post, _) = Game::<i64, _>::one_v_one(&a, &b, Outcome::winner(0, 2), &options).unwrap();
assert!(a_post.mu() > 25.0);
}
+78
View File
@@ -30,6 +30,22 @@ fn event(a: &str, b: &str, time: i64) -> Event<i64, String> {
} }
} }
/// Like [`event`], but `a` carries competitor configuration.
///
/// `prior` and `drift_scale` configure the competitor rather than the event, so
/// they are the part of ingestion most exposed to order: they are consumed once,
/// where the competitor's state is written.
fn configured_event(a: &str, b: &str, time: i64, scale: f64) -> Event<i64, String> {
Event {
time,
teams: smallvec![
Team::with_members([Member::new(a.to_string()).with_drift_scale(scale)]),
Team::with_members([Member::new(b.to_string())]),
],
outcome: Outcome::winner(0, 2),
}
}
fn converged_skills(events: Vec<Event<i64, String>>, batched: bool) -> Vec<(String, Gaussian)> { fn converged_skills(events: Vec<Event<i64, String>>, batched: bool) -> Vec<(String, Gaussian)> {
let mut h: History<i64, _, _, String> = let mut h: History<i64, _, _, String> =
History::builder_with_key().convergence(tight()).build(); History::builder_with_key().convergence(tight()).build();
@@ -145,3 +161,65 @@ fn back_dated_event_matches_batched() {
let incremental = converged_skills(events, false); let incremental = converged_skills(events, false);
assert_same(&batched, &incremental, "back-dated event"); assert_same(&batched, &incremental, "back-dated event");
} }
/// The invariant this file protects was only ever checked for *unconfigured*
/// competitors — every helper above built members with `Member::new`.
///
/// Configuration is the part most exposed to ordering, because it is consumed
/// once at the point the competitor's state is written rather than replayed per
/// event. These cover it.
#[test]
fn configured_competitors_are_order_independent() {
let events = vec![
configured_event("a", "b", 0, 0.0),
configured_event("a", "c", 1, 0.0),
configured_event("a", "b", 2, 0.0),
event("b", "c", 3),
];
assert_same(
&converged_skills(events.clone(), true),
&converged_skills(events, false),
"configuration repeated on every appearance",
);
}
/// Configuration supplied only on a *later* event is the case that used to be
/// silently dropped. It must now reach the same fit either way it is ingested.
#[test]
fn late_configuration_is_order_independent() {
let events = vec![
event("a", "b", 0),
configured_event("a", "c", 1, 0.0),
event("a", "b", 2),
];
assert_same(
&converged_skills(events.clone(), true),
&converged_skills(events, false),
"configuration supplied after first appearance",
);
}
/// And it must actually be doing something — an implementation that dropped
/// configuration entirely would pass both tests above.
#[test]
fn configuration_changes_the_fit_however_it_is_ingested() {
let configured = vec![
event("a", "b", 0),
configured_event("a", "c", 1, 0.0),
event("a", "b", 2),
];
let plain = vec![event("a", "b", 0), event("a", "c", 1), event("a", "b", 2)];
for batched in [true, false] {
let with = converged_skills(configured.clone(), batched);
let without = converged_skills(plain.clone(), batched);
assert!(
with.iter()
.zip(&without)
.any(|((_, x), (_, y))| (x.sigma() - y.sigma()).abs() > 1e-9),
"batched={batched}: configuration had no effect, so the order tests are vacuous"
);
}
}
+161
View File
@@ -0,0 +1,161 @@
//! `Observer` callbacks must actually fire.
//!
//! `on_slice_processed` (formerly `on_batch_processed`) was declared on the
//! trait and never called from anywhere, so implementors wired up a callback
//! that could not run. These tests exist so that cannot silently recur.
use std::sync::{Arc, Mutex};
use trueskill_tt::{History, Observer};
/// Plain fields. `Arc<O>` implements `Observer`, so the caller shares the
/// observer itself rather than wrapping each field in its own `Arc`.
#[derive(Default)]
struct Recorder {
iterations: Mutex<Vec<usize>>,
slices: Mutex<Vec<(i64, usize, usize)>>,
converged: Mutex<Vec<(usize, bool)>>,
}
impl Observer<i64> for Recorder {
fn on_iteration_end(&self, iter: usize, _max_step: (f64, f64)) {
self.iterations.lock().unwrap().push(iter);
}
fn on_slice_processed(&self, time: &i64, slice_idx: usize, n_events: usize) {
self.slices
.lock()
.unwrap()
.push((*time, slice_idx, n_events));
}
fn on_converged(&self, iters: usize, _final_step: (f64, f64), converged: bool) {
self.converged.lock().unwrap().push((iters, converged));
}
}
#[test]
fn every_observer_callback_fires() {
let recorder = Arc::new(Recorder::default());
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
h.record_winner(&"a", &"b", 1).unwrap();
h.record_winner(&"b", &"c", 2).unwrap();
h.record_winner(&"c", &"a", 3).unwrap();
h.converge().unwrap();
assert!(
!recorder.iterations.lock().unwrap().is_empty(),
"on_iteration_end never fired"
);
assert!(
!recorder.converged.lock().unwrap().is_empty(),
"on_converged never fired"
);
assert!(
!recorder.slices.lock().unwrap().is_empty(),
"on_slice_processed never fired — the defect this test exists for"
);
}
#[test]
fn slice_callbacks_report_the_slice_they_swept() {
let recorder = Arc::new(Recorder::default());
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
h.record_winner(&"a", &"b", 10).unwrap();
h.record_winner(&"a", &"b", 20).unwrap();
h.converge().unwrap();
let slices = recorder.slices.lock().unwrap();
// Only the times actually in the history, and each with its own events.
for &(time, idx, events) in slices.iter() {
assert!(time == 10 || time == 20, "unexpected slice time {time}");
assert!(idx < 2, "slice index {idx} out of range");
assert_eq!(events, 1, "each slice holds exactly one event");
}
// Both slices must be reported, not just one end of the sweep.
assert!(
slices.iter().any(|&(t, ..)| t == 10),
"slice 10 never reported"
);
assert!(
slices.iter().any(|&(t, ..)| t == 20),
"slice 20 never reported"
);
}
#[test]
fn a_single_slice_history_still_reports_its_sweep() {
let recorder = Arc::new(Recorder::default());
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
h.record_winner(&"a", &"b", 1).unwrap();
h.converge().unwrap();
let slices = recorder.slices.lock().unwrap();
assert!(
!slices.is_empty(),
"the single-slice path must report its sweep too"
);
assert!(slices.iter().all(|&(t, idx, _)| t == 1 && idx == 0));
}
/// The gap #40 closed: without `impl Observer for Arc<O>`, an observer that
/// accumulates anything had to wrap every field in its own `Arc` and derive
/// `Clone`, because `History` consumes the observer and never hands it back.
#[test]
fn a_shared_observer_reaches_the_callers_handle() {
let recorder = Arc::new(Recorder::default());
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
h.record_winner(&"a", &"b", 1).unwrap();
h.converge().unwrap();
assert!(!recorder.iterations.lock().unwrap().is_empty());
assert!(!recorder.slices.lock().unwrap().is_empty());
assert!(!recorder.converged.lock().unwrap().is_empty());
}
/// `?Sized` on the blanket impls means the observer can be chosen at runtime.
#[test]
fn a_trait_object_observer_works() {
let boxed: Box<dyn Observer<i64>> = Box::new(Recorder::default());
let mut h = History::builder().observer(boxed).build();
h.record_winner(&"a", &"b", 1).unwrap();
h.converge().unwrap();
let shared: Arc<dyn Observer<i64>> = Arc::new(Recorder::default());
let mut h = History::builder().observer(Arc::clone(&shared)).build();
h.record_winner(&"a", &"b", 1).unwrap();
h.converge().unwrap();
}
/// A non-shared observer can be reclaimed after convergence instead.
#[test]
fn into_observer_returns_the_accumulated_state() {
let mut h = History::builder().observer(Recorder::default()).build();
h.record_winner(&"a", &"b", 1).unwrap();
h.converge().unwrap();
// Readable in place...
assert!(!h.observer().iterations.lock().unwrap().is_empty());
// ...and reclaimable by value.
let recorder = h.into_observer();
assert!(!recorder.slices.lock().unwrap().is_empty());
}
/// Borrowing works too, for an observer that outlives the history.
#[test]
fn a_borrowed_observer_works() {
let recorder = Recorder::default();
{
let mut h = History::builder().observer(&recorder).build();
h.record_winner(&"a", &"b", 1).unwrap();
h.converge().unwrap();
}
assert!(!recorder.iterations.lock().unwrap().is_empty());
}
+291
View File
@@ -0,0 +1,291 @@
//! Prediction API: N-team outcomes, draw mass, and the error paths that used
//! to be panics or silent wrong answers.
use trueskill_tt::{History, InferenceError, MAX_PREDICTED_TEAMS};
fn history_with(names: &[&'static str], p_draw: f64) -> History {
let mut h = History::builder().p_draw(p_draw).build();
// Give every competitor a recorded skill by playing a small round robin.
for pair in names.windows(2) {
h.record_winner(&pair[0], &pair[1], 1).unwrap();
}
h.converge().unwrap();
h
}
#[test]
fn unknown_keys_are_reported_not_silently_dropped() {
let h = history_with(&["a", "b"], 0.0);
let err = h
.predict_outcome(&[&[&"a"], &[&"ghost"]])
.expect_err("an unknown key must not yield a confident prediction");
assert_eq!(err, InferenceError::UnknownKey { team: 1, member: 0 });
// Every prediction entry point, not just one.
assert!(
h.predict_win_probabilities(&[&[&"a"], &[&"ghost"]])
.is_err()
);
assert!(h.predict_quality(&[&[&"a"], &[&"ghost"]]).is_err());
assert!(h.predict_ranking(&[&[&"a"], &[&"ghost"]], &[0, 1]).is_err());
}
#[test]
fn an_entirely_unknown_team_is_an_error() {
let h = history_with(&["a", "b"], 0.0);
let err = h.predict_outcome(&[&[&"a"], &[&"x", &"y"]]).unwrap_err();
assert_eq!(err, InferenceError::UnknownKey { team: 1, member: 0 });
}
#[test]
fn degenerate_team_shapes_are_errors_rather_than_panics() {
let h = history_with(&["a", "b"], 0.0);
assert_eq!(
h.predict_outcome(&[&[&"a"]]).unwrap_err(),
InferenceError::NotEnoughTeams { got: 1 }
);
assert_eq!(
h.predict_outcome(&[]).unwrap_err(),
InferenceError::NotEnoughTeams { got: 0 }
);
assert_eq!(
h.predict_outcome(&[&[&"a"], &[]]).unwrap_err(),
InferenceError::EmptyTeam { team: 1 }
);
}
#[test]
fn more_than_two_teams_no_longer_panics() {
let h = history_with(&["a", "b", "c"], 0.0);
let p = h
.predict_outcome(&[&[&"a"], &[&"b"], &[&"c"]])
.expect("three teams must be supported");
assert!((p.total() - 1.0).abs() < 1e-6, "total = {}", p.total());
// Three teams, no draws possible: exactly the six strict orderings.
assert_eq!(p.outcomes().len(), 6);
}
#[test]
fn the_outcome_space_is_capped_rather_than_hanging() {
let names: Vec<&'static str> = vec!["a", "b", "c", "d", "e", "f", "g", "h"];
let h = history_with(&names, 0.0);
let teams: Vec<&[&&'static str]> = Vec::new();
let _ = teams;
let too_many: Vec<Vec<&&str>> = names.iter().map(|n| vec![n]).collect();
let refs: Vec<&[&&str]> = too_many.iter().map(Vec::as_slice).collect();
let err = h.predict_outcome(&refs).unwrap_err();
assert_eq!(
err,
InferenceError::TooManyTeams {
got: 8,
max: MAX_PREDICTED_TEAMS
}
);
// The cheap paths stay available at any size.
let wins = h.predict_win_probabilities(&refs).unwrap();
assert_eq!(wins.len(), 8);
assert!(
(wins.iter().sum::<f64>() - 1.0).abs() < 1e-6,
"win probabilities must still sum to one: {wins:?}"
);
}
/// The defect that made every draw-enabled prediction wrong: `[p, 1 - p]`
/// allocated no mass to a draw even with `p_draw > 0`.
#[test]
fn a_draw_carries_probability_mass_when_p_draw_is_positive() {
let h = history_with(&["a", "b"], 0.25);
let p = h.predict_outcome(&[&[&"a"], &[&"b"]]).unwrap();
let draw = p.probability_of(&[0, 0]);
assert!(draw > 0.0, "a draw-enabled model must give draws mass");
assert!((p.total() - 1.0).abs() < 1e-6, "total = {}", p.total());
let wins = p.win_probabilities();
assert!(
(wins.iter().sum::<f64>() + draw - 1.0).abs() < 1e-6,
"wins {wins:?} plus draw {draw} must be the whole space"
);
assert!(
(p.shared_first_place() - draw).abs() < 1e-12,
"a two-team draw is a shared first place"
);
}
#[test]
fn a_zero_draw_probability_admits_no_ties() {
let h = history_with(&["a", "b"], 0.0);
let p = h.predict_outcome(&[&[&"a"], &[&"b"]]).unwrap();
assert_eq!(p.probability_of(&[0, 0]), 0.0);
assert!(p.shared_first_place() < 1e-12);
}
/// The two routes to a win probability run through entirely different
/// algorithms — adaptive quadrature versus the enumerated chain recursion —
/// so agreement between them is a real cross-check, not a tautology.
#[test]
fn the_cheap_and_exhaustive_paths_agree() {
for p_draw in [0.0, 0.1] {
let h = history_with(&["a", "b", "c"], p_draw);
let teams: &[&[&&str]] = &[&[&"a"], &[&"b"], &[&"c"]];
let cheap = h.predict_win_probabilities(teams).unwrap();
let exhaustive = h.predict_outcome(teams).unwrap().win_probabilities();
for (i, (a, b)) in cheap.iter().zip(&exhaustive).enumerate() {
assert!(
(a - b).abs() < 1e-6,
"p_draw={p_draw} team {i}: quadrature {a} vs enumeration {b}"
);
}
}
}
#[test]
fn predict_ranking_agrees_with_the_distribution() {
let h = history_with(&["a", "b", "c"], 0.1);
let teams: &[&[&&str]] = &[&[&"a"], &[&"b"], &[&"c"]];
let dist = h.predict_outcome(teams).unwrap();
for (ranks, expected) in dist.outcomes() {
let direct = h.predict_ranking(teams, ranks).unwrap();
assert!(
(direct - expected).abs() < 1e-9,
"ranks {ranks:?}: {direct} vs {expected}"
);
}
}
#[test]
fn predict_ranking_checks_its_shape() {
let h = history_with(&["a", "b"], 0.0);
let err = h
.predict_ranking(&[&[&"a"], &[&"b"]], &[0, 1, 2])
.unwrap_err();
assert!(matches!(
err,
InferenceError::MismatchedShape {
expected: 2,
got: 3,
..
}
));
}
#[test]
fn the_stronger_competitor_is_favoured() {
let mut h = History::builder().build();
for t in 1..=10 {
h.record_winner(&"strong", &"weak", t).unwrap();
}
h.converge().unwrap();
let p = h.predict_outcome(&[&[&"strong"], &[&"weak"]]).unwrap();
let (best, _) = p.most_likely().expect("a most likely outcome");
assert_eq!(best, &[0, 1], "the winner should be favoured");
let wins = p.win_probabilities();
assert!(wins[0] > wins[1], "{wins:?}");
}
/// Unequal team sizes change the draw margin, because inference derives it
/// from the teams' betas. Prediction has to follow, or it describes a
/// different model than the one that will be fitted.
#[test]
fn team_size_affects_the_prediction() {
let mut h = History::builder().p_draw(0.2).build();
h.event(1)
.team(["a", "b"])
.team(["c"])
.winner(0)
.commit()
.unwrap();
h.converge().unwrap();
let p = h.predict_outcome(&[&[&"a", &"b"], &[&"c"]]).unwrap();
assert!((p.total() - 1.0).abs() < 1e-6, "total = {}", p.total());
assert!(p.probability_of(&[0, 0]) > 0.0);
}
// ---------------------------------------------------------------------------
// Expected information gain
// ---------------------------------------------------------------------------
/// The whole point of #39: "which comparison should I run next?" is a
/// different question from "who will win?" or "is this fair?".
#[test]
fn information_gain_prefers_the_uncertain_pairing() {
let mut h = History::builder().build();
// "known" and "rival" have played a lot; "newcomer" has played once.
for t in 1..=15 {
h.record_winner(&"known", &"rival", t).unwrap();
h.record_winner(&"rival", &"known", t + 100).unwrap();
}
h.record_winner(&"known", &"newcomer", 500).unwrap();
h.converge().unwrap();
let settled = h
.expected_information_gain(&[&[&"known"], &[&"rival"]])
.unwrap();
let unknown = h
.expected_information_gain(&[&[&"known"], &[&"newcomer"]])
.unwrap();
assert!(
unknown > settled,
"pairing against the newcomer should teach more: {unknown} vs {settled}"
);
}
/// The analytic ceiling, through the `History` entry point rather than the
/// standalone one.
#[test]
fn information_gain_respects_the_entropy_ceiling() {
let h = history_with(&["a", "b", "c"], 0.0);
let two = h.expected_information_gain(&[&[&"a"], &[&"b"]]).unwrap();
assert!(
(0.0..=std::f64::consts::LN_2).contains(&two),
"two-team EIG {two} outside [0, ln 2]"
);
let three = h
.expected_information_gain(&[&[&"a"], &[&"b"], &[&"c"]])
.unwrap();
assert!(
(0.0..=6.0f64.ln()).contains(&three),
"three-team EIG {three} outside [0, ln 6]"
);
}
#[test]
fn information_gain_reports_unknown_keys() {
let h = history_with(&["a", "b"], 0.0);
assert_eq!(
h.expected_information_gain(&[&[&"a"], &[&"ghost"]])
.unwrap_err(),
InferenceError::UnknownKey { team: 1, member: 0 }
);
}
/// A draw-enabled history has three outcomes to weigh rather than two, so the
/// draw branch must actually be reachable through this path.
#[test]
fn information_gain_accounts_for_draws() {
let with_draws = history_with(&["a", "b"], 0.25);
let g = with_draws
.expected_information_gain(&[&[&"a"], &[&"b"]])
.unwrap();
assert!(g > 0.0 && g <= 3.0f64.ln(), "{g}");
// The draw outcome carries mass, so it is genuinely being weighed.
let dist = with_draws.predict_outcome(&[&[&"a"], &[&"b"]]).unwrap();
assert!(dist.probability_of(&[0, 0]) > 0.0);
}
+1 -1
View File
@@ -110,7 +110,7 @@ fn history_predict_quality_supports_three_teams() {
h.record_winner(&"b", &"c", 2).unwrap(); h.record_winner(&"b", &"c", 2).unwrap();
h.converge().unwrap(); h.converge().unwrap();
let q = h.predict_quality(&[&[&"a"], &[&"b"], &[&"c"]]); let q = h.predict_quality(&[&[&"a"], &[&"b"], &[&"c"]]).unwrap();
assert!( assert!(
q.is_finite(), q.is_finite(),
"3-team predict_quality must be finite, got {q}" "3-team predict_quality must be finite, got {q}"
+186
View File
@@ -0,0 +1,186 @@
//! Input validation must hold in **release**, where `debug_assert!` is gone.
//!
//! The engine guards itself with `debug_assert!`, which documents invariants
//! but vanishes in the profile users actually ship. Anything reachable from the
//! public API has to be rejected with an `InferenceError` instead, at the
//! boundary, rather than becoming NaN or an out-of-bounds panic deep inside
//! `run_chain`.
//!
//! `GameOptions` and `ConvergenceOptions` both have public fields, so the
//! eager asserts on `HistoryBuilder` do not cover the `Game` constructors —
//! a caller can build the options struct directly.
use smallvec::smallvec;
use trueskill_tt::{
ConstantDrift, ConvergenceOptions, Event, Game, GameOptions, Gaussian, History, InferenceError,
Member, Outcome, Rating, Team,
};
type R = Rating<i64, ConstantDrift>;
fn rating() -> R {
R::new(
Gaussian::from_ms(25.0, 25.0 / 3.0),
25.0 / 6.0,
ConstantDrift(0.0),
)
}
fn options_with_alpha(alpha: f64) -> GameOptions {
GameOptions {
convergence: ConvergenceOptions {
alpha,
..ConvergenceOptions::default()
},
..GameOptions::default()
}
}
/// `alpha == 0.0` leaves every EP update unapplied, so inference silently
/// returns the priors — the worst possible failure, since the output looks
/// entirely reasonable.
#[test]
fn ranked_rejects_a_zero_damping_factor() {
let (a, b) = (rating(), rating());
let err = Game::<i64, _>::ranked(
&[&[a], &[b]],
Outcome::winner(0, 2),
&options_with_alpha(0.0),
)
.expect_err("alpha = 0 must be rejected");
assert!(
matches!(err, InferenceError::InvalidParameter { name: "alpha", .. }),
"got {err:?}"
);
}
#[test]
fn ranked_rejects_an_out_of_range_damping_factor() {
let (a, b) = (rating(), rating());
for alpha in [-0.5, 1.5, f64::NAN] {
let err = Game::<i64, _>::ranked(
&[&[a], &[b]],
Outcome::winner(0, 2),
&options_with_alpha(alpha),
)
.expect_err("alpha out of (0, 1] must be rejected");
assert!(
matches!(err, InferenceError::InvalidParameter { name: "alpha", .. }),
"alpha={alpha}: got {err:?}"
);
}
}
#[test]
fn scored_rejects_a_bad_damping_factor() {
let (a, b) = (rating(), rating());
let err = Game::<i64, _>::scored(
&[&[a], &[b]],
Outcome::scores([21.0, 9.0]),
&options_with_alpha(0.0),
)
.expect_err("alpha = 0 must be rejected");
assert!(
matches!(err, InferenceError::InvalidParameter { name: "alpha", .. }),
"got {err:?}"
);
}
/// Already covered by `Game::ranked`, asserted here so the release-mode
/// guarantee is stated in one place.
#[test]
fn ranked_rejects_an_out_of_range_draw_probability() {
let (a, b) = (rating(), rating());
for p_draw in [-0.5, 1.0, 1.5] {
let options = GameOptions {
p_draw,
..GameOptions::default()
};
assert!(
Game::<i64, _>::ranked(&[&[a], &[b]], Outcome::winner(0, 2), &options).is_err(),
"p_draw={p_draw} must be rejected"
);
}
}
#[test]
fn scored_rejects_a_non_positive_noise() {
let (a, b) = (rating(), rating());
for score_sigma in [0.0, -1.0, f64::NAN] {
let options = GameOptions {
score_sigma,
..GameOptions::default()
};
assert!(
Game::<i64, _>::scored(&[&[a], &[b]], Outcome::scores([21.0, 9.0]), &options).is_err(),
"score_sigma={score_sigma} must be rejected"
);
}
}
/// A tie with no draw probability makes the truncation margin zero and the
/// two-sided update evaluate 0/0. Ingestion must refuse it.
#[test]
fn ingestion_rejects_a_tie_without_a_draw_probability() {
let mut h = History::builder().p_draw(0.0).build();
let err = h
.add_events(vec![Event {
time: 0,
teams: smallvec![
Team::with_members([Member::new("a")]),
Team::with_members([Member::new("b")]),
],
outcome: Outcome::draw(2),
}])
.expect_err("a tie with p_draw = 0 must be rejected");
assert!(
matches!(err, InferenceError::TieWithoutDrawProbability { .. }),
"got {err:?}"
);
}
/// `Outcome::scores_with_sigma` documents that a non-positive sigma is
/// accepted at construction and rejected at ingestion.
#[test]
fn ingestion_rejects_a_non_positive_per_event_score_sigma() {
for sigma in [0.0, -1.0, f64::NAN] {
let mut h = History::builder().build();
let err = h
.add_events(vec![Event {
time: 0,
teams: smallvec![
Team::with_members([Member::new("a")]),
Team::with_members([Member::new("b")]),
],
outcome: Outcome::scores_with_sigma([21.0, 9.0], sigma),
}])
.expect_err("a non-positive per-event sigma must be rejected");
assert!(
matches!(err, InferenceError::InvalidParameter { .. }),
"sigma={sigma}: got {err:?}"
);
}
}
/// Per-team weights must match that team's membership. The top-level length
/// checks in ingestion do not cover the inner dimension.
#[test]
fn ingestion_rejects_weights_that_do_not_match_their_team() {
let mut h = History::builder().build();
let mut team = Team::with_members([Member::new("a"), Member::new("b")]);
team.members[0].weight = 1.0;
let err = h
.event(0)
.team(["a", "b"])
.team(["c"])
// Three weights for a two-member team.
.weights([1.0, 1.0, 1.0])
.winner(0)
.commit()
.expect_err("a weight/member length mismatch must be rejected");
assert!(
matches!(err, InferenceError::MismatchedShape { .. }),
"got {err:?}"
);
}