Compare commits
100
Commits
v0.3.0
...
d2ab4446ef
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d2ab4446ef | ||
|
|
e72bf3894c | ||
|
|
56193609f7 | ||
|
|
13a395fdc9 | ||
|
|
6e2ce69728 | ||
|
|
faa25fb3b1 | ||
|
|
ddbac87744 | ||
|
|
076a7ded8c | ||
|
|
8e34410db0 | ||
|
|
92d690d0f8 | ||
|
|
4c2e98c54b | ||
|
|
92ae5fca17 | ||
|
|
da55d2a7d1 | ||
|
|
6a893ffe57 | ||
|
|
f14c783c0e | ||
|
|
055575a6f4 | ||
|
|
251211f134 | ||
|
|
31564b71a0 | ||
|
|
78810c0344 | ||
|
|
9d3e002be3 | ||
|
|
9d629d0d94 | ||
|
|
7ca0daa48e | ||
|
|
c3d1afe448 | ||
|
|
e4d6dc4028 | ||
|
|
cc601c06eb | ||
|
|
86e1521f8a | ||
|
|
60fc3e9d05 | ||
|
|
e4a68ba1a7 | ||
|
|
56e8220c86 | ||
|
|
fdd1539cab | ||
|
|
85c4d0d87d | ||
|
|
a0c2f78aed | ||
|
|
4472d98b56 | ||
|
|
5f5a37090a | ||
|
|
dc1f4d5847 | ||
|
|
0ab56248bb | ||
|
|
8dff7513f7 | ||
|
|
a367155778 | ||
|
|
c69a397d80 | ||
|
|
7aa7fb62dd | ||
|
|
305f822964 | ||
|
|
ab23476aaf | ||
|
|
6139061740 | ||
|
|
f1219036b3 | ||
|
|
31cf0998b0 | ||
|
|
bbc7705c75 | ||
|
|
83bdb84152 | ||
|
|
c65373f476 | ||
|
|
7da2328692 | ||
|
|
a73afa5f24 | ||
|
|
eebf8aacd3 | ||
|
|
4e9aa6bdc1 | ||
|
|
a18df521eb | ||
|
|
1e4b589a9c | ||
|
|
8b20e0c560 | ||
|
|
862779ae34 | ||
|
|
f692906ce4 | ||
|
|
e493f47e99 | ||
|
|
4f6360128d | ||
|
|
eff63dfa2a | ||
|
|
7c6965c6a9 | ||
|
|
911b48faba | ||
|
|
f57784c141 | ||
|
|
8e4d6a637d | ||
|
|
82eff740b6 | ||
|
|
c1b1c6c7d7 | ||
|
|
1bb6bb31d8 | ||
|
|
b113385c6f | ||
|
|
f345e7690e | ||
|
|
d9e85cda1d | ||
|
|
633a503900 | ||
|
|
7cf45db5cf | ||
|
|
c866210c65 | ||
|
|
1f791bcddd | ||
|
|
c52e2550af | ||
|
|
4924bc8b57 | ||
|
|
36eacf5f67 | ||
|
|
71554fd944 | ||
|
|
2cf21a753d | ||
|
|
35d7512557 | ||
|
|
e1bddf2474 | ||
|
|
5d36fa1008 | ||
|
|
c12bc830a5 | ||
|
|
901f60972e | ||
|
|
8116fd081f | ||
|
|
17d072b2ae | ||
|
|
3dd659307a | ||
|
|
7e289ee834 | ||
|
|
564969ee5d | ||
|
|
683813ec10 | ||
|
|
2a48d10aa9 | ||
|
|
d4f91fd221 | ||
|
|
8c087ad015 | ||
|
|
7341669d1a | ||
|
|
2fff745c3b | ||
|
|
3c2f9ac64c | ||
|
|
507894dae7 | ||
|
|
bb2a845882 | ||
|
|
87fca8dcca | ||
|
|
ef62b57a08 |
+153
@@ -2,6 +2,158 @@
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
## 0.8.0 - 2026-09-08
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- feat!: make a short fit an error and raise the default iteration cap
|
||||
- feat!: validate mu, sigma and beta on HistoryBuilder
|
||||
- feat!: add History::register and History::rating, and reject config conflicts across batches
|
||||
- fix!: reject non-finite weights at ingestion
|
||||
- fix!: reject malformed games at the Game boundary too
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- fix: reject malformed events at the ingestion boundary
|
||||
|
||||
### Documentation
|
||||
|
||||
- docs: record the rayon opt-in deviation in spec section 6
|
||||
- docs: state what the joint's cost actually scales in
|
||||
|
||||
### Features
|
||||
|
||||
- feat: add EventBuilder::members for per-member configuration
|
||||
|
||||
### Other (unconventional)
|
||||
|
||||
- Merge branch 'fix/ingestion-shape'
|
||||
- Merge branch 'feat/convergence-strictness'
|
||||
- Merge branch 'fix/non-finite-weights'
|
||||
- Merge branch 'test/close-coverage-gaps'
|
||||
- Merge branch 'fix/game-boundary'
|
||||
|
||||
### Testing
|
||||
|
||||
- test: cover non-finite results and color-group disjointness
|
||||
|
||||
## 0.7.0 - 2026-09-08
|
||||
|
||||
### Features
|
||||
|
||||
- feat: factorise the joint once with History::joint
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- chore: Release trueskill-tt version 0.7.0
|
||||
|
||||
### Other (unconventional)
|
||||
|
||||
- Merge branch 'feat/joint-handle'
|
||||
|
||||
## 0.6.0 - 2026-09-08
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- fix!: make the joint span slices, not just the latest one
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- chore: Release trueskill-tt version 0.6.0
|
||||
|
||||
## 0.5.0 - 2026-09-08
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- feat!: name the unknown key, expose tail probabilities, flag short fits
|
||||
- refactor!: remove the factor-graph surface nothing used, add try_winner
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- fix(test): the ingestion-order property was comparing two truncated fits
|
||||
|
||||
### Documentation
|
||||
|
||||
- docs: record that the event log is the source of truth, and why
|
||||
|
||||
### Features
|
||||
|
||||
- feat: add UnknownKeys::Prior, and explain why there is no Skip
|
||||
- feat: add History::posterior_of for a linear combination of competitors
|
||||
- feat: add History::predict_margin for scored matchups
|
||||
- feat: add expected_variance_reduction for scored active learning
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- chore: Release trueskill-tt version 0.5.0
|
||||
|
||||
### Styling
|
||||
|
||||
- style: factor the event-pair type out of the reconvergence fixture
|
||||
- style: use arrays rather than vec! in the calibration fixture
|
||||
|
||||
### Testing
|
||||
|
||||
- test: pin that re-convergence is path-independent
|
||||
- test: calibrate the marginals against the exact posterior
|
||||
- test: pin what an additive model does to combined uncertainty
|
||||
|
||||
## 0.4.2 - 2026-09-07
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- fix: replace the erfc approximation with libm, for free
|
||||
- fix: route every transcendental through libm, and combine sigmas with hypot
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- chore: Release trueskill-tt version 0.4.2
|
||||
|
||||
### Testing
|
||||
|
||||
- test: localise the erfc_inv tail residual to the caller's argument
|
||||
|
||||
## 0.4.1 - 2026-09-07
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- fix: correct erfc_inv's sign error and keep evidence in log space
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- chore: Release trueskill-tt version 0.4.1
|
||||
|
||||
### Testing
|
||||
|
||||
- test: pin quality()'s N-group closed form, closing the README cross-check
|
||||
|
||||
## 0.4.0 - 2026-09-07
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- feat!: N-team outcome prediction with draw mass, replacing the 2-team panic
|
||||
- refactor!: close the remaining API gaps from #21
|
||||
- fix!: apply competitor configuration whenever it is supplied
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- fix(release): skip the changelog hook during a dry run
|
||||
- fix: stop destroying tail precision in evidence and truncation
|
||||
- fix: reject convergence options that silently disable inference
|
||||
|
||||
### Documentation
|
||||
|
||||
- docs: correct drifted documentation and compile the README in CI
|
||||
|
||||
### Features
|
||||
|
||||
- feat: add expected information gain for active matchup selection
|
||||
- feat: let observers be shared, boxed, or borrowed
|
||||
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- chore: Release trueskill-tt version 0.4.0
|
||||
|
||||
## 0.3.0 - 2026-09-01
|
||||
|
||||
### Breaking Changes
|
||||
@@ -25,6 +177,7 @@ All notable changes to this project will be documented in this file.
|
||||
### Miscellaneous Tasks
|
||||
|
||||
- chore: ignore proptest regression seed files
|
||||
- chore: Release trueskill-tt version 0.3.0
|
||||
|
||||
### Performance
|
||||
|
||||
|
||||
@@ -24,6 +24,20 @@ is where several defects have hidden — a debug-only run is not evidence.
|
||||
- `approx` — `approx::AbsDiffEq` etc. for `Gaussian`. Most numerical goldens need it.
|
||||
- `rayon` — opt-in parallel within-slice sweep and per-slice query passes.
|
||||
|
||||
## Working rules
|
||||
|
||||
- **Investigate before implementing.** Measure the actual behaviour first —
|
||||
against an analytic reference where one exists. Several "obvious" fixes in
|
||||
this repo turned out to be wrong in sign or unnecessary, and the measurement
|
||||
is what caught them.
|
||||
- **Fix the root issue, not the symptom.** A clamp that hides an underflow, or
|
||||
a tolerance loosened to make a test pass, is a defect deferred.
|
||||
- **Scout crates.io before hand-rolling numerics.** Check accuracy against an
|
||||
independent reference rather than trusting downloads: `puruspe` has 1.4M
|
||||
downloads and is 346 ULP off in the tail, where `libm` is 1. Fewer
|
||||
dependencies is preferable, not mandatory — take the dependency when it is
|
||||
measurably better.
|
||||
|
||||
## Architecture
|
||||
|
||||
A Rust port of [TrueSkillThroughTime.py](https://github.com/glandfried/TrueSkillThroughTime.py):
|
||||
@@ -32,8 +46,17 @@ evidence both forward and backward across a history.
|
||||
|
||||
### Data flow
|
||||
|
||||
Ingestion (public types, `event.rs`):
|
||||
|
||||
```
|
||||
History → TimeSlice[] → Event[] → Team[] → Item[]
|
||||
Event<T, K> → Team<K>[] → Member<K>[]
|
||||
```
|
||||
|
||||
`History::add_events` flattens that into indices; teams survive only as
|
||||
grouping, not as a value. Inference then runs on the internal shapes:
|
||||
|
||||
```
|
||||
History → TimeSlice[] → Event[] → Item[]
|
||||
↓
|
||||
Game (factor graph) → Schedule → BuiltinFactor[]
|
||||
```
|
||||
@@ -45,30 +68,35 @@ History → TimeSlice[] → Event[] → Team[] → Item[]
|
||||
- **`TimeSlice`** (`time_slice.rs`) — all events at one time. Owns a
|
||||
`SkillStore` and a `ScratchArena`; `iteration()` sweeps its events, using
|
||||
`ColorGroups` to partition independent ones.
|
||||
- **`Event`** (`time_slice.rs`) — one match. `compute()` runs inference reading
|
||||
skills immutably; `apply()` folds the result back. The split is what lets a
|
||||
color group run in parallel with no `unsafe`.
|
||||
- **`Event`** — two distinct types, do not confuse them. The *public* ingestion
|
||||
`Event<T, K>` is in `event.rs` (with `Team`/`Member`); the *internal*
|
||||
`pub(crate) Event` in `time_slice.rs` is one match during inference, where
|
||||
`compute()` runs inference reading skills immutably and `apply()` folds the
|
||||
result back. That split is what lets a color group run in parallel with no
|
||||
`unsafe`.
|
||||
- **`Game`** (`game.rs`) — a single match's factor graph. `run_chain` builds the
|
||||
diff chain between rank-adjacent teams and drives it to convergence.
|
||||
- **`Gaussian`** (`gaussian.rs`) — natural parameters (`pi = 1/sigma²`,
|
||||
`tau = mu/sigma²`). `Mul`/`Div` are the EP product/cavity: pure adds and
|
||||
subtracts. Variance-space ops (`Add`, `Sub`, `exclude`, `forget`) go through
|
||||
`from_mv`/`variance()` and take no square root.
|
||||
- **`factor/`** — `TeamSumFactor`, `RankDiffFactor`, `TruncFactor` (ranked),
|
||||
`MarginFactor` (scored), over a flat `VarStore`. `BuiltinFactor` dispatches
|
||||
by enum rather than `dyn`.
|
||||
- **`Schedule`** (`schedule.rs`) — drives factor propagation. `EpsilonOrMax` is
|
||||
the only implementation.
|
||||
- **`factor/`** — `TruncFactor` (ranked) and `MarginFactor` (scored) over a
|
||||
flat `VarStore`. `Game::run_chain` drives them directly through a local
|
||||
`DiffFactor` enum; there is no `Schedule` indirection and no generic `Factor`
|
||||
trait. Both were removed once measurement showed nothing had ever used them
|
||||
— see #42.
|
||||
- **`Competitor`** (`competitor.rs`) — per-history temporal state (`message`,
|
||||
`last_time`). **`Rating`** (`rating.rs`) — static config (prior, `beta`, drift).
|
||||
- **`storage/`** — `SkillStore` (per slice) and `CompetitorStore` (per history),
|
||||
both dense `Vec`s indexed by `Index`.
|
||||
- **`storage/`** — `SkillStore` (per slice, `pub(crate)`) and `CompetitorStore`
|
||||
(per history, public), both indexed by `Index`. The module is `pub`, but only
|
||||
`CompetitorStore` is reachable from outside the crate.
|
||||
- **`KeyTable`** (`key_table.rs`) — user key ↔ `Index`, both directions O(1).
|
||||
- **`Drift`** (`drift.rs`) / **`Time`** (`time.rs`) — traits. `Time` is a *trait*
|
||||
(`i64`, `Untimed`), not an enum.
|
||||
- **`lib.rs`** — public exports, global defaults (`MU`, `SIGMA`, `BETA`,
|
||||
`GAMMA`, `P_DRAW`, `EPSILON`, `ITERATIONS`), and the standalone `quality()`,
|
||||
`cdf()`, `erfc()`.
|
||||
`GAMMA`, `P_DRAW`, `EPSILON`, `ITERATIONS`), and the standalone `quality()`.
|
||||
The `cdf()` / `erfc()` helpers live here too but are `pub(crate)` and private
|
||||
respectively — not public API.
|
||||
|
||||
### Invariants worth knowing
|
||||
|
||||
@@ -83,6 +111,12 @@ History → TimeSlice[] → Event[] → Team[] → Item[]
|
||||
chain underflows to zero, and `ln(0)` is `-inf`.
|
||||
- **Colors are contiguous.** `recompute_color_groups` reorders events so each
|
||||
color occupies one range; `ColorGroups::groups_are_contiguous` asserts it.
|
||||
- **Transcendentals go through `libm`, not `std`.** IEEE 754 pins the basic
|
||||
operations and `sqrt` but says nothing about `exp`/`log`/`erf`, and `std`
|
||||
delegates to the *system* math library — measured, `f64::exp` and `libm::exp`
|
||||
disagree on 9.7% of inputs by one ULP. Since inference is an iterative fixed
|
||||
point, one ULP can change an iteration count. Use `libm::exp` / `libm::log` in
|
||||
inference code; `f64::sqrt` is fine (IEEE specifies it). Tests may use either.
|
||||
- **The crate is `#![forbid(unsafe_code)]`.** Keep it that way.
|
||||
- **Ingestion order must not change the answer.** Events added one at a time
|
||||
must converge to the same fixed point as the same events batched — see
|
||||
|
||||
+6
-5
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "trueskill-tt"
|
||||
version = "0.3.0"
|
||||
version = "0.8.0"
|
||||
edition = "2024"
|
||||
rust-version = "1.85"
|
||||
description = "TrueSkill Through Time: Bayesian skill rating that tracks how skill evolves over time, via Gaussian message passing"
|
||||
@@ -33,10 +33,6 @@ bench = false
|
||||
name = "batch"
|
||||
harness = false
|
||||
|
||||
[[bench]]
|
||||
name = "gaussian"
|
||||
harness = false
|
||||
|
||||
[[bench]]
|
||||
name = "history_converge"
|
||||
harness = false
|
||||
@@ -51,6 +47,7 @@ harness = false
|
||||
|
||||
[dependencies]
|
||||
approx = { version = "0.5.1", optional = true }
|
||||
libm = "0.2.16"
|
||||
rayon = { version = "1", optional = true }
|
||||
smallvec = "1"
|
||||
|
||||
@@ -78,3 +75,7 @@ debug = true
|
||||
|
||||
[profile.dev]
|
||||
debug = true
|
||||
|
||||
[[bench]]
|
||||
name = "joint"
|
||||
harness = false
|
||||
|
||||
@@ -1,82 +1,238 @@
|
||||
# TrueSkill - Through Time
|
||||
|
||||
Rust port of [TrueSkillThroughTime.py](https://github.com/glandfried/TrueSkillThroughTime.py).
|
||||
Bayesian skill rating over a time axis.
|
||||
|
||||
## Other implementations
|
||||
Where plain TrueSkill gives each competitor one running estimate, TrueSkill
|
||||
Through Time treats a whole history as a single model and infers skill *at every
|
||||
point in time*. Evidence flows both directions: a result today sharpens the
|
||||
estimate of who someone was last year, so early estimates stop being frozen
|
||||
guesses and comparisons across eras become meaningful.
|
||||
|
||||
- [ttt-scala](https://github.com/ankurdave/ttt-scala)
|
||||
- [ChessAnalysis #F](https://github.com/lucasmaystre/ChessAnalysis)
|
||||
- [TrueSkillThroughTime.jl](https://github.com/glandfried/TrueSkillThroughTime.jl)
|
||||
- [TrueSkillThroughTime.R](https://github.com/glandfried/TrueSkillThroughTime.R)
|
||||
- [TrueSkill Through Time: Revisiting the History of Chess](https://www.microsoft.com/en-us/research/wp-content/uploads/2008/01/NIPS2007_0931.pdf)
|
||||
- [TrueSkill Through Time. The full scientific documentation](https://glandfried.github.io/publication/landfried2021-learning/)
|
||||
A Rust port of
|
||||
[TrueSkillThroughTime.py](https://github.com/glandfried/TrueSkillThroughTime.py).
|
||||
|
||||
## Install
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
trueskill-tt = "0.8"
|
||||
```
|
||||
|
||||
Optional features, both off by default:
|
||||
|
||||
- `approx` — `approx`'s equality traits for `Gaussian`. Useful in tests.
|
||||
- `rayon` — parallelises the within-slice sweep and the per-slice passes of
|
||||
`learning_curves` / `log_evidence`. Results stay bit-identical regardless of
|
||||
worker count; `just determinism` asserts it at 1, 2, 4 and 8 threads.
|
||||
|
||||
## Quickstart
|
||||
|
||||
Record results, converge, then read off skills.
|
||||
|
||||
```rust
|
||||
use trueskill_tt::History;
|
||||
|
||||
let mut history = History::default();
|
||||
|
||||
history.record_winner(&"alice", &"bob", 1)?;
|
||||
history.record_winner(&"bob", &"carol", 2)?;
|
||||
history.record_winner(&"alice", &"carol", 3)?;
|
||||
|
||||
history.converge()?;
|
||||
|
||||
let alice = history.current_skill("alice").unwrap();
|
||||
assert!(alice.mu() > 0.0, "alice won every game she played");
|
||||
# Ok::<(), trueskill_tt::InferenceError>(())
|
||||
```
|
||||
|
||||
The third argument is the time. It is what makes this Through Time rather than
|
||||
plain TrueSkill: skill is inferred at each of those moments, not once at the
|
||||
end. `learning_curve` reads the whole trajectory back.
|
||||
|
||||
```rust
|
||||
# use trueskill_tt::History;
|
||||
# let mut history = History::default();
|
||||
# history.record_winner(&"alice", &"bob", 1)?;
|
||||
# history.record_winner(&"bob", &"carol", 2)?;
|
||||
# history.record_winner(&"alice", &"carol", 3)?;
|
||||
# history.converge()?;
|
||||
// `None` means the key is unknown; `Some(vec![])` means known but unplayed.
|
||||
let curve = history.learning_curve("alice").unwrap();
|
||||
for (time, skill) in &curve {
|
||||
println!("t={time}: {:.2} ± {:.2}", skill.mu(), skill.sigma());
|
||||
}
|
||||
|
||||
// Everyone's latest posterior in one pass — the leaderboard query.
|
||||
let latest = history.current_skills();
|
||||
assert_eq!(latest.len(), 3);
|
||||
# Ok::<(), trueskill_tt::InferenceError>(())
|
||||
```
|
||||
|
||||
## Teams, rankings and draws
|
||||
|
||||
Anything beyond one-versus-one goes through the fluent event builder. An event
|
||||
is only recorded by the terminal `.commit()`.
|
||||
|
||||
```rust
|
||||
use trueskill_tt::History;
|
||||
|
||||
let mut history = History::builder().p_draw(0.1).build();
|
||||
|
||||
history
|
||||
.event(1)
|
||||
.team(["alice", "bob"])
|
||||
.team(["carol", "dave"])
|
||||
.ranking([0, 1]) // lower is better; equal values are a tie
|
||||
.commit()?;
|
||||
|
||||
history.converge()?;
|
||||
# Ok::<(), trueskill_tt::InferenceError>(())
|
||||
```
|
||||
|
||||
**A tie needs a positive `p_draw`.** A `p_draw` of zero asserts draws cannot
|
||||
happen, so a tied result has no representable likelihood and is rejected rather
|
||||
than fitted to something else:
|
||||
|
||||
```rust
|
||||
use trueskill_tt::{History, InferenceError};
|
||||
|
||||
let mut history = History::default(); // p_draw defaults to 0.0
|
||||
let err = history.record_draw(&"alice", &"bob", 1).unwrap_err();
|
||||
assert!(matches!(err, InferenceError::TieWithoutDrawProbability { .. }));
|
||||
```
|
||||
|
||||
This also catches `Outcome::winner(w, n)` for three or more teams, which ties
|
||||
every loser.
|
||||
|
||||
## Which entry point?
|
||||
|
||||
| You want to | Use |
|
||||
|---|---|
|
||||
| One match, two competitors | `record_winner` / `record_draw` |
|
||||
| Teams, explicit ranks, scores, per-member weights | `history.event(t)…commit()` |
|
||||
| A batch you already have as values | `add_events(iter)` |
|
||||
| Score a hypothetical with no history at all | `Game` |
|
||||
|
||||
`Game` is the odd one out and worth being explicit about: it is a single match's
|
||||
factor graph, it does not participate in a `History`, and nothing it computes is
|
||||
remembered. Reach for it to evaluate a matchup in isolation; reach for `History`
|
||||
for everything that accumulates.
|
||||
|
||||
## `converge` is strict
|
||||
|
||||
`converge` returns `Err(NotConverged)` if the sweep hits `max_iter` with the
|
||||
step still above `epsilon`, and `Err(NonFiniteResult)` if a sweep produces NaN.
|
||||
|
||||
It used to return `Ok` with `converged: false`, which was the worst available
|
||||
shape. A fit that stops short is *wrong by a little*: every posterior is finite,
|
||||
the ordering looks sensible, and nothing about the output says the numbers were
|
||||
still moving. Detection was opt-in, and `let _ = h.converge()` silently opted
|
||||
out — which is how a real defect hid in this crate's own test suite.
|
||||
|
||||
The default `max_iter` is high enough that reaching it means something is
|
||||
genuinely wrong rather than that the history is large; the loop exits at
|
||||
`epsilon` long before, so raising the cap costs nothing when it is not needed.
|
||||
Use `converge_partial` when a deliberately capped, unconverged fit is the point.
|
||||
|
||||
Predictions are strict for the same reason: every `predict_*` method reads
|
||||
skills through one gate that refuses a NaN-poisoned fit, rather than returning a
|
||||
plausible number computed from it.
|
||||
|
||||
## Drift
|
||||
|
||||
Skill drift models how a player's true skill can change between appearances. Each time a player reappears after a gap, their skill uncertainty is widened by the drift model before the new evidence is incorporated.
|
||||
Skill drift models how a competitor's true skill can change between appearances.
|
||||
Each time they reappear after a gap, their skill uncertainty is widened by the
|
||||
drift model before the new evidence is incorporated.
|
||||
|
||||
Drift is represented by the `Drift` trait:
|
||||
Drift is represented by the `Drift` trait (`src/drift.rs`), generic over the
|
||||
history's time type:
|
||||
|
||||
```rust
|
||||
pub trait Drift: Copy + Debug {
|
||||
fn variance_delta(&self, elapsed: i64) -> f64;
|
||||
```text
|
||||
pub trait Drift<T: Time>: Copy + Debug + Send + Sync {
|
||||
fn variance_delta(&self, from: &T, to: &T) -> f64;
|
||||
fn variance_for_elapsed(&self, elapsed: i64) -> f64;
|
||||
}
|
||||
```
|
||||
|
||||
`variance_delta` returns the amount to add to `σ²` given the elapsed time since the player last played. Internally, `Gaussian::forget` uses this to compute the new sigma: `σ_new = sqrt(σ² + variance_delta)`.
|
||||
Both methods return the amount to add to `σ²`, not to `σ`. `variance_delta`
|
||||
works from two timestamps; `variance_for_elapsed` takes an already-computed
|
||||
elapsed count, and is used on the paths that cache it. `Gaussian::forget`
|
||||
applies the result entirely in variance space — `from_mv(mu, variance() +
|
||||
variance_delta)` — taking no square root.
|
||||
|
||||
That block is a quotation rather than a doctest. The custom-drift example below
|
||||
is compiled by CI, so it is what actually pins the signature.
|
||||
|
||||
### ConstantDrift
|
||||
|
||||
The built-in `ConstantDrift` implements a linear random walk — skill uncertainty grows proportionally to time:
|
||||
The built-in `ConstantDrift` implements a linear random walk — skill uncertainty
|
||||
grows proportionally to time:
|
||||
|
||||
```
|
||||
```text
|
||||
variance_delta = elapsed * γ²
|
||||
```
|
||||
|
||||
This is the standard TrueSkill Through Time model. Use it by passing a `ConstantDrift(gamma)` when constructing a `Player`:
|
||||
This is the standard TrueSkill Through Time model. Pass a `ConstantDrift::new(gamma)`
|
||||
when constructing a `Rating`:
|
||||
|
||||
```rust
|
||||
use trueskill_tt::{Player, Gaussian, drift::ConstantDrift};
|
||||
use trueskill_tt::{ConstantDrift, Gaussian, Rating};
|
||||
|
||||
// gamma = 0.1 means skill can shift ~0.1 per time unit
|
||||
let player = Player::new(Gaussian::from_ms(0.0, 6.0), 1.0, ConstantDrift(0.1));
|
||||
// gamma = 0.1 means skill can shift ~0.1 per time unit.
|
||||
let rating: Rating<i64, ConstantDrift> =
|
||||
Rating::new(Gaussian::from_ms(0.0, 6.0), 1.0, ConstantDrift::new(0.1));
|
||||
|
||||
assert_eq!(rating.drift().gamma(), 0.1);
|
||||
```
|
||||
|
||||
The type annotation is load-bearing: `ConstantDrift` implements `Drift<T>` for
|
||||
every `T: Time`, so without it `T` is ambiguous.
|
||||
|
||||
### Custom drift
|
||||
|
||||
Implement `Drift` to express any other model. For example, a drift that saturates after a long absence (uncertainty grows with the square root of elapsed time instead of linearly):
|
||||
Implement `Drift<T>` to express any other model. For example, a drift that
|
||||
saturates after a long absence, with uncertainty growing as the square root of
|
||||
elapsed time instead of linearly:
|
||||
|
||||
```rust
|
||||
use trueskill_tt::drift::Drift;
|
||||
use trueskill_tt::{Drift, Gaussian, History, Rating, Time};
|
||||
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
struct SqrtDrift {
|
||||
gamma: f64,
|
||||
}
|
||||
|
||||
impl Drift for SqrtDrift {
|
||||
fn variance_delta(&self, elapsed: i64) -> f64 {
|
||||
(elapsed as f64).sqrt() * self.gamma * self.gamma
|
||||
impl<T: Time> Drift<T> for SqrtDrift {
|
||||
fn variance_delta(&self, from: &T, to: &T) -> f64 {
|
||||
let elapsed = from.elapsed_to(to).max(0) as f64;
|
||||
elapsed.sqrt() * self.gamma * self.gamma
|
||||
}
|
||||
|
||||
fn variance_for_elapsed(&self, elapsed: i64) -> f64 {
|
||||
(elapsed.max(0) as f64).sqrt() * self.gamma * self.gamma
|
||||
}
|
||||
}
|
||||
|
||||
let player = Player::new(Gaussian::from_ms(0.0, 6.0), 1.0, SqrtDrift { gamma: 0.5 });
|
||||
// On a single Rating:
|
||||
let rating: Rating<i64, SqrtDrift> =
|
||||
Rating::new(Gaussian::from_ms(0.0, 6.0), 1.0, SqrtDrift { gamma: 0.5 });
|
||||
|
||||
// Or for a whole History, via the builder:
|
||||
let history = History::builder().drift(SqrtDrift { gamma: 0.5 }).build();
|
||||
|
||||
assert_eq!(rating.beta(), 1.0);
|
||||
assert_eq!(history.log_evidence(), 0.0);
|
||||
```
|
||||
|
||||
To use a custom drift type with `History`, use the `.drift()` builder method instead of `.gamma()`:
|
||||
|
||||
```rust
|
||||
let h = History::builder()
|
||||
.drift(SqrtDrift { gamma: 0.5 })
|
||||
.build();
|
||||
```
|
||||
`HistoryBuilder::drift` is the only way to set a history's drift model; there is
|
||||
no `gamma()` shorthand. The default is `ConstantDrift::new(GAMMA)`.
|
||||
|
||||
### Per-competitor drift
|
||||
|
||||
A `History` has one drift model, but individual competitors can scale it.
|
||||
`Member::with_drift_scale(s)` multiplies the drift *variance* that competitor
|
||||
accumulates, so `s` is in the same units as `gamma`: `ConstantDrift(g)` at
|
||||
scale `s` behaves exactly as `ConstantDrift(g * s)` would, for that competitor
|
||||
accumulates, so `s` is in the same units as `gamma`: `ConstantDrift::new(g)` at
|
||||
scale `s` behaves exactly as `ConstantDrift::new(g * s)` would, for that competitor
|
||||
alone.
|
||||
|
||||
`0.0` pins a competitor still. That is what makes a **fixed reference point**
|
||||
@@ -84,22 +240,41 @@ expressible in the same graph as moving competitors — a bot at a known
|
||||
strength, a rating floor, a course difficulty:
|
||||
|
||||
```rust
|
||||
let events = vec![Event {
|
||||
use trueskill_tt::{ConstantDrift, Event, History, Member, Outcome, Team};
|
||||
|
||||
let mut h = History::builder().drift(ConstantDrift::new(0.1)).build();
|
||||
|
||||
h.add_events(vec![Event {
|
||||
time: 0,
|
||||
teams: smallvec![
|
||||
teams: [
|
||||
Team::with_members([Member::new("player")]),
|
||||
// A course does not improve. Pin it, and the round's evidence
|
||||
// lands on the player instead of being split between the two.
|
||||
Team::with_members([Member::new("layout_7").with_drift_scale(0.0)]),
|
||||
],
|
||||
]
|
||||
.into_iter()
|
||||
.collect(),
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}];
|
||||
}])
|
||||
.unwrap();
|
||||
|
||||
h.converge().unwrap();
|
||||
```
|
||||
|
||||
Like `with_prior`, the scale is **competitor configuration captured at first
|
||||
appearance** — setting it on a key the history already knows has no effect. It
|
||||
must be finite and non-negative; ingestion otherwise fails with
|
||||
`InferenceError::InvalidParameter`.
|
||||
Like `with_prior`, the scale is **competitor configuration, not a per-event
|
||||
value**: it applies to the competitor for the whole history, and it applies
|
||||
whenever it is supplied — including on a key the history already knows.
|
||||
Configuring one late still refits the whole history rather than taking effect
|
||||
only from that event onward, because `converge` refits from competitor state.
|
||||
Repeating the same value is inert; supplying two *different* values for one
|
||||
competitor within a single batch is `InferenceError::ConflictingCompetitorConfig`,
|
||||
since events in a batch have no order. The scale must be finite and
|
||||
non-negative; ingestion otherwise fails with `InferenceError::InvalidParameter`.
|
||||
|
||||
The fluent `EventBuilder` reaches this too: `.team([...])` is the common case
|
||||
and leaves both unset, while `.members([...])` takes `Member` values directly,
|
||||
so `h.event(t).members([Member::new("layout_7").with_drift_scale(0.0)])` is
|
||||
equivalent to the typed shape above.
|
||||
|
||||
## Scored outcomes
|
||||
|
||||
@@ -110,7 +285,7 @@ soft Gaussian evidence about the latent performance diff. Configure
|
||||
(smaller σ = more trust).
|
||||
|
||||
```rust
|
||||
use trueskill_tt::{History, Outcome};
|
||||
use trueskill_tt::History;
|
||||
|
||||
let mut h = History::builder().score_sigma(2.0).build();
|
||||
h.event(1)
|
||||
@@ -122,15 +297,129 @@ h.event(1)
|
||||
h.converge().unwrap();
|
||||
```
|
||||
|
||||
## Todo
|
||||
## Prediction
|
||||
|
||||
- [x] Implement approx for Gaussian
|
||||
- [x] Add more tests from `TrueSkillThroughTime.jl`
|
||||
- [x] Generalise a time axis — `Time` is now a trait (`Untimed`, `i64`), not an enum
|
||||
- [x] Add examples (`examples/atp.rs`, `examples/scored.rs`)
|
||||
- [x] Add Observer (`Observer` / `NullObserver`)
|
||||
- [x] Benchmark the inference loop (`benches/batch.rs`, `benches/history_converge.rs`, `benches/ingest.rs`)
|
||||
- [ ] Cross-check `quality()` against [sublee/trueskill](https://github.com/sublee/trueskill/tree/master) — N-group support works and is covered by invariants, but no reference values are asserted
|
||||
`predict_outcome` gives the full distribution over finishing orders. Each entry
|
||||
is a rank vector in the same shape `Outcome::ranking` takes — equal ranks mean a
|
||||
tie — so an outcome feeds straight back into inference.
|
||||
|
||||
```rust
|
||||
use trueskill_tt::History;
|
||||
|
||||
let mut h = History::builder().p_draw(0.1).build();
|
||||
h.record_winner(&"alice", &"bob", 1).unwrap();
|
||||
h.converge().unwrap();
|
||||
|
||||
let p = h.predict_outcome(&[&[&"alice"], &[&"bob"]]).unwrap();
|
||||
|
||||
// Probabilities are exhaustive and disjoint, so they sum to one.
|
||||
assert!((p.total() - 1.0).abs() < 1e-6);
|
||||
|
||||
let (best, likelihood) = p.most_likely().unwrap();
|
||||
println!("most likely: {best:?} at {likelihood:.3}");
|
||||
println!("draw: {:.3}", p.probability_of(&[0, 0]));
|
||||
```
|
||||
|
||||
Supports any number of teams. Because the outcome space grows factorially, the
|
||||
full distribution is capped at `MAX_PREDICTED_TEAMS`; two cheaper entry points
|
||||
stay available at any size:
|
||||
|
||||
- `predict_win_probabilities(teams)` — `P(team i finishes strictly first)`,
|
||||
quadratic in team count.
|
||||
- `predict_ranking(teams, ranks)` — one specific finishing order.
|
||||
|
||||
Unknown keys are an error by default, not a silent omission: a team the history
|
||||
has never seen cannot produce a confident-looking probability. The error names
|
||||
the key, and every key must already be known — pre-filter with
|
||||
`current_skill` if your caller cannot guarantee that.
|
||||
|
||||
If predicting for competitors you have never seen is the point rather than a
|
||||
mistake, say so once:
|
||||
|
||||
```rust
|
||||
use trueskill_tt::{History, UnknownKeys};
|
||||
|
||||
let h = History::builder().unknown_keys(UnknownKeys::Prior).build();
|
||||
```
|
||||
|
||||
An unknown competitor is then answered from the configured prior, which is the
|
||||
honest reading — you have no evidence about them — and correctly *widens* a team
|
||||
that contains one. There is deliberately no "skip the member" mode: a team's
|
||||
performance is the sum of its members, so dropping one would make the model more
|
||||
certain because it knows less.
|
||||
|
||||
### Asking about one competitor
|
||||
|
||||
`Gaussian` answers tail questions directly, which is what a stopping rule needs:
|
||||
|
||||
```rust
|
||||
use trueskill_tt::History;
|
||||
|
||||
let mut h = History::default();
|
||||
h.record_winner(&"alice", &"bob", 1).unwrap();
|
||||
h.converge().unwrap();
|
||||
|
||||
let skill = h.current_skill("alice").unwrap();
|
||||
|
||||
// "How sure am I that this is below the cutoff?" — a probability, not a
|
||||
// `mu + z * sigma` band whose confidence drifts as sigma changes.
|
||||
let _ = skill.probability_below(20.0);
|
||||
|
||||
// Use this rather than `1.0 - probability_below(x)`: the complement cancels
|
||||
// away every digit in the upper tail, which is where a stopping rule lives.
|
||||
let _ = skill.probability_above(30.0);
|
||||
```
|
||||
|
||||
## Which match to play next
|
||||
|
||||
`quality()` measures whether a matchup is *fair*. That is not the same as
|
||||
whether it is *informative*, and the two only coincide for two evenly matched
|
||||
competitors. When each observation costs something, ask
|
||||
`expected_information_gain` instead — the outcome-weighted divergence between
|
||||
what you believe now and what you would believe afterwards.
|
||||
|
||||
```rust
|
||||
use trueskill_tt::History;
|
||||
|
||||
let mut h = History::default();
|
||||
for t in 1..=10 {
|
||||
h.record_winner(&"veteran", &"regular", t).unwrap();
|
||||
h.record_winner(&"regular", &"veteran", t + 100).unwrap();
|
||||
}
|
||||
h.record_winner(&"veteran", &"newcomer", 500).unwrap();
|
||||
h.converge().unwrap();
|
||||
|
||||
let settled = h.expected_information_gain(&[&[&"veteran"], &[&"regular"]]).unwrap();
|
||||
let unknown = h.expected_information_gain(&[&[&"veteran"], &[&"newcomer"]]).unwrap();
|
||||
|
||||
// Playing the newcomer teaches you more than replaying a settled rivalry.
|
||||
assert!(unknown > settled);
|
||||
```
|
||||
|
||||
The result is in nats, and is bounded by the entropy of the outcome: at most
|
||||
`ln 2 ≈ 0.693` for a two-way result, `ln 3` once draws are possible, `ln k` for
|
||||
`k` outcomes. A value near zero means you already know how it ends.
|
||||
|
||||
This costs one full inference pass **per possible outcome**, so it is far more
|
||||
expensive than `quality()`. Scoring every pairing among `n` competitors is
|
||||
`O(n² × outcomes)` passes — shortlist with `quality()` or
|
||||
`predict_win_probabilities` first, then score only the shortlist.
|
||||
|
||||
## Other implementations
|
||||
|
||||
- [ttt-scala](https://github.com/ankurdave/ttt-scala)
|
||||
- [ChessAnalysis #F](https://github.com/lucasmaystre/ChessAnalysis)
|
||||
- [TrueSkillThroughTime.jl](https://github.com/glandfried/TrueSkillThroughTime.jl)
|
||||
- [TrueSkillThroughTime.R](https://github.com/glandfried/TrueSkillThroughTime.R)
|
||||
- [TrueSkill Through Time: Revisiting the History of Chess](https://www.microsoft.com/en-us/research/wp-content/uploads/2008/01/NIPS2007_0931.pdf)
|
||||
- [TrueSkill Through Time. The full scientific documentation](https://glandfried.github.io/publication/landfried2021-learning/)
|
||||
|
||||
## Status
|
||||
|
||||
Every box on the old todo list is ticked, so it has been retired; open work
|
||||
lives in the issue tracker instead. The crate is in use and the API is still
|
||||
moving — breaking changes are batched into minor releases rather than dribbled
|
||||
out, and `CHANGELOG.md` records them.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
+45
-35
@@ -1,45 +1,55 @@
|
||||
//! One slice's event sweep.
|
||||
//!
|
||||
//! Written against the public API rather than against `TimeSlice` directly.
|
||||
//! It used to reach for `TimeSlice`, `KeyTable`, `CompetitorStore`,
|
||||
//! `Competitor` and `EventKind`, and was the *only* thing outside `src/`
|
||||
//! that did — so a benchmark was dictating five public types that no test,
|
||||
//! example or consumer could otherwise obtain.
|
||||
//!
|
||||
//! A single-slice history's `converge` calls exactly the same per-slice sweep,
|
||||
//! so capping at one iteration measures the same code path.
|
||||
|
||||
use criterion::{Criterion, criterion_group, criterion_main};
|
||||
use trueskill_tt::{
|
||||
BETA, Competitor, ConvergenceOptions, EventKind, GAMMA, KeyTable, MU, P_DRAW, Rating, SIGMA,
|
||||
TimeSlice, drift::ConstantDrift, gaussian::Gaussian, storage::CompetitorStore,
|
||||
};
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{ConstantDrift, ConvergenceOptions, Event, History, Member, Outcome, Team};
|
||||
|
||||
fn criterion_benchmark(criterion: &mut Criterion) {
|
||||
let mut index_map = KeyTable::new();
|
||||
let build = || {
|
||||
let mut h = History::builder()
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 1,
|
||||
epsilon: 0.0,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.build();
|
||||
|
||||
let a = index_map.get_or_create("a");
|
||||
let b = index_map.get_or_create("b");
|
||||
let c = index_map.get_or_create("c");
|
||||
// 100 events, all at one time, so the history has a single slice.
|
||||
let events: Vec<Event<i64, &'static str>> = (0..100)
|
||||
.map(|_| Event {
|
||||
time: 1,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("a")]),
|
||||
Team::with_members([Member::new("b")]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
})
|
||||
.collect();
|
||||
h.add_events(events).expect("fixture ingests");
|
||||
h
|
||||
};
|
||||
|
||||
let mut agents: CompetitorStore<i64, ConstantDrift> = CompetitorStore::new();
|
||||
|
||||
for agent in [a, b, c] {
|
||||
agents.insert(
|
||||
agent,
|
||||
Competitor {
|
||||
rating: Rating::new(Gaussian::from_ms(MU, SIGMA), BETA, ConstantDrift(GAMMA)),
|
||||
..Default::default()
|
||||
criterion.bench_function("slice_sweep_100_events", |b| {
|
||||
b.iter_batched(
|
||||
build,
|
||||
|mut h| {
|
||||
// `converge_partial`, not `converge`: one iteration is
|
||||
// deliberately short of convergence and `converge` reports that
|
||||
// as an error.
|
||||
let _ = h.converge_partial();
|
||||
},
|
||||
criterion::BatchSize::SmallInput,
|
||||
);
|
||||
}
|
||||
|
||||
let mut composition = Vec::new();
|
||||
let mut results = Vec::new();
|
||||
let mut weights = Vec::new();
|
||||
|
||||
for _ in 0..100 {
|
||||
composition.push(vec![vec![a], vec![b]]);
|
||||
results.push(vec![1.0, 0.0]);
|
||||
weights.push(vec![vec![1.0], vec![1.0]]);
|
||||
}
|
||||
|
||||
let kinds = vec![EventKind::Ranked; composition.len()];
|
||||
|
||||
let mut time_slice = TimeSlice::new(1, P_DRAW, ConvergenceOptions::default());
|
||||
time_slice.add_events(composition, Some(results), Some(weights), kinds, &agents);
|
||||
|
||||
criterion.bench_function("Batch::iteration", |b| {
|
||||
b.iter(|| time_slice.iteration(0, &agents))
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -1,53 +0,0 @@
|
||||
use criterion::{Criterion, criterion_group, criterion_main};
|
||||
use trueskill_tt::gaussian::Gaussian;
|
||||
|
||||
fn benchmark_gaussian_arithmetic(criterion: &mut Criterion) {
|
||||
// Define test Gaussians
|
||||
let g1 = Gaussian::from_ms(25.0, 25.0 / 3.0);
|
||||
let g2 = Gaussian::from_ms(0.0, 1.0);
|
||||
let g3 = Gaussian::from_ms(1.0, 1.0);
|
||||
|
||||
// Benchmark addition
|
||||
criterion.bench_function("Gaussian::add", |bencher| {
|
||||
bencher.iter(|| g1 + g2);
|
||||
});
|
||||
|
||||
// Benchmark subtraction
|
||||
criterion.bench_function("Gaussian::sub", |bencher| {
|
||||
bencher.iter(|| g1 - g3);
|
||||
});
|
||||
|
||||
// Benchmark multiplication
|
||||
criterion.bench_function("Gaussian::mul", |bencher| {
|
||||
bencher.iter(|| g1 * g2);
|
||||
});
|
||||
|
||||
// Benchmark division
|
||||
// NOTE: numerator must have higher precision (smaller sigma) than the
|
||||
// denominator in this representation; g2 (sigma=1) / g1 (sigma=8.33) is
|
||||
// well-defined, whereas g1 / g2 underflows and panics in mu_sigma.
|
||||
criterion.bench_function("Gaussian::div", |bencher| {
|
||||
bencher.iter(|| g2 / g1);
|
||||
});
|
||||
|
||||
// Benchmark natural parameter conversions
|
||||
criterion.bench_function("Gaussian::pi", |bencher| {
|
||||
bencher.iter(|| g1.pi());
|
||||
});
|
||||
|
||||
criterion.bench_function("Gaussian::tau", |bencher| {
|
||||
bencher.iter(|| g1.tau());
|
||||
});
|
||||
|
||||
// Benchmark combined pi/tau operations (used in mul/div)
|
||||
criterion.bench_function("Gaussian::pi_tau_combined", |bencher| {
|
||||
bencher.iter(|| {
|
||||
let pi = g1.pi();
|
||||
let tau = g1.tau();
|
||||
(pi, tau)
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
criterion_group!(benches, benchmark_gaussian_arithmetic);
|
||||
criterion_main!(benches);
|
||||
@@ -43,11 +43,12 @@ fn build_history_1v1(
|
||||
rng
|
||||
};
|
||||
|
||||
let mut h = History::<i64, _, _, String>::builder_with_key()
|
||||
let mut h = History::builder()
|
||||
.key_type::<String>()
|
||||
.mu(25.0)
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.drift(ConstantDrift(25.0 / 300.0))
|
||||
.drift(ConstantDrift::new(25.0 / 300.0))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 30,
|
||||
epsilon: 1e-6,
|
||||
@@ -82,7 +83,7 @@ fn bench_converge(c: &mut Criterion) {
|
||||
b.iter_batched(
|
||||
|| build_history_1v1(500, 100, 10, 42),
|
||||
|mut h| {
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
},
|
||||
BatchSize::SmallInput,
|
||||
);
|
||||
@@ -92,7 +93,7 @@ fn bench_converge(c: &mut Criterion) {
|
||||
b.iter_batched(
|
||||
|| build_history_1v1(2000, 200, 20, 42),
|
||||
|mut h| {
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
},
|
||||
BatchSize::SmallInput,
|
||||
);
|
||||
@@ -106,7 +107,7 @@ fn bench_converge(c: &mut Criterion) {
|
||||
b.iter_batched(
|
||||
|| build_history_1v1(5000, 50000, 5000, 42),
|
||||
|mut h| {
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
},
|
||||
BatchSize::SmallInput,
|
||||
);
|
||||
|
||||
+4
-2
@@ -32,7 +32,8 @@ fn bench_ingest(c: &mut Criterion) {
|
||||
b.iter_batched(
|
||||
|| events(n, 0),
|
||||
|evs| {
|
||||
let mut h: History<i64, _, _, String> = History::builder_with_key().build();
|
||||
let mut h: History<i64, _, _, String> =
|
||||
History::builder().key_type::<String>().build();
|
||||
for ev in evs {
|
||||
h.add_events(std::iter::once(ev)).unwrap();
|
||||
}
|
||||
@@ -46,7 +47,8 @@ fn bench_ingest(c: &mut Criterion) {
|
||||
b.iter_batched(
|
||||
|| events(n, 0),
|
||||
|evs| {
|
||||
let mut h: History<i64, _, _, String> = History::builder_with_key().build();
|
||||
let mut h: History<i64, _, _, String> =
|
||||
History::builder().key_type::<String>().build();
|
||||
h.add_events(evs).unwrap();
|
||||
black_box(h.time_slices_len())
|
||||
},
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
//! Cost of the joint posterior: factorising versus querying.
|
||||
//!
|
||||
//! The split is the whole point of `History::joint`. Factorising is `O(n^3)` in
|
||||
//! the history's appearances and depends only on the fit; a query is `O(n^2)`
|
||||
//! and depends only on the question. `posterior_of_one_shot` pays both every
|
||||
//! time, `joint_query` pays only the second.
|
||||
|
||||
use criterion::{Criterion, criterion_group, criterion_main};
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{ConstantDrift, ConvergenceOptions, Event, History, Member, Outcome, Team};
|
||||
|
||||
/// 30 slices of 8 duels: 480 appearances over 100 competitors.
|
||||
fn fitted() -> History<i64, ConstantDrift, trueskill_tt::NullObserver, String> {
|
||||
let mut h: History<i64, ConstantDrift, _, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.05))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 30,
|
||||
epsilon: 1e-10,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
|
||||
let mut events: Vec<Event<i64, String>> = Vec::new();
|
||||
let mut k = 0usize;
|
||||
for t in 0..30i64 {
|
||||
for _ in 0..8 {
|
||||
k += 1;
|
||||
events.push(Event {
|
||||
time: t,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(format!("p{}", k % 100))]),
|
||||
Team::with_members([Member::new(format!("p{}", (k + 37) % 100))]),
|
||||
],
|
||||
outcome: Outcome::scores([
|
||||
(k as f64 * 0.3).sin().abs() * 20.0,
|
||||
(k as f64 * 0.3).cos().abs() * 20.0,
|
||||
]),
|
||||
});
|
||||
}
|
||||
}
|
||||
h.add_events(events).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h
|
||||
}
|
||||
|
||||
fn bench_joint(c: &mut Criterion) {
|
||||
let h = fitted();
|
||||
let a = "p0".to_string();
|
||||
let b = "p1".to_string();
|
||||
let terms = [(&a, 1.0), (&b, -1.0)];
|
||||
|
||||
c.bench_function("joint_factorise_480_appearances", |bencher| {
|
||||
bencher.iter(|| std::hint::black_box(h.joint().unwrap().variables()));
|
||||
});
|
||||
|
||||
// Factorise-and-query, the cost the deleted `History::posterior_of`
|
||||
// wrapper paid on every call. Kept as the baseline the cached query below
|
||||
// is measured against.
|
||||
c.bench_function("posterior_of_one_shot_480_appearances", |bencher| {
|
||||
bencher.iter(|| std::hint::black_box(h.joint().unwrap().posterior_of(&terms).unwrap()));
|
||||
});
|
||||
|
||||
let joint = h.joint().unwrap();
|
||||
c.bench_function("joint_query_480_appearances", |bencher| {
|
||||
bencher.iter(|| std::hint::black_box(joint.posterior_of(&terms).unwrap()));
|
||||
});
|
||||
}
|
||||
|
||||
criterion_group!(benches, bench_joint);
|
||||
criterion_main!(benches);
|
||||
+4
-3
@@ -5,11 +5,12 @@ use trueskill_tt::{ConstantDrift, Event, History, Member, Outcome, Team};
|
||||
fn bench_scored_history(c: &mut Criterion) {
|
||||
c.bench_function("scored_history_60_events_30_iter", |bencher| {
|
||||
bencher.iter(|| {
|
||||
let mut h: History<i64, ConstantDrift, _, String> = History::builder_with_key()
|
||||
let mut h: History<i64, ConstantDrift, _, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.mu(25.0)
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.drift(ConstantDrift(0.03))
|
||||
.drift(ConstantDrift::new(0.03))
|
||||
.score_sigma(2.0)
|
||||
.build();
|
||||
|
||||
@@ -29,7 +30,7 @@ fn bench_scored_history(c: &mut Criterion) {
|
||||
});
|
||||
}
|
||||
h.add_events(events).unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
@@ -500,6 +500,26 @@ All public traits (`Time`, `Drift`, `Observer`, `Factor`, `Schedule`) require `S
|
||||
|
||||
`rayon` as default-on feature; with `default-features = false`, parallel paths fall back to sequential iterators behind `cfg(feature = "rayon")`.
|
||||
|
||||
> **Not implemented. Deliberate deviation, decided 2026-09-08 (issue #5).**
|
||||
>
|
||||
> `rayon` ships **opt-in**: `Cargo.toml` has no `default = [...]` key. The
|
||||
> measured speedups are 1.0x on realistic workloads and 1.3x on a pathological
|
||||
> one (issue #4), because typical slices hold too few events to amortize
|
||||
> rayon's task-spawn overhead. Default-on would hand every downstream user a
|
||||
> thread pool and a dependency for approximately no gain.
|
||||
>
|
||||
> This section made the trade conditional on cross-slice dirty-bit skipping
|
||||
> landing and changing the parallel story. It did not land: #4 was closed on
|
||||
> 2026-08-27 by removing the inert `ConvergenceReport::slices_skipped` field
|
||||
> rather than by implementing the mechanism, so the re-measurement this was
|
||||
> waiting on will not arrive.
|
||||
>
|
||||
> The "Trade-offs" note below also cited an `unsafe` concurrent-write path
|
||||
> through `SkillStore` as a cost of default-on. That cost does not exist: the
|
||||
> crate is `#![forbid(unsafe_code)]`, and the compute/apply split on the
|
||||
> internal `Event` is what lets a color group run in parallel without it. The
|
||||
> case for opt-in rests on the measurements alone.
|
||||
|
||||
### Expected speedup ballpark
|
||||
|
||||
For 1000 players, 60 events/slice × 1000 slices, 30 convergence iterations:
|
||||
@@ -521,7 +541,7 @@ These are pre-implementation estimates. Each tier validates with criterion.
|
||||
- Color-group parallelism requires up-front graph coloring at ingestion. Cost: linear in events, run once per `add_events`. Cheap.
|
||||
- Default = asynchronous EP (preserves current semantics). Synchronous opt-in only.
|
||||
- Cross-slice sweep stays sequential; no speculative parallel sweeps.
|
||||
- Rayon default-on but feature-gated.
|
||||
- Rayon default-on but feature-gated. **Superseded — shipped opt-in; see the deviation note in Section 6.**
|
||||
|
||||
### Open question
|
||||
|
||||
|
||||
+29
-8
@@ -1,7 +1,8 @@
|
||||
use plotters::prelude::*;
|
||||
use smallvec::smallvec;
|
||||
use time::{Date, Month};
|
||||
use trueskill_tt::{Event, History, Member, Outcome, Team, drift::ConstantDrift};
|
||||
use trueskill_tt::{
|
||||
Event, History, Member, Outcome, Team, drift::ConstantDrift, smallvec::smallvec,
|
||||
};
|
||||
|
||||
fn main() {
|
||||
let mut csv = csv::Reader::open("examples/atp.csv").unwrap();
|
||||
@@ -42,18 +43,38 @@ fn main() {
|
||||
}
|
||||
}
|
||||
|
||||
let mut hist: History<i64, _, _, String> = History::builder_with_key()
|
||||
let mut hist: History<i64, _, _, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.sigma(1.6)
|
||||
.drift(ConstantDrift(0.036))
|
||||
.drift(ConstantDrift::new(0.036))
|
||||
.convergence(trueskill_tt::ConvergenceOptions {
|
||||
max_iter: 10,
|
||||
// This history needs 30 sweeps to reach the epsilon below. It was
|
||||
// capped at 10 until the `#[must_use]` on `ConvergenceReport`
|
||||
// surfaced that the example had been shipping a short fit.
|
||||
max_iter: 100,
|
||||
epsilon: 0.01,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
|
||||
hist.add_events(events).unwrap();
|
||||
hist.converge().unwrap();
|
||||
|
||||
// Read the report rather than discarding it. A fit that hits `max_iter`
|
||||
// without reaching `epsilon` is not an error and does not look wrong — every
|
||||
// rating comes back finite and sensibly ordered — so this flag is the only
|
||||
// thing that says the numbers were still moving when the sweep stopped.
|
||||
let report = hist.converge().unwrap();
|
||||
eprintln!(
|
||||
"converged={} after {} sweeps, final step {:?}",
|
||||
report.converged, report.iterations, report.final_step
|
||||
);
|
||||
if !report.converged {
|
||||
eprintln!(
|
||||
"warning: stopped after {} sweeps with a final step of {:?}, \
|
||||
short of epsilon — raise ConvergenceOptions::max_iter",
|
||||
report.iterations, report.final_step
|
||||
);
|
||||
}
|
||||
|
||||
let players = [
|
||||
("aggasi", "a092", 38800i64),
|
||||
@@ -77,7 +98,7 @@ fn main() {
|
||||
let mut y_spec = (f64::MAX, f64::MIN);
|
||||
|
||||
for &(_, id, cutoff) in &players {
|
||||
for (ts, gs) in hist.learning_curve(id) {
|
||||
for (ts, gs) in hist.learning_curve(id).unwrap() {
|
||||
if ts >= cutoff {
|
||||
continue;
|
||||
}
|
||||
@@ -123,7 +144,7 @@ fn main() {
|
||||
let mut upper = Vec::new();
|
||||
let mut lower = Vec::new();
|
||||
|
||||
for (ts, gs) in hist.learning_curve(id) {
|
||||
for (ts, gs) in hist.learning_curve(id).unwrap() {
|
||||
if ts >= cutoff {
|
||||
continue;
|
||||
}
|
||||
|
||||
+2
-3
@@ -6,15 +6,14 @@
|
||||
//!
|
||||
//! Run with: `cargo run --example scored --release`
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{ConstantDrift, Event, History, Member, Outcome, Team};
|
||||
use trueskill_tt::{ConstantDrift, Event, History, Member, Outcome, Team, smallvec::smallvec};
|
||||
|
||||
fn main() {
|
||||
let mut h = History::builder()
|
||||
.mu(25.0)
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.drift(ConstantDrift(0.03))
|
||||
.drift(ConstantDrift::new(0.03))
|
||||
.score_sigma(2.0) // tune to data; smaller = trust margins more
|
||||
.build();
|
||||
|
||||
|
||||
+9
-1
@@ -3,4 +3,12 @@ publish = true
|
||||
# Hold off pushing until tags and publish have both succeeded; `just release`
|
||||
# pushes last.
|
||||
push = false
|
||||
pre-release-hook = ["sh", "-c", "git cliff -o CHANGELOG.md --tag {{version}} && git add CHANGELOG.md"]
|
||||
# Regenerate the changelog and stage it so it lands in the release commit.
|
||||
#
|
||||
# Guarded on DRY_RUN because cargo-release runs pre-release hooks during a dry
|
||||
# run too (verified against cargo-release 1.1.5, which exports DRY_RUN=true,
|
||||
# CRATE_NAME, PREV_VERSION and NEW_VERSION to the hook). Without the guard,
|
||||
# `just release-plan` — documented as a preview that writes nothing — writes and
|
||||
# `git add`s CHANGELOG.md, and the clean-tree check in `just release` then
|
||||
# refuses to run. That check is load-bearing: publishing is irreversible.
|
||||
pre-release-hook = ["sh", "-c", '[ "$DRY_RUN" = "true" ] || (git cliff -o CHANGELOG.md --tag {{version}} && git add CHANGELOG.md)']
|
||||
|
||||
@@ -0,0 +1,391 @@
|
||||
//! Active learning: which comparison teaches you the most.
|
||||
//!
|
||||
//! [`quality`](crate::quality) answers "is this matchup *fair*". That is a
|
||||
//! different question from "is this matchup *informative*", and the two
|
||||
//! coincide only for two evenly matched competitors. When each observation
|
||||
//! costs something — a human click, a scheduled fixture — the question worth
|
||||
//! asking is the second one.
|
||||
//!
|
||||
//! The quantity here is expected information gain: the outcome-weighted
|
||||
//! divergence between what you believe now and what you would believe after
|
||||
//! seeing the result.
|
||||
//!
|
||||
//! ```text
|
||||
//! EIG(matchup) = SUM P(outcome) * KL( posterior_after(outcome) || prior )
|
||||
//! outcome
|
||||
//! ```
|
||||
//!
|
||||
//! It is the mutual information between the observed outcome and the skills,
|
||||
//! which is worth remembering because it pins the scale: information gain
|
||||
//! cannot exceed the entropy of the thing you are about to observe. A contest
|
||||
//! with `k` distinguishable outcomes can teach you at most `ln k` nats,
|
||||
//! whatever the ratings. That ceiling is the sharpest available test of an
|
||||
//! implementation — see [`expected_information_gain`].
|
||||
|
||||
use crate::{
|
||||
GameOptions, Gaussian, InferenceError, Outcome, Rating, drift::Drift, predict, time::Time,
|
||||
};
|
||||
|
||||
/// Outcomes below this probability contribute nothing measurable and are not
|
||||
/// worth an inference pass.
|
||||
///
|
||||
/// The contribution of an outcome is `P * KL`, and `KL` is bounded in practice
|
||||
/// by tens of nats, so a probability this small moves the total by less than
|
||||
/// the quadrature error already present in `P` itself.
|
||||
const NEGLIGIBLE: f64 = 1e-12;
|
||||
|
||||
/// `KL(q || p)` for two univariate Gaussians, in nats.
|
||||
///
|
||||
/// Both arguments are proper posteriors from inference, so the degenerate
|
||||
/// cases guarded here (zero or infinite variance) indicate that inference has
|
||||
/// broken down rather than anything a caller did.
|
||||
fn kl_divergence(q: Gaussian, p: Gaussian) -> f64 {
|
||||
let (var_q, var_p) = (q.sigma().powi(2), p.sigma().powi(2));
|
||||
|
||||
if !(var_q.is_finite() && var_p.is_finite()) || var_q <= 0.0 || var_p <= 0.0 {
|
||||
return 0.0;
|
||||
}
|
||||
|
||||
let mean_gap = q.mu() - p.mu();
|
||||
|
||||
// Algebraically `0.5 * (ln(var_p/var_q) + (var_q + gap^2)/var_p - 1)`, but
|
||||
// written so that neither term can go negative.
|
||||
//
|
||||
// The direct form cancels against its `- 1.0` for two near-identical
|
||||
// distributions and returns a *negative* divergence — measured, 762 082 of
|
||||
// 3 000 000 near-identical pairs, worst `-5.55e-17`, which is exactly one
|
||||
// ULP of the 1.0. It also loses the answer entirely where it is small:
|
||||
// at `var_q/var_p - 1 = 1e-9` the direct form gives `0.0` where the true
|
||||
// value is `2.5e-19`.
|
||||
//
|
||||
// With `u = var_q/var_p - 1` the variance part is `0.5 * (u - ln(1+u))`,
|
||||
// which is non-negative for every `u > -1`, and the mean part is a square
|
||||
// over a positive variance. Non-negativity is then structural rather than
|
||||
// incidental.
|
||||
let u = var_q / var_p - 1.0;
|
||||
0.5 * u_minus_ln1p(u) + mean_gap * mean_gap / (2.0 * var_p)
|
||||
}
|
||||
|
||||
/// `u - ln(1 + u)`, without the cancellation that spelling invites.
|
||||
///
|
||||
/// Both terms are approximately `u` for small `u`, so the subtraction loses
|
||||
/// everything just where the result matters. The Taylor series
|
||||
/// `u^2/2 - u^3/3 + u^4/4 - ...` is exact in that regime and manifestly
|
||||
/// non-negative, since `u^2/2` dominates.
|
||||
fn u_minus_ln1p(u: f64) -> f64 {
|
||||
if u.abs() < 1e-4 {
|
||||
let u2 = u * u;
|
||||
u2 * (0.5 - u / 3.0 + u2 / 4.0)
|
||||
} else {
|
||||
u - libm::log1p(u)
|
||||
}
|
||||
}
|
||||
|
||||
/// Expected information gain of a hypothetical matchup, in nats.
|
||||
///
|
||||
/// Enumerates the outcomes this matchup could have, runs inference for each to
|
||||
/// get the belief it would produce, and weights the resulting divergence by
|
||||
/// that outcome's probability. A higher value means the result would teach you
|
||||
/// more.
|
||||
///
|
||||
/// # Interpreting the value
|
||||
///
|
||||
/// Nats. The upper bound is the entropy of the outcome variable: at most
|
||||
/// `ln 2 ≈ 0.693` for a two-way result, `ln 3 ≈ 1.099` once draws are
|
||||
/// possible, `ln k` for `k` outcomes. A value near the ceiling means the
|
||||
/// result is close to a coin flip *and* would move the posteriors a long way;
|
||||
/// a value near zero means you already know what will happen, or that the
|
||||
/// result would barely change your beliefs if you saw it.
|
||||
///
|
||||
/// This is not a monotone transform of [`quality`](crate::quality). A lopsided
|
||||
/// matchup between two uncertain competitors scores well on quality-times-
|
||||
/// variance heuristics and poorly here, because the near-certain outcome
|
||||
/// carries almost no information.
|
||||
///
|
||||
/// # Cost
|
||||
///
|
||||
/// One full inference pass per possible outcome, so this is far more expensive
|
||||
/// than `quality()` — which is one closed-form evaluation. The outcome count
|
||||
/// grows quickly with team count (3 outcomes for two teams that can draw, 13
|
||||
/// for three, 75 for four), and scoring every candidate pairing among `n`
|
||||
/// competitors is `O(n² × outcomes)` inference passes.
|
||||
///
|
||||
/// For a selector over many candidates, shortlist with the cheap
|
||||
/// [`quality`](crate::quality) or
|
||||
/// [`predict_win_probabilities`](crate::History::predict_win_probabilities)
|
||||
/// first and score only the shortlist here. The expected-variance-reduction
|
||||
/// proxy sometimes suggested as a cheaper alternative is *not* cheaper: it
|
||||
/// needs the same hypothetical posteriors, so it shares the dominant cost.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// - `NotEnoughTeams` if fewer than two teams are supplied.
|
||||
/// - `EmptyTeam` if any team has no members.
|
||||
/// - `TooManyTeams` if the outcome space is too large to enumerate; see
|
||||
/// [`MAX_PREDICTED_TEAMS`](crate::MAX_PREDICTED_TEAMS).
|
||||
/// - `InvalidProbability` if `options.p_draw` is outside `[0.0, 1.0)`.
|
||||
/// - `GridTooCoarse` when the performance sigmas are too far apart to
|
||||
/// integrate on one grid. This comes from `outcome_distribution`, which runs
|
||||
/// before any inference — so it is not covered by "anything `Game::ranked`
|
||||
/// returns" below.
|
||||
/// - Anything [`Game::ranked`](crate::Game::ranked) returns for a hypothetical
|
||||
/// outcome.
|
||||
pub fn expected_information_gain<T: Time, D: Drift<T>>(
|
||||
teams: &[&[Rating<T, D>]],
|
||||
options: &GameOptions,
|
||||
) -> Result<f64, InferenceError> {
|
||||
if teams.len() < 2 {
|
||||
return Err(InferenceError::NotEnoughTeams { got: teams.len() });
|
||||
}
|
||||
if teams.len() > crate::MAX_PREDICTED_TEAMS {
|
||||
return Err(InferenceError::TooManyTeams {
|
||||
got: teams.len(),
|
||||
max: crate::MAX_PREDICTED_TEAMS,
|
||||
});
|
||||
}
|
||||
if !(0.0..1.0).contains(&options.p_draw) {
|
||||
return Err(InferenceError::InvalidProbability {
|
||||
value: options.p_draw,
|
||||
});
|
||||
}
|
||||
for (idx, team) in teams.iter().enumerate() {
|
||||
if team.is_empty() {
|
||||
return Err(InferenceError::EmptyTeam { team: idx });
|
||||
}
|
||||
}
|
||||
|
||||
// Prediction runs on performances: skill inflated by each member's beta.
|
||||
let performances: Vec<Gaussian> = teams
|
||||
.iter()
|
||||
.map(|team| {
|
||||
team.iter()
|
||||
.fold(crate::N00, |acc, rating| acc.convolve(rating.performance()))
|
||||
})
|
||||
.collect();
|
||||
|
||||
// Draw margins per pair, derived from the teams' betas exactly as
|
||||
// inference derives them, so the outcomes weighted here are the outcomes
|
||||
// that would actually be fitted.
|
||||
let beta_sq: Vec<f64> = teams
|
||||
.iter()
|
||||
.map(|team| team.iter().map(|r| r.beta().powi(2)).sum())
|
||||
.collect();
|
||||
let p_draw = options.p_draw;
|
||||
let margins = predict::Margins::new(teams.len(), |i, j| {
|
||||
if p_draw == 0.0 {
|
||||
0.0
|
||||
} else {
|
||||
crate::compute_margin(p_draw, (beta_sq[i] + beta_sq[j]).sqrt())
|
||||
}
|
||||
});
|
||||
|
||||
let mut gain = 0.0;
|
||||
|
||||
for (ranks, probability) in predict::outcome_distribution(&performances, &margins)? {
|
||||
if probability <= NEGLIGIBLE {
|
||||
continue;
|
||||
}
|
||||
|
||||
let game = crate::Game::ranked(teams, Outcome::ranking(ranks), options)?;
|
||||
let posteriors = game.posteriors();
|
||||
|
||||
// Beliefs factorise across competitors, so the joint divergence is the
|
||||
// sum of the per-competitor ones.
|
||||
let divergence: f64 = teams
|
||||
.iter()
|
||||
.zip(&posteriors)
|
||||
.flat_map(|(team, posterior)| team.iter().zip(posterior))
|
||||
.map(|(rating, &after)| kl_divergence(after, rating.prior()))
|
||||
.sum();
|
||||
|
||||
gain += probability * divergence;
|
||||
}
|
||||
|
||||
Ok(gain)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::{BETA, ConstantDrift, GAMMA};
|
||||
|
||||
type R = Rating<i64, ConstantDrift>;
|
||||
|
||||
fn rating(mu: f64, sigma: f64) -> R {
|
||||
R::new(
|
||||
Gaussian::from_ms(mu, sigma),
|
||||
BETA,
|
||||
ConstantDrift::new(GAMMA),
|
||||
)
|
||||
}
|
||||
|
||||
fn options(p_draw: f64) -> GameOptions {
|
||||
GameOptions {
|
||||
p_draw,
|
||||
..GameOptions::default()
|
||||
}
|
||||
}
|
||||
|
||||
fn eig(teams: &[&[R]], p_draw: f64) -> f64 {
|
||||
expected_information_gain(teams, &options(p_draw)).unwrap()
|
||||
}
|
||||
|
||||
/// The analytic ceiling. Information gain is the mutual information between
|
||||
/// the outcome and the skills, so it cannot exceed the entropy of the
|
||||
/// outcome variable — whatever the ratings. This is the check a subtly
|
||||
/// wrong implementation fails while still returning plausible numbers: an
|
||||
/// early prototype of this returned 4.77 nats from a sign error and passed
|
||||
/// every monotonicity test.
|
||||
#[test]
|
||||
fn never_exceeds_the_entropy_of_the_outcome() {
|
||||
let ceiling_two = std::f64::consts::LN_2;
|
||||
|
||||
for (a, b) in [
|
||||
(rating(0.0, 6.0), rating(0.0, 6.0)),
|
||||
(rating(0.0, 0.5), rating(0.0, 0.5)),
|
||||
(rating(12.0, 6.0), rating(-12.0, 6.0)),
|
||||
(rating(40.0, 1.0), rating(-40.0, 1.0)),
|
||||
(rating(3.0, 6.0), rating(-2.0, 0.1)),
|
||||
(rating(0.0, 25.0), rating(0.0, 25.0)),
|
||||
] {
|
||||
let g = eig(&[&[a], &[b]], 0.0);
|
||||
assert!(
|
||||
g >= 0.0 && g <= ceiling_two,
|
||||
"EIG {g} outside [0, ln 2] for mu=({}, {}) sigma=({}, {})",
|
||||
a.prior().mu(),
|
||||
b.prior().mu(),
|
||||
a.prior().sigma(),
|
||||
b.prior().sigma()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// With draws enabled there are three outcomes, so the ceiling rises to
|
||||
/// `ln 3` — and the two-outcome bound no longer applies.
|
||||
#[test]
|
||||
fn the_ceiling_follows_the_outcome_count() {
|
||||
let ceiling_three = 3.0f64.ln();
|
||||
for sigma in [0.5, 3.0, 6.0, 25.0] {
|
||||
let g = eig(&[&[rating(0.0, sigma)], &[rating(0.0, sigma)]], 0.25);
|
||||
assert!(
|
||||
g >= 0.0 && g <= ceiling_three,
|
||||
"EIG {g} outside [0, ln 3] at sigma {sigma}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// An even matchup between uncertain competitors is the informative one.
|
||||
/// A hopelessly lopsided matchup teaches you almost nothing, because you
|
||||
/// already know how it ends.
|
||||
#[test]
|
||||
fn an_even_matchup_beats_a_lopsided_one() {
|
||||
let even = eig(&[&[rating(0.0, 6.0)], &[rating(0.0, 6.0)]], 0.0);
|
||||
let lopsided = eig(&[&[rating(12.0, 6.0)], &[rating(-12.0, 6.0)]], 0.0);
|
||||
assert!(
|
||||
even > lopsided,
|
||||
"even {even} should beat lopsided {lopsided}"
|
||||
);
|
||||
}
|
||||
|
||||
/// Certainty is the thing information gain is measuring the absence of:
|
||||
/// the less you know, the more there is to learn.
|
||||
#[test]
|
||||
fn gain_falls_as_certainty_rises() {
|
||||
let mut previous = f64::INFINITY;
|
||||
for sigma in [12.0, 6.0, 3.0, 1.0, 0.5, 0.1] {
|
||||
let g = eig(&[&[rating(0.0, sigma)], &[rating(0.0, sigma)]], 0.0);
|
||||
assert!(
|
||||
g < previous,
|
||||
"sigma {sigma}: {g} did not fall below {previous}"
|
||||
);
|
||||
previous = g;
|
||||
}
|
||||
assert!(previous >= 0.0);
|
||||
}
|
||||
|
||||
/// The heuristic this replaces is `quality * sigma_a^2 * sigma_b^2`. It is
|
||||
/// not a monotone transform of information gain — it ranks a lopsided
|
||||
/// matchup above a confident even one, and EIG ranks them the other way.
|
||||
/// Pinning the disagreement down is what stops a future "simplification"
|
||||
/// from quietly reverting to the heuristic.
|
||||
#[test]
|
||||
fn disagrees_with_the_quality_times_variance_heuristic() {
|
||||
let heuristic = |a: &R, b: &R| {
|
||||
crate::quality(&[&[a.prior()], &[b.prior()]], BETA)
|
||||
* a.prior().sigma().powi(2)
|
||||
* b.prior().sigma().powi(2)
|
||||
};
|
||||
|
||||
let (confident_a, confident_b) = (rating(0.0, 0.5), rating(0.0, 0.5));
|
||||
let (lopsided_a, lopsided_b) = (rating(12.0, 6.0), rating(-12.0, 6.0));
|
||||
|
||||
assert!(
|
||||
heuristic(&lopsided_a, &lopsided_b) > heuristic(&confident_a, &confident_b),
|
||||
"the heuristic should prefer the lopsided matchup"
|
||||
);
|
||||
assert!(
|
||||
eig(&[&[confident_a], &[confident_b]], 0.0) > eig(&[&[lopsided_a], &[lopsided_b]], 0.0),
|
||||
"information gain should prefer the even matchup"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn supports_more_than_two_teams() {
|
||||
let teams: Vec<Vec<R>> = vec![
|
||||
vec![rating(0.0, 6.0)],
|
||||
vec![rating(0.0, 6.0)],
|
||||
vec![rating(0.0, 6.0)],
|
||||
];
|
||||
let refs: Vec<&[R]> = teams.iter().map(Vec::as_slice).collect();
|
||||
let g = expected_information_gain(&refs, &options(0.0)).unwrap();
|
||||
// Six distinguishable orderings with no draws.
|
||||
assert!(
|
||||
g > 0.0 && g <= 6.0f64.ln(),
|
||||
"three-team EIG {g} out of range"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn multi_member_teams_are_supported() {
|
||||
let a = [rating(0.0, 6.0), rating(1.0, 4.0)];
|
||||
let b = [rating(0.0, 6.0)];
|
||||
let g = expected_information_gain(&[&a, &b], &options(0.0)).unwrap();
|
||||
assert!(g > 0.0 && g <= std::f64::consts::LN_2, "{g}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn degenerate_shapes_are_errors() {
|
||||
let a = [rating(0.0, 6.0)];
|
||||
assert!(matches!(
|
||||
expected_information_gain(&[&a], &options(0.0)),
|
||||
Err(InferenceError::NotEnoughTeams { got: 1 })
|
||||
));
|
||||
let empty: [R; 0] = [];
|
||||
assert!(matches!(
|
||||
expected_information_gain(&[&a, &empty], &options(0.0)),
|
||||
Err(InferenceError::EmptyTeam { team: 1 })
|
||||
));
|
||||
assert!(matches!(
|
||||
expected_information_gain(&[&a, &a], &options(1.5)),
|
||||
Err(InferenceError::InvalidProbability { .. })
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn kl_divergence_is_zero_for_identical_beliefs() {
|
||||
let g = Gaussian::from_ms(3.0, 2.0);
|
||||
assert!(kl_divergence(g, g).abs() < 1e-15);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn kl_divergence_is_non_negative_and_grows_with_separation() {
|
||||
let prior = Gaussian::from_ms(0.0, 3.0);
|
||||
let mut previous = 0.0;
|
||||
for mu in [0.0, 0.5, 1.0, 2.0, 4.0] {
|
||||
let d = kl_divergence(Gaussian::from_ms(mu, 3.0), prior);
|
||||
assert!(d >= 0.0, "negative divergence at mu {mu}: {d}");
|
||||
assert!(d >= previous, "not increasing at mu {mu}");
|
||||
previous = d;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -191,3 +191,121 @@ mod tests {
|
||||
assert_eq!(cg.total_events(), 4);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod properties {
|
||||
use std::collections::HashSet;
|
||||
|
||||
use proptest::prelude::*;
|
||||
|
||||
use super::*;
|
||||
|
||||
/// The property the whole parallel sweep rests on: two events sharing a
|
||||
/// competitor must never land in the same color, because a color group is
|
||||
/// run concurrently and two events touching one competitor would race.
|
||||
///
|
||||
/// Hand-written cases cover the shapes someone thought of. This covers the
|
||||
/// ones nobody did — the correctness of `sweep_color_groups` depends on it
|
||||
/// holding for every input, not for five.
|
||||
fn check(events: &[Vec<usize>]) {
|
||||
let groups = color_greedy(events.len(), |ev| {
|
||||
events[ev]
|
||||
.iter()
|
||||
.copied()
|
||||
.map(Index::from)
|
||||
.collect::<Vec<_>>()
|
||||
});
|
||||
|
||||
// Disjointness *between events* within a color. Deduplicated per
|
||||
// event, because one event legitimately naming a competitor twice is
|
||||
// not a collision — `color_greedy` collects each event's members into
|
||||
// a set for exactly that reason.
|
||||
for color in 0..groups.n_colors() {
|
||||
let mut seen: HashSet<usize> = HashSet::new();
|
||||
for &ev in &groups.groups[color] {
|
||||
let members: HashSet<usize> = events[ev].iter().copied().collect();
|
||||
for competitor in members {
|
||||
assert!(
|
||||
seen.insert(competitor),
|
||||
"competitor {competitor} shared by two events in color {color}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Every event is assigned exactly once. Without this, a partition that
|
||||
// dropped events would satisfy disjointness trivially.
|
||||
let mut assigned: Vec<usize> = groups.groups.iter().flatten().copied().collect();
|
||||
assigned.sort_unstable();
|
||||
assert_eq!(assigned, (0..events.len()).collect::<Vec<_>>());
|
||||
assert_eq!(groups.total_events(), events.len());
|
||||
|
||||
// No empty colors: one would waste a sweep and make `n_colors`
|
||||
// misleading.
|
||||
for (color, group) in groups.groups.iter().enumerate() {
|
||||
assert!(!group.is_empty(), "color {color} is empty");
|
||||
}
|
||||
|
||||
// Contiguity is not a property of `color_greedy` — it holds only after
|
||||
// `recompute_color_groups` reorders the events so each color occupies
|
||||
// one range. What must always hold is that the reorder is *possible*:
|
||||
// relabelling events in group order yields contiguous groups. The
|
||||
// parallel sweep slices `&mut` sub-ranges from those, so if this ever
|
||||
// failed the reorder would produce overlapping ranges.
|
||||
let mut next = 0usize;
|
||||
let relabelled: Vec<Vec<usize>> = groups
|
||||
.groups
|
||||
.iter()
|
||||
.map(|group| {
|
||||
group
|
||||
.iter()
|
||||
.map(|_| {
|
||||
let i = next;
|
||||
next += 1;
|
||||
i
|
||||
})
|
||||
.collect()
|
||||
})
|
||||
.collect();
|
||||
assert!(ColorGroups { groups: relabelled }.groups_are_contiguous());
|
||||
}
|
||||
|
||||
proptest! {
|
||||
#![proptest_config(ProptestConfig::with_cases(512))]
|
||||
|
||||
/// Small competitor pool, so collisions are common and colors are
|
||||
/// forced to multiply.
|
||||
#[test]
|
||||
fn colors_are_disjoint_on_a_dense_pool(
|
||||
events in prop::collection::vec(
|
||||
prop::collection::vec(0usize..6, 1..4),
|
||||
0..20,
|
||||
)
|
||||
) {
|
||||
check(&events);
|
||||
}
|
||||
|
||||
/// Wide pool, so most events are independent and land in one color.
|
||||
#[test]
|
||||
fn colors_are_disjoint_on_a_sparse_pool(
|
||||
events in prop::collection::vec(
|
||||
prop::collection::vec(0usize..200, 1..6),
|
||||
0..30,
|
||||
)
|
||||
) {
|
||||
check(&events);
|
||||
}
|
||||
|
||||
/// Repeated competitors within one event must not confuse the
|
||||
/// member-set bookkeeping.
|
||||
#[test]
|
||||
fn colors_are_disjoint_with_repeated_members(
|
||||
events in prop::collection::vec(
|
||||
prop::collection::vec(0usize..3, 1..8),
|
||||
0..15,
|
||||
)
|
||||
) {
|
||||
check(&events);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+2
-2
@@ -7,8 +7,8 @@ use crate::{
|
||||
|
||||
/// Per-history, temporal state for someone competing.
|
||||
///
|
||||
/// Renamed from `Agent` in T2; the former `.player` field is now
|
||||
/// `.rating` to match the `Player → Rating` rename.
|
||||
/// The mutable half of a competitor: `Rating` holds their static
|
||||
/// configuration, this holds what inference learns as it sweeps.
|
||||
#[derive(Debug)]
|
||||
pub struct Competitor<T: Time = i64, D: Drift<T> = ConstantDrift> {
|
||||
pub rating: Rating<T, D>,
|
||||
|
||||
+101
-2
@@ -4,9 +4,38 @@ use std::time::Duration;
|
||||
|
||||
use smallvec::SmallVec;
|
||||
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
/// The stopping rule for the fixed-point loops, plus how hard they are damped.
|
||||
///
|
||||
/// Set once per history through
|
||||
/// [`HistoryBuilder::convergence`](crate::HistoryBuilder::convergence), and
|
||||
/// carried by `GameOptions` for a single match scored without a history. The
|
||||
/// defaults are the crate's globals: [`ITERATIONS`](crate::ITERATIONS),
|
||||
/// [`EPSILON`](crate::EPSILON), and undamped EP.
|
||||
///
|
||||
/// Deliberately **not** `#[non_exhaustive]`, unlike [`ConvergenceReport`]. The
|
||||
/// usual argument for marking an options struct is that `..Default::default()`
|
||||
/// makes a future field additive — but `Default::default` is not a `const fn`,
|
||||
/// so marking it would make
|
||||
/// `const OPTS: ConvergenceOptions = ConvergenceOptions { .. }` impossible from
|
||||
/// outside the crate, with no workaround. This type is `Copy` and a natural
|
||||
/// const; that cost is permanent, and adding a field is a one-time major bump.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct ConvergenceOptions {
|
||||
/// Hard cap on full forward+backward sweeps.
|
||||
///
|
||||
/// A runaway guard, not a budget: the loop exits as soon as the step falls
|
||||
/// to `epsilon`, so raising this costs nothing on a history that converges.
|
||||
/// Reaching it is
|
||||
/// [`InferenceError::NotConverged`](crate::InferenceError::NotConverged).
|
||||
pub max_iter: usize,
|
||||
/// Convergence threshold, in skill units.
|
||||
///
|
||||
/// The sweep stops once *both* components of the step — the largest change
|
||||
/// a whole iteration made to any competitor's posterior mean, and to any
|
||||
/// posterior standard deviation — are at or below this. Larger values stop
|
||||
/// sooner and further from the fixed point. Must be non-negative; NaN is
|
||||
/// rejected, since every comparison against it is false and the loop would
|
||||
/// read it as converged.
|
||||
pub epsilon: f64,
|
||||
/// EP damping factor in natural-parameter space: each per-factor
|
||||
/// update inside a single game writes `α·new + (1−α)·old`. `1.0` is
|
||||
@@ -20,6 +49,37 @@ pub struct ConvergenceOptions {
|
||||
pub alpha: f64,
|
||||
}
|
||||
|
||||
impl ConvergenceOptions {
|
||||
/// Reject values that would make inference silently meaningless.
|
||||
///
|
||||
/// `HistoryBuilder::convergence` asserts these eagerly, but the fields are
|
||||
/// public and `GameOptions` carries a `ConvergenceOptions` — so a caller
|
||||
/// can hand `Game::ranked` a set the builder never saw. In release the
|
||||
/// engine's `debug_assert!`s are gone, and an `alpha` of zero leaves every
|
||||
/// EP update unapplied: inference returns the priors, with every likelihood
|
||||
/// uninformative and nothing to indicate anything went wrong.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// `InvalidParameter` if `alpha` is outside `(0.0, 1.0]` or `epsilon` is
|
||||
/// negative. NaN fails both comparisons and is rejected.
|
||||
pub(crate) fn validate(&self) -> Result<(), crate::InferenceError> {
|
||||
if !(self.alpha > 0.0 && self.alpha <= 1.0) {
|
||||
return Err(crate::InferenceError::InvalidParameter {
|
||||
name: "alpha",
|
||||
value: self.alpha,
|
||||
});
|
||||
}
|
||||
if self.epsilon.is_nan() || self.epsilon < 0.0 {
|
||||
return Err(crate::InferenceError::InvalidParameter {
|
||||
name: "epsilon",
|
||||
value: self.epsilon,
|
||||
});
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for ConvergenceOptions {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
@@ -31,12 +91,51 @@ impl Default for ConvergenceOptions {
|
||||
}
|
||||
|
||||
/// Post-hoc summary of a `History::converge` call.
|
||||
#[derive(Clone, Debug)]
|
||||
///
|
||||
/// From [`History::converge`](crate::History::converge) this always describes a
|
||||
/// converged fit — stopping at `max_iter` is
|
||||
/// [`InferenceError::NotConverged`](crate::InferenceError::NotConverged) there.
|
||||
/// From [`History::converge_partial`](crate::History::converge_partial) it may
|
||||
/// not be, and `converged` is what says so.
|
||||
/// Constructed only by `converge` / `converge_partial`, never by a caller, so
|
||||
/// `#[non_exhaustive]` costs nothing here and lets a future field be additive.
|
||||
/// The two *options* structs deliberately do not carry it — see the note on
|
||||
/// [`ConvergenceOptions`].
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
#[non_exhaustive]
|
||||
pub struct ConvergenceReport {
|
||||
/// Full forward+backward sweeps actually run. `0` for a history with no
|
||||
/// time slices, which is converged trivially.
|
||||
pub iterations: usize,
|
||||
/// How far the last sweep still moved the fit, as `(mean, standard
|
||||
/// deviation)`.
|
||||
///
|
||||
/// Not natural parameters: each component is a componentwise maximum of
|
||||
/// `|Δmu|` and `|Δsigma|` over every competitor posterior the sweep
|
||||
/// touched, so both are in skill units and both are non-negative. Each is
|
||||
/// compared against `epsilon` separately — `converged` means neither
|
||||
/// exceeds it. `(0.0, 0.0)` for a history with no time slices.
|
||||
pub final_step: (f64, f64),
|
||||
/// Natural log of the model evidence for the whole history at this fit,
|
||||
/// summed over every time slice.
|
||||
///
|
||||
/// The same quantity
|
||||
/// [`History::log_evidence`](crate::History::log_evidence) returns, taken
|
||||
/// once the sweep has stopped. Only comparable between fits of the same
|
||||
/// events; higher means the model explains them better.
|
||||
pub log_evidence: f64,
|
||||
/// Whether the sweep reached `epsilon` rather than stopping at `max_iter`.
|
||||
///
|
||||
/// Always `true` from [`History::converge`](crate::History::converge),
|
||||
/// which reports the other case as `NotConverged`. From
|
||||
/// [`History::converge_partial`](crate::History::converge_partial) this is
|
||||
/// the only thing that distinguishes a finished fit from a capped one.
|
||||
pub converged: bool,
|
||||
/// Wall-clock time each sweep took, in the order they ran.
|
||||
///
|
||||
/// One entry per iteration, so its length equals `iterations`; empty for a
|
||||
/// history with no time slices. It times the sweeps only, so the final
|
||||
/// log-evidence pass is not in any entry.
|
||||
pub per_iteration_time: SmallVec<[Duration; 32]>,
|
||||
}
|
||||
|
||||
|
||||
+52
-2
@@ -21,8 +21,58 @@ pub trait Drift<T: Time>: Copy + Debug + Send + Sync {
|
||||
///
|
||||
/// For `Time = i64`: variance added is `(to - from) * gamma^2`.
|
||||
/// For `Time = Untimed`: elapsed is always 0, so drift is always 0.
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub struct ConstantDrift(pub f64);
|
||||
///
|
||||
/// # Why the field is private
|
||||
///
|
||||
/// `gamma` enters only as `gamma * gamma`, so a negative value is squared away:
|
||||
/// measured against the old public-field form, `ConstantDrift(-0.0833)` produced
|
||||
/// results **bit identical** to `ConstantDrift(0.0833)`. The sign was neither
|
||||
/// rejected nor honoured — it vanished. That is the same sign-absorption `HistoryBuilder::sigma`,
|
||||
/// `HistoryBuilder::beta`, `Gaussian::from_ms` and `Rating::new` all reject.
|
||||
///
|
||||
/// It could not be checked while the field was a public tuple position, because
|
||||
/// there was no constructor to intercept. Validating inside
|
||||
/// `variance_for_elapsed` would have been worse: it runs inside the sweep, so a
|
||||
/// construction-time mistake would panic mid-inference — and `Gaussian::from_ms`
|
||||
/// is a worked example of why that is the wrong place for a guard, where
|
||||
/// rejecting NaN turned the `NonFiniteResult` reporting path into a crash.
|
||||
///
|
||||
/// So [`ConstantDrift::new`] is the only way in, and it checks. Read the value
|
||||
/// back with [`ConstantDrift::gamma`].
|
||||
///
|
||||
/// A non-finite gamma is caught a second time regardless:
|
||||
/// `History::converge` validates the drift variance each competitor actually
|
||||
/// accumulates, which also covers a custom [`Drift`] implementation.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct ConstantDrift(f64);
|
||||
|
||||
impl ConstantDrift {
|
||||
/// Drift of `gamma` standard deviations per unit time.
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics unless `gamma` is finite and non-negative.
|
||||
///
|
||||
/// The field is private and this is the only constructor precisely so that
|
||||
/// there is somewhere to check. While it was a public tuple field there was
|
||||
/// nothing to intercept, and a negative gamma was silently squared away —
|
||||
/// see the type docs.
|
||||
#[must_use]
|
||||
pub fn new(gamma: f64) -> Self {
|
||||
assert!(
|
||||
gamma.is_finite() && gamma >= 0.0,
|
||||
"gamma must be finite and non-negative (got {gamma}); it is only ever \
|
||||
squared, so a negative value would silently behave as its absolute value"
|
||||
);
|
||||
Self(gamma)
|
||||
}
|
||||
|
||||
/// Standard deviations of drift accumulated per unit time.
|
||||
#[must_use]
|
||||
pub fn gamma(&self) -> f64 {
|
||||
self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: Time> Drift<T> for ConstantDrift {
|
||||
fn variance_delta(&self, from: &T, to: &T) -> f64 {
|
||||
|
||||
+284
-15
@@ -1,45 +1,267 @@
|
||||
use std::fmt;
|
||||
|
||||
/// How a prediction should treat a key the history has never seen.
|
||||
///
|
||||
/// Configured once per history via
|
||||
/// [`HistoryBuilder::unknown_keys`](crate::HistoryBuilder::unknown_keys).
|
||||
/// Neither known consumer wants this to vary between queries — one predicts
|
||||
/// thousands of candidate matchups in a loop, the other's headline feature is
|
||||
/// predicting a competitor nobody has faced — so it is a property of how you
|
||||
/// intend to use the model rather than an argument on five call sites.
|
||||
///
|
||||
/// # There is deliberately no `Skip`
|
||||
///
|
||||
/// Dropping an unknown member is the obvious third option and it is wrong. A
|
||||
/// team's performance is the *sum* of its members, so removing one removes its
|
||||
/// variance too: measured on a two-member team with one unknown, skipping gives
|
||||
/// a performance sigma of 2.37 where treating the member as unknown gives 6.53.
|
||||
/// An unknown competitor would make the model *more* certain, which is
|
||||
/// backwards. `Prior` is also the answer the model already gives for a
|
||||
/// competitor it knows about but has no evidence for, so it corresponds to a
|
||||
/// state the model can actually be in; skipping does not.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
|
||||
#[non_exhaustive]
|
||||
pub enum UnknownKeys {
|
||||
/// Reject the prediction with [`InferenceError::UnknownKey`].
|
||||
///
|
||||
/// The default, and the right one when every key is expected to be known:
|
||||
/// a team of strangers should not silently produce a confident-looking
|
||||
/// answer.
|
||||
#[default]
|
||||
Reject,
|
||||
/// Treat an unknown competitor as one sitting at the history's configured
|
||||
/// prior.
|
||||
///
|
||||
/// This is the honest Bayesian reading — a competitor you have never
|
||||
/// observed is exactly the prior — and it makes "predict a matchup
|
||||
/// involving someone new" a first-class question rather than something a
|
||||
/// caller fakes with a neutral constant.
|
||||
Prior,
|
||||
}
|
||||
|
||||
/// Every way ingestion, inference or prediction can refuse to answer.
|
||||
///
|
||||
/// The crate reports rather than repairs. An input it cannot represent, a fit
|
||||
/// that never reached its fixed point, a quadrature it cannot resolve — each
|
||||
/// comes back here instead of as a clamped, skipped or truncated result that
|
||||
/// would still look like a number. Several variants exist precisely because the
|
||||
/// silent version was measured and found to return a plausible wrong answer.
|
||||
///
|
||||
/// The enum and most of its variants are `#[non_exhaustive]`: new cases and new
|
||||
/// fields are additive, so match with a `_` arm and construct through the
|
||||
/// library rather than by literal.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
#[non_exhaustive]
|
||||
pub enum InferenceError {
|
||||
/// Expected and actual lengths of some array-shaped input differ.
|
||||
#[non_exhaustive]
|
||||
MismatchedShape {
|
||||
/// Which input disagreed, as a short label — `"ranks vs teams"`,
|
||||
/// `"weights"`, `"times"`.
|
||||
kind: &'static str,
|
||||
/// The length it had to have, taken from whatever it must line up with
|
||||
/// (usually the event's team count).
|
||||
expected: usize,
|
||||
/// The length actually supplied.
|
||||
got: usize,
|
||||
},
|
||||
/// An `Outcome` of the wrong variant was supplied for the requested inference.
|
||||
#[non_exhaustive]
|
||||
WrongOutcomeKind {
|
||||
/// The call that rejected the outcome, e.g. `"Game::ranked"`.
|
||||
context: &'static str,
|
||||
/// The [`Outcome`](crate::Outcome) variant that call needs, by name.
|
||||
expected: &'static str,
|
||||
/// The variant actually supplied, by name.
|
||||
got: &'static str,
|
||||
},
|
||||
/// A probability value is outside `[0, 1]`.
|
||||
InvalidProbability { value: f64 },
|
||||
#[non_exhaustive]
|
||||
InvalidProbability {
|
||||
/// The value supplied, as it fell outside `[0, 1]`. Today only
|
||||
/// `p_draw` reaches here.
|
||||
value: f64,
|
||||
},
|
||||
/// A scalar parameter is outside its valid range.
|
||||
InvalidParameter { name: &'static str, value: f64 },
|
||||
#[non_exhaustive]
|
||||
InvalidParameter {
|
||||
/// The parameter, spelled as the API spells it — `"alpha"`,
|
||||
/// `"epsilon"`, `"score_sigma"`, `"drift_scale"`, `"drift variance"`.
|
||||
name: &'static str,
|
||||
/// The value supplied for it. Out of that parameter's range, or NaN,
|
||||
/// which fails every range comparison and is rejected on that basis.
|
||||
value: f64,
|
||||
},
|
||||
/// An event contains tied teams, but the draw probability is zero.
|
||||
///
|
||||
/// A zero draw probability asserts that draws cannot occur, so a tied
|
||||
/// result has no representable likelihood. Configure a positive `p_draw`
|
||||
/// (via `HistoryBuilder::p_draw` or `GameOptions::p_draw`) to admit ties.
|
||||
TieWithoutDrawProbability { teams: (usize, usize) },
|
||||
/// Convergence exceeded `max_iter` without falling below `epsilon`.
|
||||
ConvergenceFailed {
|
||||
last_step: (f64, f64),
|
||||
#[non_exhaustive]
|
||||
TieWithoutDrawProbability {
|
||||
/// Positions in the event's team list of the first tied pair, lowest
|
||||
/// index first. Only one pair is reported — the event is rejected
|
||||
/// whole, so enumerating the rest would add nothing.
|
||||
teams: (usize, usize),
|
||||
},
|
||||
/// The convergence sweep hit `max_iter` with the step still above
|
||||
/// `epsilon`.
|
||||
///
|
||||
/// A fit that stops short is wrong by a little, which is the worst
|
||||
/// available failure: every posterior is finite, the ordering looks sensible,
|
||||
/// and nothing in the numbers says they were still moving. Reported rather
|
||||
/// than returned as a flag on an `Ok`, because a flag has to be checked
|
||||
/// and `let _ = h.converge()` is the natural way not to.
|
||||
///
|
||||
/// Either the history needs more iterations — raise `max_iter` — or it is
|
||||
/// oscillating rather than converging, in which case `alpha < 1.0` damps
|
||||
/// the within-game EP loop. [`History::converge_partial`](crate::History::converge_partial)
|
||||
/// returns the short fit instead when that is genuinely what is wanted.
|
||||
#[non_exhaustive]
|
||||
NotConverged {
|
||||
/// Full forward+backward sweeps run before the loop gave up.
|
||||
iterations: usize,
|
||||
/// How far the last sweep still moved the fit, as
|
||||
/// `(largest change in a mean, largest change in a standard
|
||||
/// deviation)` over every competitor posterior it touched — the same
|
||||
/// quantity as
|
||||
/// [`ConvergenceReport::final_step`](crate::ConvergenceReport).
|
||||
final_step: (f64, f64),
|
||||
/// The threshold both components of `final_step` had to reach.
|
||||
epsilon: f64,
|
||||
},
|
||||
/// Inference produced a non-finite value (NaN or infinity).
|
||||
///
|
||||
/// Indicates numerical breakdown; the resulting skills are meaningless
|
||||
/// and must not be treated as a converged estimate.
|
||||
#[non_exhaustive]
|
||||
NonFiniteResult {
|
||||
/// Where the breakdown was caught — `"History::converge"` for a sweep,
|
||||
/// or a phrase naming the prediction that read an unusable skill.
|
||||
context: &'static str,
|
||||
/// The offending pair, at least one component of which is NaN or
|
||||
/// infinite. From `converge` it is the sweep's step; from a prediction
|
||||
/// it is the skill's own `(mu, sigma)`.
|
||||
step: (f64, f64),
|
||||
},
|
||||
/// Negative precision: a Gaussian with `pi < 0` slipped into an API call.
|
||||
NegativePrecision { pi: f64 },
|
||||
/// One batch declared two different values for the same competitor's
|
||||
/// configuration.
|
||||
///
|
||||
/// `prior` and `drift_scale` configure a competitor, not an event, so a
|
||||
/// batch that sets one of them twice with different values has no
|
||||
/// well-defined meaning: events within a batch are not ordered, so
|
||||
/// "last one wins" would make the result depend on iteration order.
|
||||
/// Declaring the same value repeatedly is fine and is the expected shape
|
||||
/// when a competitor's configuration is a property of the domain.
|
||||
#[non_exhaustive]
|
||||
ConflictingCompetitorConfig {
|
||||
/// The competitor's interned slot as a raw `usize`,
|
||||
/// not the user key — the batch is already flattened to indices by the
|
||||
/// time the conflict is detectable.
|
||||
competitor: usize,
|
||||
/// Which piece of configuration was declared twice: `"prior"` or
|
||||
/// `"drift_scale"`.
|
||||
field: &'static str,
|
||||
},
|
||||
/// A prediction referenced a key the history has no skill for.
|
||||
///
|
||||
/// Reported rather than skipped: dropping unknown keys turns a team of
|
||||
/// strangers into a confident-looking probability about nobody.
|
||||
///
|
||||
/// `key` is the offending key's `Debug` rendering. It is carried because
|
||||
/// the indices alone are not actionable: a caller that logs
|
||||
/// `UnknownKey { team: 0, member: 0 }` learns nothing about *which* of its
|
||||
/// keys the history has not seen, and the natural handling — fall back to a
|
||||
/// neutral value — turns the whole thing into a plausible constant.
|
||||
#[non_exhaustive]
|
||||
UnknownKey {
|
||||
/// Position of the offending team in the supplied matchup. `0` on the
|
||||
/// queries that take a flat list of keys rather than teams, where
|
||||
/// there is only one list to index into.
|
||||
team: usize,
|
||||
/// Position of the offending key within that team, or within the flat
|
||||
/// key list.
|
||||
member: usize,
|
||||
/// The key's `Debug` rendering, captured because `K` is only required
|
||||
/// to be `Debug` — see the variant docs for why the indices alone are
|
||||
/// not enough.
|
||||
key: String,
|
||||
},
|
||||
/// `History::register` was called for a competitor that already exists.
|
||||
///
|
||||
/// Registration states a competitor's configuration before anything has
|
||||
/// been observed about them, so a competitor that already exists has
|
||||
/// already been configured — by an earlier `register`, or by an event that
|
||||
/// created them. Silently overwriting would reintroduce exactly the
|
||||
/// order-dependence registration exists to remove.
|
||||
///
|
||||
/// To change an existing competitor's configuration, supply it on an event
|
||||
/// through `Member`; that refits the whole history.
|
||||
#[non_exhaustive]
|
||||
AlreadyRegistered {
|
||||
/// The already-known competitor's key, in its `Debug` rendering.
|
||||
key: String,
|
||||
},
|
||||
/// A prediction was given a team with no members.
|
||||
#[non_exhaustive]
|
||||
EmptyTeam {
|
||||
/// Position of the memberless team in the supplied list.
|
||||
team: usize,
|
||||
},
|
||||
/// The prediction grid cannot resolve the narrowest feature in the matchup.
|
||||
///
|
||||
/// `predict_outcome` and `predict_ranking` integrate every team's density
|
||||
/// on one shared grid, whose resolution is set by the narrowest sigma (or a
|
||||
/// narrower draw margin). When the widest and narrowest are far enough
|
||||
/// apart, resolving the narrow one across the wide one's support needs more
|
||||
/// nodes than the grid is allowed to hold.
|
||||
///
|
||||
/// Reported rather than clamped. Clamping is what this replaced, and it
|
||||
/// returned probabilities greater than one — measured, a `P` of 2.79 and a
|
||||
/// `Prediction::total()` of 5.41 — because the trapezoid rule stops
|
||||
/// resolving a density once the step exceeds roughly 1.7 of its sigma.
|
||||
///
|
||||
/// `predict_win_probabilities` answers the same matchup through adaptive
|
||||
/// quadrature and is accurate here; use it when only the per-team win
|
||||
/// probabilities are needed.
|
||||
#[non_exhaustive]
|
||||
GridTooCoarse {
|
||||
/// Nodes required to resolve the narrowest feature.
|
||||
needed: usize,
|
||||
/// Nodes the grid may hold.
|
||||
max: usize,
|
||||
},
|
||||
/// A joint posterior was requested where one cannot be formed exactly.
|
||||
#[non_exhaustive]
|
||||
JointUnavailable {
|
||||
/// Why no exact joint exists here: the history has no events, it holds
|
||||
/// ranked events whose EP factors are not retained past convergence, or
|
||||
/// the assembled precision matrix is not positive-definite.
|
||||
reason: &'static str,
|
||||
},
|
||||
/// Fewer than two teams were supplied to a prediction.
|
||||
#[non_exhaustive]
|
||||
NotEnoughTeams {
|
||||
/// How many teams the prediction was actually given. Two is the
|
||||
/// minimum: there is nothing to compare against with fewer.
|
||||
got: usize,
|
||||
},
|
||||
/// The full outcome distribution was requested for too many teams.
|
||||
///
|
||||
/// Each realisation sorts into exactly one (order, tie-pattern) event, so
|
||||
/// the space holds `n! * 2^(n-1)` members — 1_920 at five teams, 23_040 at
|
||||
/// six, 322_560 at seven. Past `max` this stops being something to
|
||||
/// enumerate on a caller's behalf; ask for individual rankings with
|
||||
/// `predict_ranking`, or for `predict_win_probabilities`, both of which
|
||||
/// stay cheap at any team count.
|
||||
#[non_exhaustive]
|
||||
TooManyTeams {
|
||||
/// How many teams the outcome distribution was asked for.
|
||||
got: usize,
|
||||
/// The largest team count that will be enumerated,
|
||||
/// [`MAX_PREDICTED_TEAMS`](crate::MAX_PREDICTED_TEAMS).
|
||||
max: usize,
|
||||
},
|
||||
}
|
||||
|
||||
impl fmt::Display for InferenceError {
|
||||
@@ -69,6 +291,18 @@ impl fmt::Display for InferenceError {
|
||||
teams.0, teams.1
|
||||
)
|
||||
}
|
||||
Self::NotConverged {
|
||||
iterations,
|
||||
final_step,
|
||||
epsilon,
|
||||
} => {
|
||||
write!(
|
||||
f,
|
||||
"did not converge in {iterations} iterations: final step {final_step:?} \
|
||||
is still above epsilon {epsilon}; raise max_iter, or damp with \
|
||||
alpha < 1.0 if it is oscillating"
|
||||
)
|
||||
}
|
||||
Self::NonFiniteResult { context, step } => {
|
||||
write!(
|
||||
f,
|
||||
@@ -78,17 +312,52 @@ impl fmt::Display for InferenceError {
|
||||
Self::InvalidParameter { name, value } => {
|
||||
write!(f, "{name} is invalid: {value}")
|
||||
}
|
||||
Self::ConvergenceFailed {
|
||||
last_step,
|
||||
iterations,
|
||||
} => {
|
||||
Self::ConflictingCompetitorConfig { competitor, field } => {
|
||||
write!(
|
||||
f,
|
||||
"convergence failed after {iterations} iterations; last step = {last_step:?}"
|
||||
"competitor {competitor}: this batch sets {field} to two different values"
|
||||
)
|
||||
}
|
||||
Self::NegativePrecision { pi } => {
|
||||
write!(f, "precision must be non-negative; got {pi}")
|
||||
Self::UnknownKey { team, member, key } => {
|
||||
write!(
|
||||
f,
|
||||
"team {team}, member {member}: no skill recorded for key {key} \
|
||||
(every key must already be known to the history; pre-filter \
|
||||
with `current_skill` if that is not guaranteed)"
|
||||
)
|
||||
}
|
||||
Self::AlreadyRegistered { key } => {
|
||||
write!(
|
||||
f,
|
||||
"competitor {key} is already registered; registration states \
|
||||
configuration before anything is observed, so re-registering \
|
||||
would silently overwrite it"
|
||||
)
|
||||
}
|
||||
Self::EmptyTeam { team } => {
|
||||
write!(f, "team {team} has no members")
|
||||
}
|
||||
Self::GridTooCoarse { needed, max } => {
|
||||
write!(
|
||||
f,
|
||||
"the prediction grid needs {needed} nodes to resolve the narrowest \
|
||||
team's density across the widest team's support, but may hold only \
|
||||
{max}; the sigmas in this matchup are too far apart to integrate on \
|
||||
one grid. Use predict_win_probabilities, which is accurate here"
|
||||
)
|
||||
}
|
||||
Self::JointUnavailable { reason } => {
|
||||
write!(f, "no exact joint posterior is available: {reason}")
|
||||
}
|
||||
Self::NotEnoughTeams { got } => {
|
||||
write!(f, "prediction needs at least 2 teams, got {got}")
|
||||
}
|
||||
Self::TooManyTeams { got, max } => {
|
||||
write!(
|
||||
f,
|
||||
"the outcome distribution over {got} teams is too large to enumerate (limit {max}); \
|
||||
use predict_ranking or predict_win_probabilities instead"
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+86
-14
@@ -1,35 +1,69 @@
|
||||
//! Typed event description for bulk ingestion.
|
||||
//!
|
||||
//! `Event<T, K>` is the new public event shape (spec Section 4). Replaces
|
||||
//! the nested `Vec<Vec<Vec<Index>>>`, `Vec<Vec<f64>>`, `Vec<Vec<Vec<f64>>>`
|
||||
//! that the old `add_events_with_prior` took.
|
||||
//! `Event<T, K>` is the public event shape taken by `History::add_events`. It
|
||||
//! is a typed front end, not a replacement: `add_events` flattens it into the
|
||||
//! nested `Vec<Vec<Vec<Index>>>` / `Vec<Vec<f64>>` / `Vec<Vec<Vec<f64>>>` that
|
||||
//! the internal `add_events_with_prior` chokepoint still takes, and which
|
||||
//! `record_winner` and `record_draw` also route through.
|
||||
|
||||
use smallvec::SmallVec;
|
||||
|
||||
use crate::{gaussian::Gaussian, outcome::Outcome, time::Time};
|
||||
|
||||
/// A single match at time `time` involving some number of teams.
|
||||
#[derive(Clone, Debug)]
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct Event<T: Time, K> {
|
||||
/// When the match happened, on the history's time axis.
|
||||
///
|
||||
/// Events sharing a `time` land in the same time slice and are fitted
|
||||
/// together, so nothing distinguishes their order. Drift is driven by the
|
||||
/// gap between a competitor's *consecutive appearances*, not by the gap
|
||||
/// between slices, so a competitor idle across several slices accumulates
|
||||
/// the whole span at once when it next plays.
|
||||
pub time: T,
|
||||
/// The teams that took part, positionally aligned with `outcome`: team `i`
|
||||
/// here is the team `outcome` ranks or scores at index `i`.
|
||||
///
|
||||
/// Ingestion rejects fewer than two teams (`NotEnoughTeams`) and any team
|
||||
/// with no members (`EmptyTeam`).
|
||||
pub teams: SmallVec<[Team<K>; 4]>,
|
||||
/// How the match ended: ranks (lower is better) or per-team scores (higher
|
||||
/// is better), one entry per entry of `teams`.
|
||||
///
|
||||
/// A tie — two equal ranks — needs a positive `p_draw`, otherwise
|
||||
/// ingestion fails with `TieWithoutDrawProbability`.
|
||||
pub outcome: Outcome,
|
||||
}
|
||||
|
||||
/// A team: list of members competing together.
|
||||
#[derive(Clone, Debug)]
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
#[must_use]
|
||||
pub struct Team<K> {
|
||||
/// The competitors playing together, in no significant order: the team's
|
||||
/// performance is the weight-scaled sum over its members, which does not
|
||||
/// depend on how they are listed.
|
||||
///
|
||||
/// Must be non-empty — an empty team contributes no performance at all, so
|
||||
/// ingestion rejects it with `EmptyTeam` rather than returning a plausible
|
||||
/// posterior for whoever it was matched against.
|
||||
pub members: SmallVec<[Member<K>; 4]>,
|
||||
}
|
||||
|
||||
impl<K> Team<K> {
|
||||
#[must_use]
|
||||
/// A team with no members yet, to be filled through the public `members`
|
||||
/// field.
|
||||
///
|
||||
/// Committing it while still empty is an `EmptyTeam` error.
|
||||
pub fn new() -> Self {
|
||||
Self {
|
||||
members: SmallVec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// A team of exactly these competitors.
|
||||
///
|
||||
/// Members must be built already — `Member::from(key)` covers the common
|
||||
/// case of a plain key at default weight with no overrides.
|
||||
pub fn with_members<I: IntoIterator<Item = Member<K>>>(members: I) -> Self {
|
||||
Self {
|
||||
members: members.into_iter().collect(),
|
||||
@@ -48,13 +82,40 @@ impl<K> Default for Team<K> {
|
||||
/// `weight` applies per event and defaults to 1.0.
|
||||
///
|
||||
/// `prior` and `drift_scale` are **competitor configuration**, not per-event
|
||||
/// values: both are captured when the competitor is first created and ignored
|
||||
/// on every later appearance. Setting either on a key the history already knows
|
||||
/// has no effect.
|
||||
#[derive(Clone, Debug)]
|
||||
/// values. Setting either applies to the competitor for the whole history, not
|
||||
/// just to this event, and applies whenever it is supplied — including on a key
|
||||
/// the history already knows. Because configuration lives on the competitor and
|
||||
/// `converge` refits from competitor state, configuring one late still refits
|
||||
/// the whole history rather than taking effect only from that event onward.
|
||||
///
|
||||
/// Repeating the same value is inert, which is the expected shape when the
|
||||
/// configuration is a property of the domain. Supplying two *different* values
|
||||
/// for one competitor within a single batch is
|
||||
/// `InferenceError::ConflictingCompetitorConfig`: events in a batch have no
|
||||
/// order, so there would be no well-defined winner.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
#[must_use]
|
||||
pub struct Member<K> {
|
||||
/// The competitor's identity. Equal keys across events are the same
|
||||
/// competitor: `History` interns each distinct key to an internal `Index`
|
||||
/// the first time it sees it, and every later appearance resolves to that
|
||||
/// same competitor's temporal state.
|
||||
pub key: K,
|
||||
/// This member's share of the team's performance, for this event only.
|
||||
///
|
||||
/// The team's performance is the sum of `weight × member performance`, so
|
||||
/// `1.0` is a full share and `0.5` counts the member half; the message
|
||||
/// coming back to the member is divided by the same weight. Defaults to
|
||||
/// `1.0`.
|
||||
///
|
||||
/// Must be finite — a NaN or infinite weight is `InvalidParameter` at
|
||||
/// ingestion. Zero and negative are accepted, both being expressible in
|
||||
/// the same arithmetic.
|
||||
pub weight: f64,
|
||||
/// Starting skill for this competitor, replacing the history's `mu`/`sigma`
|
||||
/// default. `None` keeps the history default.
|
||||
///
|
||||
/// Competitor configuration, not a per-event value; see the type docs.
|
||||
pub prior: Option<Gaussian>,
|
||||
/// Multiplier on the drift *variance* this competitor accumulates.
|
||||
/// `None` means 1.0.
|
||||
@@ -62,6 +123,8 @@ pub struct Member<K> {
|
||||
}
|
||||
|
||||
impl<K> Member<K> {
|
||||
/// A competitor taking a full share of its team's performance, with no
|
||||
/// configuration overrides: the history's prior and drift apply.
|
||||
pub fn new(key: K) -> Self {
|
||||
Self {
|
||||
key,
|
||||
@@ -71,6 +134,12 @@ impl<K> Member<K> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Change how much of the team's performance this member accounts for.
|
||||
///
|
||||
/// Unlike `prior` and `drift_scale`, this is genuinely per-event: the same
|
||||
/// key can carry a different weight in every event it appears in, which is
|
||||
/// what makes it usable for partial participation — a substitute who
|
||||
/// played half the match, a doubles partner credited unequally.
|
||||
pub fn with_weight(mut self, weight: f64) -> Self {
|
||||
self.weight = weight;
|
||||
self
|
||||
@@ -78,7 +147,9 @@ impl<K> Member<K> {
|
||||
|
||||
/// Set this competitor's starting skill estimate.
|
||||
///
|
||||
/// Captured at the competitor's first appearance; see the type docs.
|
||||
/// Competitor configuration, not a per-event value: it applies for the
|
||||
/// whole history and applies whenever it is supplied, including on a key
|
||||
/// the history already knows. See the type docs.
|
||||
pub fn with_prior(mut self, prior: Gaussian) -> Self {
|
||||
self.prior = Some(prior);
|
||||
self
|
||||
@@ -87,14 +158,15 @@ impl<K> Member<K> {
|
||||
/// Scale how fast this competitor drifts, relative to the history's drift.
|
||||
///
|
||||
/// The scale multiplies the drift *variance*, so it is in the same units as
|
||||
/// `gamma`: `ConstantDrift(g)` at `scale = s` behaves exactly as
|
||||
/// `ConstantDrift(g * s)` would for this competitor alone.
|
||||
/// `gamma`: `ConstantDrift::new(g)` at `scale = s` behaves exactly as
|
||||
/// `ConstantDrift::new(g * s)` would for this competitor alone.
|
||||
///
|
||||
/// `0.0` pins the competitor still — useful for a reference point that
|
||||
/// shares a scale with moving competitors but should not itself move: a bot
|
||||
/// at a known strength, a rating floor, a course difficulty.
|
||||
///
|
||||
/// Captured at the competitor's first appearance; see the type docs.
|
||||
/// Applies for the whole history and whenever it is supplied, including on
|
||||
/// a key the history already knows; see the type docs.
|
||||
/// Must be finite and non-negative, or ingestion fails with
|
||||
/// [`InferenceError::InvalidParameter`](crate::InferenceError::InvalidParameter).
|
||||
pub fn with_drift_scale(mut self, scale: f64) -> Self {
|
||||
|
||||
+80
-4
@@ -9,6 +9,35 @@ use crate::{
|
||||
time::Time,
|
||||
};
|
||||
|
||||
/// One match under construction, handed back by [`History::event`].
|
||||
///
|
||||
/// Describes a single event a piece at a time — teams, then per-member weights
|
||||
/// if they differ, then how it ended — instead of assembling an
|
||||
/// [`Event`] value and passing it to [`History::add_events`]. The two routes
|
||||
/// ingest through the same chokepoint and accept the same things; this one just
|
||||
/// reads better for a single match written by hand.
|
||||
///
|
||||
/// The builder borrows the history mutably and nothing reaches it until
|
||||
/// [`EventBuilder::commit`]. A builder that is dropped instead ingests
|
||||
/// nothing at all, silently — hence the `#[must_use]`, which is the only
|
||||
/// warning you get. `commit` is also where validation surfaces: the setters
|
||||
/// return `Self` to keep the chain fluent, so a mismatch such as a weight list
|
||||
/// the wrong length is recorded while building and returned as an error from
|
||||
/// `commit`.
|
||||
///
|
||||
/// ```
|
||||
/// # use trueskill_tt::History;
|
||||
/// let mut h = History::builder().build();
|
||||
/// h.event(1)
|
||||
/// .team(["alice", "bob"])
|
||||
/// .team(["carol"])
|
||||
/// .ranking([0, 1])
|
||||
/// .commit()?;
|
||||
/// assert_eq!(h.event_count(), 1);
|
||||
/// # Ok::<(), trueskill_tt::InferenceError>(())
|
||||
/// ```
|
||||
#[must_use = "an event is only recorded by `.commit()`; a dropped builder \
|
||||
silently ingests nothing"]
|
||||
pub struct EventBuilder<'h, T, D, O, K>
|
||||
where
|
||||
T: Time,
|
||||
@@ -50,6 +79,8 @@ where
|
||||
}
|
||||
|
||||
/// Add a team by its member keys (weight 1.0 each, no prior overrides).
|
||||
///
|
||||
/// Use [`EventBuilder::members`] to set `prior` or `drift_scale`.
|
||||
pub fn team<I: IntoIterator<Item = K>>(mut self, keys: I) -> Self {
|
||||
let members: SmallVec<[Member<K>; 4]> = keys.into_iter().map(Member::new).collect();
|
||||
self.event.teams.push(Team { members });
|
||||
@@ -57,6 +88,40 @@ where
|
||||
self
|
||||
}
|
||||
|
||||
/// Add a team from fully-specified [`Member`] values.
|
||||
///
|
||||
/// [`EventBuilder::team`] is the common case and builds members with
|
||||
/// `Member::new`, which leaves `prior` and `drift_scale` unset. This is the
|
||||
/// escape hatch for when they matter:
|
||||
///
|
||||
/// ```
|
||||
/// # use trueskill_tt::{Gaussian, History, Member};
|
||||
/// # let mut h = History::builder().build();
|
||||
/// h.event(0)
|
||||
/// .team(["player"])
|
||||
/// .members([Member::new("layout_7")
|
||||
/// .with_drift_scale(0.0)
|
||||
/// .with_prior(Gaussian::from_ms(0.0, 1.0))])
|
||||
/// .ranking([0, 1])
|
||||
/// .commit()?;
|
||||
/// # Ok::<(), trueskill_tt::InferenceError>(())
|
||||
/// ```
|
||||
///
|
||||
/// One method rather than a `priors` and a `drift_scales` setter beside
|
||||
/// `weights`: those would have to grow a parallel array — and a parallel
|
||||
/// length check — every time `Member` gains a field, and each one would be
|
||||
/// a new way to get the lengths wrong. `Member`'s own builder already
|
||||
/// expresses all of it.
|
||||
///
|
||||
/// `prior` and `drift_scale` are competitor configuration rather than
|
||||
/// per-event values; see [`Member`] for what that means for a key the
|
||||
/// history already knows.
|
||||
pub fn members<I: IntoIterator<Item = Member<K>>>(mut self, members: I) -> Self {
|
||||
self.event.teams.push(Team::with_members(members));
|
||||
self.current_team_idx = Some(self.event.teams.len() - 1);
|
||||
self
|
||||
}
|
||||
|
||||
/// Set per-member weights for the most recently added team.
|
||||
///
|
||||
/// A length mismatch is recorded and returned by [`EventBuilder::commit`]
|
||||
@@ -106,10 +171,21 @@ where
|
||||
|
||||
/// Set explicit per-team continuous scores with a per-event noise override.
|
||||
///
|
||||
/// `sigma` overrides `HistoryBuilder::score_sigma` for this event only.
|
||||
/// Must be `> 0.0`; debug-asserts otherwise via `Outcome::scores_with_sigma`.
|
||||
pub fn scores_with_sigma<I: IntoIterator<Item = f64>>(mut self, scores: I, sigma: f64) -> Self {
|
||||
self.event.outcome = crate::Outcome::scores_with_sigma(scores, sigma);
|
||||
/// `score_sigma` is the observation noise on the *score margin*, not a
|
||||
/// skill sigma, and it overrides `HistoryBuilder::score_sigma` for this
|
||||
/// event only. A small value takes the margin near-literally; a large one
|
||||
/// barely moves the ratings.
|
||||
///
|
||||
/// Must be `> 0.0`. Building the outcome with a non-positive or NaN value
|
||||
/// is allowed; it is rejected with `InferenceError::InvalidParameter` when
|
||||
/// the event is ingested, so callers get an error from `commit` rather
|
||||
/// than a panic.
|
||||
pub fn scores_with_noise<I: IntoIterator<Item = f64>>(
|
||||
mut self,
|
||||
scores: I,
|
||||
score_sigma: f64,
|
||||
) -> Self {
|
||||
self.event.outcome = crate::Outcome::scores_with_noise(scores, score_sigma);
|
||||
self
|
||||
}
|
||||
|
||||
|
||||
@@ -20,6 +20,8 @@ pub struct VarStore {
|
||||
}
|
||||
|
||||
impl VarStore {
|
||||
/// Test-only: inference allocates its store through `ScratchArena`.
|
||||
#[cfg(test)]
|
||||
#[must_use]
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
@@ -29,23 +31,19 @@ impl VarStore {
|
||||
self.marginals.clear();
|
||||
}
|
||||
|
||||
/// Test-only, as `new`.
|
||||
#[cfg(test)]
|
||||
#[must_use]
|
||||
pub fn len(&self) -> usize {
|
||||
self.marginals.len()
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.marginals.is_empty()
|
||||
}
|
||||
|
||||
pub fn alloc(&mut self, init: Gaussian) -> VarId {
|
||||
let id = VarId(self.marginals.len() as u32);
|
||||
self.marginals.push(init);
|
||||
id
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn get(&self, id: VarId) -> Gaussian {
|
||||
self.marginals[id.0 as usize]
|
||||
}
|
||||
@@ -55,58 +53,7 @@ impl VarStore {
|
||||
}
|
||||
}
|
||||
|
||||
/// A factor in the EP graph.
|
||||
///
|
||||
/// Factors hold their own outgoing messages and propagate them by reading
|
||||
/// connected variable marginals from a `VarStore` and writing back updated
|
||||
/// marginals.
|
||||
pub trait Factor: Send + Sync {
|
||||
/// Update outgoing messages and write back to the var store.
|
||||
///
|
||||
/// Returns the max delta `(|Δmu|, |Δsigma|)` across writes this
|
||||
/// propagation. Used by the `Schedule` to detect convergence.
|
||||
fn propagate(&mut self, vars: &mut VarStore) -> (f64, f64);
|
||||
|
||||
/// Optional log-evidence contribution. Default 0.0 (no contribution).
|
||||
fn log_evidence(&self, _vars: &VarStore) -> f64 {
|
||||
0.0
|
||||
}
|
||||
}
|
||||
|
||||
/// Enum dispatcher for the built-in factor types.
|
||||
///
|
||||
/// Using an enum instead of `Box<dyn Factor>` keeps factor data inline and
|
||||
/// avoids virtual-call overhead in the hot inference loop.
|
||||
#[derive(Debug)]
|
||||
pub enum BuiltinFactor {
|
||||
TeamSum(team_sum::TeamSumFactor),
|
||||
RankDiff(rank_diff::RankDiffFactor),
|
||||
Trunc(trunc::TruncFactor),
|
||||
Margin(margin::MarginFactor),
|
||||
}
|
||||
|
||||
impl Factor for BuiltinFactor {
|
||||
fn propagate(&mut self, vars: &mut VarStore) -> (f64, f64) {
|
||||
match self {
|
||||
Self::TeamSum(f) => f.propagate(vars),
|
||||
Self::RankDiff(f) => f.propagate(vars),
|
||||
Self::Trunc(f) => f.propagate(vars),
|
||||
Self::Margin(f) => f.propagate(vars),
|
||||
}
|
||||
}
|
||||
|
||||
fn log_evidence(&self, vars: &VarStore) -> f64 {
|
||||
match self {
|
||||
Self::Trunc(f) => f.log_evidence(vars),
|
||||
Self::Margin(f) => f.log_evidence(vars),
|
||||
Self::TeamSum(_) | Self::RankDiff(_) => 0.0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub mod margin;
|
||||
pub mod rank_diff;
|
||||
pub mod team_sum;
|
||||
pub mod trunc;
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -153,20 +100,4 @@ mod tests {
|
||||
assert_eq!(store.len(), 0);
|
||||
assert_eq!(store.marginals.capacity(), cap);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn builtin_factor_dispatches_to_margin() {
|
||||
use super::margin::MarginFactor;
|
||||
let mut vars = VarStore::new();
|
||||
let diff = vars.alloc(Gaussian::from_ms(0.0, 6.0));
|
||||
let mut f = BuiltinFactor::Margin(MarginFactor::new(diff, 5.0, 1.0));
|
||||
|
||||
f.propagate(&mut vars);
|
||||
|
||||
let result = vars.get(diff);
|
||||
assert!((result.mu() - 4.864864864864865).abs() < 1e-12);
|
||||
|
||||
let logz = f.log_evidence(&vars);
|
||||
assert!((logz - (-3.062235327364623)).abs() < 1e-10);
|
||||
}
|
||||
}
|
||||
+43
-24
@@ -1,8 +1,8 @@
|
||||
use crate::{
|
||||
N_INF,
|
||||
factor::{Factor, VarId, VarStore},
|
||||
factor::{VarId, VarStore},
|
||||
gaussian::Gaussian,
|
||||
pdf,
|
||||
ln_pdf,
|
||||
};
|
||||
|
||||
/// Gaussian observation factor on a diff variable.
|
||||
@@ -16,7 +16,7 @@ pub struct MarginFactor {
|
||||
pub m_obs: f64,
|
||||
pub sigma: f64,
|
||||
pub(crate) msg: Gaussian,
|
||||
pub(crate) evidence_cached: Option<f64>,
|
||||
pub(crate) log_evidence_cached: Option<f64>,
|
||||
}
|
||||
|
||||
impl MarginFactor {
|
||||
@@ -28,7 +28,7 @@ impl MarginFactor {
|
||||
m_obs,
|
||||
sigma,
|
||||
msg: N_INF,
|
||||
evidence_cached: None,
|
||||
log_evidence_cached: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -39,39 +39,58 @@ impl MarginFactor {
|
||||
/// exactly; `alpha < 1.0` writes `α·new_msg + (1−α)·old_msg`.
|
||||
pub(crate) fn propagate_with_alpha(&mut self, vars: &mut VarStore, alpha: f64) -> (f64, f64) {
|
||||
let marginal = vars.get(self.diff);
|
||||
let cavity = marginal / self.msg;
|
||||
let cavity = marginal.cavity(self.msg);
|
||||
|
||||
if self.evidence_cached.is_none() {
|
||||
self.evidence_cached = Some(cavity_evidence(cavity, self.m_obs, self.sigma));
|
||||
if self.log_evidence_cached.is_none() {
|
||||
self.log_evidence_cached = Some(cavity_log_evidence(cavity, self.m_obs, self.sigma));
|
||||
}
|
||||
|
||||
let new_msg = Gaussian::from_ms(self.m_obs, self.sigma);
|
||||
let damped = self.msg.damp_natural(new_msg, alpha);
|
||||
let old_msg = self.msg;
|
||||
self.msg = damped;
|
||||
vars.set(self.diff, cavity * damped);
|
||||
vars.set(self.diff, cavity.ep_product(damped));
|
||||
|
||||
old_msg.delta(damped)
|
||||
}
|
||||
}
|
||||
|
||||
impl Factor for MarginFactor {
|
||||
fn propagate(&mut self, vars: &mut VarStore) -> (f64, f64) {
|
||||
/// Undamped wrappers, used by this module's tests. Inference drives these
|
||||
/// factors through `propagate_with_alpha` and reads the cached log evidence
|
||||
/// directly, so these are not on any production path.
|
||||
#[cfg(test)]
|
||||
impl MarginFactor {
|
||||
pub(crate) fn propagate(&mut self, vars: &mut VarStore) -> (f64, f64) {
|
||||
self.propagate_with_alpha(vars, 1.0)
|
||||
}
|
||||
|
||||
fn log_evidence(&self, _vars: &VarStore) -> f64 {
|
||||
self.evidence_cached.unwrap_or(1.0).ln()
|
||||
pub(crate) fn log_evidence(&self) -> f64 {
|
||||
self.log_evidence_cached.unwrap_or(0.0)
|
||||
}
|
||||
}
|
||||
|
||||
/// Density of the observed margin under the cavity, clamped to a positive
|
||||
/// floor so a far-out observation cannot underflow to `0.0` and make
|
||||
/// `log_evidence` `-inf`.
|
||||
fn cavity_evidence(cavity: Gaussian, m_obs: f64, sigma: f64) -> f64 {
|
||||
let combined_sigma = (cavity.sigma().powi(2) + sigma.powi(2)).sqrt();
|
||||
/// `ln` of the observed margin's density under the cavity.
|
||||
///
|
||||
/// Computed in log space rather than as `pdf(..).ln()`. The density underflows
|
||||
/// to zero past about 38 sigma of separation, and clamping that to
|
||||
/// `f64::MIN_POSITIVE` reported -708 nats however far out the observation
|
||||
/// actually was — 4292 nats adrift at 100 sigma, and unbounded beyond. A score
|
||||
/// far from what the model expected is exactly the observation a log-evidence
|
||||
/// figure exists to notice.
|
||||
fn cavity_log_evidence(cavity: Gaussian, m_obs: f64, sigma: f64) -> f64 {
|
||||
// `hypot`, not `sqrt(a^2 + b^2)`: squaring overflows to infinity above a
|
||||
// sigma of ~1.3e154 and flushes to zero below ~1.5e-154, and `Gaussian`'s
|
||||
// constructors are public so a caller can reach both.
|
||||
let combined_sigma = libm::hypot(cavity.sigma(), sigma);
|
||||
let value = ln_pdf(m_obs, cavity.mu(), combined_sigma);
|
||||
|
||||
pdf(m_obs, cavity.mu(), combined_sigma).max(f64::MIN_POSITIVE)
|
||||
// A degenerate cavity (infinite sigma) is the only way to reach a
|
||||
// non-finite result; fall back to the old floor rather than emit -inf.
|
||||
if value.is_finite() {
|
||||
value
|
||||
} else {
|
||||
libm::log(f64::MIN_POSITIVE)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -113,16 +132,16 @@ mod tests {
|
||||
let mut vars = VarStore::new();
|
||||
let diff = vars.alloc(Gaussian::from_ms(0.0, 6.0));
|
||||
let mut f = MarginFactor::new(diff, 5.0, 1.0);
|
||||
assert!(f.evidence_cached.is_none());
|
||||
assert!(f.log_evidence_cached.is_none());
|
||||
|
||||
f.propagate(&mut vars);
|
||||
let z = f.evidence_cached.unwrap();
|
||||
// pdf(5, 0, sqrt(37)) ≈ 0.046783
|
||||
assert!((z - 0.04678300292616668).abs() < 1e-10);
|
||||
let z = f.log_evidence_cached.unwrap();
|
||||
// ln pdf(5, 0, sqrt(37)) = ln(0.046783...)
|
||||
assert!((z.exp() - 0.04678300292616668).abs() < 1e-10);
|
||||
|
||||
// Subsequent propagations don't change it.
|
||||
f.propagate(&mut vars);
|
||||
assert_eq!(f.evidence_cached.unwrap(), z);
|
||||
assert_eq!(f.log_evidence_cached.unwrap(), z);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -131,7 +150,7 @@ mod tests {
|
||||
let diff = vars.alloc(Gaussian::from_ms(0.0, 6.0));
|
||||
let mut f = MarginFactor::new(diff, 5.0, 1.0);
|
||||
f.propagate(&mut vars);
|
||||
let logz = f.log_evidence(&vars);
|
||||
let logz = f.log_evidence();
|
||||
assert!((logz - (-3.062235327364623)).abs() < 1e-10);
|
||||
}
|
||||
|
||||
|
||||
@@ -1,95 +0,0 @@
|
||||
use crate::factor::{Factor, VarId, VarStore};
|
||||
|
||||
/// Maintains the constraint `diff = team_a - team_b` between three vars.
|
||||
///
|
||||
/// On each propagation:
|
||||
/// - Reads marginals at `team_a` and `team_b` (which already incorporate any
|
||||
/// incoming messages from neighboring factors).
|
||||
/// - Computes `new_diff = team_a - team_b` (variance addition; see `Gaussian::Sub`).
|
||||
/// - Writes the new marginal to `diff`.
|
||||
/// - Returns the delta against the previous diff value.
|
||||
///
|
||||
/// This factor does NOT store an outgoing message; the diff variable is
|
||||
/// effectively replaced on each propagation. The `TruncFactor` on the same diff
|
||||
/// var holds the EP-divide message that produces the cavity.
|
||||
#[derive(Debug)]
|
||||
pub struct RankDiffFactor {
|
||||
pub team_a: VarId,
|
||||
pub team_b: VarId,
|
||||
pub diff: VarId,
|
||||
}
|
||||
|
||||
impl Factor for RankDiffFactor {
|
||||
fn propagate(&mut self, vars: &mut VarStore) -> (f64, f64) {
|
||||
let a = vars.get(self.team_a);
|
||||
let b = vars.get(self.team_b);
|
||||
let new_diff = a - b;
|
||||
let old = vars.get(self.diff);
|
||||
vars.set(self.diff, new_diff);
|
||||
old.delta(new_diff)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::{N_INF, gaussian::Gaussian};
|
||||
|
||||
#[test]
|
||||
fn diff_of_two_known_gaussians() {
|
||||
let mut vars = VarStore::new();
|
||||
let team_a = vars.alloc(Gaussian::from_ms(25.0, 3.0));
|
||||
let team_b = vars.alloc(Gaussian::from_ms(20.0, 4.0));
|
||||
let diff = vars.alloc(N_INF);
|
||||
|
||||
let mut f = RankDiffFactor {
|
||||
team_a,
|
||||
team_b,
|
||||
diff,
|
||||
};
|
||||
f.propagate(&mut vars);
|
||||
|
||||
let result = vars.get(diff);
|
||||
// mu = 25 - 20 = 5; var = 9 + 16 = 25; sigma = 5
|
||||
assert!((result.mu() - 5.0).abs() < 1e-12);
|
||||
assert!((result.sigma() - 5.0).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delta_zero_on_repeat() {
|
||||
let mut vars = VarStore::new();
|
||||
let team_a = vars.alloc(Gaussian::from_ms(10.0, 2.0));
|
||||
let team_b = vars.alloc(Gaussian::from_ms(8.0, 1.0));
|
||||
let diff = vars.alloc(N_INF);
|
||||
|
||||
let mut f = RankDiffFactor {
|
||||
team_a,
|
||||
team_b,
|
||||
diff,
|
||||
};
|
||||
f.propagate(&mut vars);
|
||||
let (dmu, dsig) = f.propagate(&mut vars);
|
||||
assert!(dmu < 1e-12);
|
||||
assert!(dsig < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delta_reflects_team_change() {
|
||||
let mut vars = VarStore::new();
|
||||
let team_a = vars.alloc(Gaussian::from_ms(10.0, 1.0));
|
||||
let team_b = vars.alloc(Gaussian::from_ms(0.0, 1.0));
|
||||
let diff = vars.alloc(N_INF);
|
||||
|
||||
let mut f = RankDiffFactor {
|
||||
team_a,
|
||||
team_b,
|
||||
diff,
|
||||
};
|
||||
f.propagate(&mut vars);
|
||||
|
||||
// change team_a, repropagate; delta should be positive
|
||||
vars.set(team_a, Gaussian::from_ms(15.0, 1.0));
|
||||
let (dmu, _dsig) = f.propagate(&mut vars);
|
||||
assert!(dmu > 4.0, "expected ~5 delta, got {}", dmu);
|
||||
}
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
use crate::{
|
||||
N00,
|
||||
factor::{Factor, VarId, VarStore},
|
||||
gaussian::Gaussian,
|
||||
};
|
||||
|
||||
/// Computes the weighted sum of player performances into a team-perf var.
|
||||
///
|
||||
/// Inputs are pre-computed player performance Gaussians (i.e., rating priors
|
||||
/// already with beta² noise added via `Rating::performance()`). The factor
|
||||
/// runs once per game and writes the weighted sum to the output var.
|
||||
#[derive(Debug)]
|
||||
pub struct TeamSumFactor {
|
||||
pub inputs: Vec<(Gaussian, f64)>,
|
||||
pub out: VarId,
|
||||
}
|
||||
|
||||
impl Factor for TeamSumFactor {
|
||||
fn propagate(&mut self, vars: &mut VarStore) -> (f64, f64) {
|
||||
let perf = self.inputs.iter().fold(N00, |acc, (g, w)| acc + (*g * *w));
|
||||
let old = vars.get(self.out);
|
||||
vars.set(self.out, perf);
|
||||
old.delta(perf)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::N_INF;
|
||||
|
||||
#[test]
|
||||
fn single_player_unit_weight() {
|
||||
let mut vars = VarStore::new();
|
||||
let out = vars.alloc(N_INF);
|
||||
let g = Gaussian::from_ms(25.0, 5.0);
|
||||
let mut f = TeamSumFactor {
|
||||
inputs: vec![(g, 1.0)],
|
||||
out,
|
||||
};
|
||||
|
||||
f.propagate(&mut vars);
|
||||
let result = vars.get(out);
|
||||
assert!((result.mu() - 25.0).abs() < 1e-12);
|
||||
assert!((result.sigma() - 5.0).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn two_players_summed() {
|
||||
let mut vars = VarStore::new();
|
||||
let out = vars.alloc(N_INF);
|
||||
let g1 = Gaussian::from_ms(20.0, 3.0);
|
||||
let g2 = Gaussian::from_ms(30.0, 4.0);
|
||||
let mut f = TeamSumFactor {
|
||||
inputs: vec![(g1, 1.0), (g2, 1.0)],
|
||||
out,
|
||||
};
|
||||
|
||||
f.propagate(&mut vars);
|
||||
let result = vars.get(out);
|
||||
// sum: mu = 20 + 30 = 50, var = 9 + 16 = 25, sigma = 5
|
||||
assert!((result.mu() - 50.0).abs() < 1e-12);
|
||||
assert!((result.sigma() - 5.0).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn weighted_inputs() {
|
||||
let mut vars = VarStore::new();
|
||||
let out = vars.alloc(N_INF);
|
||||
let g = Gaussian::from_ms(10.0, 2.0);
|
||||
let mut f = TeamSumFactor {
|
||||
inputs: vec![(g, 2.0)],
|
||||
out,
|
||||
};
|
||||
|
||||
f.propagate(&mut vars);
|
||||
let result = vars.get(out);
|
||||
// g * 2.0: mu = 10*2 = 20, sigma = 2*2 = 4
|
||||
assert!((result.mu() - 20.0).abs() < 1e-12);
|
||||
assert!((result.sigma() - 4.0).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delta_is_zero_on_repeat_propagate() {
|
||||
let mut vars = VarStore::new();
|
||||
let out = vars.alloc(N_INF);
|
||||
let g = Gaussian::from_ms(5.0, 1.0);
|
||||
let mut f = TeamSumFactor {
|
||||
inputs: vec![(g, 1.0)],
|
||||
out,
|
||||
};
|
||||
|
||||
f.propagate(&mut vars);
|
||||
let (dmu, dsig) = f.propagate(&mut vars);
|
||||
assert!(dmu < 1e-12, "expected ~0 delta on repeat, got {}", dmu);
|
||||
assert!(dsig < 1e-12);
|
||||
}
|
||||
}
|
||||
+114
-33
@@ -1,7 +1,8 @@
|
||||
use crate::{
|
||||
N_INF, approx, cdf,
|
||||
factor::{Factor, VarId, VarStore},
|
||||
N_INF, approx,
|
||||
factor::{VarId, VarStore},
|
||||
gaussian::Gaussian,
|
||||
ln_interval, ln_sf,
|
||||
};
|
||||
|
||||
/// EP truncation factor on a diff variable.
|
||||
@@ -18,7 +19,7 @@ pub struct TruncFactor {
|
||||
/// Outgoing message to the diff variable (initial: `N_INF`, the EP identity).
|
||||
pub(crate) msg: Gaussian,
|
||||
/// Cached evidence (linear, not log) computed from the cavity on first propagation.
|
||||
pub(crate) evidence_cached: Option<f64>,
|
||||
pub(crate) log_evidence_cached: Option<f64>,
|
||||
}
|
||||
|
||||
impl TruncFactor {
|
||||
@@ -29,7 +30,7 @@ impl TruncFactor {
|
||||
margin,
|
||||
tie,
|
||||
msg: N_INF,
|
||||
evidence_cached: None,
|
||||
log_evidence_cached: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -40,14 +41,14 @@ impl TruncFactor {
|
||||
/// exactly; `alpha < 1.0` writes `α·new_msg + (1−α)·old_msg`.
|
||||
pub(crate) fn propagate_with_alpha(&mut self, vars: &mut VarStore, alpha: f64) -> (f64, f64) {
|
||||
let marginal = vars.get(self.diff);
|
||||
let cavity = marginal / self.msg;
|
||||
let cavity = marginal.cavity(self.msg);
|
||||
|
||||
if self.evidence_cached.is_none() {
|
||||
self.evidence_cached = Some(cavity_evidence(cavity, self.margin, self.tie));
|
||||
if self.log_evidence_cached.is_none() {
|
||||
self.log_evidence_cached = Some(cavity_log_evidence(cavity, self.margin, self.tie));
|
||||
}
|
||||
|
||||
let trunc = approx(cavity, self.margin, self.tie);
|
||||
let new_msg = trunc / cavity;
|
||||
let new_msg = trunc.cavity(cavity);
|
||||
|
||||
let damped = self.msg.damp_natural(new_msg, alpha);
|
||||
let old_msg = self.msg;
|
||||
@@ -56,37 +57,46 @@ impl TruncFactor {
|
||||
// marginal_new = cavity * stored_msg. With alpha = 1.0 this equals
|
||||
// `trunc` (since cavity * new_msg = trunc by construction); with
|
||||
// alpha < 1.0 it reflects the partially-applied update.
|
||||
vars.set(self.diff, cavity * damped);
|
||||
vars.set(self.diff, cavity.ep_product(damped));
|
||||
|
||||
old_msg.delta(damped)
|
||||
}
|
||||
}
|
||||
|
||||
impl Factor for TruncFactor {
|
||||
fn propagate(&mut self, vars: &mut VarStore) -> (f64, f64) {
|
||||
/// Undamped wrappers, used by this module's tests. Inference drives these
|
||||
/// factors through `propagate_with_alpha` and reads the cached log evidence
|
||||
/// directly, so these are not on any production path.
|
||||
#[cfg(test)]
|
||||
impl TruncFactor {
|
||||
pub(crate) fn propagate(&mut self, vars: &mut VarStore) -> (f64, f64) {
|
||||
self.propagate_with_alpha(vars, 1.0)
|
||||
}
|
||||
|
||||
fn log_evidence(&self, _vars: &VarStore) -> f64 {
|
||||
self.evidence_cached.unwrap_or(1.0).ln()
|
||||
}
|
||||
}
|
||||
|
||||
/// P(diff > margin) for non-tie, P(|diff| < margin) for tie.
|
||||
/// `ln P(diff > margin)` for a win, `ln P(|diff| < margin)` for a tie.
|
||||
///
|
||||
/// Clamped to a positive floor: for a near-certain outcome the tail rounds to
|
||||
/// exactly 0.0, and the `erfc` approximation used by `cdf` carries ~1e-7 error
|
||||
/// so it can even return slightly more than 1.0, making the difference
|
||||
/// negative. Either would send `log_evidence` to `-inf` or NaN and poison the
|
||||
/// sum across the whole history.
|
||||
fn cavity_evidence(diff: Gaussian, margin: f64, tie: bool) -> f64 {
|
||||
let raw = if tie {
|
||||
cdf(margin, diff.mu(), diff.sigma()) - cdf(-margin, diff.mu(), diff.sigma())
|
||||
/// Computed in log space throughout. Two earlier shapes both lost the tail:
|
||||
/// `1 - cdf(..)` cancelled away every digit of an unlikely outcome, and even
|
||||
/// once that was fixed the linear probability underflows to zero past about 38
|
||||
/// sigma, where clamping reported -708 nats regardless of the truth. An upset
|
||||
/// is the observation a log-evidence figure exists to notice, so it has to stay
|
||||
/// exact precisely where it is smallest.
|
||||
fn cavity_log_evidence(diff: Gaussian, margin: f64, tie: bool) -> f64 {
|
||||
let (mu, sigma) = (diff.mu(), diff.sigma());
|
||||
|
||||
let value = if tie {
|
||||
ln_interval(-margin, margin, mu, sigma)
|
||||
} else {
|
||||
1.0 - cdf(margin, diff.mu(), diff.sigma())
|
||||
ln_sf(margin, mu, sigma)
|
||||
};
|
||||
|
||||
raw.clamp(f64::MIN_POSITIVE, 1.0)
|
||||
// A degenerate cavity is the only route to a non-finite result; keep the
|
||||
// old floor for it rather than letting -inf poison the whole history's sum.
|
||||
if value.is_finite() {
|
||||
value
|
||||
} else {
|
||||
libm::log(f64::MIN_POSITIVE)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -117,19 +127,90 @@ mod tests {
|
||||
let diff = vars.alloc(Gaussian::from_ms(2.0, 3.0));
|
||||
|
||||
let mut f = TruncFactor::new(diff, 0.0, false);
|
||||
assert!(f.evidence_cached.is_none());
|
||||
assert!(f.log_evidence_cached.is_none());
|
||||
|
||||
f.propagate(&mut vars);
|
||||
assert!(f.evidence_cached.is_some());
|
||||
let first = f.evidence_cached.unwrap();
|
||||
assert!(f.log_evidence_cached.is_some());
|
||||
let first = f.log_evidence_cached.unwrap();
|
||||
|
||||
// Evidence should be P(diff > 0) for diff ~ N(2, 9) ≈ 0.748
|
||||
assert!(first > 0.7);
|
||||
assert!(first < 0.8);
|
||||
assert!(first.exp() > 0.7);
|
||||
assert!(first.exp() < 0.8);
|
||||
|
||||
// Subsequent propagations don't change it.
|
||||
f.propagate(&mut vars);
|
||||
assert_eq!(f.evidence_cached.unwrap(), first);
|
||||
assert_eq!(f.log_evidence_cached.unwrap(), first);
|
||||
}
|
||||
|
||||
/// The defect this guards: `1 - cdf` collapsed to zero for a surprising
|
||||
/// result, the clamp turned that into `f64::MIN_POSITIVE`, and
|
||||
/// `log_evidence` reported ln of *that* — about -708 whatever the truth
|
||||
/// was. An upset is the observation a model-comparison score exists to
|
||||
/// notice, so it was wrong exactly where it mattered.
|
||||
#[test]
|
||||
fn evidence_of_an_upset_is_not_flattened_to_the_clamp_floor() {
|
||||
// diff ~ N(-9, 1) with margin 0: the favoured side lost by nine sigma.
|
||||
let evidence = cavity_log_evidence(Gaussian::from_ms(-9.0, 1.0), 0.0, false).exp();
|
||||
|
||||
assert!(
|
||||
evidence > f64::MIN_POSITIVE,
|
||||
"evidence collapsed onto the clamp floor: {evidence}"
|
||||
);
|
||||
// P(X > 0) for X ~ N(-9, 1) is the standard normal tail at 9 sigma.
|
||||
assert!(
|
||||
(evidence - 1.128_588e-19).abs() / 1.128_588e-19 < 1e-6,
|
||||
"expected ~1.13e-19, got {evidence}"
|
||||
);
|
||||
assert!(
|
||||
(evidence.ln() + 43.628).abs() < 1e-2,
|
||||
"log evidence {} should be about -43.6, not -708",
|
||||
evidence.ln()
|
||||
);
|
||||
}
|
||||
|
||||
/// Evidence must stay finite and positive however extreme the mismatch,
|
||||
/// since `log_evidence` sums across the whole history and one `-inf` or
|
||||
/// `NaN` poisons all of it.
|
||||
///
|
||||
/// Finiteness alone is too weak a bar — the clamped version was finite too,
|
||||
/// and wrong by hundreds of nats. `log_evidence_tracks_the_analytic_tail`
|
||||
/// below is the assertion that actually holds this up.
|
||||
#[test]
|
||||
fn evidence_stays_positive_and_finite_at_any_separation() {
|
||||
for mu in [-300.0f64, -50.0, -9.0, 0.0, 9.0, 50.0, 300.0] {
|
||||
for tie in [false, true] {
|
||||
let ln_e = cavity_log_evidence(Gaussian::from_ms(mu, 1.0), 1.0, tie);
|
||||
assert!(
|
||||
ln_e.is_finite() && ln_e <= 0.0,
|
||||
"mu={mu} tie={tie}: log evidence {ln_e} is not a log-probability"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The clamp used to floor everything past ~38 sigma at `ln(MIN_POSITIVE)`
|
||||
/// = -708, however far out the real observation was. In log space the
|
||||
/// answer is a polynomial and stays exact: at 1000 sigma the truth is about
|
||||
/// -500_000 nats, and -708 is not a rounding error.
|
||||
#[test]
|
||||
fn log_evidence_tracks_the_analytic_tail() {
|
||||
for mu in [-40.0f64, -60.0, -100.0, -1000.0] {
|
||||
// P(diff > 0) for diff ~ N(mu, 1), mu far below zero.
|
||||
let got = cavity_log_evidence(Gaussian::from_ms(mu, 1.0), 0.0, false);
|
||||
|
||||
// ln Phi(mu) ~ -mu^2/2 - ln(-mu) - ln(sqrt(2 pi)) for mu << 0.
|
||||
let z = -mu;
|
||||
let approx = -0.5 * z * z - z.ln() - (2.0 * std::f64::consts::PI).sqrt().ln();
|
||||
|
||||
assert!(
|
||||
got < libm::log(f64::MIN_POSITIVE),
|
||||
"mu={mu}: {got} is still stuck on the old clamp floor"
|
||||
);
|
||||
assert!(
|
||||
(got - approx).abs() / approx.abs() < 1e-3,
|
||||
"mu={mu}: got {got}, asymptotic expectation {approx}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -141,7 +222,7 @@ mod tests {
|
||||
f.propagate(&mut vars);
|
||||
|
||||
// For diff ~ N(0, 4), tie=true with margin=1: P(-1 < diff < 1) ≈ 0.383
|
||||
let ev = f.evidence_cached.unwrap();
|
||||
let ev = f.log_evidence_cached.unwrap().exp();
|
||||
assert!(ev > 0.35 && ev < 0.42);
|
||||
}
|
||||
|
||||
|
||||
@@ -1,13 +0,0 @@
|
||||
//! Factor-graph public API.
|
||||
//!
|
||||
//! Power users can construct custom factor graphs via `Game::custom` (T2
|
||||
//! minimal; full ergonomics in T4) and drive them with custom `Schedule`
|
||||
//! implementations.
|
||||
|
||||
pub use crate::{
|
||||
factor::{
|
||||
BuiltinFactor, Factor, VarId, VarStore, margin::MarginFactor, rank_diff::RankDiffFactor,
|
||||
team_sum::TeamSumFactor, trunc::TruncFactor,
|
||||
},
|
||||
schedule::{EpsilonOrMax, Schedule, ScheduleReport},
|
||||
};
|
||||
+335
-149
File diff suppressed because it is too large
Load Diff
+304
-67
@@ -1,5 +1,3 @@
|
||||
use std::ops;
|
||||
|
||||
use crate::{MU, N_INF, SIGMA};
|
||||
|
||||
/// A Gaussian distribution stored in natural parameters.
|
||||
@@ -11,6 +9,7 @@ use crate::{MU, N_INF, SIGMA};
|
||||
/// the stored fields with no `sqrt` or reciprocal in the hot path. `mu()` and
|
||||
/// `sigma()` are accessors computed on demand.
|
||||
#[derive(Clone, Copy, PartialEq, Debug)]
|
||||
#[must_use]
|
||||
pub struct Gaussian {
|
||||
pi: f64,
|
||||
tau: f64,
|
||||
@@ -18,8 +17,43 @@ pub struct Gaussian {
|
||||
|
||||
impl Gaussian {
|
||||
/// Construct from mean and standard deviation.
|
||||
#[must_use]
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics if `sigma` is negative. NaN is deliberately allowed through: a
|
||||
/// broken fit produces one, and `converge` reports that as
|
||||
/// `NonFiniteResult` rather than panicking mid-inference.
|
||||
///
|
||||
/// A negative sigma used to be accepted and returned results **bit
|
||||
/// identical** to its absolute value, because sigma only ever enters as
|
||||
/// `sigma * sigma`. The sign was not rejected and not honoured; it simply
|
||||
/// vanished. That is the same defect `HistoryBuilder::sigma`,
|
||||
/// `HistoryBuilder::beta` and `Member::with_drift_scale` already reject.
|
||||
///
|
||||
/// # Very small sigma
|
||||
///
|
||||
/// `pi = 1 / sigma^2` leaves `f64`'s range below about `1.5e-154`, and
|
||||
/// `tau = mu * pi` overflows sooner still — at a threshold that depends on
|
||||
/// `mu`, so there is a band where `pi` is finite and only `tau` is not.
|
||||
/// Both land on the same point-mass representation the `sigma == 0.0`
|
||||
/// branch produces, and a point mass with a non-zero mean has `mu() = NaN`,
|
||||
/// because `tau / pi` is `inf / inf`.
|
||||
///
|
||||
/// This is not rejected, because `approx` legitimately produces a very
|
||||
/// small truncated sigma and inference must not panic. It is worth knowing
|
||||
/// that such a `Gaussian` is not equal to itself, so two identical
|
||||
/// declarations of one can be reported as conflicting.
|
||||
pub const fn from_ms(mu: f64, sigma: f64) -> Self {
|
||||
// NaN is admitted on purpose. A broken fit legitimately produces a NaN
|
||||
// sigma — `sqrt` of a negative truncated variance — and the design is
|
||||
// to propagate that to `converge`'s `NonFiniteResult` guard, not to
|
||||
// panic inside inference. Rejecting it here turned that reporting path
|
||||
// into a crash, which two tests caught immediately.
|
||||
assert!(
|
||||
sigma >= 0.0 || sigma.is_nan(),
|
||||
"sigma must not be negative; it is only ever squared, so a negative \
|
||||
value would silently behave as its absolute value"
|
||||
);
|
||||
if sigma == f64::INFINITY {
|
||||
Self { pi: 0.0, tau: 0.0 }
|
||||
} else if sigma == 0.0 {
|
||||
@@ -39,11 +73,12 @@ impl Gaussian {
|
||||
/// Construct from mean and *variance*, skipping the square-root round trip.
|
||||
///
|
||||
/// `from_ms(mu, var.sqrt())` immediately squares the root away again to
|
||||
/// recover `pi = 1/var`. Variance-combining operations (`Add`, `Sub`,
|
||||
/// `exclude`, `forget`) work in variance space throughout, so they go
|
||||
/// through here instead and never take a root.
|
||||
/// recover `pi = 1/var`. Variance-combining operations work in variance
|
||||
/// space throughout, so they go through here instead and never take a
|
||||
/// root. Use it whenever you already hold a variance —
|
||||
/// [`variance`](Gaussian::variance) is its inverse.
|
||||
#[inline]
|
||||
pub(crate) fn from_mv(mu: f64, var: f64) -> Self {
|
||||
pub fn from_mv(mu: f64, var: f64) -> Self {
|
||||
if var == f64::INFINITY {
|
||||
Self { pi: 0.0, tau: 0.0 }
|
||||
} else if var == 0.0 {
|
||||
@@ -64,18 +99,32 @@ impl Gaussian {
|
||||
Self { pi, tau }
|
||||
}
|
||||
|
||||
/// Precision, `1 / sigma^2` — one of the two natural parameters.
|
||||
///
|
||||
/// This is the representation the type actually stores, which is why the EP
|
||||
/// product and cavity (`Mul` / `Div`) are plain adds and subtracts. Larger
|
||||
/// means more certain; `0.0` is an improper, uninformative message and
|
||||
/// `inf` is a point mass.
|
||||
#[inline]
|
||||
#[must_use]
|
||||
pub fn pi(&self) -> f64 {
|
||||
pub(crate) fn pi(&self) -> f64 {
|
||||
self.pi
|
||||
}
|
||||
|
||||
/// Precision-adjusted mean, `mu / sigma^2` — the other natural parameter.
|
||||
///
|
||||
/// Stored rather than derived, for the same reason as [`Gaussian::pi`].
|
||||
/// Meaningful only alongside `pi`: on its own it is not a location.
|
||||
#[inline]
|
||||
#[must_use]
|
||||
pub fn tau(&self) -> f64 {
|
||||
pub(crate) fn tau(&self) -> f64 {
|
||||
self.tau
|
||||
}
|
||||
|
||||
/// Mean skill: the point estimate.
|
||||
///
|
||||
/// Derived from the natural parameters as `tau / pi`. An improper message
|
||||
/// (`pi <= 0`) has no defined mean and reports `0.0` — see
|
||||
/// [`Gaussian::sigma`], which reports `inf` for the same state, and read
|
||||
/// the two together before treating a mean as informative.
|
||||
#[inline]
|
||||
#[must_use]
|
||||
pub fn mu(&self) -> f64 {
|
||||
@@ -90,12 +139,14 @@ impl Gaussian {
|
||||
}
|
||||
}
|
||||
|
||||
/// Variance, `1 / pi`, without the root-and-square of `sigma().powi(2)`.
|
||||
/// Variance, without the root-and-square of `sigma().powi(2)`.
|
||||
///
|
||||
/// Mirrors `sigma()`'s treatment of the improper (`pi <= 0`) and point-mass
|
||||
/// (`pi == inf`) cases.
|
||||
/// Mirrors [`sigma`](Gaussian::sigma)'s treatment of the improper
|
||||
/// (infinite) and point-mass (zero) cases, and is the inverse of
|
||||
/// [`from_mv`](Gaussian::from_mv).
|
||||
#[inline]
|
||||
pub(crate) fn variance(&self) -> f64 {
|
||||
#[must_use]
|
||||
pub fn variance(&self) -> f64 {
|
||||
if self.pi <= 0.0 {
|
||||
f64::INFINITY
|
||||
} else if self.pi.is_infinite() {
|
||||
@@ -105,6 +156,12 @@ impl Gaussian {
|
||||
}
|
||||
}
|
||||
|
||||
/// Standard deviation: how unsure this estimate is.
|
||||
///
|
||||
/// Derived as `1 / sqrt(pi)`. An improper message (`pi <= 0`) reports
|
||||
/// `inf`, and a point mass (`pi == inf`) reports `0.0` — both are real
|
||||
/// states rather than error codes, and both are legitimate for a converged
|
||||
/// fit with degenerate parameters.
|
||||
#[inline]
|
||||
#[must_use]
|
||||
pub fn sigma(&self) -> f64 {
|
||||
@@ -120,7 +177,25 @@ impl Gaussian {
|
||||
}
|
||||
}
|
||||
|
||||
/// How far this Gaussian moved from `other`, as `(|d mu|, |d sigma|)`.
|
||||
///
|
||||
/// Identical messages have not moved, whatever their parameters, and that
|
||||
/// case is answered in natural space before touching `mu()`/`sigma()`. An
|
||||
/// improper message has `pi == 0`, so `sigma()` is infinite — and
|
||||
/// `inf - inf` is NaN, a NaN *change* for a message that did not change at
|
||||
/// all. (`mu()` is guarded and returns 0.0 here, so the mean component was
|
||||
/// never the problem; the sigma component alone produced `(0.0, NaN)`.)
|
||||
///
|
||||
/// That is reachable in ordinary inference: once a pairing is more than
|
||||
/// about nine cavity-sigma apart the truncation is a no-op, `trunc / cavity`
|
||||
/// is exactly the identity message, and the chain compares one identity
|
||||
/// against another. Before this guard that produced `(0.0, NaN)`, which
|
||||
/// silently disabled the sigma half of the convergence test.
|
||||
pub(crate) fn delta(&self, other: Gaussian) -> (f64, f64) {
|
||||
if self.pi == other.pi && self.tau == other.tau {
|
||||
return (0.0, 0.0);
|
||||
}
|
||||
|
||||
(
|
||||
(self.mu() - other.mu()).abs(),
|
||||
(self.sigma() - other.sigma()).abs(),
|
||||
@@ -145,13 +220,51 @@ impl Gaussian {
|
||||
Self::from_mv(self.mu(), self.variance() + variance_delta)
|
||||
}
|
||||
|
||||
/// `P(X < x)` under this Gaussian.
|
||||
///
|
||||
/// The question a stopping rule asks: *how sure am I that this competitor's
|
||||
/// true skill is below the cutoff?* Expressing that as a probability keeps
|
||||
/// its meaning as sigma changes, where a `mu + z * sigma` band silently
|
||||
/// means different confidence at different uncertainties — which is exactly
|
||||
/// the regime a stopping rule operates in.
|
||||
///
|
||||
/// Accurate in the *lower* tail. For the upper tail use
|
||||
/// [`Gaussian::probability_above`] rather than `1.0 - probability_below(x)`,
|
||||
/// which cancels away every significant digit once the result is small.
|
||||
///
|
||||
/// An improper Gaussian (non-positive precision) has no defined mean, so
|
||||
/// this returns `0.5` — the same convention `mu()` and `sigma()` follow.
|
||||
#[must_use]
|
||||
pub fn probability_below(&self, x: f64) -> f64 {
|
||||
if self.pi <= 0.0 {
|
||||
return 0.5;
|
||||
}
|
||||
crate::cdf(x, self.mu(), self.sigma())
|
||||
}
|
||||
|
||||
/// `P(X > x)` under this Gaussian.
|
||||
///
|
||||
/// Computed as a survival function rather than `1 - cdf`, so it keeps full
|
||||
/// relative precision in the upper tail: `1 - cdf` returns exactly zero
|
||||
/// past about 8.3 sigma, where the true value is still 1e-19 and perfectly
|
||||
/// representable. A stopping rule is evaluated precisely there — the
|
||||
/// interesting cases are the ones near certainty.
|
||||
///
|
||||
/// An improper Gaussian returns `0.5`, as [`Gaussian::probability_below`].
|
||||
#[must_use]
|
||||
pub fn probability_above(&self, x: f64) -> f64 {
|
||||
if self.pi <= 0.0 {
|
||||
return 0.5;
|
||||
}
|
||||
crate::sf(x, self.mu(), self.sigma())
|
||||
}
|
||||
|
||||
/// EP damping in natural-parameter space: `α·new + (1−α)·self`.
|
||||
///
|
||||
/// Used by within-game inference to stabilise oscillating fixed-point
|
||||
/// loops on hard graphs. `alpha = 1.0` returns `new` exactly;
|
||||
/// `alpha < 1.0` shrinks each per-step update.
|
||||
#[must_use]
|
||||
pub fn damp_natural(self, new: Gaussian, alpha: f64) -> Gaussian {
|
||||
pub(crate) fn damp_natural(self, new: Gaussian, alpha: f64) -> Gaussian {
|
||||
Gaussian::from_natural(
|
||||
alpha * new.pi() + (1.0 - alpha) * self.pi(),
|
||||
alpha * new.tau() + (1.0 - alpha) * self.tau(),
|
||||
@@ -165,34 +278,65 @@ impl Default for Gaussian {
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Add<Gaussian> for Gaussian {
|
||||
type Output = Gaussian;
|
||||
/// Variance addition: (mu1 + mu2, sqrt(σ1² + σ2²)).
|
||||
/// Used for combining performance and noise; rare relative to mul/div.
|
||||
fn add(self, rhs: Gaussian) -> Self::Output {
|
||||
Self::from_mv(self.mu() + rhs.mu(), self.variance() + rhs.variance())
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Sub<Gaussian> for Gaussian {
|
||||
type Output = Gaussian;
|
||||
/// (mu1 - mu2, sqrt(σ1² + σ2²)). Same sigma combination as Add.
|
||||
fn sub(self, rhs: Gaussian) -> Self::Output {
|
||||
Self::from_mv(self.mu() - rhs.mu(), self.variance() + rhs.variance())
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Mul<Gaussian> for Gaussian {
|
||||
type Output = Gaussian;
|
||||
/// Factor product: nat-param add. Hot path — two f64 additions, no sqrt.
|
||||
fn mul(self, rhs: Gaussian) -> Self::Output {
|
||||
impl Gaussian {
|
||||
/// The EP factor **product**: multiply two messages about the same
|
||||
/// variable.
|
||||
///
|
||||
/// Two natural-parameter additions and no square root, which is why the
|
||||
/// type stores `pi` and `tau` rather than `mu` and `sigma`. This is the
|
||||
/// hot path.
|
||||
///
|
||||
/// Not arithmetic — `N(10, 2).ep_product(N(4, 3))` is `N(8.15, 1.66)`,
|
||||
/// nowhere near 40. It used to be spelled `a * b`, on a public `Mul` impl,
|
||||
/// where that was a trap rather than a shorthand.
|
||||
#[inline]
|
||||
pub(crate) fn ep_product(self, rhs: Gaussian) -> Gaussian {
|
||||
Self::from_natural(self.pi + rhs.pi, self.tau + rhs.tau)
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Mul<f64> for Gaussian {
|
||||
type Output = Gaussian;
|
||||
fn mul(self, scalar: f64) -> Self::Output {
|
||||
/// The EP **cavity**: divide out a message this belief already absorbed.
|
||||
///
|
||||
/// The inverse of [`ep_product`](Gaussian::ep_product), and two
|
||||
/// subtractions rather than two additions.
|
||||
///
|
||||
/// **May return an improper result.** Cancelling a message that carried
|
||||
/// most of the precision leaves `pi <= 0`, which is not a distribution.
|
||||
/// `mu()` reports `0.0` and `sigma()` reports `inf` for such a value —
|
||||
/// both are the accessors' policy for "undefined", not answers. Measured:
|
||||
/// `N(10, 2).cavity(N(1, 1))` has `pi = -0.75`, and its `mu()` prints a
|
||||
/// confident `0`. That is why this is not a public operator.
|
||||
#[inline]
|
||||
pub(crate) fn cavity(self, rhs: Gaussian) -> Gaussian {
|
||||
Self::from_natural(self.pi - rhs.pi, self.tau - rhs.tau)
|
||||
}
|
||||
|
||||
/// Convolve two independent Gaussians: `N(mu1 + mu2, sqrt(v1 + v2))`.
|
||||
///
|
||||
/// The distribution of a *sum* of independent variables, so the variances
|
||||
/// add — the result is always wider than either input. Used to combine a
|
||||
/// skill with performance noise. Goes through `from_mv` and takes no root.
|
||||
#[inline]
|
||||
pub(crate) fn convolve(self, rhs: Gaussian) -> Gaussian {
|
||||
Self::from_mv(self.mu() + rhs.mu(), self.variance() + rhs.variance())
|
||||
}
|
||||
|
||||
/// Convolve a *difference*: `N(mu1 - mu2, sqrt(v1 + v2))`.
|
||||
///
|
||||
/// The means subtract and the variances still **add**, because a
|
||||
/// difference of independent variables is no more certain than a sum. That
|
||||
/// is the half that made the old `Sub` impl misleading: `a - b` grew the
|
||||
/// sigma from 2 to `sqrt(4 + 9)`.
|
||||
#[inline]
|
||||
pub(crate) fn convolve_diff(self, rhs: Gaussian) -> Gaussian {
|
||||
Self::from_mv(self.mu() - rhs.mu(), self.variance() + rhs.variance())
|
||||
}
|
||||
|
||||
/// Scale by a constant: `mu` by `scalar`, `sigma` by `|scalar|`.
|
||||
///
|
||||
/// The one operation that *is* ordinary arithmetic — it is the
|
||||
/// distribution of `scalar * X`. Used for per-member weights.
|
||||
#[inline]
|
||||
pub(crate) fn scale(self, scalar: f64) -> Gaussian {
|
||||
if !scalar.is_finite() {
|
||||
return N_INF;
|
||||
}
|
||||
@@ -207,16 +351,44 @@ impl ops::Mul<f64> for Gaussian {
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Div<Gaussian> for Gaussian {
|
||||
type Output = Gaussian;
|
||||
/// Cavity: nat-param sub. Hot path — two f64 subtractions, no sqrt.
|
||||
fn div(self, rhs: Gaussian) -> Self::Output {
|
||||
Self::from_natural(self.pi - rhs.pi, self.tau - rhs.tau)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
/// A message that did not change must report no change, even when it is
|
||||
/// improper. `mu()` of an improper Gaussian is `0/0 = NaN` and `sigma()` is
|
||||
/// infinite, so the mean/sigma form reported `(NaN, NaN)` for two identical
|
||||
/// identity messages — which silently disabled the sigma half of the
|
||||
/// convergence test in `run_chain`.
|
||||
#[test]
|
||||
fn delta_of_two_identical_improper_messages_is_zero() {
|
||||
let improper = crate::N_INF;
|
||||
// `mu()` is guarded and returns 0.0 for an improper Gaussian, so the
|
||||
// mean component was always fine. The NaN came from the sigma
|
||||
// component alone: `inf - inf`. The pre-fix value was `(0.0, NaN)`.
|
||||
assert!(improper.sigma().is_infinite(), "premise: sigma is infinite");
|
||||
assert_eq!(improper.mu(), 0.0, "premise: mu is guarded, not NaN");
|
||||
assert!(
|
||||
(improper.sigma() - improper.sigma()).is_nan(),
|
||||
"premise: the unguarded sigma difference is NaN"
|
||||
);
|
||||
assert_eq!(improper.delta(improper), (0.0, 0.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delta_of_identical_proper_messages_is_zero() {
|
||||
let g = Gaussian::from_ms(25.0, 8.0);
|
||||
assert_eq!(g.delta(g), (0.0, 0.0));
|
||||
}
|
||||
|
||||
/// The shortcut must not swallow a real difference.
|
||||
#[test]
|
||||
fn delta_still_measures_a_real_move() {
|
||||
let a = Gaussian::from_ms(25.0, 8.0);
|
||||
let b = Gaussian::from_ms(26.0, 9.0);
|
||||
let (dmu, dsigma) = a.delta(b);
|
||||
assert!((dmu - 1.0).abs() < 1e-12, "{dmu}");
|
||||
assert!((dsigma - 1.0).abs() < 1e-12, "{dsigma}");
|
||||
}
|
||||
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
@@ -236,64 +408,64 @@ mod tests {
|
||||
|
||||
// Subtracting such a message must not produce NaN (the original failure path).
|
||||
let proper = Gaussian::from_ms(9.75, 1.256);
|
||||
let diff = proper - tiny_neg;
|
||||
let diff = proper.convolve_diff(tiny_neg);
|
||||
assert!(diff.pi().is_finite() && !diff.pi().is_nan());
|
||||
assert!(diff.tau().is_finite() && !diff.tau().is_nan());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_add() {
|
||||
fn convolve_adds_variances() {
|
||||
let n = Gaussian::from_ms(25.0, 25.0 / 3.0);
|
||||
let m = Gaussian::from_ms(0.0, 1.0);
|
||||
let r = n + m;
|
||||
let r = n.convolve(m);
|
||||
assert!((r.mu() - 25.0).abs() < 1e-12);
|
||||
assert!((r.sigma() - 8.393118874676116).abs() < 1e-10);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_sub() {
|
||||
fn convolve_diff_subtracts_means_and_adds_variances() {
|
||||
let n = Gaussian::from_ms(25.0, 25.0 / 3.0);
|
||||
let m = Gaussian::from_ms(1.0, 1.0);
|
||||
let r = n - m;
|
||||
let r = n.convolve_diff(m);
|
||||
assert!((r.mu() - 24.0).abs() < 1e-12);
|
||||
assert!((r.sigma() - 8.393118874676116).abs() < 1e-10);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_mul() {
|
||||
fn ep_product_is_not_arithmetic() {
|
||||
let n = Gaussian::from_ms(25.0, 25.0 / 3.0);
|
||||
let m = Gaussian::from_ms(0.0, 1.0);
|
||||
let r = n * m;
|
||||
let r = n.ep_product(m);
|
||||
assert!((r.mu() - 0.35488958990536273).abs() < 1e-10);
|
||||
assert!((r.sigma() - 0.992876838486922).abs() < 1e-10);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_div() {
|
||||
fn cavity_undoes_a_product() {
|
||||
let n = Gaussian::from_ms(25.0, 25.0 / 3.0);
|
||||
let m = Gaussian::from_ms(0.0, 1.0);
|
||||
let r = m / n;
|
||||
let r = m.cavity(n);
|
||||
assert!((r.mu() - (-0.3652597402597402)).abs() < 1e-10);
|
||||
assert!((r.sigma() - 1.0072787050317253).abs() < 1e-10);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_n00_is_add_identity() {
|
||||
// N00 (sigma=0) is the additive identity for the variance-convolution Add op.
|
||||
// N_INF (sigma=inf) is the identity for the EP-product Mul op.
|
||||
// N00 (sigma=0) is the identity for `convolve`.
|
||||
// N_INF (sigma=inf) is the identity for `ep_product`.
|
||||
let g = Gaussian::from_ms(3.0, 2.0);
|
||||
let n00 = Gaussian::from_ms(0.0, 0.0);
|
||||
let r = n00 + g;
|
||||
let r = n00.convolve(g);
|
||||
assert!((r.mu() - g.mu()).abs() < 1e-12);
|
||||
assert!((r.sigma() - g.sigma()).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_mul_is_factor_product() {
|
||||
// n * m in nat-params should be pi_n + pi_m, tau_n + tau_m
|
||||
fn ep_product_adds_natural_parameters() {
|
||||
// `ep_product` in nat-params should be pi_n + pi_m, tau_n + tau_m
|
||||
let n = Gaussian::from_ms(2.0, 3.0);
|
||||
let m = Gaussian::from_ms(1.0, 2.0);
|
||||
let r = n * m;
|
||||
let r = n.ep_product(m);
|
||||
let expected_pi = n.pi() + m.pi();
|
||||
let expected_tau = n.tau() + m.tau();
|
||||
assert!((r.pi() - expected_pi).abs() < 1e-15);
|
||||
@@ -301,10 +473,10 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_div_is_cavity() {
|
||||
fn cavity_subtracts_natural_parameters() {
|
||||
let n = Gaussian::from_ms(2.0, 1.0);
|
||||
let m = Gaussian::from_ms(1.0, 2.0);
|
||||
let r = n / m;
|
||||
let r = n.cavity(m);
|
||||
let expected_pi = n.pi() - m.pi();
|
||||
let expected_tau = n.tau() - m.tau();
|
||||
assert!((r.pi() - expected_pi).abs() < 1e-15);
|
||||
@@ -340,3 +512,68 @@ mod tests {
|
||||
assert!((damped.tau() - expected_tau).abs() < 1e-12);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tail_probability_tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn probability_below_matches_published_quantiles() {
|
||||
let g = Gaussian::from_ms(0.0, 1.0);
|
||||
for (x, expected) in [
|
||||
(-1.959_963_984_540_054, 0.025),
|
||||
(0.0, 0.5),
|
||||
(1.281_551_565_544_6, 0.9),
|
||||
(1.959_963_984_540_054, 0.975),
|
||||
] {
|
||||
let got = g.probability_below(x);
|
||||
assert!(
|
||||
(got - expected).abs() < 1e-12,
|
||||
"P(X < {x}) = {got}, expected {expected}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_two_tails_partition_the_mass() {
|
||||
let g = Gaussian::from_ms(3.0, 2.0);
|
||||
for x in [-4.0f64, 0.0, 3.0, 7.5] {
|
||||
let total = g.probability_below(x) + g.probability_above(x);
|
||||
assert!((total - 1.0).abs() < 1e-15, "at {x}: {total}");
|
||||
}
|
||||
}
|
||||
|
||||
/// The reason `probability_above` exists rather than `1 - probability_below`.
|
||||
#[test]
|
||||
fn probability_above_keeps_precision_where_the_complement_collapses() {
|
||||
let g = Gaussian::from_ms(0.0, 1.0);
|
||||
for (x, expected) in [(9.0f64, 1.128_588e-19), (20.0, 2.753_624e-89)] {
|
||||
let got = g.probability_above(x);
|
||||
assert!(
|
||||
(got - expected).abs() / expected < 1e-6,
|
||||
"P(X > {x}) = {got}, expected ~{expected}"
|
||||
);
|
||||
assert_eq!(
|
||||
1.0 - g.probability_below(x),
|
||||
0.0,
|
||||
"the complement should still collapse at {x}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_scaled_gaussian_shifts_and_stretches() {
|
||||
let g = Gaussian::from_ms(25.0, 6.0);
|
||||
assert!((g.probability_below(25.0) - 0.5).abs() < 1e-15);
|
||||
// One sigma either side of the mean.
|
||||
assert!((g.probability_below(31.0) - 0.841_344_746_068_543).abs() < 1e-12);
|
||||
assert!((g.probability_above(19.0) - 0.841_344_746_068_543).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_improper_gaussian_is_uninformative_rather_than_nan() {
|
||||
let improper = Gaussian::from_ms(0.0, f64::INFINITY);
|
||||
assert_eq!(improper.probability_below(5.0), 0.5);
|
||||
assert_eq!(improper.probability_above(5.0), 0.5);
|
||||
}
|
||||
}
|
||||
|
||||
+2123
-243
File diff suppressed because it is too large
Load Diff
+152
@@ -0,0 +1,152 @@
|
||||
//! Cholesky factorisation of a joint precision matrix.
|
||||
//!
|
||||
//! Every question the joint answers is a *bilinear form* in the precision
|
||||
//! matrix's inverse — the variance of a contrast is `c^T L^-1 c`, and the
|
||||
//! covariance of two contrasts is `c^T L^-1 a`. None of them wants `L^-1 c`
|
||||
//! itself, which is what makes the shape here worth stating explicitly.
|
||||
//!
|
||||
//! Writing the precision as `A = L L^T`,
|
||||
//!
|
||||
//! ```text
|
||||
//! c^T A^-1 a = c^T L^-T L^-1 a = (L^-1 c) . (L^-1 a)
|
||||
//! ```
|
||||
//!
|
||||
//! so a single forward substitution per contrast answers everything, and the
|
||||
//! back substitution a general solve would do is wasted work. That halves the
|
||||
//! cost of a query, and it removes a failure mode: a variance computed as
|
||||
//! `c . (A^-1 c)` is a difference of products that can round to a small
|
||||
//! negative number, where the same quantity as `|L^-1 c|^2` is a sum of
|
||||
//! squares and cannot.
|
||||
//!
|
||||
//! Factorising is `O(n^3)` and whitening is `O(n^2)`, so the split also
|
||||
//! matters structurally: the expensive half depends only on the fit, and is
|
||||
//! shared across every query a [`Joint`](crate::Joint) answers.
|
||||
|
||||
/// A factorised symmetric positive-definite matrix, reusable across queries.
|
||||
pub(crate) struct Cholesky {
|
||||
/// Lower triangle of `L`, row-major `n * n`. The upper triangle is
|
||||
/// leftover scratch from the factorisation and is never read.
|
||||
l: Vec<f64>,
|
||||
n: usize,
|
||||
}
|
||||
|
||||
impl Cholesky {
|
||||
/// Factorise `a` (row-major, `n * n`, symmetric) into `L L^T`.
|
||||
///
|
||||
/// `a` is consumed as scratch.
|
||||
///
|
||||
/// Returns `None` if the matrix is not positive-definite, which for a
|
||||
/// precision matrix means the model is improper — a competitor with
|
||||
/// neither a proper prior nor any evidence.
|
||||
pub(crate) fn factor(mut a: Vec<f64>, n: usize) -> Option<Self> {
|
||||
debug_assert_eq!(a.len(), n * n);
|
||||
|
||||
for j in 0..n {
|
||||
let mut d = a[j * n + j];
|
||||
for k in 0..j {
|
||||
d -= a[j * n + k] * a[j * n + k];
|
||||
}
|
||||
// Explicit rather than `!(d > 0.0)`: a NaN pivot must fail here
|
||||
// too, and a negated comparison would let it through as "not
|
||||
// positive".
|
||||
if d.is_nan() || d <= 0.0 {
|
||||
return None;
|
||||
}
|
||||
let d = d.sqrt();
|
||||
a[j * n + j] = d;
|
||||
|
||||
for i in j + 1..n {
|
||||
let mut s = a[i * n + j];
|
||||
for k in 0..j {
|
||||
s -= a[i * n + k] * a[j * n + k];
|
||||
}
|
||||
a[i * n + j] = s / d;
|
||||
}
|
||||
}
|
||||
|
||||
Some(Self { l: a, n })
|
||||
}
|
||||
|
||||
/// Whiten a contrast: `y = L^-1 b`.
|
||||
///
|
||||
/// The point of the result is the dot product, not the vector: for two
|
||||
/// contrasts `b` and `b'`, `y . y'` is `b^T A^-1 b'`. See the module docs.
|
||||
pub(crate) fn whiten(&self, b: &[f64]) -> Vec<f64> {
|
||||
debug_assert_eq!(b.len(), self.n);
|
||||
let n = self.n;
|
||||
let mut y = b.to_vec();
|
||||
for i in 0..n {
|
||||
// Folded from `y[i]` rather than summed and subtracted once, so the
|
||||
// accumulation order matches a plain substitution loop exactly.
|
||||
let row = &self.l[i * n..i * n + i];
|
||||
let s = row
|
||||
.iter()
|
||||
.zip(&y[..i])
|
||||
.fold(y[i], |acc, (l, v)| acc - l * v);
|
||||
y[i] = s / self.l[i * n + i];
|
||||
}
|
||||
y
|
||||
}
|
||||
}
|
||||
|
||||
/// `b^T A^-1 b'`, given the two whitened contrasts.
|
||||
pub(crate) fn bilinear(y: &[f64], y_prime: &[f64]) -> f64 {
|
||||
y.iter().zip(y_prime).map(|(a, b)| a * b).sum()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// `[[4, 1], [1, 3]] z = [1, 2]` has `z = [1/11, 7/11]`, so the quadratic
|
||||
/// form `b^T A^-1 b` is `1 * 1/11 + 2 * 7/11 = 15/11`.
|
||||
#[test]
|
||||
fn reproduces_a_known_quadratic_form() {
|
||||
let c = Cholesky::factor(vec![4.0, 1.0, 1.0, 3.0], 2).unwrap();
|
||||
let y = c.whiten(&[1.0, 2.0]);
|
||||
assert!((bilinear(&y, &y) - 15.0 / 11.0).abs() < 1e-12);
|
||||
}
|
||||
|
||||
/// Whitening `e_i` recovers the inverse's diagonal, which is the variance
|
||||
/// of a single variable.
|
||||
#[test]
|
||||
fn recovers_the_inverse_diagonal() {
|
||||
// A = [[2, -1, 0], [-1, 2, -1], [0, -1, 2]]; inverse diagonal is
|
||||
// [0.75, 1.0, 0.75].
|
||||
let a = vec![2.0, -1.0, 0.0, -1.0, 2.0, -1.0, 0.0, -1.0, 2.0];
|
||||
let c = Cholesky::factor(a, 3).unwrap();
|
||||
for (i, expected) in [0.75, 1.0, 0.75].into_iter().enumerate() {
|
||||
let mut e = vec![0.0; 3];
|
||||
e[i] = 1.0;
|
||||
let y = c.whiten(&e);
|
||||
assert!((bilinear(&y, &y) - expected).abs() < 1e-12, "row {i}");
|
||||
}
|
||||
}
|
||||
|
||||
/// The off-diagonal bilinear form is symmetric and matches the inverse.
|
||||
#[test]
|
||||
fn recovers_an_off_diagonal_covariance() {
|
||||
// Same A; (A^-1)_{0,1} = 0.5.
|
||||
let a = vec![2.0, -1.0, 0.0, -1.0, 2.0, -1.0, 0.0, -1.0, 2.0];
|
||||
let c = Cholesky::factor(a, 3).unwrap();
|
||||
let y0 = c.whiten(&[1.0, 0.0, 0.0]);
|
||||
let y1 = c.whiten(&[0.0, 1.0, 0.0]);
|
||||
assert!((bilinear(&y0, &y1) - 0.5).abs() < 1e-12);
|
||||
assert!((bilinear(&y1, &y0) - 0.5).abs() < 1e-12);
|
||||
}
|
||||
|
||||
/// A variance can never come out negative, because it is a sum of squares.
|
||||
#[test]
|
||||
fn a_quadratic_form_is_never_negative() {
|
||||
let a = vec![1e12, 1e12 - 1.0, 1e12 - 1.0, 1e12];
|
||||
let c = Cholesky::factor(a, 2).unwrap();
|
||||
let y = c.whiten(&[1.0, -1.0]);
|
||||
assert!(bilinear(&y, &y) >= 0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_a_non_positive_definite_matrix() {
|
||||
// Singular: the second row is a multiple of the first.
|
||||
assert!(Cholesky::factor(vec![1.0, 2.0, 2.0, 4.0], 2).is_none());
|
||||
}
|
||||
}
|
||||
+16
-12
@@ -26,21 +26,24 @@ where
|
||||
K: Eq + Hash + Clone,
|
||||
{
|
||||
#[must_use]
|
||||
pub fn new() -> Self {
|
||||
pub(crate) fn new() -> Self {
|
||||
Self {
|
||||
forward: HashMap::new(),
|
||||
reverse: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn get<Q: ?Sized + Hash + Eq>(&self, k: &Q) -> Option<Index>
|
||||
pub(crate) fn get<Q: ?Sized + Hash + Eq>(&self, k: &Q) -> Option<Index>
|
||||
where
|
||||
K: Borrow<Q>,
|
||||
{
|
||||
self.forward.get(k).cloned()
|
||||
}
|
||||
|
||||
pub fn get_or_create<Q: ?Sized + Hash + Eq + ToOwned<Owned = K>>(&mut self, k: &Q) -> Index
|
||||
pub(crate) fn get_or_create<Q: ?Sized + Hash + Eq + ToOwned<Owned = K>>(
|
||||
&mut self,
|
||||
k: &Q,
|
||||
) -> Index
|
||||
where
|
||||
K: Borrow<Q>,
|
||||
{
|
||||
@@ -56,23 +59,24 @@ where
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn key(&self, idx: Index) -> Option<&K> {
|
||||
pub(crate) fn key(&self, idx: Index) -> Option<&K> {
|
||||
self.reverse.get(idx.0)
|
||||
}
|
||||
|
||||
pub fn keys(&self) -> impl Iterator<Item = &K> {
|
||||
self.forward.keys()
|
||||
/// Every key, in the order they were first interned.
|
||||
///
|
||||
/// Iterates the dense reverse table rather than the forward `HashMap`.
|
||||
/// Rust seeds its default hasher per process, so a `HashMap` walk yields a
|
||||
/// different order on every run — which is fine for membership but not for
|
||||
/// anything a caller might sum, sort or print.
|
||||
pub(crate) fn keys(&self) -> impl ExactSizeIterator<Item = &K> {
|
||||
self.reverse.iter()
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn len(&self) -> usize {
|
||||
pub(crate) fn len(&self) -> usize {
|
||||
self.reverse.len()
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.reverse.is_empty()
|
||||
}
|
||||
}
|
||||
|
||||
impl<K> Default for KeyTable<K>
|
||||
|
||||
+974
-67
File diff suppressed because it is too large
Load Diff
+45
-4
@@ -91,6 +91,29 @@ impl Lu {
|
||||
det
|
||||
}
|
||||
|
||||
/// `ln |det|`, accumulated term by term rather than multiplied out.
|
||||
///
|
||||
/// The determinant of an `n x n` Gram matrix is a product of `n` diagonal
|
||||
/// entries, so it leaves `f64`'s range long before the quantities built
|
||||
/// from it do. `quality()` only ever wants a *ratio* of two determinants,
|
||||
/// and that ratio is perfectly representable while the determinants
|
||||
/// themselves are not — measured, at 250 rating groups both overflow and
|
||||
/// the ratio came back `NaN` where the true answer is `9.51e-88`.
|
||||
///
|
||||
/// Returns `-inf` for a singular matrix, so `exp` of it is zero.
|
||||
fn ln_abs_determinant(&self) -> f64 {
|
||||
if self.sign == 0.0 {
|
||||
return f64::NEG_INFINITY;
|
||||
}
|
||||
|
||||
let mut acc = 0.0;
|
||||
for i in 0..self.n {
|
||||
acc += libm::log(self.lu[i * self.n + i].abs());
|
||||
}
|
||||
|
||||
acc
|
||||
}
|
||||
|
||||
/// Solve `Ax = b` for a single column of the identity, giving one column
|
||||
/// of the inverse.
|
||||
fn solve_column(&self, col: usize, out: &mut [f64]) {
|
||||
@@ -117,7 +140,7 @@ impl Lu {
|
||||
}
|
||||
|
||||
impl Matrix {
|
||||
pub fn new(height: usize, width: usize) -> Matrix {
|
||||
pub(crate) fn new(height: usize, width: usize) -> Matrix {
|
||||
Matrix {
|
||||
data: vec![0.0; height * width].into_boxed_slice(),
|
||||
height,
|
||||
@@ -125,7 +148,7 @@ impl Matrix {
|
||||
}
|
||||
}
|
||||
|
||||
pub fn transpose(&self) -> Matrix {
|
||||
pub(crate) fn transpose(&self) -> Matrix {
|
||||
let mut matrix = Matrix::new(self.width, self.height);
|
||||
|
||||
for c in 0..self.width {
|
||||
@@ -143,7 +166,7 @@ impl Matrix {
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics if the matrix is not square.
|
||||
pub fn determinant(&self) -> f64 {
|
||||
pub(crate) fn determinant(&self) -> f64 {
|
||||
assert_eq!(
|
||||
self.width, self.height,
|
||||
"determinant requires a square matrix, got {}x{}",
|
||||
@@ -157,12 +180,30 @@ impl Matrix {
|
||||
Lu::decompose(self).determinant()
|
||||
}
|
||||
|
||||
/// `ln |det|` of a square matrix; `-inf` when singular.
|
||||
///
|
||||
/// See [`Lu::ln_abs_determinant`] for why a ratio of determinants must be
|
||||
/// taken this way.
|
||||
pub(crate) fn ln_abs_determinant(&self) -> f64 {
|
||||
assert_eq!(
|
||||
self.width, self.height,
|
||||
"determinant requires a square matrix, got {}x{}",
|
||||
self.height, self.width
|
||||
);
|
||||
|
||||
if self.width == 0 {
|
||||
return 0.0;
|
||||
}
|
||||
|
||||
Lu::decompose(self).ln_abs_determinant()
|
||||
}
|
||||
|
||||
/// Matrix inverse via LU decomposition.
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics if the matrix is not square or is singular.
|
||||
pub fn inverse(&self) -> Matrix {
|
||||
pub(crate) fn inverse(&self) -> Matrix {
|
||||
assert_eq!(
|
||||
self.width, self.height,
|
||||
"inverse requires a square matrix, got {}x{}",
|
||||
|
||||
+85
-2
@@ -14,13 +14,95 @@ pub trait Observer<T: Time>: Send + Sync {
|
||||
/// Called after each convergence iteration across the whole history.
|
||||
fn on_iteration_end(&self, _iter: usize, _max_step: (f64, f64)) {}
|
||||
|
||||
/// Called after each time slice is processed within an iteration.
|
||||
fn on_batch_processed(&self, _time: &T, _slice_idx: usize, _n_events: usize) {}
|
||||
/// Called after each time slice is swept within an iteration.
|
||||
///
|
||||
/// A convergence iteration sweeps every slice twice — once travelling
|
||||
/// backward through the history and once forward — so a multi-slice
|
||||
/// history fires this twice per slice per iteration. A single-slice
|
||||
/// history is swept once and fires once.
|
||||
fn on_slice_processed(&self, _time: &T, _slice_idx: usize, _n_events: usize) {}
|
||||
|
||||
/// Called once when convergence completes (or max iters is reached).
|
||||
fn on_converged(&self, _iters: usize, _final_step: (f64, f64), _converged: bool) {}
|
||||
}
|
||||
|
||||
/// Shared and boxed observers forward to what they point at.
|
||||
///
|
||||
/// `History` takes its observer by value, so a caller who wants to *read* what
|
||||
/// an observer recorded has to keep a handle to it. Without these impls the
|
||||
/// natural spelling does not compile:
|
||||
///
|
||||
/// ```
|
||||
/// # use std::sync::{Arc, Mutex};
|
||||
/// # use trueskill_tt::{History, Observer};
|
||||
/// #[derive(Default)]
|
||||
/// struct Recorder {
|
||||
/// iterations: Mutex<Vec<usize>>,
|
||||
/// }
|
||||
///
|
||||
/// impl Observer<i64> for Recorder {
|
||||
/// fn on_iteration_end(&self, iter: usize, _step: (f64, f64)) {
|
||||
/// self.iterations.lock().unwrap().push(iter);
|
||||
/// }
|
||||
/// }
|
||||
///
|
||||
/// let recorder = Arc::new(Recorder::default());
|
||||
/// let mut h = History::builder().observer(Arc::clone(&recorder)).build();
|
||||
/// h.record_winner(&"a", &"b", 1).unwrap();
|
||||
/// h.converge().unwrap();
|
||||
///
|
||||
/// // The caller's handle sees what the history's copy recorded.
|
||||
/// assert!(!recorder.iterations.lock().unwrap().is_empty());
|
||||
/// ```
|
||||
///
|
||||
/// The alternative was for every observer to wrap each of its own fields in an
|
||||
/// `Arc` and derive `Clone` — one allocation and one lock per field, and a
|
||||
/// pattern each implementor had to rediscover.
|
||||
///
|
||||
/// `?Sized` is deliberate: it makes `Arc<dyn Observer<T>>` and
|
||||
/// `Box<dyn Observer<T>>` work, so observers can be chosen at runtime.
|
||||
impl<T: Time, O: Observer<T> + ?Sized> Observer<T> for std::sync::Arc<O> {
|
||||
fn on_iteration_end(&self, iter: usize, max_step: (f64, f64)) {
|
||||
(**self).on_iteration_end(iter, max_step);
|
||||
}
|
||||
|
||||
fn on_slice_processed(&self, time: &T, slice_idx: usize, n_events: usize) {
|
||||
(**self).on_slice_processed(time, slice_idx, n_events);
|
||||
}
|
||||
|
||||
fn on_converged(&self, iters: usize, final_step: (f64, f64), converged: bool) {
|
||||
(**self).on_converged(iters, final_step, converged);
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: Time, O: Observer<T> + ?Sized> Observer<T> for Box<O> {
|
||||
fn on_iteration_end(&self, iter: usize, max_step: (f64, f64)) {
|
||||
(**self).on_iteration_end(iter, max_step);
|
||||
}
|
||||
|
||||
fn on_slice_processed(&self, time: &T, slice_idx: usize, n_events: usize) {
|
||||
(**self).on_slice_processed(time, slice_idx, n_events);
|
||||
}
|
||||
|
||||
fn on_converged(&self, iters: usize, final_step: (f64, f64), converged: bool) {
|
||||
(**self).on_converged(iters, final_step, converged);
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: Time, O: Observer<T> + ?Sized> Observer<T> for &O {
|
||||
fn on_iteration_end(&self, iter: usize, max_step: (f64, f64)) {
|
||||
(**self).on_iteration_end(iter, max_step);
|
||||
}
|
||||
|
||||
fn on_slice_processed(&self, time: &T, slice_idx: usize, n_events: usize) {
|
||||
(**self).on_slice_processed(time, slice_idx, n_events);
|
||||
}
|
||||
|
||||
fn on_converged(&self, iters: usize, final_step: (f64, f64), converged: bool) {
|
||||
(**self).on_converged(iters, final_step, converged);
|
||||
}
|
||||
}
|
||||
|
||||
/// ZST no-op observer; the default when none is configured.
|
||||
#[derive(Copy, Clone, Debug, Default)]
|
||||
pub struct NullObserver;
|
||||
@@ -35,6 +117,7 @@ mod tests {
|
||||
fn null_observer_compiles_for_i64() {
|
||||
let o = NullObserver;
|
||||
<NullObserver as Observer<i64>>::on_iteration_end(&o, 1, (0.0, 0.0));
|
||||
<NullObserver as Observer<i64>>::on_slice_processed(&o, &7, 0, 3);
|
||||
<NullObserver as Observer<i64>>::on_converged(&o, 5, (1e-6, 1e-6), true);
|
||||
}
|
||||
|
||||
|
||||
+79
-19
@@ -1,6 +1,6 @@
|
||||
//! Outcome of a match.
|
||||
//!
|
||||
//! `Ranked(ranks)` for ordinal results; `Scored { scores, sigma }` for
|
||||
//! `Ranked(ranks)` for ordinal results; `Scored { scores, score_sigma }` for
|
||||
//! continuous per-team scores (engages `MarginFactor` in the engine).
|
||||
|
||||
use smallvec::SmallVec;
|
||||
@@ -10,19 +10,41 @@ use smallvec::SmallVec;
|
||||
/// `Ranked(ranks)`: lower rank = better. Equal ranks mean a tie between those
|
||||
/// teams. `ranks.len()` must equal the number of teams in the event.
|
||||
///
|
||||
/// `Scored { scores, sigma }`: higher score = better. Adjacent (sorted) pairs
|
||||
/// `Scored { scores, score_sigma }`: higher score = better. Adjacent (sorted) pairs
|
||||
/// feed observed margins to `MarginFactor`. `scores.len()` must equal the
|
||||
/// number of teams in the event. `sigma` overrides `HistoryBuilder::score_sigma`
|
||||
/// when `Some`; `None` inherits the history default.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
#[non_exhaustive]
|
||||
#[must_use]
|
||||
pub enum Outcome {
|
||||
/// An ordinal finish: one rank per team, in the order the teams were given.
|
||||
///
|
||||
/// Lower is better, `0` is first, and equal values are a tie between those
|
||||
/// teams — which needs `p_draw > 0`, or ingestion rejects the event with
|
||||
/// [`InferenceError::TieWithoutDrawProbability`](crate::InferenceError::TieWithoutDrawProbability).
|
||||
///
|
||||
/// Only the ordering and the equalities are used. Ranks need not be dense
|
||||
/// or start at zero: inference sorts the teams and compares rank-adjacent
|
||||
/// pairs against a margin set by `p_draw`, so `[0, 1, 2]` and `[0, 5, 90]`
|
||||
/// are the same observation. A gap does not mean a bigger win — use
|
||||
/// `Scored` when the size of the difference is evidence.
|
||||
Ranked(SmallVec<[u32; 4]>),
|
||||
/// A continuous finish: one score per team, higher is better.
|
||||
///
|
||||
/// Unlike `Ranked`, the *sizes* of the differences are evidence. Teams are
|
||||
/// sorted by score and each adjacent pair's observed gap is fed to a
|
||||
/// `MarginFactor` as a measurement with standard deviation `score_sigma`,
|
||||
/// so
|
||||
/// beating a team by ten says more than beating them by one.
|
||||
#[non_exhaustive]
|
||||
Scored {
|
||||
/// Per-team scores, in the order the teams were given; higher is
|
||||
/// better. Must have one entry per team, and every entry finite.
|
||||
scores: SmallVec<[f64; 4]>,
|
||||
/// Per-event noise override. `None` means inherit
|
||||
/// `HistoryBuilder::score_sigma`. Must be `> 0.0` if `Some`.
|
||||
sigma: Option<f64>,
|
||||
score_sigma: Option<f64>,
|
||||
},
|
||||
}
|
||||
|
||||
@@ -34,16 +56,43 @@ impl Outcome {
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics if `winner >= n`.
|
||||
#[must_use]
|
||||
/// Panics if `winner >= n`. Use [`Outcome::try_winner`] when the index
|
||||
/// comes from data rather than a literal.
|
||||
///
|
||||
/// This is the one constructor here that validates, and deliberately so.
|
||||
/// Its siblings build freely and let ingestion reject what it cannot use,
|
||||
/// which works because a malformed rank vector stays recognisable. An
|
||||
/// out-of-range winner does not: `winner(5, 2)` would produce ranks
|
||||
/// `[1, 1]`, an all-tied draw that ingestion accepts without complaint when
|
||||
/// `p_draw > 0`. Asking "team 5 won" and silently getting "everyone drew"
|
||||
/// is exactly the class of quiet wrong answer this crate keeps removing, so
|
||||
/// the check happens here where the mistake is.
|
||||
pub fn winner(winner: u32, n: u32) -> Self {
|
||||
assert!(winner < n, "winner index {winner} out of range 0..{n}");
|
||||
Self::try_winner(winner, n)
|
||||
.unwrap_or_else(|_| panic!("winner index {winner} out of range 0..{n}"))
|
||||
}
|
||||
|
||||
/// `n`-team outcome where team `winner` won, or an error if `winner` is not
|
||||
/// a valid team index.
|
||||
///
|
||||
/// The fallible form of [`Outcome::winner`], for when the index is computed
|
||||
/// or parsed rather than written literally.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// `InvalidParameter` if `winner >= n`.
|
||||
pub fn try_winner(winner: u32, n: u32) -> Result<Self, crate::InferenceError> {
|
||||
if winner >= n {
|
||||
return Err(crate::InferenceError::InvalidParameter {
|
||||
name: "winner",
|
||||
value: f64::from(winner),
|
||||
});
|
||||
}
|
||||
let ranks: SmallVec<[u32; 4]> = (0..n).map(|i| if i == winner { 0 } else { 1 }).collect();
|
||||
Self::Ranked(ranks)
|
||||
Ok(Self::Ranked(ranks))
|
||||
}
|
||||
|
||||
/// All `n` teams tied.
|
||||
#[must_use]
|
||||
pub fn draw(n: u32) -> Self {
|
||||
Self::Ranked(SmallVec::from_vec(vec![0; n as usize]))
|
||||
}
|
||||
@@ -58,23 +107,34 @@ impl Outcome {
|
||||
pub fn scores<I: IntoIterator<Item = f64>>(scores: I) -> Self {
|
||||
Self::Scored {
|
||||
scores: scores.into_iter().collect(),
|
||||
sigma: None,
|
||||
score_sigma: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Explicit per-team continuous scores with a per-event noise override.
|
||||
///
|
||||
/// `sigma` must be `> 0.0`. Constructing an `Outcome` with a non-positive
|
||||
/// or NaN sigma is allowed; the value is rejected with
|
||||
/// The noise is on the *observed score margin*, in the units of the scores
|
||||
/// themselves — it is not a skill sigma, which is what the old name
|
||||
/// `scores_with_sigma` read as. It overrides `HistoryBuilder::score_sigma`
|
||||
/// for this event only.
|
||||
///
|
||||
/// `score_sigma` must be `> 0.0`. Constructing an `Outcome` with a
|
||||
/// non-positive or NaN value is allowed; the value is rejected with
|
||||
/// `InferenceError::InvalidParameter` when the event is ingested, so
|
||||
/// callers get an error rather than a panic.
|
||||
pub fn scores_with_sigma<I: IntoIterator<Item = f64>>(scores: I, sigma: f64) -> Self {
|
||||
pub fn scores_with_noise<I: IntoIterator<Item = f64>>(scores: I, score_sigma: f64) -> Self {
|
||||
Self::Scored {
|
||||
scores: scores.into_iter().collect(),
|
||||
sigma: Some(sigma),
|
||||
score_sigma: Some(score_sigma),
|
||||
}
|
||||
}
|
||||
|
||||
/// How many teams this outcome describes — the number of ranks, or of
|
||||
/// scores.
|
||||
///
|
||||
/// Ingestion checks it against the event's own team list and rejects a
|
||||
/// disagreement with `MismatchedShape`, so this is the cheap way to check
|
||||
/// an outcome built elsewhere before committing the event.
|
||||
#[must_use]
|
||||
pub fn team_count(&self) -> usize {
|
||||
match self {
|
||||
@@ -156,7 +216,7 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn scores_with_sigma_round_trips() {
|
||||
let o = Outcome::scores_with_sigma([10.0, 4.0], 0.5);
|
||||
let o = Outcome::scores_with_noise([10.0, 4.0], 0.5);
|
||||
assert_eq!(o.team_count(), 2);
|
||||
assert_eq!(o.as_scores(), Some(&[10.0, 4.0][..]));
|
||||
}
|
||||
@@ -165,16 +225,16 @@ mod tests {
|
||||
fn scores_constructor_leaves_sigma_unset() {
|
||||
let o = Outcome::scores([3.0, 1.0]);
|
||||
match o {
|
||||
Outcome::Scored { scores: _, sigma } => assert!(sigma.is_none()),
|
||||
Outcome::Scored { score_sigma, .. } => assert!(score_sigma.is_none()),
|
||||
Outcome::Ranked(_) => panic!("expected Scored variant"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scores_with_sigma_sets_sigma_some() {
|
||||
let o = Outcome::scores_with_sigma([3.0, 1.0], 2.0);
|
||||
let o = Outcome::scores_with_noise([3.0, 1.0], 2.0);
|
||||
match o {
|
||||
Outcome::Scored { scores: _, sigma } => assert_eq!(sigma, Some(2.0)),
|
||||
Outcome::Scored { score_sigma, .. } => assert_eq!(score_sigma, Some(2.0)),
|
||||
Outcome::Ranked(_) => panic!("expected Scored variant"),
|
||||
}
|
||||
}
|
||||
@@ -184,9 +244,9 @@ mod tests {
|
||||
/// `tests/degenerate_inputs.rs::scored_event_rejects_non_positive_sigma`.
|
||||
#[test]
|
||||
fn scores_with_sigma_defers_validation_to_ingestion() {
|
||||
let o = Outcome::scores_with_sigma([3.0, 1.0], 0.0);
|
||||
let o = Outcome::scores_with_noise([3.0, 1.0], 0.0);
|
||||
match o {
|
||||
Outcome::Scored { sigma, .. } => assert_eq!(sigma, Some(0.0)),
|
||||
Outcome::Scored { score_sigma, .. } => assert_eq!(score_sigma, Some(0.0)),
|
||||
Outcome::Ranked(_) => panic!("expected Scored variant"),
|
||||
}
|
||||
}
|
||||
|
||||
+760
@@ -0,0 +1,760 @@
|
||||
//! Outcome prediction: who wins, and how likely is a given finishing order.
|
||||
//!
|
||||
//! Prediction runs on *performances*, not skills. A competitor's skill is
|
||||
//! inflated by their performance noise `beta` before any comparison, which is
|
||||
//! what separates "how good are they" from "how will they do today".
|
||||
//!
|
||||
//! Two questions, two algorithms:
|
||||
//!
|
||||
//! - **Who finishes first.** Because performances are independent Gaussians,
|
||||
//! the probability that team `i` beats every other team separates into a
|
||||
//! *one-dimensional* integral — no multivariate orthant integral is
|
||||
//! involved. [`quadrature::integrate`] evaluates it to near machine
|
||||
//! precision for a few hundred `cdf` calls.
|
||||
//! - **A specific finishing order.** The factor graph only ever constrains
|
||||
//! rank-*adjacent* teams (see `Game::run_chain`), so the joint probability
|
||||
//! of a full order is a chain of local constraints rather than a general
|
||||
//! orthant probability. That chain collapses into a sequential recursion:
|
||||
//! one cumulative integral per adjacent pair, `O(teams * grid)` overall.
|
||||
//!
|
||||
//! Both are deterministic. A sampler would have been easier to write and
|
||||
//! would have made every `predict_*` call return a slightly different number,
|
||||
//! which is not a property a rating library should have.
|
||||
|
||||
use crate::{Gaussian, InferenceError, quadrature};
|
||||
|
||||
/// Teams beyond this count make the outcome enumeration impractical.
|
||||
///
|
||||
/// Each realisation sorts into exactly one (permutation, tie-pattern) event,
|
||||
/// so the space has `n! * 2^(n-1)` members: 24 at 3 teams, 192 at 4, 1_920 at
|
||||
/// 5, 23_040 at 6. The jump to 322_560 at 7 is where enumerating stops being
|
||||
/// a reasonable thing to do on a caller's behalf.
|
||||
pub(crate) const MAX_TEAMS_FOR_DISTRIBUTION: usize = 6;
|
||||
|
||||
/// Relative tolerance for the first-place integrals.
|
||||
///
|
||||
/// The adaptive integrator reaches the exact two-team closed form to ~1e-15 at
|
||||
/// this tolerance, which is round-off for a probability. `cdf` is no longer the
|
||||
/// limit — it went to ~1 ULP when `erfc` moved to `libm` — so this is the
|
||||
/// integrator's own floor.
|
||||
const WIN_TOLERANCE: f64 = 1e-8;
|
||||
|
||||
/// Nodes for the ranking grid, and the floor below which a grid is pointless.
|
||||
///
|
||||
/// The recursion converges as O(h^2), so this trades nodes against accuracy
|
||||
/// directly. Measured against the exact two-team closed form, 2_048 nodes leave
|
||||
/// ~1.2e-6 of discretisation error and 8_192 reach ~1e-7.
|
||||
///
|
||||
/// Unlike the adaptive path there is no approximation floor underneath this any
|
||||
/// more — `cdf` is accurate to ~1 ULP since `erfc` moved to `libm` — so the
|
||||
/// error here is purely the grid, and a caller who needs more can only get it
|
||||
/// by paying for more nodes. 8_192 is the accuracy/cost point chosen, not a
|
||||
/// point where refining stops helping.
|
||||
const MIN_GRID_POINTS: usize = 8_192;
|
||||
const MAX_GRID_POINTS: usize = 262_144;
|
||||
/// Nodes requested across the narrowest feature the recursion must resolve.
|
||||
const NODES_PER_FEATURE: f64 = 12.0;
|
||||
/// Nodes below which the trapezoid rule stops resolving that feature at all.
|
||||
const MIN_NODES_PER_FEATURE: f64 = 4.0;
|
||||
|
||||
/// How many standard deviations of support the grid and integrals cover.
|
||||
///
|
||||
/// The normal density is below 1e-18 of its peak past nine sigma, far under
|
||||
/// the precision of everything else here.
|
||||
const SUPPORT_SIGMAS: f64 = 9.0;
|
||||
|
||||
/// Standard normal CDF at `z`.
|
||||
fn phi(z: f64) -> f64 {
|
||||
crate::cdf(z, 0.0, 1.0)
|
||||
}
|
||||
|
||||
/// Normal density of `x` under `g`.
|
||||
fn density(g: Gaussian, x: f64) -> f64 {
|
||||
let sigma = g.sigma();
|
||||
let z = (x - g.mu()) / sigma;
|
||||
libm::exp(-0.5 * z * z) / (sigma * (2.0 * std::f64::consts::PI).sqrt())
|
||||
}
|
||||
|
||||
/// Per-pair draw margins.
|
||||
///
|
||||
/// The margin is *not* a single number for the whole game: inference derives
|
||||
/// it per rank-adjacent pair from those two teams' betas (`Game::likelihoods`).
|
||||
/// Prediction has to use the same per-pair values or it answers a question
|
||||
/// about a different model than the one that will actually be fitted.
|
||||
pub(crate) struct Margins {
|
||||
n: usize,
|
||||
values: Vec<f64>,
|
||||
}
|
||||
|
||||
impl Margins {
|
||||
/// Build from a per-pair margin function.
|
||||
pub(crate) fn new<F: Fn(usize, usize) -> f64>(n: usize, f: F) -> Self {
|
||||
let mut values = vec![0.0; n * n];
|
||||
for i in 0..n {
|
||||
for j in 0..n {
|
||||
if i != j {
|
||||
values[i * n + j] = f(i, j);
|
||||
}
|
||||
}
|
||||
}
|
||||
Self { n, values }
|
||||
}
|
||||
|
||||
fn get(&self, i: usize, j: usize) -> f64 {
|
||||
self.values[i * self.n + j]
|
||||
}
|
||||
|
||||
/// True when no pair can draw, so every tie has probability zero.
|
||||
fn all_zero(&self) -> bool {
|
||||
self.values.iter().all(|&v| v == 0.0)
|
||||
}
|
||||
}
|
||||
|
||||
/// `P(team i finishes strictly first)` for every team.
|
||||
///
|
||||
/// Strictly means beating each rival by more than that pair's draw margin, so
|
||||
/// with a non-zero margin these sum to less than one; the shortfall is the
|
||||
/// probability that the top place is shared.
|
||||
pub(crate) fn win_probabilities(perf: &[Gaussian], margins: &Margins) -> Vec<f64> {
|
||||
(0..perf.len())
|
||||
.map(|i| {
|
||||
let (mu, sigma) = (perf[i].mu(), perf[i].sigma());
|
||||
let (lo, hi) = (mu - SUPPORT_SIGMAS * sigma, mu + SUPPORT_SIGMAS * sigma);
|
||||
|
||||
// Each rival's CDF turns over near its own mean plus the margin.
|
||||
// Seeding there is what keeps a rival with a tiny sigma — a step
|
||||
// function in disguise — from being stepped over.
|
||||
let mut seeds = Vec::with_capacity(3 * perf.len());
|
||||
for (j, rival) in perf.iter().enumerate().filter(|&(j, _)| j != i) {
|
||||
let centre = rival.mu() + margins.get(i, j);
|
||||
seeds.extend_from_slice(&[centre - rival.sigma(), centre, centre + rival.sigma()]);
|
||||
}
|
||||
|
||||
quadrature::integrate(
|
||||
|x| {
|
||||
let d = density(perf[i], x);
|
||||
if d == 0.0 {
|
||||
return 0.0;
|
||||
}
|
||||
let beaten: f64 = (0..perf.len())
|
||||
.filter(|&j| j != i)
|
||||
.map(|j| phi((x - margins.get(i, j) - perf[j].mu()) / perf[j].sigma()))
|
||||
.product();
|
||||
d * beaten
|
||||
},
|
||||
lo,
|
||||
hi,
|
||||
&seeds,
|
||||
WIN_TOLERANCE,
|
||||
)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Grid bounds and resolution covering every team's support.
|
||||
///
|
||||
/// Resolution is set by the *smallest* feature in play — the narrowest sigma,
|
||||
/// or a draw margin narrower still — because that is what the recursion has to
|
||||
/// resolve. A grid sized off the widest team would step over the narrow one.
|
||||
fn grid_shape(perf: &[Gaussian], margins: &Margins) -> Result<(f64, f64, usize), InferenceError> {
|
||||
let lo = perf
|
||||
.iter()
|
||||
.map(|g| g.mu() - SUPPORT_SIGMAS * g.sigma())
|
||||
.fold(f64::INFINITY, f64::min);
|
||||
let hi = perf
|
||||
.iter()
|
||||
.map(|g| g.mu() + SUPPORT_SIGMAS * g.sigma())
|
||||
.fold(f64::NEG_INFINITY, f64::max);
|
||||
|
||||
let narrowest = perf
|
||||
.iter()
|
||||
.map(Gaussian::sigma)
|
||||
.fold(f64::INFINITY, f64::min);
|
||||
let smallest_margin = margins
|
||||
.values
|
||||
.iter()
|
||||
.copied()
|
||||
.filter(|&m| m > 0.0)
|
||||
.fold(f64::INFINITY, f64::min);
|
||||
|
||||
let feature = narrowest.min(smallest_margin);
|
||||
let wanted = if feature.is_finite() && feature > 0.0 {
|
||||
((hi - lo) / (feature / NODES_PER_FEATURE)).ceil()
|
||||
} else {
|
||||
MIN_GRID_POINTS as f64
|
||||
};
|
||||
|
||||
if !wanted.is_finite() {
|
||||
return Ok((lo, hi, MIN_GRID_POINTS));
|
||||
}
|
||||
|
||||
// Report rather than clamp. Clamping is what this replaced: it silently
|
||||
// handed the recursion a grid too coarse for the narrowest density, and the
|
||||
// trapezoid rule then returned probabilities greater than one — measured, a
|
||||
// `P` of 2.79 and a total of 5.41. Trapezoid error on a Gaussian is
|
||||
// `~exp(-2 pi^2 (sigma/h)^2)`, which is 1e-12 at `h/sigma = 0.86` and O(1)
|
||||
// by `h/sigma = 17`, so the cliff is sharp and there is no useful answer on
|
||||
// the far side of it.
|
||||
//
|
||||
// The floor is `MIN_NODES_PER_FEATURE` rather than the `NODES_PER_FEATURE`
|
||||
// asked for, because the request carries a large margin: measured accurate
|
||||
// to 2.2e-12 at 1.4 nodes per sigma, and wrong by 1.2e-3 at 0.7.
|
||||
let needed = wanted as usize;
|
||||
let floor = ((hi - lo) / (feature / MIN_NODES_PER_FEATURE)).ceil();
|
||||
if floor.is_finite() && floor as usize > MAX_GRID_POINTS {
|
||||
return Err(InferenceError::GridTooCoarse {
|
||||
needed,
|
||||
max: MAX_GRID_POINTS,
|
||||
});
|
||||
}
|
||||
|
||||
Ok((lo, hi, needed.clamp(MIN_GRID_POINTS, MAX_GRID_POINTS)))
|
||||
}
|
||||
|
||||
/// Densities of each team sampled on the shared grid.
|
||||
struct Sampled {
|
||||
lo: f64,
|
||||
step: f64,
|
||||
points: usize,
|
||||
density: Vec<Vec<f64>>,
|
||||
}
|
||||
|
||||
impl Sampled {
|
||||
fn new(perf: &[Gaussian], margins: &Margins) -> Result<Self, InferenceError> {
|
||||
let (lo, hi, points) = grid_shape(perf, margins)?;
|
||||
let step = (hi - lo) / (points - 1) as f64;
|
||||
let density = perf
|
||||
.iter()
|
||||
.map(|&g| {
|
||||
(0..points)
|
||||
.map(|i| density(g, lo + i as f64 * step))
|
||||
.collect()
|
||||
})
|
||||
.collect();
|
||||
Ok(Self {
|
||||
lo,
|
||||
step,
|
||||
points,
|
||||
density,
|
||||
})
|
||||
}
|
||||
|
||||
fn node(&self, i: usize) -> f64 {
|
||||
self.lo + i as f64 * self.step
|
||||
}
|
||||
}
|
||||
|
||||
/// `P(order[0] >= order[1] >= ... )` with the given adjacency pattern.
|
||||
///
|
||||
/// `tied[k]` says whether `order[k]` and `order[k + 1]` finish within that
|
||||
/// pair's draw margin. The recursion runs bottom-up: `carry` holds, for each
|
||||
/// grid node, the probability that everything *below* the current team holds
|
||||
/// given that team landed on that node. A strict gap reads a cumulative
|
||||
/// integral; a tie reads a window. Both are O(1) against one prefix array,
|
||||
/// so each level costs O(grid) and the whole order costs O(teams * grid).
|
||||
fn order_probability(margins: &Margins, sampled: &Sampled, order: &[usize], tied: &[bool]) -> f64 {
|
||||
let mut carry = vec![1.0; sampled.points];
|
||||
|
||||
for k in (0..order.len() - 1).rev() {
|
||||
let below = order[k + 1];
|
||||
let above = order[k];
|
||||
let margin = margins.get(above, below);
|
||||
|
||||
let integrand: Vec<f64> = (0..sampled.points)
|
||||
.map(|i| sampled.density[below][i] * carry[i])
|
||||
.collect();
|
||||
let cumulative = quadrature::Grid::from_values(sampled.lo, sampled.step, integrand);
|
||||
|
||||
carry = (0..sampled.points)
|
||||
.map(|i| {
|
||||
let x = sampled.node(i);
|
||||
if tied[k] {
|
||||
// Sorted order already implies `below <= above`, so the
|
||||
// tie window is one-sided: [x - margin, x].
|
||||
cumulative.integral_between(x - margin, x)
|
||||
} else {
|
||||
cumulative.integral_to(x - margin)
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
}
|
||||
|
||||
let top = order[0];
|
||||
let integrand: Vec<f64> = (0..sampled.points)
|
||||
.map(|i| sampled.density[top][i] * carry[i])
|
||||
.collect();
|
||||
quadrature::Grid::from_values(sampled.lo, sampled.step, integrand).total()
|
||||
}
|
||||
|
||||
/// Dense ranks implied by a sorted order and its tie pattern.
|
||||
fn ranks_of(order: &[usize], tied: &[bool], n: usize) -> Vec<u32> {
|
||||
let mut ranks = vec![0u32; n];
|
||||
let mut rank = 0u32;
|
||||
ranks[order[0]] = 0;
|
||||
for k in 0..order.len() - 1 {
|
||||
if !tied[k] {
|
||||
rank += 1;
|
||||
}
|
||||
ranks[order[k + 1]] = rank;
|
||||
}
|
||||
ranks
|
||||
}
|
||||
|
||||
/// Every (order, tie-pattern) event, or only the strict ones when no pair can
|
||||
/// draw — a tie then has probability exactly zero and is not worth integrating.
|
||||
fn events(n: usize, strict_only: bool) -> Vec<(Vec<usize>, Vec<bool>)> {
|
||||
fn permute(current: &mut Vec<usize>, k: usize, out: &mut Vec<Vec<usize>>) {
|
||||
if k == current.len() {
|
||||
out.push(current.clone());
|
||||
return;
|
||||
}
|
||||
for i in k..current.len() {
|
||||
current.swap(k, i);
|
||||
permute(current, k + 1, out);
|
||||
current.swap(k, i);
|
||||
}
|
||||
}
|
||||
|
||||
let mut orders = Vec::new();
|
||||
permute(&mut (0..n).collect(), 0, &mut orders);
|
||||
|
||||
let patterns: Vec<Vec<bool>> = if strict_only {
|
||||
vec![vec![false; n - 1]]
|
||||
} else {
|
||||
(0..(1u32 << (n - 1)))
|
||||
.map(|mask| (0..n - 1).map(|i| mask >> i & 1 == 1).collect())
|
||||
.collect()
|
||||
};
|
||||
|
||||
let mut out = Vec::with_capacity(orders.len() * patterns.len());
|
||||
for order in orders {
|
||||
for pattern in &patterns {
|
||||
out.push((order.clone(), pattern.clone()));
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// The full distribution over finishing orders, aggregated by rank vector.
|
||||
///
|
||||
/// Orders that differ only *within* a tied group describe the same finishing
|
||||
/// order, so their probabilities are summed into one entry.
|
||||
pub(crate) fn outcome_distribution(
|
||||
perf: &[Gaussian],
|
||||
margins: &Margins,
|
||||
) -> Result<Vec<(Vec<u32>, f64)>, InferenceError> {
|
||||
let n = perf.len();
|
||||
let sampled = Sampled::new(perf, margins)?;
|
||||
|
||||
let mut aggregated: Vec<(Vec<u32>, f64)> = Vec::new();
|
||||
for (order, tied) in events(n, margins.all_zero()) {
|
||||
let p = order_probability(margins, &sampled, &order, &tied);
|
||||
let ranks = ranks_of(&order, &tied, n);
|
||||
match aggregated.iter_mut().find(|(r, _)| *r == ranks) {
|
||||
Some((_, acc)) => *acc += p,
|
||||
None => aggregated.push((ranks, p)),
|
||||
}
|
||||
}
|
||||
|
||||
aggregated.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal));
|
||||
Ok(aggregated)
|
||||
}
|
||||
|
||||
/// All permutations of `items`.
|
||||
fn permutations(items: &[usize]) -> Vec<Vec<usize>> {
|
||||
fn go(current: &mut Vec<usize>, k: usize, out: &mut Vec<Vec<usize>>) {
|
||||
if k == current.len() {
|
||||
out.push(current.clone());
|
||||
return;
|
||||
}
|
||||
for i in k..current.len() {
|
||||
current.swap(k, i);
|
||||
go(current, k + 1, out);
|
||||
current.swap(k, i);
|
||||
}
|
||||
}
|
||||
let mut out = Vec::new();
|
||||
go(&mut items.to_vec(), 0, &mut out);
|
||||
out
|
||||
}
|
||||
|
||||
/// Every (order, tie-pattern) event consistent with a grouping by rank.
|
||||
///
|
||||
/// Teams sharing a rank may finish in any internal order, so this is the
|
||||
/// product of each group's permutations. Adjacencies inside a group are ties;
|
||||
/// the adjacency joining one group to the next is not.
|
||||
fn orders_for_groups(groups: &[Vec<usize>]) -> Vec<(Vec<usize>, Vec<bool>)> {
|
||||
let per_group: Vec<Vec<Vec<usize>>> = groups.iter().map(|g| permutations(g)).collect();
|
||||
|
||||
let mut out = Vec::new();
|
||||
let mut choice = vec![0usize; groups.len()];
|
||||
|
||||
loop {
|
||||
let mut order = Vec::new();
|
||||
let mut tied = Vec::new();
|
||||
for (gi, group) in per_group.iter().enumerate() {
|
||||
for (offset, &member) in group[choice[gi]].iter().enumerate() {
|
||||
if !order.is_empty() {
|
||||
tied.push(offset != 0);
|
||||
}
|
||||
order.push(member);
|
||||
}
|
||||
}
|
||||
out.push((order, tied));
|
||||
|
||||
let mut k = 0;
|
||||
loop {
|
||||
if k == choice.len() {
|
||||
return out;
|
||||
}
|
||||
choice[k] += 1;
|
||||
if choice[k] < per_group[k].len() {
|
||||
break;
|
||||
}
|
||||
choice[k] = 0;
|
||||
k += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Probability of one specific rank vector.
|
||||
///
|
||||
/// Ties in `ranks` mean the tied teams may finish in any internal order, so
|
||||
/// this sums the orders consistent with the requested ranking rather than
|
||||
/// picking one.
|
||||
pub(crate) fn ranking_probability(
|
||||
perf: &[Gaussian],
|
||||
margins: &Margins,
|
||||
ranks: &[u32],
|
||||
) -> Result<f64, InferenceError> {
|
||||
let n = perf.len();
|
||||
let sampled = Sampled::new(perf, margins)?;
|
||||
|
||||
let mut distinct: Vec<u32> = ranks.to_vec();
|
||||
distinct.sort_unstable();
|
||||
distinct.dedup();
|
||||
|
||||
let groups: Vec<Vec<usize>> = distinct
|
||||
.iter()
|
||||
.map(|&r| (0..n).filter(|&i| ranks[i] == r).collect())
|
||||
.collect();
|
||||
|
||||
Ok(orders_for_groups(&groups)
|
||||
.iter()
|
||||
.map(|(order, tied)| order_probability(margins, &sampled, order, tied))
|
||||
.sum())
|
||||
}
|
||||
|
||||
/// A distribution over the ways a contest could finish.
|
||||
///
|
||||
/// Each entry pairs a rank vector — the same shape [`crate::Outcome::ranking`]
|
||||
/// takes, with equal ranks meaning a tie — against its probability. Entries
|
||||
/// are ordered most likely first, and cover the whole outcome space, so the
|
||||
/// probabilities sum to one.
|
||||
///
|
||||
/// The rank vectors compose directly with inference: feeding one to
|
||||
/// `Game::ranked` asks "what would we believe if *this* happened", which is
|
||||
/// what an expected-information-gain calculation needs alongside the weight.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
#[must_use]
|
||||
pub struct Prediction {
|
||||
outcomes: Vec<(Vec<u32>, f64)>,
|
||||
}
|
||||
|
||||
impl Prediction {
|
||||
pub(crate) fn new(outcomes: Vec<(Vec<u32>, f64)>) -> Self {
|
||||
Self { outcomes }
|
||||
}
|
||||
|
||||
/// Every possible finishing order and its probability, most likely first.
|
||||
#[must_use]
|
||||
pub fn outcomes(&self) -> impl ExactSizeIterator<Item = (&[u32], f64)> {
|
||||
self.outcomes.iter().map(|(r, p)| (r.as_slice(), *p))
|
||||
}
|
||||
|
||||
/// The single most likely finishing order.
|
||||
#[must_use]
|
||||
pub fn most_likely(&self) -> Option<(&[u32], f64)> {
|
||||
self.outcomes.first().map(|(r, p)| (r.as_slice(), *p))
|
||||
}
|
||||
|
||||
/// Probability of one specific finishing order, or zero if it cannot occur.
|
||||
#[must_use]
|
||||
pub fn probability_of(&self, ranks: &[u32]) -> f64 {
|
||||
self.outcomes
|
||||
.iter()
|
||||
.find(|(r, _)| r.as_slice() == ranks)
|
||||
.map_or(0.0, |(_, p)| *p)
|
||||
}
|
||||
|
||||
/// `P(team i finishes strictly first)`, for each team.
|
||||
///
|
||||
/// Sums to less than one exactly when the top place can be shared; the
|
||||
/// shortfall is [`Prediction::shared_first_place`].
|
||||
#[must_use]
|
||||
pub fn win_probabilities(&self) -> Vec<f64> {
|
||||
let n = self.outcomes.first().map_or(0, |(r, _)| r.len());
|
||||
let mut wins = vec![0.0; n];
|
||||
for (ranks, p) in &self.outcomes {
|
||||
let leaders = ranks.iter().filter(|&&r| r == 0).count();
|
||||
if leaders == 1 {
|
||||
let winner = ranks.iter().position(|&r| r == 0).expect("a rank-0 team");
|
||||
wins[winner] += p;
|
||||
}
|
||||
}
|
||||
wins
|
||||
}
|
||||
|
||||
/// Probability that two or more teams share first place.
|
||||
#[must_use]
|
||||
pub fn shared_first_place(&self) -> f64 {
|
||||
self.outcomes
|
||||
.iter()
|
||||
.filter(|(r, _)| r.iter().filter(|&&x| x == 0).count() > 1)
|
||||
.map(|(_, p)| p)
|
||||
.sum()
|
||||
}
|
||||
|
||||
/// Total probability mass, which should be one.
|
||||
///
|
||||
/// Exposed because it is a genuine check on the numerics rather than a
|
||||
/// formality: the outcome space is exhaustive and disjoint by construction,
|
||||
/// so any drift from one is integration error and nothing else.
|
||||
#[must_use]
|
||||
pub fn total(&self) -> f64 {
|
||||
self.outcomes.iter().map(|(_, p)| p).sum()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn g(mu: f64, sigma: f64) -> Gaussian {
|
||||
Gaussian::from_ms(mu, sigma)
|
||||
}
|
||||
|
||||
fn flat(n: usize, eps: f64) -> Margins {
|
||||
Margins::new(n, |_, _| eps)
|
||||
}
|
||||
|
||||
/// Exact two-team result: `P(a first) = Phi((mu_a - mu_b - eps) / sd)`.
|
||||
fn closed_form_two(a: Gaussian, b: Gaussian, eps: f64) -> (f64, f64) {
|
||||
let sd = a.sigma().hypot(b.sigma());
|
||||
(
|
||||
phi((a.mu() - b.mu() - eps) / sd),
|
||||
phi((b.mu() - a.mu() - eps) / sd),
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn two_team_win_probabilities_match_the_closed_form() {
|
||||
for (ma, sa, mb, sb, eps) in [
|
||||
(0.0, 6.0, 0.0, 6.0, 0.0),
|
||||
(3.0, 6.0, -2.0, 1.0, 0.0),
|
||||
(0.0, 6.0, 0.0, 6.0, 2.0),
|
||||
(3.0, 6.0, -2.0, 1.0, 1.5),
|
||||
(40.0, 1.0, 0.0, 1.0, 0.0),
|
||||
] {
|
||||
let perf = [g(ma, sa), g(mb, sb)];
|
||||
let got = win_probabilities(&perf, &flat(2, eps));
|
||||
let (wa, wb) = closed_form_two(perf[0], perf[1], eps);
|
||||
assert!(
|
||||
(got[0] - wa).abs() < 1e-12 && (got[1] - wb).abs() < 1e-12,
|
||||
"mu=({ma},{mb}) sigma=({sa},{sb}) eps={eps}: got {got:?}, want [{wa}, {wb}]"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// The identity that a wrong-but-plausible implementation cannot fake:
|
||||
/// with no draw margin, exactly one team finishes first.
|
||||
#[test]
|
||||
fn win_probabilities_sum_to_one_without_a_draw_margin() {
|
||||
for perf in [
|
||||
vec![g(0.0, 6.0), g(0.0, 6.0)],
|
||||
vec![g(5.0, 6.0), g(0.0, 3.0), g(-5.0, 1.0)],
|
||||
vec![
|
||||
g(8.0, 2.0),
|
||||
g(3.0, 6.0),
|
||||
g(0.0, 1.0),
|
||||
g(-3.0, 4.0),
|
||||
g(-8.0, 6.0),
|
||||
],
|
||||
] {
|
||||
let sum: f64 = win_probabilities(&perf, &flat(perf.len(), 0.0))
|
||||
.iter()
|
||||
.sum();
|
||||
assert!(
|
||||
(sum - 1.0).abs() < 1e-7,
|
||||
"{} teams: sum = {sum}",
|
||||
perf.len()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// A rival with a tiny sigma is a step function in disguise. Fixed-node
|
||||
/// quadrature steps over it and lands ~1e-2 out while still looking like a
|
||||
/// probability; this is the case that rules that approach out.
|
||||
#[test]
|
||||
fn win_probabilities_survive_a_rival_with_a_tiny_sigma() {
|
||||
let perf = [g(0.0, 0.001), g(0.5, 6.0), g(-0.5, 6.0)];
|
||||
let got = win_probabilities(&perf, &flat(3, 0.0));
|
||||
let sum: f64 = got.iter().sum();
|
||||
assert!((sum - 1.0).abs() < 1e-6, "sum = {sum}, probs = {got:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_stronger_team_is_more_likely_to_win() {
|
||||
let perf = [g(10.0, 3.0), g(0.0, 3.0), g(-10.0, 3.0)];
|
||||
let p = win_probabilities(&perf, &flat(3, 0.0));
|
||||
assert!(p[0] > p[1] && p[1] > p[2], "not monotone: {p:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn identical_teams_are_equally_likely_to_win() {
|
||||
let perf = [g(1.0, 4.0), g(1.0, 4.0), g(1.0, 4.0)];
|
||||
let p = win_probabilities(&perf, &flat(3, 0.0));
|
||||
for probs in p.windows(2) {
|
||||
assert!((probs[0] - probs[1]).abs() < 1e-9, "asymmetric: {p:?}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Every realisation sorts into exactly one finishing order, so the whole
|
||||
/// distribution must sum to one — with or without a draw margin.
|
||||
#[test]
|
||||
fn outcome_distribution_sums_to_one() {
|
||||
for (perf, eps) in [
|
||||
(vec![g(0.0, 6.0), g(0.0, 6.0)], 0.0),
|
||||
(vec![g(0.0, 6.0), g(0.0, 6.0)], 2.0),
|
||||
(vec![g(0.0, 6.0), g(0.0, 6.0), g(0.0, 6.0)], 0.0),
|
||||
(vec![g(5.0, 6.0), g(0.0, 3.0), g(-5.0, 1.0)], 1.5),
|
||||
(vec![g(0.0, 0.05), g(0.5, 6.0), g(-0.5, 6.0)], 1.0),
|
||||
(
|
||||
vec![g(6.0, 2.0), g(2.0, 6.0), g(-2.0, 1.0), g(-6.0, 4.0)],
|
||||
1.0,
|
||||
),
|
||||
] {
|
||||
let n = perf.len();
|
||||
let dist = outcome_distribution(&perf, &flat(n, eps)).unwrap();
|
||||
let sum: f64 = dist.iter().map(|(_, p)| p).sum();
|
||||
assert!(
|
||||
(sum - 1.0).abs() < 1e-6,
|
||||
"{n} teams, eps={eps}: sum = {sum} over {} outcomes",
|
||||
dist.len()
|
||||
);
|
||||
assert!(dist.iter().all(|(_, p)| *p >= 0.0), "negative probability");
|
||||
}
|
||||
}
|
||||
|
||||
/// With two teams the distribution is the exact win/draw/loss triple.
|
||||
#[test]
|
||||
fn two_team_distribution_matches_the_closed_form() {
|
||||
let perf = [g(3.0, 6.0), g(-2.0, 1.0)];
|
||||
let eps = 1.5;
|
||||
let dist = outcome_distribution(&perf, &flat(2, eps)).unwrap();
|
||||
let (wa, wb) = closed_form_two(perf[0], perf[1], eps);
|
||||
|
||||
let find = |ranks: &[u32]| {
|
||||
dist.iter()
|
||||
.find(|(r, _)| r == ranks)
|
||||
.map_or(0.0, |(_, p)| *p)
|
||||
};
|
||||
|
||||
assert!(
|
||||
(find(&[0, 1]) - wa).abs() < 1e-6,
|
||||
"a wins: {}",
|
||||
find(&[0, 1])
|
||||
);
|
||||
assert!(
|
||||
(find(&[1, 0]) - wb).abs() < 1e-6,
|
||||
"b wins: {}",
|
||||
find(&[1, 0])
|
||||
);
|
||||
assert!(
|
||||
(find(&[0, 0]) - (1.0 - wa - wb)).abs() < 1e-6,
|
||||
"draw: {}",
|
||||
find(&[0, 0])
|
||||
);
|
||||
}
|
||||
|
||||
/// Asking for one ranking must agree with that ranking's entry in the
|
||||
/// full distribution — the two use different code paths to the same value.
|
||||
#[test]
|
||||
fn ranking_probability_agrees_with_the_distribution() {
|
||||
let perf = [g(5.0, 6.0), g(0.0, 3.0), g(-5.0, 1.0)];
|
||||
let eps = 1.5;
|
||||
let margins = flat(3, eps);
|
||||
let dist = outcome_distribution(&perf, &margins).unwrap();
|
||||
|
||||
for (ranks, expected) in &dist {
|
||||
let direct = ranking_probability(&perf, &margins, ranks).unwrap();
|
||||
assert!(
|
||||
(direct - expected).abs() < 1e-9,
|
||||
"ranks {ranks:?}: direct {direct} vs distribution {expected}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Tie mass is controlled by the draw margin. Only the *all-tied* outcome
|
||||
/// is monotone in it: every one of its constraints is a window that widens
|
||||
/// with the margin. A partially-tied outcome like `[0, 0, 1]` is not, and
|
||||
/// must not be asserted to be — widening the margin makes its tie easier
|
||||
/// but its "and the last team is strictly behind by more than the margin"
|
||||
/// clause harder, so it peaks and then falls.
|
||||
#[test]
|
||||
fn all_tied_probability_grows_with_the_draw_margin() {
|
||||
let perf = [g(0.0, 4.0), g(0.0, 4.0), g(-8.0, 2.0)];
|
||||
let mut previous = 0.0;
|
||||
for eps in [0.0, 0.5, 1.0, 2.0, 4.0, 8.0, 24.0] {
|
||||
let p = ranking_probability(&perf, &flat(3, eps), &[0, 0, 0]).unwrap();
|
||||
assert!(p >= previous, "eps={eps}: {p} < {previous}");
|
||||
if eps == 0.0 {
|
||||
assert!(p < 1e-12, "a tie needs a margin, got {p}");
|
||||
}
|
||||
previous = p;
|
||||
}
|
||||
assert!(
|
||||
previous > 0.9,
|
||||
"a very wide margin ties everyone: {previous}"
|
||||
);
|
||||
}
|
||||
|
||||
/// The converse, stated as the non-property it is: a partially-tied
|
||||
/// outcome is non-monotone in the margin. Pinning this down stops a future
|
||||
/// change from "fixing" it into monotonicity and quietly breaking the model.
|
||||
#[test]
|
||||
fn a_partially_tied_outcome_peaks_in_the_middle() {
|
||||
let perf = [g(0.0, 4.0), g(0.0, 4.0), g(-8.0, 2.0)];
|
||||
let sweep: Vec<f64> = [0.5, 2.0, 4.0, 8.0, 16.0]
|
||||
.iter()
|
||||
.map(|&eps| ranking_probability(&perf, &flat(3, eps), &[0, 0, 1]).unwrap())
|
||||
.collect();
|
||||
let peak = sweep
|
||||
.iter()
|
||||
.enumerate()
|
||||
.fold(
|
||||
(0, 0.0),
|
||||
|(bi, bv), (i, &v)| if v > bv { (i, v) } else { (bi, bv) },
|
||||
)
|
||||
.0;
|
||||
assert!(
|
||||
peak > 0 && peak < sweep.len() - 1,
|
||||
"expected an interior peak: {sweep:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// With no draw margin a tie has probability exactly zero, and the
|
||||
/// enumeration must not waste work pretending otherwise.
|
||||
#[test]
|
||||
fn ties_are_impossible_without_a_draw_margin() {
|
||||
let perf = [g(0.0, 4.0), g(0.0, 4.0), g(0.0, 4.0)];
|
||||
let dist = outcome_distribution(&perf, &flat(3, 0.0)).unwrap();
|
||||
assert_eq!(dist.len(), 6, "expected only the 6 strict orders: {dist:?}");
|
||||
assert!(dist.iter().all(|(r, _)| {
|
||||
let mut seen = r.clone();
|
||||
seen.sort_unstable();
|
||||
seen.dedup();
|
||||
seen.len() == r.len()
|
||||
}));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,322 @@
|
||||
//! Deterministic numerical integration for the prediction paths.
|
||||
//!
|
||||
//! Prediction asks two questions that have no closed form beyond two teams:
|
||||
//! "who finishes first" and "how likely is this exact finishing order". Both
|
||||
//! reduce to integrals over a single performance variable, so neither needs a
|
||||
//! sampler — and that matters, because a Monte Carlo predictor would make
|
||||
//! `predict_*` non-reproducible and would answer a slightly different question
|
||||
//! on every call.
|
||||
//!
|
||||
//! Two routines live here:
|
||||
//!
|
||||
//! - [`integrate`], adaptive Gauss-Kronrod G7-K15, for the first-place
|
||||
//! marginals. It carries its own error estimate, so it can refine where the
|
||||
//! integrand actually bends instead of guessing a node count up front.
|
||||
//! - [`Grid`], a uniform grid with trapezoid prefix sums, for the ranking
|
||||
//! chain recursion, where each level needs the *running* integral of the
|
||||
//! level below at arbitrary points rather than one definite integral.
|
||||
//!
|
||||
//! Fixed-node Gauss-Hermite is the obvious tool for the first of these and is
|
||||
//! a trap: the integrand is a product of normal CDFs, and when one team's
|
||||
//! sigma is much smaller than the integrating team's, that product turns into
|
||||
//! a near-step function narrower than the node spacing. The nodes step over
|
||||
//! it and the result is wrong by ~1e-2 while still looking like a probability.
|
||||
//! Adaptive refinement is what makes the small-sigma case safe.
|
||||
|
||||
/// Kronrod 15-point abscissae, non-negative half, descending.
|
||||
const XGK: [f64; 8] = [
|
||||
0.991_455_371_120_813,
|
||||
0.949_107_912_342_759,
|
||||
0.864_864_423_359_769,
|
||||
0.741_531_185_599_394,
|
||||
0.586_087_235_467_691,
|
||||
0.405_845_151_377_397,
|
||||
0.207_784_955_007_898,
|
||||
0.0,
|
||||
];
|
||||
|
||||
/// Kronrod 15-point weights, matching [`XGK`].
|
||||
const WGK: [f64; 8] = [
|
||||
0.022_935_322_010_529,
|
||||
0.063_092_092_629_979,
|
||||
0.104_790_010_322_250,
|
||||
0.140_653_259_715_525,
|
||||
0.169_004_726_639_267,
|
||||
0.190_350_578_064_785,
|
||||
0.204_432_940_075_298,
|
||||
0.209_482_141_084_728,
|
||||
];
|
||||
|
||||
/// Gauss 7-point weights, applying to the odd-indexed [`XGK`] entries.
|
||||
const WG: [f64; 4] = [
|
||||
0.129_484_966_168_870,
|
||||
0.279_705_391_489_277,
|
||||
0.381_830_050_505_119,
|
||||
0.417_959_183_673_469,
|
||||
];
|
||||
|
||||
/// Panels are bisected worst-first; this bounds the work on a pathological
|
||||
/// integrand rather than letting it spin.
|
||||
const MAX_SUBDIVISIONS: usize = 200;
|
||||
|
||||
/// One G7-K15 panel over `[a, b]`: `(integral, absolute error estimate)`.
|
||||
///
|
||||
/// The error estimate is the gap between the embedded 7-point Gauss rule and
|
||||
/// the 15-point Kronrod extension. It is the only reason this is preferable
|
||||
/// to a fixed rule: it tells the caller *where* the integrand is hard.
|
||||
fn gk15<F: Fn(f64) -> f64>(f: &F, a: f64, b: f64) -> (f64, f64) {
|
||||
let centre = 0.5 * (a + b);
|
||||
let half = 0.5 * (b - a);
|
||||
|
||||
let mut kronrod = 0.0;
|
||||
let mut gauss = 0.0;
|
||||
|
||||
for i in 0..8 {
|
||||
let offset = XGK[i] * half;
|
||||
// XGK[7] is the centre node and must not be counted twice.
|
||||
let sum = if i == 7 {
|
||||
f(centre)
|
||||
} else {
|
||||
f(centre - offset) + f(centre + offset)
|
||||
};
|
||||
kronrod += WGK[i] * sum;
|
||||
if i % 2 == 1 {
|
||||
gauss += WG[i / 2] * sum;
|
||||
}
|
||||
}
|
||||
|
||||
(kronrod * half, ((kronrod - gauss) * half).abs())
|
||||
}
|
||||
|
||||
/// Adaptively integrate `f` over `[a, b]` to relative tolerance `tol`.
|
||||
///
|
||||
/// `seeds` are interior points where the integrand is known to bend sharply —
|
||||
/// for a product of normal CDFs, each rival's transition centre. Splitting
|
||||
/// there up front costs nothing and saves the adaptive loop from having to
|
||||
/// discover a step by bisection.
|
||||
///
|
||||
/// Returns the integral. The error estimate is consumed internally rather
|
||||
/// than returned: callers here integrate probability densities, where the
|
||||
/// meaningful check is the sum-to-one identity over a whole outcome space,
|
||||
/// not a per-integral residual.
|
||||
pub(crate) fn integrate<F: Fn(f64) -> f64>(f: F, a: f64, b: f64, seeds: &[f64], tol: f64) -> f64 {
|
||||
// Explicit rather than `!(b > a)`: a NaN bound must fall through to zero
|
||||
// rather than being read as a valid ordering.
|
||||
if a.partial_cmp(&b) != Some(std::cmp::Ordering::Less) {
|
||||
return 0.0;
|
||||
}
|
||||
|
||||
let mut edges: Vec<f64> = Vec::with_capacity(seeds.len() + 2);
|
||||
edges.push(a);
|
||||
edges.push(b);
|
||||
for &s in seeds {
|
||||
if s > a && s < b {
|
||||
edges.push(s);
|
||||
}
|
||||
}
|
||||
edges.sort_by(|p, q| p.partial_cmp(q).expect("integration bounds are finite"));
|
||||
edges.dedup();
|
||||
|
||||
// (lo, hi, integral, error)
|
||||
let mut panels: Vec<(f64, f64, f64, f64)> = edges
|
||||
.windows(2)
|
||||
.map(|w| {
|
||||
let (v, e) = gk15(&f, w[0], w[1]);
|
||||
(w[0], w[1], v, e)
|
||||
})
|
||||
.collect();
|
||||
|
||||
for _ in 0..MAX_SUBDIVISIONS {
|
||||
let total: f64 = panels.iter().map(|p| p.2).sum();
|
||||
let error: f64 = panels.iter().map(|p| p.3).sum();
|
||||
|
||||
// Absolute floor as well as relative: these integrands are
|
||||
// probabilities, so an absolute 1e-15 is already past the useful
|
||||
// precision of the underlying `cdf`.
|
||||
if error <= tol * total.abs().max(1e-12) || error < 1e-15 {
|
||||
break;
|
||||
}
|
||||
|
||||
let worst = panels
|
||||
.iter()
|
||||
.enumerate()
|
||||
.fold((0usize, f64::NEG_INFINITY), |(bi, be), (i, p)| {
|
||||
if p.3 > be { (i, p.3) } else { (bi, be) }
|
||||
})
|
||||
.0;
|
||||
|
||||
let (lo, hi, _, _) = panels[worst];
|
||||
let mid = 0.5 * (lo + hi);
|
||||
// Bisection has hit the floating-point floor; refining further would
|
||||
// loop without reducing the error.
|
||||
if !(mid > lo && mid < hi) {
|
||||
break;
|
||||
}
|
||||
|
||||
let (v1, e1) = gk15(&f, lo, mid);
|
||||
let (v2, e2) = gk15(&f, mid, hi);
|
||||
panels[worst] = (lo, mid, v1, e1);
|
||||
panels.push((mid, hi, v2, e2));
|
||||
}
|
||||
|
||||
panels.iter().map(|p| p.2).sum()
|
||||
}
|
||||
|
||||
/// A uniform grid carrying trapezoid prefix sums of one integrand.
|
||||
///
|
||||
/// The ranking recursion needs, at every level, the running integral of the
|
||||
/// level below evaluated at arbitrary points — a cumulative integral, not a
|
||||
/// definite one. Prefix sums give that in O(1) per query after an O(G) build,
|
||||
/// which is what keeps a full ranking probability linear in the team count.
|
||||
pub(crate) struct Grid {
|
||||
lo: f64,
|
||||
step: f64,
|
||||
/// Integrand sampled at each node.
|
||||
values: Vec<f64>,
|
||||
/// `prefix[i]` is the integral from `lo` to node `i`.
|
||||
prefix: Vec<f64>,
|
||||
}
|
||||
|
||||
impl Grid {
|
||||
/// Build directly from already-sampled values.
|
||||
///
|
||||
/// The ranking recursion evaluates every level on the same nodes, so the
|
||||
/// per-team densities are sampled once and reused; re-evaluating `exp`
|
||||
/// per level would dominate the cost.
|
||||
pub(crate) fn from_values(lo: f64, step: f64, values: Vec<f64>) -> Self {
|
||||
let mut prefix = vec![0.0; values.len()];
|
||||
for i in 1..values.len() {
|
||||
prefix[i] = prefix[i - 1] + 0.5 * step * (values[i - 1] + values[i]);
|
||||
}
|
||||
Self {
|
||||
lo,
|
||||
step,
|
||||
values,
|
||||
prefix,
|
||||
}
|
||||
}
|
||||
|
||||
/// Integral from the grid's lower bound up to `x`.
|
||||
///
|
||||
/// Clamped at both ends: the caller sizes the grid to cover the whole
|
||||
/// support, so a query outside it is asking for a tail that is zero (below)
|
||||
/// or the whole mass (above).
|
||||
pub(crate) fn integral_to(&self, x: f64) -> f64 {
|
||||
let last = self.values.len() - 1;
|
||||
if x <= self.lo {
|
||||
return 0.0;
|
||||
}
|
||||
if x >= self.lo + last as f64 * self.step {
|
||||
return self.prefix[last];
|
||||
}
|
||||
|
||||
let scaled = (x - self.lo) / self.step;
|
||||
let i = scaled.floor() as usize;
|
||||
let frac = scaled - i as f64;
|
||||
|
||||
// Whole cells, plus the trapezoid over the partial cell. The integrand
|
||||
// is linear within a cell under the trapezoid rule, so the partial
|
||||
// piece is exact with respect to that same approximation.
|
||||
self.prefix[i]
|
||||
+ frac
|
||||
* self.step
|
||||
* (self.values[i] + 0.5 * frac * (self.values[i + 1] - self.values[i]))
|
||||
}
|
||||
|
||||
/// Integral over `[from, to]`.
|
||||
pub(crate) fn integral_between(&self, from: f64, to: f64) -> f64 {
|
||||
(self.integral_to(to) - self.integral_to(from)).max(0.0)
|
||||
}
|
||||
|
||||
/// Total integral over the whole grid.
|
||||
pub(crate) fn total(&self) -> f64 {
|
||||
self.prefix[self.values.len() - 1]
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
const TOL: f64 = 1e-10;
|
||||
|
||||
/// Sample `f` over `[lo, hi]` at `points` nodes.
|
||||
fn sample<F: FnMut(f64) -> f64>(lo: f64, hi: f64, points: usize, mut f: F) -> Grid {
|
||||
let step = (hi - lo) / (points - 1) as f64;
|
||||
Grid::from_values(
|
||||
lo,
|
||||
step,
|
||||
(0..points).map(|i| f(lo + i as f64 * step)).collect(),
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn integrates_a_polynomial_exactly() {
|
||||
// G7-K15 is exact for polynomials well past cubic, so a single panel
|
||||
// should already be at round-off.
|
||||
let v = integrate(|x| 3.0 * x * x + 2.0 * x + 1.0, 0.0, 2.0, &[], TOL);
|
||||
assert!((v - 14.0).abs() < 1e-12, "got {v}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn integrates_a_gaussian_density_to_one() {
|
||||
let f = |x: f64| (-0.5 * x * x).exp() / (2.0 * std::f64::consts::PI).sqrt();
|
||||
let v = integrate(f, -10.0, 10.0, &[], TOL);
|
||||
assert!((v - 1.0).abs() < 1e-12, "got {v}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resolves_a_step_far_narrower_than_the_initial_panel() {
|
||||
// The failure mode that rules out fixed-node quadrature: a transition
|
||||
// 1e-4 wide inside a range of 20. A fixed rule steps over it.
|
||||
let f = |x: f64| if x < 0.5 { 0.0 } else { 1.0 };
|
||||
let v = integrate(f, -10.0, 10.0, &[0.5], TOL);
|
||||
assert!((v - 9.5).abs() < 1e-6, "got {v}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn seeds_do_not_change_the_value_of_a_smooth_integrand() {
|
||||
let f = |x: f64| (-0.5 * x * x).exp();
|
||||
let plain = integrate(f, -8.0, 8.0, &[], TOL);
|
||||
let seeded = integrate(f, -8.0, 8.0, &[-3.0, 0.25, 5.5], TOL);
|
||||
assert!((plain - seeded).abs() < 1e-12, "{plain} vs {seeded}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_or_inverted_range_integrates_to_zero() {
|
||||
assert_eq!(integrate(|_| 1.0, 1.0, 1.0, &[], TOL), 0.0);
|
||||
assert_eq!(integrate(|_| 1.0, 2.0, 1.0, &[], TOL), 0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn grid_prefix_matches_a_known_cumulative_integral() {
|
||||
// f(x) = x over [0, 4]; integral to x is x^2/2.
|
||||
let g = sample(0.0, 4.0, 4001, |x| x);
|
||||
for probe in [0.0, 0.5, 1.0, 2.5, 3.75, 4.0] {
|
||||
let want = probe * probe / 2.0;
|
||||
let got = g.integral_to(probe);
|
||||
assert!(
|
||||
(got - want).abs() < 1e-9,
|
||||
"at {probe}: got {got}, want {want}"
|
||||
);
|
||||
}
|
||||
assert!((g.total() - 8.0).abs() < 1e-9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn grid_between_is_the_difference_of_two_prefixes() {
|
||||
let g = sample(-5.0, 5.0, 8001, |x| (-0.5 * x * x).exp());
|
||||
let whole = g.integral_between(-5.0, 5.0);
|
||||
let split = g.integral_between(-5.0, 0.3) + g.integral_between(0.3, 5.0);
|
||||
assert!((whole - split).abs() < 1e-12, "{whole} vs {split}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn grid_clamps_queries_outside_its_support() {
|
||||
let g = sample(0.0, 1.0, 101, |_| 1.0);
|
||||
assert_eq!(g.integral_to(-3.0), 0.0);
|
||||
assert!((g.integral_to(9.0) - 1.0).abs() < 1e-12);
|
||||
// Reversed bounds must not produce negative probability mass.
|
||||
assert_eq!(g.integral_between(0.8, 0.2), 0.0);
|
||||
}
|
||||
}
|
||||
+21
-5
@@ -9,9 +9,9 @@ use crate::{
|
||||
|
||||
/// Static rating configuration: prior skill, performance noise `beta`, drift.
|
||||
///
|
||||
/// Renamed from `Player` in T2; `Rating` better describes the data
|
||||
/// (a configuration) vs. a person (who's a `Competitor` with state).
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
/// A configuration rather than a person: the per-history temporal state
|
||||
/// (messages, last appearance) lives on `Competitor`.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct Rating<T: Time = i64, D: Drift<T> = ConstantDrift> {
|
||||
pub(crate) prior: Gaussian,
|
||||
pub(crate) beta: f64,
|
||||
@@ -23,7 +23,24 @@ pub struct Rating<T: Time = i64, D: Drift<T> = ConstantDrift> {
|
||||
}
|
||||
|
||||
impl<T: Time, D: Drift<T>> Rating<T, D> {
|
||||
/// # Panics
|
||||
///
|
||||
/// Panics unless `beta` is finite and non-negative, matching
|
||||
/// `HistoryBuilder::beta`.
|
||||
///
|
||||
/// Zero is allowed and meaningful — performance is then exactly skill, and
|
||||
/// the fit differs measurably from a positive beta rather than degenerating.
|
||||
/// Negative is rejected because `beta` enters only as `beta^2`: measured, a
|
||||
/// negative beta returned results **bit identical** to its absolute value,
|
||||
/// and a NaN beta reached `Game::ranked`, which returned `Ok` carrying a
|
||||
/// `Gaussian { pi: NaN, tau: NaN }` — there is no `converge` on that path to
|
||||
/// catch it.
|
||||
pub fn new(prior: Gaussian, beta: f64, drift: D) -> Self {
|
||||
assert!(
|
||||
beta.is_finite() && beta >= 0.0,
|
||||
"beta must be finite and non-negative (got {beta}); it is only ever \
|
||||
squared, so a negative value would silently behave as its absolute value"
|
||||
);
|
||||
Self {
|
||||
prior,
|
||||
beta,
|
||||
@@ -44,7 +61,6 @@ impl<T: Time, D: Drift<T>> Rating<T, D> {
|
||||
}
|
||||
|
||||
/// The configured prior skill estimate.
|
||||
#[must_use]
|
||||
pub fn prior(&self) -> Gaussian {
|
||||
self.prior
|
||||
}
|
||||
@@ -93,7 +109,7 @@ impl Default for Rating<i64, ConstantDrift> {
|
||||
Self {
|
||||
prior: Gaussian::default(),
|
||||
beta: BETA,
|
||||
drift: ConstantDrift(GAMMA),
|
||||
drift: ConstantDrift::new(GAMMA),
|
||||
drift_scale: 1.0,
|
||||
_time: PhantomData,
|
||||
}
|
||||
|
||||
-152
@@ -1,152 +0,0 @@
|
||||
//! Schedule trait and built-in implementations.
|
||||
//!
|
||||
//! A schedule drives factor propagation to convergence. The default
|
||||
//! `EpsilonOrMax` performs one `TeamSum` sweep (setup) then alternating
|
||||
//! forward/backward sweeps over the iterating factors until the max
|
||||
//! delta drops below epsilon or `max` iterations is reached.
|
||||
|
||||
use crate::factor::{BuiltinFactor, Factor, VarStore};
|
||||
|
||||
/// Result returned by a `Schedule::run` call.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct ScheduleReport {
|
||||
pub iterations: usize,
|
||||
pub final_step: (f64, f64),
|
||||
pub converged: bool,
|
||||
}
|
||||
|
||||
/// Drives factor propagation to convergence.
|
||||
pub trait Schedule: Send + Sync {
|
||||
fn run(&self, factors: &mut [BuiltinFactor], vars: &mut VarStore) -> ScheduleReport;
|
||||
}
|
||||
|
||||
/// Default schedule: sweep forward then backward until step ≤ eps or iter == max.
|
||||
///
|
||||
/// Matches the existing `Game::likelihoods` loop bit-for-bit when given the
|
||||
/// same factor layout (`TeamSums` first, then alternating RankDiff/Trunc pairs).
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct EpsilonOrMax {
|
||||
pub eps: f64,
|
||||
pub max: usize,
|
||||
}
|
||||
|
||||
impl Default for EpsilonOrMax {
|
||||
fn default() -> Self {
|
||||
// Derived from `ConvergenceOptions` so there is one source of truth for
|
||||
// the tolerance and iteration cap. These previously disagreed: this
|
||||
// default capped at 10 iterations while `ConvergenceOptions` allowed 30,
|
||||
// and which applied depended on whether inference went through
|
||||
// `run_chain` or a `Schedule`.
|
||||
let defaults = crate::ConvergenceOptions::default();
|
||||
|
||||
Self {
|
||||
eps: defaults.epsilon,
|
||||
max: defaults.max_iter,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Schedule for EpsilonOrMax {
|
||||
fn run(&self, factors: &mut [BuiltinFactor], vars: &mut VarStore) -> ScheduleReport {
|
||||
// Partition: leading run of TeamSum factors run exactly once (setup).
|
||||
let n_setup = factors
|
||||
.iter()
|
||||
.position(|f| !matches!(f, BuiltinFactor::TeamSum(_)))
|
||||
.unwrap_or(factors.len());
|
||||
|
||||
for f in factors[..n_setup].iter_mut() {
|
||||
f.propagate(vars);
|
||||
}
|
||||
|
||||
let mut iterations = 0;
|
||||
// With no iterating factors the graph is already at its fixed point:
|
||||
// the setup pass above is all there is to do. Reporting `converged:
|
||||
// false` with an infinite step for that case gave callers a false
|
||||
// negative.
|
||||
let mut final_step = (0.0, 0.0);
|
||||
let mut converged = true;
|
||||
|
||||
if n_setup < factors.len() {
|
||||
final_step = (f64::INFINITY, f64::INFINITY);
|
||||
converged = false;
|
||||
for _ in 0..self.max {
|
||||
let mut step = (0.0_f64, 0.0_f64);
|
||||
|
||||
// Forward sweep over iterating factors.
|
||||
for f in factors[n_setup..].iter_mut() {
|
||||
let d = f.propagate(vars);
|
||||
step.0 = step.0.max(d.0);
|
||||
step.1 = step.1.max(d.1);
|
||||
}
|
||||
|
||||
// Backward sweep.
|
||||
for f in factors[n_setup..].iter_mut().rev() {
|
||||
let d = f.propagate(vars);
|
||||
step.0 = step.0.max(d.0);
|
||||
step.1 = step.1.max(d.1);
|
||||
}
|
||||
|
||||
iterations += 1;
|
||||
final_step = step;
|
||||
|
||||
if step.0 <= self.eps && step.1 <= self.eps {
|
||||
converged = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
ScheduleReport {
|
||||
iterations,
|
||||
final_step,
|
||||
converged,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::{N_INF, factor::team_sum::TeamSumFactor, gaussian::Gaussian};
|
||||
|
||||
#[test]
|
||||
fn schedule_runs_setup_factors_once() {
|
||||
// Single TeamSum factor; schedule should propagate it exactly once and report 0 iterations.
|
||||
let mut vars = VarStore::new();
|
||||
let out = vars.alloc(N_INF);
|
||||
let mut factors = vec![BuiltinFactor::TeamSum(TeamSumFactor {
|
||||
inputs: vec![(Gaussian::from_ms(5.0, 1.0), 1.0)],
|
||||
out,
|
||||
})];
|
||||
let schedule = EpsilonOrMax::default();
|
||||
let report = schedule.run(&mut factors, &mut vars);
|
||||
assert_eq!(report.iterations, 0);
|
||||
// The team-perf var should hold the sum.
|
||||
let result = vars.get(out);
|
||||
assert!((result.mu() - 5.0).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn report_marks_converged_when_no_iterating_factors() {
|
||||
// A graph of only setup factors has nothing to iterate, so it is at its
|
||||
// fixed point after the setup pass: 0 iterations, and converged.
|
||||
let mut vars = VarStore::new();
|
||||
let out = vars.alloc(N_INF);
|
||||
let mut factors = vec![BuiltinFactor::TeamSum(TeamSumFactor {
|
||||
inputs: vec![(Gaussian::from_ms(0.0, 1.0), 1.0)],
|
||||
out,
|
||||
})];
|
||||
let report = EpsilonOrMax::default().run(&mut factors, &mut vars);
|
||||
assert_eq!(report.iterations, 0);
|
||||
assert!(report.converged);
|
||||
assert_eq!(report.final_step, (0.0, 0.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn default_matches_convergence_options() {
|
||||
let schedule = EpsilonOrMax::default();
|
||||
let options = crate::ConvergenceOptions::default();
|
||||
assert_eq!(schedule.max, options.max_iter);
|
||||
assert_eq!(schedule.eps, options.epsilon);
|
||||
}
|
||||
}
|
||||
@@ -56,16 +56,16 @@ impl<T: Time, D: Drift<T>> CompetitorStore<T, D> {
|
||||
self.get(idx).is_some()
|
||||
}
|
||||
|
||||
/// Test-only: no code path in the crate needs a count.
|
||||
#[cfg(test)]
|
||||
#[must_use]
|
||||
pub fn len(&self) -> usize {
|
||||
self.n_present
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.n_present == 0
|
||||
}
|
||||
|
||||
/// Test-only: iterating every competitor is an assertion helper, not part
|
||||
/// of inference, which walks slices rather than the store.
|
||||
#[cfg(test)]
|
||||
pub fn iter(&self) -> impl Iterator<Item = (Index, &Competitor<T, D>)> {
|
||||
self.competitors
|
||||
.iter()
|
||||
@@ -73,13 +73,6 @@ impl<T: Time, D: Drift<T>> CompetitorStore<T, D> {
|
||||
.filter_map(|(i, slot)| slot.as_ref().map(|a| (Index(i), a)))
|
||||
}
|
||||
|
||||
pub fn iter_mut(&mut self) -> impl Iterator<Item = (Index, &mut Competitor<T, D>)> {
|
||||
self.competitors
|
||||
.iter_mut()
|
||||
.enumerate()
|
||||
.filter_map(|(i, slot)| slot.as_mut().map(|a| (Index(i), a)))
|
||||
}
|
||||
|
||||
pub fn values_mut(&mut self) -> impl Iterator<Item = &mut Competitor<T, D>> {
|
||||
self.competitors.iter_mut().filter_map(|s| s.as_mut())
|
||||
}
|
||||
|
||||
+242
-121
@@ -9,7 +9,7 @@ use crate::{
|
||||
arena::ScratchArena,
|
||||
color_group::ColorGroups,
|
||||
drift::Drift,
|
||||
game::Game,
|
||||
game::GameRef,
|
||||
gaussian::Gaussian,
|
||||
rating::Rating,
|
||||
storage::{CompetitorStore, SkillStore},
|
||||
@@ -26,7 +26,9 @@ pub(crate) struct Skill {
|
||||
|
||||
impl Skill {
|
||||
pub(crate) fn posterior(&self) -> Gaussian {
|
||||
self.likelihood * self.backward * self.forward
|
||||
self.likelihood
|
||||
.ep_product(self.backward)
|
||||
.ep_product(self.forward)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -50,12 +52,12 @@ pub enum EventKind {
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
struct Item {
|
||||
agent: Index,
|
||||
competitor: Index,
|
||||
/// This competitor's slot in the owning slice's `SkillStore`, resolved
|
||||
/// once at ingestion.
|
||||
///
|
||||
/// The convergence loop reaches skills through this rather than through
|
||||
/// `agent`, which is what keeps `HashMap` hashing out of the hot path now
|
||||
/// `competitor`, which is what keeps `HashMap` hashing out of the hot path now
|
||||
/// that the store is compact rather than indexed by the global `Index`.
|
||||
slot: u32,
|
||||
likelihood: Gaussian,
|
||||
@@ -66,15 +68,15 @@ impl Item {
|
||||
&self,
|
||||
forward: bool,
|
||||
skills: &SkillStore,
|
||||
agents: &CompetitorStore<T, D>,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
) -> Rating<T, D> {
|
||||
let r = &agents[self.agent].rating;
|
||||
let r = &competitors[self.competitor].rating;
|
||||
let skill = skills.at(self.slot);
|
||||
|
||||
if forward {
|
||||
Rating::new(skill.forward, r.beta, r.drift).with_drift_scale(r.drift_scale)
|
||||
} else {
|
||||
Rating::new(skill.posterior() / self.likelihood, r.beta, r.drift)
|
||||
Rating::new(skill.posterior().cavity(self.likelihood), r.beta, r.drift)
|
||||
.with_drift_scale(r.drift_scale)
|
||||
}
|
||||
}
|
||||
@@ -95,10 +97,10 @@ pub(crate) struct Event {
|
||||
}
|
||||
|
||||
impl Event {
|
||||
pub(crate) fn iter_agents(&self) -> impl Iterator<Item = Index> + '_ {
|
||||
pub(crate) fn iter_competitors(&self) -> impl Iterator<Item = Index> + '_ {
|
||||
self.teams
|
||||
.iter()
|
||||
.flat_map(|t| t.items.iter().map(|it| it.agent))
|
||||
.flat_map(|t| t.items.iter().map(|it| it.competitor))
|
||||
}
|
||||
|
||||
fn outputs(&self) -> Vec<f64> {
|
||||
@@ -112,14 +114,14 @@ impl Event {
|
||||
&self,
|
||||
forward: bool,
|
||||
skills: &SkillStore,
|
||||
agents: &CompetitorStore<T, D>,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
) -> Vec<Vec<Rating<T, D>>> {
|
||||
self.teams
|
||||
.iter()
|
||||
.map(|team| {
|
||||
team.items
|
||||
.iter()
|
||||
.map(|item| item.within_prior(forward, skills, agents))
|
||||
.map(|item| item.within_prior(forward, skills, competitors))
|
||||
.collect::<Vec<_>>()
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
@@ -133,18 +135,23 @@ impl Event {
|
||||
fn compute<T: Time, D: Drift<T>>(
|
||||
&self,
|
||||
skills: &SkillStore,
|
||||
agents: &CompetitorStore<T, D>,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
p_draw: f64,
|
||||
convergence: crate::ConvergenceOptions,
|
||||
arena: &mut ScratchArena,
|
||||
) -> EventUpdate {
|
||||
let teams = self.within_priors(false, skills, agents);
|
||||
let teams = self.within_priors(false, skills, competitors);
|
||||
let result = self.outputs();
|
||||
let g = match self.kind {
|
||||
EventKind::Ranked => {
|
||||
Game::ranked_with_arena(teams, &result, &self.weights, p_draw, convergence, arena)
|
||||
}
|
||||
EventKind::Scored { score_sigma } => Game::scored_with_arena(
|
||||
EventKind::Ranked => GameRef::ranked_with_arena(
|
||||
teams,
|
||||
&result,
|
||||
&self.weights,
|
||||
p_draw,
|
||||
convergence,
|
||||
arena,
|
||||
),
|
||||
EventKind::Scored { score_sigma } => GameRef::scored_with_arena(
|
||||
teams,
|
||||
&result,
|
||||
&self.weights,
|
||||
@@ -166,7 +173,7 @@ impl Event {
|
||||
for (i, item) in team.items.iter_mut().enumerate() {
|
||||
let fresh = update.likelihoods[t][i];
|
||||
let old_likelihood = skills.at(item.slot).likelihood;
|
||||
let new_likelihood = (old_likelihood / item.likelihood) * fresh;
|
||||
let new_likelihood = old_likelihood.cavity(item.likelihood).ep_product(fresh);
|
||||
skills.at_mut(item.slot).likelihood = new_likelihood;
|
||||
item.likelihood = fresh;
|
||||
}
|
||||
@@ -179,12 +186,12 @@ impl Event {
|
||||
fn iteration_direct<T: Time, D: Drift<T>>(
|
||||
&mut self,
|
||||
skills: &mut SkillStore,
|
||||
agents: &CompetitorStore<T, D>,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
p_draw: f64,
|
||||
convergence: crate::ConvergenceOptions,
|
||||
arena: &mut ScratchArena,
|
||||
) {
|
||||
let update = self.compute(skills, agents, p_draw, convergence, arena);
|
||||
let update = self.compute(skills, competitors, p_draw, convergence, arena);
|
||||
self.apply(skills, update);
|
||||
}
|
||||
}
|
||||
@@ -228,7 +235,7 @@ pub struct TimeSlice<T: Time = i64> {
|
||||
}
|
||||
|
||||
impl<T: Time> TimeSlice<T> {
|
||||
pub fn new(time: T, p_draw: f64, convergence: crate::ConvergenceOptions) -> Self {
|
||||
pub(crate) fn new(time: T, p_draw: f64, convergence: crate::ConvergenceOptions) -> Self {
|
||||
Self {
|
||||
events: Vec::new(),
|
||||
skills: SkillStore::new(),
|
||||
@@ -255,7 +262,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
}
|
||||
|
||||
let cg = color_greedy(n, |ev_idx| {
|
||||
self.events[ev_idx].iter_agents().collect::<Vec<_>>()
|
||||
self.events[ev_idx].iter_competitors().collect::<Vec<_>>()
|
||||
});
|
||||
|
||||
let mut reordered: Vec<Event> = Vec::with_capacity(n);
|
||||
@@ -282,17 +289,17 @@ impl<T: Time> TimeSlice<T> {
|
||||
);
|
||||
}
|
||||
|
||||
pub fn add_events<D: Drift<T>>(
|
||||
pub(crate) fn add_events<D: Drift<T>>(
|
||||
&mut self,
|
||||
composition: Vec<Vec<Vec<Index>>>,
|
||||
results: Option<Vec<Vec<f64>>>,
|
||||
weights: Option<Vec<Vec<Vec<f64>>>>,
|
||||
kinds: Vec<EventKind>,
|
||||
agents: &CompetitorStore<T, D>,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
) {
|
||||
let mut unique = Vec::with_capacity(10);
|
||||
|
||||
let this_agent = composition.iter().flatten().flatten().filter(|idx| {
|
||||
let these_competitors = composition.iter().flatten().flatten().filter(|idx| {
|
||||
if !unique.contains(idx) {
|
||||
unique.push(*idx);
|
||||
|
||||
@@ -302,10 +309,10 @@ impl<T: Time> TimeSlice<T> {
|
||||
false
|
||||
});
|
||||
|
||||
for idx in this_agent {
|
||||
let elapsed = compute_elapsed(agents[*idx].last_time.as_ref(), &self.time);
|
||||
for idx in these_competitors {
|
||||
let elapsed = compute_elapsed(competitors[*idx].last_time.as_ref(), &self.time);
|
||||
|
||||
let forward = agents[*idx].receive(&self.time);
|
||||
let forward = competitors[*idx].receive(&self.time);
|
||||
|
||||
if let Some(skill) = self.skills.get_mut(*idx) {
|
||||
skill.elapsed = elapsed;
|
||||
@@ -332,12 +339,12 @@ impl<T: Time> TimeSlice<T> {
|
||||
.map(|(t, team)| {
|
||||
let items = team
|
||||
.iter()
|
||||
.map(|&agent| Item {
|
||||
agent,
|
||||
.map(|&competitor| Item {
|
||||
competitor,
|
||||
// Every participant was inserted into `skills`
|
||||
// just above, so the slot always resolves.
|
||||
slot: skills
|
||||
.slot_of(agent)
|
||||
.slot_of(competitor)
|
||||
.expect("participant must be present in the slice store"),
|
||||
likelihood: N_INF,
|
||||
})
|
||||
@@ -376,7 +383,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
|
||||
self.color_groups_dirty = true;
|
||||
|
||||
self.iteration(from, agents);
|
||||
self.iteration(from, competitors);
|
||||
}
|
||||
|
||||
pub(crate) fn posteriors(&self) -> HashMap<Index, Gaussian> {
|
||||
@@ -393,7 +400,11 @@ impl<T: Time> TimeSlice<T> {
|
||||
/// Panics if an event references a competitor with no entry in this
|
||||
/// slice's skill store. `add_events` inserts one for every participant, so
|
||||
/// this cannot happen for slices built through the public API.
|
||||
pub fn iteration<D: Drift<T>>(&mut self, from: usize, agents: &CompetitorStore<T, D>) {
|
||||
pub(crate) fn iteration<D: Drift<T>>(
|
||||
&mut self,
|
||||
from: usize,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
) {
|
||||
if from == 0 && self.color_groups_dirty {
|
||||
self.recompute_color_groups();
|
||||
}
|
||||
@@ -401,11 +412,11 @@ impl<T: Time> TimeSlice<T> {
|
||||
if from > 0 || self.color_groups.is_empty() {
|
||||
// Initial pass (add_events) or no color groups yet: simple sequential sweep.
|
||||
for event in self.events.iter_mut().skip(from) {
|
||||
let teams = event.within_priors(false, &self.skills, agents);
|
||||
let teams = event.within_priors(false, &self.skills, competitors);
|
||||
let result = event.outputs();
|
||||
|
||||
let g = match event.kind {
|
||||
EventKind::Ranked => Game::ranked_with_arena(
|
||||
EventKind::Ranked => GameRef::ranked_with_arena(
|
||||
teams,
|
||||
&result,
|
||||
&event.weights,
|
||||
@@ -413,7 +424,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
self.convergence,
|
||||
&mut self.arena,
|
||||
),
|
||||
EventKind::Scored { score_sigma } => Game::scored_with_arena(
|
||||
EventKind::Scored { score_sigma } => GameRef::scored_with_arena(
|
||||
teams,
|
||||
&result,
|
||||
&event.weights,
|
||||
@@ -426,8 +437,9 @@ impl<T: Time> TimeSlice<T> {
|
||||
for (t, team) in event.teams.iter_mut().enumerate() {
|
||||
for (i, item) in team.items.iter_mut().enumerate() {
|
||||
let old_likelihood = self.skills.at(item.slot).likelihood;
|
||||
let new_likelihood =
|
||||
(old_likelihood / item.likelihood) * g.likelihoods[t][i];
|
||||
let new_likelihood = old_likelihood
|
||||
.cavity(item.likelihood)
|
||||
.ep_product(g.likelihoods[t][i]);
|
||||
self.skills.at_mut(item.slot).likelihood = new_likelihood;
|
||||
item.likelihood = g.likelihoods[t][i];
|
||||
}
|
||||
@@ -436,14 +448,14 @@ impl<T: Time> TimeSlice<T> {
|
||||
event.log_evidence = g.log_evidence;
|
||||
}
|
||||
} else {
|
||||
self.sweep_color_groups(agents);
|
||||
self.sweep_color_groups(competitors);
|
||||
}
|
||||
}
|
||||
|
||||
/// Full event sweep using the color-group partition. Colors are processed
|
||||
/// sequentially; within each color the inner loop is parallel under rayon.
|
||||
///
|
||||
/// Events in one color group touch disjoint agent sets, so none of them
|
||||
/// Events in one color group touch disjoint competitor sets, so none of them
|
||||
/// can observe another's writes. That makes the sweep separable: inference
|
||||
/// runs concurrently over shared `&self.skills`, and the resulting updates
|
||||
/// are folded in afterwards in index order. Splitting it this way needs no
|
||||
@@ -451,7 +463,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
/// across thread counts because the apply order does not depend on which
|
||||
/// worker finished first.
|
||||
#[cfg(feature = "rayon")]
|
||||
fn sweep_color_groups<D: Drift<T>>(&mut self, agents: &CompetitorStore<T, D>) {
|
||||
fn sweep_color_groups<D: Drift<T>>(&mut self, competitors: &CompetitorStore<T, D>) {
|
||||
use rayon::prelude::*;
|
||||
|
||||
thread_local! {
|
||||
@@ -483,7 +495,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
let mut arena = cell.borrow_mut();
|
||||
arena.reset();
|
||||
|
||||
ev.compute(skills, agents, p_draw, convergence, &mut arena)
|
||||
ev.compute(skills, competitors, p_draw, convergence, &mut arena)
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
@@ -495,7 +507,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
for ev in &mut self.events[range] {
|
||||
ev.iteration_direct(
|
||||
&mut self.skills,
|
||||
agents,
|
||||
competitors,
|
||||
p_draw,
|
||||
self.convergence,
|
||||
&mut self.arena,
|
||||
@@ -509,7 +521,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
/// Events within each color group are updated inline — no EventOutput allocation —
|
||||
/// matching the T2 performance profile.
|
||||
#[cfg(not(feature = "rayon"))]
|
||||
fn sweep_color_groups<D: Drift<T>>(&mut self, agents: &CompetitorStore<T, D>) {
|
||||
fn sweep_color_groups<D: Drift<T>>(&mut self, competitors: &CompetitorStore<T, D>) {
|
||||
for color_idx in 0..self.color_groups.groups.len() {
|
||||
if self.color_groups.groups[color_idx].is_empty() {
|
||||
continue;
|
||||
@@ -523,7 +535,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
for ev in &mut self.events[range] {
|
||||
ev.iteration_direct(
|
||||
&mut self.skills,
|
||||
agents,
|
||||
competitors,
|
||||
p_draw,
|
||||
self.convergence,
|
||||
&mut self.arena,
|
||||
@@ -544,7 +556,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
/// schedule default.
|
||||
pub(crate) fn iterate_to_convergence<D: Drift<T>>(
|
||||
&mut self,
|
||||
agents: &CompetitorStore<T, D>,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
) -> usize {
|
||||
use crate::{tuple_gt, tuple_max};
|
||||
|
||||
@@ -557,7 +569,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
while tuple_gt(step, epsilon) && i < max_iter {
|
||||
let old = self.posteriors();
|
||||
|
||||
self.iteration(0, agents);
|
||||
self.iteration(0, competitors);
|
||||
|
||||
let new = self.posteriors();
|
||||
|
||||
@@ -575,37 +587,37 @@ impl<T: Time> TimeSlice<T> {
|
||||
i
|
||||
}
|
||||
|
||||
pub(crate) fn forward_prior_out(&self, agent: &Index) -> Gaussian {
|
||||
let skill = self.skills.get(*agent).unwrap();
|
||||
skill.forward * skill.likelihood
|
||||
pub(crate) fn forward_prior_out(&self, competitor: &Index) -> Gaussian {
|
||||
let skill = self.skills.get(*competitor).unwrap();
|
||||
skill.forward.ep_product(skill.likelihood)
|
||||
}
|
||||
|
||||
pub(crate) fn backward_prior_out<D: Drift<T>>(
|
||||
&self,
|
||||
agent: &Index,
|
||||
agents: &CompetitorStore<T, D>,
|
||||
competitor: &Index,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
) -> Gaussian {
|
||||
let skill = self.skills.get(*agent).unwrap();
|
||||
let n = skill.likelihood * skill.backward;
|
||||
let skill = self.skills.get(*competitor).unwrap();
|
||||
let n = skill.likelihood.ep_product(skill.backward);
|
||||
n.forget(
|
||||
agents[*agent]
|
||||
competitors[*competitor]
|
||||
.rating
|
||||
.drift_variance_for_elapsed(skill.elapsed),
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn new_backward_info<D: Drift<T>>(&mut self, agents: &CompetitorStore<T, D>) {
|
||||
for (agent, skill) in self.skills.iter_mut() {
|
||||
skill.backward = agents[agent].message.unwrap_or(N_INF);
|
||||
pub(crate) fn new_backward_info<D: Drift<T>>(&mut self, competitors: &CompetitorStore<T, D>) {
|
||||
for (competitor, skill) in self.skills.iter_mut() {
|
||||
skill.backward = competitors[competitor].message.unwrap_or(N_INF);
|
||||
}
|
||||
self.iteration(0, agents);
|
||||
self.iteration(0, competitors);
|
||||
}
|
||||
|
||||
pub(crate) fn new_forward_info<D: Drift<T>>(&mut self, agents: &CompetitorStore<T, D>) {
|
||||
for (agent, skill) in self.skills.iter_mut() {
|
||||
skill.forward = agents[agent].receive_for_elapsed(skill.elapsed);
|
||||
pub(crate) fn new_forward_info<D: Drift<T>>(&mut self, competitors: &CompetitorStore<T, D>) {
|
||||
for (competitor, skill) in self.skills.iter_mut() {
|
||||
skill.forward = competitors[competitor].receive_for_elapsed(skill.elapsed);
|
||||
}
|
||||
self.iteration(0, agents);
|
||||
self.iteration(0, competitors);
|
||||
}
|
||||
|
||||
/// Run this slice's events on forward (filtering) information alone.
|
||||
@@ -615,10 +627,18 @@ impl<T: Time> TimeSlice<T> {
|
||||
/// configured prior. The sweep runs on a scratch copy, so the real slice
|
||||
/// is untouched — which is what makes the filtered estimates independent
|
||||
/// of whether `History::converge` has run.
|
||||
/// One forward-only step for this slice.
|
||||
///
|
||||
/// `targets` restricts only the *evidence sum*, to events in which at
|
||||
/// least one target competitor appears; an empty set means no restriction.
|
||||
/// The forward messages are always built from every event in the slice —
|
||||
/// restricting those instead would answer a different question (a history
|
||||
/// in which the other events never happened), not a held-out one.
|
||||
pub(crate) fn filtered_step<D: Drift<T>>(
|
||||
&self,
|
||||
incoming: &HashMap<Index, Gaussian>,
|
||||
agents: &CompetitorStore<T, D>,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
targets: &std::collections::HashSet<Index>,
|
||||
) -> FilteredStep {
|
||||
let mut scratch = TimeSlice {
|
||||
events: self.events.clone(),
|
||||
@@ -641,16 +661,16 @@ impl<T: Time> TimeSlice<T> {
|
||||
event.log_evidence = 0.0;
|
||||
}
|
||||
|
||||
for (agent, skill) in self.skills.iter() {
|
||||
let rating = &agents[agent].rating;
|
||||
for (competitor, skill) in self.skills.iter() {
|
||||
let rating = &competitors[competitor].rating;
|
||||
|
||||
let forward = match incoming.get(&agent) {
|
||||
let forward = match incoming.get(&competitor) {
|
||||
Some(message) => message.forget(rating.drift_variance_for_elapsed(skill.elapsed)),
|
||||
None => rating.prior,
|
||||
};
|
||||
|
||||
let slot = scratch.skills.insert(
|
||||
agent,
|
||||
competitor,
|
||||
Skill {
|
||||
forward,
|
||||
backward: N_INF,
|
||||
@@ -666,19 +686,31 @@ impl<T: Time> TimeSlice<T> {
|
||||
// than leave it to be rediscovered after it breaks.
|
||||
debug_assert_eq!(
|
||||
Some(slot),
|
||||
self.skills.slot_of(agent),
|
||||
"scratch slot must match the real slice's slot for {agent:?}"
|
||||
self.skills.slot_of(competitor),
|
||||
"scratch slot must match the real slice's slot for {competitor:?}"
|
||||
);
|
||||
}
|
||||
|
||||
scratch.iterate_to_convergence(agents);
|
||||
scratch.iterate_to_convergence(competitors);
|
||||
|
||||
FilteredStep {
|
||||
log_evidence: scratch.events.iter().map(|event| event.log_evidence).sum(),
|
||||
log_evidence: scratch
|
||||
.events
|
||||
.iter()
|
||||
.filter(|event| {
|
||||
targets.is_empty()
|
||||
|| event
|
||||
.teams
|
||||
.iter()
|
||||
.flat_map(|team| &team.items)
|
||||
.any(|item| targets.contains(&item.competitor))
|
||||
})
|
||||
.map(|event| event.log_evidence)
|
||||
.sum(),
|
||||
posteriors: scratch
|
||||
.skills
|
||||
.iter()
|
||||
.map(|(agent, skill)| (agent, skill.posterior()))
|
||||
.map(|(competitor, skill)| (competitor, skill.posterior()))
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
@@ -687,7 +719,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
&self,
|
||||
targets: &[Index],
|
||||
forward: bool,
|
||||
agents: &CompetitorStore<T, D>,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
) -> f64 {
|
||||
// Hashed once rather than scanned per player per event, so a
|
||||
// `log_evidence_for` with many keys is not quadratic.
|
||||
@@ -696,11 +728,11 @@ impl<T: Time> TimeSlice<T> {
|
||||
let mut arena = ScratchArena::new();
|
||||
|
||||
let run_event = |event: &Event, arena: &mut ScratchArena| -> f64 {
|
||||
let teams = event.within_priors(forward, &self.skills, agents);
|
||||
let teams = event.within_priors(forward, &self.skills, competitors);
|
||||
let result = event.outputs();
|
||||
match event.kind {
|
||||
EventKind::Ranked => {
|
||||
Game::ranked_with_arena(
|
||||
GameRef::ranked_with_arena(
|
||||
teams,
|
||||
&result,
|
||||
&event.weights,
|
||||
@@ -711,7 +743,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
.log_evidence
|
||||
}
|
||||
EventKind::Scored { score_sigma } => {
|
||||
Game::scored_with_arena(
|
||||
GameRef::scored_with_arena(
|
||||
teams,
|
||||
&result,
|
||||
&event.weights,
|
||||
@@ -741,7 +773,7 @@ impl<T: Time> TimeSlice<T> {
|
||||
.teams
|
||||
.iter()
|
||||
.flat_map(|team| &team.items)
|
||||
.any(|item| target_set.contains(&item.agent))
|
||||
.any(|item| target_set.contains(&item.competitor))
|
||||
})
|
||||
.map(|event| run_event(event, &mut arena))
|
||||
.sum()
|
||||
@@ -753,27 +785,36 @@ impl<T: Time> TimeSlice<T> {
|
||||
.teams
|
||||
.iter()
|
||||
.flat_map(|team| &team.items)
|
||||
.any(|item| target_set.contains(&item.agent))
|
||||
.any(|item| target_set.contains(&item.competitor))
|
||||
})
|
||||
.map(|event| event.log_evidence)
|
||||
.sum()
|
||||
}
|
||||
}
|
||||
|
||||
pub fn get_composition(&self) -> Vec<Vec<Vec<Index>>> {
|
||||
/// Test-only: reads the slice's shape back for assertions.
|
||||
#[cfg(test)]
|
||||
pub(crate) fn get_composition(&self) -> Vec<Vec<Vec<Index>>> {
|
||||
self.events
|
||||
.iter()
|
||||
.map(|event| {
|
||||
event
|
||||
.teams
|
||||
.iter()
|
||||
.map(|team| team.items.iter().map(|item| item.agent).collect::<Vec<_>>())
|
||||
.map(|team| {
|
||||
team.items
|
||||
.iter()
|
||||
.map(|item| item.competitor)
|
||||
.collect::<Vec<_>>()
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
}
|
||||
|
||||
pub fn get_results(&self) -> Vec<Vec<f64>> {
|
||||
/// Test-only: reads the slice's shape back for assertions.
|
||||
#[cfg(test)]
|
||||
pub(crate) fn get_results(&self) -> Vec<Vec<f64>> {
|
||||
self.events
|
||||
.iter()
|
||||
.map(|event| {
|
||||
@@ -809,13 +850,85 @@ pub(crate) fn compute_elapsed<T: Time>(last: Option<&T>, current: &T) -> i64 {
|
||||
elapsed.max(0)
|
||||
}
|
||||
|
||||
impl<T: Time> TimeSlice<T> {
|
||||
/// This slice's scored event factors, as contrasts over competitors.
|
||||
///
|
||||
/// Message passing produces per-competitor marginals and throws the
|
||||
/// correlation away — `Item::likelihood` is already the projection of an
|
||||
/// event's factor onto one competitor. So a joint has to be rebuilt from
|
||||
/// the factor structure rather than recovered from the messages.
|
||||
///
|
||||
/// Usefully, a precision matrix depends only on *structure* — who played
|
||||
/// whom, with what weights and what observation noise — and not on the
|
||||
/// observed outcomes. The means are already exact, so only the second
|
||||
/// moment needs rebuilding.
|
||||
///
|
||||
/// Each entry is a contrast and the observation variance that sits on it.
|
||||
/// Ranked events contribute nothing: their truncation factors are EP
|
||||
/// approximations that inference does not retain.
|
||||
pub(crate) fn scored_contrasts<D: Drift<T>>(
|
||||
&self,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
) -> Vec<(Vec<(Index, f64)>, f64)> {
|
||||
let mut out = Vec::new();
|
||||
|
||||
for event in &self.events {
|
||||
let EventKind::Scored { score_sigma } = event.kind else {
|
||||
continue;
|
||||
};
|
||||
|
||||
// Teams best-first, matching the diff chain inference builds.
|
||||
let mut order: Vec<usize> = (0..event.teams.len()).collect();
|
||||
order.sort_by(|&a, &b| {
|
||||
event.teams[b]
|
||||
.output
|
||||
.partial_cmp(&event.teams[a].output)
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
});
|
||||
|
||||
for pair in order.windows(2) {
|
||||
let (hi, lo) = (pair[0], pair[1]);
|
||||
let mut contrast: Vec<(Index, f64)> = Vec::new();
|
||||
let mut noise = score_sigma * score_sigma;
|
||||
|
||||
for (team, sign) in [(hi, 1.0), (lo, -1.0)] {
|
||||
for (m, item) in event.teams[team].items.iter().enumerate() {
|
||||
let w = event.weights[team][m];
|
||||
noise += w * w * competitors[item.competitor].rating.beta.powi(2);
|
||||
contrast.push((item.competitor, sign * w));
|
||||
}
|
||||
}
|
||||
|
||||
out.push((contrast, noise));
|
||||
}
|
||||
}
|
||||
|
||||
out
|
||||
}
|
||||
|
||||
/// True when every event here is scored, so the joint is exact.
|
||||
pub(crate) fn all_scored(&self) -> bool {
|
||||
self.events
|
||||
.iter()
|
||||
.all(|e| matches!(e.kind, EventKind::Scored { .. }))
|
||||
}
|
||||
|
||||
/// The competitors appearing in this slice, with the elapsed count since
|
||||
/// each one's previous appearance.
|
||||
pub(crate) fn appearances(&self) -> impl Iterator<Item = (Index, i64)> + '_ {
|
||||
self.skills
|
||||
.keys()
|
||||
.map(|idx| (idx, self.skills.get(idx).expect("slice key").elapsed))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use approx::assert_ulps_eq;
|
||||
|
||||
use super::*;
|
||||
use crate::{
|
||||
KeyTable, competitor::Competitor, drift::ConstantDrift, rating::Rating,
|
||||
competitor::Competitor, drift::ConstantDrift, key_table::KeyTable, rating::Rating,
|
||||
storage::CompetitorStore,
|
||||
};
|
||||
|
||||
@@ -830,16 +943,16 @@ mod tests {
|
||||
let e = index_map.get_or_create("e");
|
||||
let f = index_map.get_or_create("f");
|
||||
|
||||
let mut agents: CompetitorStore<i64, ConstantDrift> = CompetitorStore::new();
|
||||
let mut competitors: CompetitorStore<i64, ConstantDrift> = CompetitorStore::new();
|
||||
|
||||
for agent in [a, b, c, d, e, f] {
|
||||
agents.insert(
|
||||
agent,
|
||||
for competitor in [a, b, c, d, e, f] {
|
||||
competitors.insert(
|
||||
competitor,
|
||||
Competitor {
|
||||
rating: Rating::new(
|
||||
Gaussian::from_ms(25.0, 25.0 / 3.0),
|
||||
25.0 / 6.0,
|
||||
ConstantDrift(25.0 / 300.0),
|
||||
ConstantDrift::new(25.0 / 300.0),
|
||||
),
|
||||
..Default::default()
|
||||
},
|
||||
@@ -857,7 +970,7 @@ mod tests {
|
||||
Some(vec![vec![1.0, 0.0], vec![0.0, 1.0], vec![1.0, 0.0]]),
|
||||
None,
|
||||
vec![EventKind::Ranked; 3],
|
||||
&agents,
|
||||
&competitors,
|
||||
);
|
||||
|
||||
let post = time_slice.posteriors();
|
||||
@@ -893,7 +1006,7 @@ mod tests {
|
||||
epsilon = 1e-6
|
||||
);
|
||||
|
||||
assert_eq!(time_slice.iterate_to_convergence(&agents), 1);
|
||||
assert_eq!(time_slice.iterate_to_convergence(&competitors), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -907,16 +1020,16 @@ mod tests {
|
||||
let e = index_map.get_or_create("e");
|
||||
let f = index_map.get_or_create("f");
|
||||
|
||||
let mut agents: CompetitorStore<i64, ConstantDrift> = CompetitorStore::new();
|
||||
let mut competitors: CompetitorStore<i64, ConstantDrift> = CompetitorStore::new();
|
||||
|
||||
for agent in [a, b, c, d, e, f] {
|
||||
agents.insert(
|
||||
agent,
|
||||
for competitor in [a, b, c, d, e, f] {
|
||||
competitors.insert(
|
||||
competitor,
|
||||
Competitor {
|
||||
rating: Rating::new(
|
||||
Gaussian::from_ms(25.0, 25.0 / 3.0),
|
||||
25.0 / 6.0,
|
||||
ConstantDrift(25.0 / 300.0),
|
||||
ConstantDrift::new(25.0 / 300.0),
|
||||
),
|
||||
..Default::default()
|
||||
},
|
||||
@@ -934,7 +1047,7 @@ mod tests {
|
||||
Some(vec![vec![1.0, 0.0], vec![0.0, 1.0], vec![1.0, 0.0]]),
|
||||
None,
|
||||
vec![EventKind::Ranked; 3],
|
||||
&agents,
|
||||
&competitors,
|
||||
);
|
||||
|
||||
let post = time_slice.posteriors();
|
||||
@@ -955,7 +1068,7 @@ mod tests {
|
||||
epsilon = 1e-6
|
||||
);
|
||||
|
||||
assert!(time_slice.iterate_to_convergence(&agents) > 1);
|
||||
assert!(time_slice.iterate_to_convergence(&competitors) > 1);
|
||||
|
||||
let post = time_slice.posteriors();
|
||||
|
||||
@@ -987,16 +1100,16 @@ mod tests {
|
||||
let e = index_map.get_or_create("e");
|
||||
let f = index_map.get_or_create("f");
|
||||
|
||||
let mut agents: CompetitorStore<i64, ConstantDrift> = CompetitorStore::new();
|
||||
let mut competitors: CompetitorStore<i64, ConstantDrift> = CompetitorStore::new();
|
||||
|
||||
for agent in [a, b, c, d, e, f] {
|
||||
agents.insert(
|
||||
agent,
|
||||
for competitor in [a, b, c, d, e, f] {
|
||||
competitors.insert(
|
||||
competitor,
|
||||
Competitor {
|
||||
rating: Rating::new(
|
||||
Gaussian::from_ms(25.0, 25.0 / 3.0),
|
||||
25.0 / 6.0,
|
||||
ConstantDrift(25.0 / 300.0),
|
||||
ConstantDrift::new(25.0 / 300.0),
|
||||
),
|
||||
..Default::default()
|
||||
},
|
||||
@@ -1014,10 +1127,10 @@ mod tests {
|
||||
Some(vec![vec![1.0, 0.0], vec![0.0, 1.0], vec![1.0, 0.0]]),
|
||||
None,
|
||||
vec![EventKind::Ranked; 3],
|
||||
&agents,
|
||||
&competitors,
|
||||
);
|
||||
|
||||
time_slice.iterate_to_convergence(&agents);
|
||||
time_slice.iterate_to_convergence(&competitors);
|
||||
|
||||
let post = time_slice.posteriors();
|
||||
|
||||
@@ -1046,12 +1159,12 @@ mod tests {
|
||||
Some(vec![vec![1.0, 0.0], vec![0.0, 1.0], vec![1.0, 0.0]]),
|
||||
None,
|
||||
vec![EventKind::Ranked; 3],
|
||||
&agents,
|
||||
&competitors,
|
||||
);
|
||||
|
||||
assert_eq!(time_slice.events.len(), 6);
|
||||
|
||||
time_slice.iterate_to_convergence(&agents);
|
||||
time_slice.iterate_to_convergence(&competitors);
|
||||
|
||||
let post = time_slice.posteriors();
|
||||
|
||||
@@ -1090,16 +1203,16 @@ mod tests {
|
||||
let c = index_map.get_or_create("c");
|
||||
let d = index_map.get_or_create("d");
|
||||
|
||||
let mut agents: CompetitorStore<i64, ConstantDrift> = CompetitorStore::new();
|
||||
let mut competitors: CompetitorStore<i64, ConstantDrift> = CompetitorStore::new();
|
||||
|
||||
for agent in [a, b, c, d] {
|
||||
agents.insert(
|
||||
agent,
|
||||
for competitor in [a, b, c, d] {
|
||||
competitors.insert(
|
||||
competitor,
|
||||
Competitor {
|
||||
rating: Rating::new(
|
||||
Gaussian::from_ms(25.0, 25.0 / 3.0),
|
||||
25.0 / 6.0,
|
||||
ConstantDrift(25.0 / 300.0),
|
||||
ConstantDrift::new(25.0 / 300.0),
|
||||
),
|
||||
..Default::default()
|
||||
},
|
||||
@@ -1117,7 +1230,7 @@ mod tests {
|
||||
Some(vec![vec![1.0, 0.0], vec![1.0, 0.0], vec![1.0, 0.0]]),
|
||||
None,
|
||||
vec![EventKind::Ranked; 3],
|
||||
&agents,
|
||||
&competitors,
|
||||
);
|
||||
|
||||
assert_eq!(ts.color_groups.n_colors(), 2);
|
||||
@@ -1128,16 +1241,24 @@ mod tests {
|
||||
assert_eq!(ts.color_groups.color_range(1), 2..3);
|
||||
|
||||
// Events at positions 0 and 1 (color 0) must be disjoint — verify by
|
||||
// checking that the agent sets of self.events[0] and self.events[1] do
|
||||
// not include the agent at self.events[2].
|
||||
let agents_in_ev2: Vec<Index> = ts.events[2].iter_agents().collect();
|
||||
let agents_in_ev0: Vec<Index> = ts.events[0].iter_agents().collect();
|
||||
let agents_in_ev1: Vec<Index> = ts.events[1].iter_agents().collect();
|
||||
// checking that the competitor sets of self.events[0] and self.events[1] do
|
||||
// not include the competitor at self.events[2].
|
||||
let competitors_in_ev2: Vec<Index> = ts.events[2].iter_competitors().collect();
|
||||
let competitors_in_ev0: Vec<Index> = ts.events[0].iter_competitors().collect();
|
||||
let competitors_in_ev1: Vec<Index> = ts.events[1].iter_competitors().collect();
|
||||
// ev0 and ev1 must be disjoint from each other (color-0 invariant).
|
||||
assert!(agents_in_ev0.iter().all(|ag| !agents_in_ev1.contains(ag)));
|
||||
// ev2 must share an agent with ev0 or ev1 (it needed its own color).
|
||||
let ev2_overlaps_ev0 = agents_in_ev2.iter().any(|ag| agents_in_ev0.contains(ag));
|
||||
let ev2_overlaps_ev1 = agents_in_ev2.iter().any(|ag| agents_in_ev1.contains(ag));
|
||||
assert!(
|
||||
competitors_in_ev0
|
||||
.iter()
|
||||
.all(|ag| !competitors_in_ev1.contains(ag))
|
||||
);
|
||||
// ev2 must share an competitor with ev0 or ev1 (it needed its own color).
|
||||
let ev2_overlaps_ev0 = competitors_in_ev2
|
||||
.iter()
|
||||
.any(|ag| competitors_in_ev0.contains(ag));
|
||||
let ev2_overlaps_ev1 = competitors_in_ev2
|
||||
.iter()
|
||||
.any(|ag| competitors_in_ev1.contains(ag));
|
||||
assert!(ev2_overlaps_ev0 || ev2_overlaps_ev1);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
//! What an additive model does to uncertainty, and why "add the marginals" is
|
||||
//! unsafe in one direction and merely wasteful in the other.
|
||||
//!
|
||||
//! Structurally this is the shape a joint player/layout model takes: every
|
||||
//! observation measures a *sum* of nodes against a reference, so the data pins
|
||||
//! differences and leaves the overall level to the prior. That is the classic
|
||||
//! rating-scale indeterminacy, not a defect.
|
||||
//!
|
||||
//! The consequence for a consumer is that combining marginals is wrong in
|
||||
//! opposite directions depending on the combination, which is worth pinning
|
||||
//! because the unsafe direction is not the one you would guess:
|
||||
//!
|
||||
//! - **Differences** (`a - b`): the shared level cancels, so the exact width is
|
||||
//! small — and adding marginals lands within a couple of percent of it here,
|
||||
//! because the loopy underestimate offsets the ignored correlation.
|
||||
//! - **Sums** (`a + b`): the shared level does *not* cancel, so the exact width
|
||||
//! is large, and adding marginals is roughly five times too narrow. That is
|
||||
//! overconfident, and it is the direction that publishes a claim the data
|
||||
//! does not support.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{ConstantDrift, ConvergenceOptions, Event, History, Member, Outcome, Team};
|
||||
|
||||
#[test]
|
||||
fn additive_structure_makes_sums_wide_and_differences_tight() {
|
||||
// Structurally like ustat: every round is (player + hole) measured against
|
||||
// a fixed reference. Only SUMS are pinned by the data; the split between
|
||||
// player and hole is pinned only by the prior.
|
||||
let players = ["p0", "p1", "p2"];
|
||||
let holes = ["h0", "h1"];
|
||||
|
||||
let mut h: History<i64, _, _, &'static str> = History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-12,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
|
||||
let mut seed = 3u64;
|
||||
let mut rnd = move || {
|
||||
seed ^= seed << 13;
|
||||
seed ^= seed >> 7;
|
||||
seed ^= seed << 17;
|
||||
seed
|
||||
};
|
||||
// true skills, so we know what the data encodes
|
||||
let truth_p = [2.0, 0.0, -2.0];
|
||||
let truth_h = [1.0, -1.0];
|
||||
|
||||
let mut events = Vec::new();
|
||||
for _ in 0..60 {
|
||||
let p = (rnd() as usize) % 3;
|
||||
let q = (rnd() as usize) % 2;
|
||||
let noise = ((rnd() % 1000) as f64 / 1000.0 - 0.5) * 2.0;
|
||||
let score = truth_p[p] + truth_h[q] + noise;
|
||||
events.push(Event {
|
||||
time: 1,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(players[p]), Member::new(holes[q])]),
|
||||
Team::with_members([Member::new("reference")]),
|
||||
],
|
||||
outcome: Outcome::scores([score, 0.0]),
|
||||
});
|
||||
}
|
||||
h.add_events(events).unwrap();
|
||||
let r = h.converge().unwrap();
|
||||
assert!(r.converged, "{:?}", r.final_step);
|
||||
|
||||
println!("\n== marginals (what current_skill reports) ==");
|
||||
for k in players.iter().chain(holes.iter()) {
|
||||
let g = h.current_skill(k).unwrap();
|
||||
println!(" {k}: mu {:>8.4} sigma {:>8.4}", g.mu(), g.sigma());
|
||||
}
|
||||
|
||||
println!("\n== the same nodes via posterior_of (exact marginal) ==");
|
||||
for k in players.iter().chain(holes.iter()) {
|
||||
let g = h.joint().unwrap().posterior_of(&[(k, 1.0)]).unwrap();
|
||||
println!(" {k}: mu {:>8.4} sigma {:>8.4}", g.mu(), g.sigma());
|
||||
}
|
||||
|
||||
println!("\n== combinations the data actually pins ==");
|
||||
for (label, terms) in [
|
||||
("p0 + h0 (a round)", vec![(&"p0", 1.0), (&"h0", 1.0)]),
|
||||
(
|
||||
"p0 - p1 (rank two players)",
|
||||
vec![(&"p0", 1.0), (&"p1", -1.0)],
|
||||
),
|
||||
("p0 - p2", vec![(&"p0", 1.0), (&"p2", -1.0)]),
|
||||
(
|
||||
"h0 - h1 (rank two holes)",
|
||||
vec![(&"h0", 1.0), (&"h1", -1.0)],
|
||||
),
|
||||
] {
|
||||
let joint = h.joint().unwrap().posterior_of(&terms).unwrap();
|
||||
// what a consumer gets today by adding marginals
|
||||
let naive: f64 = terms
|
||||
.iter()
|
||||
.map(|(k, c)| c * c * h.current_skill(*k).unwrap().sigma().powi(2))
|
||||
.sum::<f64>()
|
||||
.sqrt();
|
||||
println!(
|
||||
" {label:<28} exact sigma {:>7.4} adding marginals {:>7.4} {:>5.2}x over",
|
||||
joint.sigma(),
|
||||
naive,
|
||||
naive / joint.sigma()
|
||||
);
|
||||
|
||||
let ratio = naive / joint.sigma();
|
||||
if label.contains('+') {
|
||||
assert!(
|
||||
ratio < 0.5,
|
||||
"{label}: adding marginals should be badly OVERconfident for a \
|
||||
sum, got {ratio:.3}x"
|
||||
);
|
||||
} else {
|
||||
assert!(
|
||||
(0.8..1.25).contains(&ratio),
|
||||
"{label}: adding marginals happens to be close for a difference, \
|
||||
got {ratio:.3}x"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// A single node in an additive model is weakly identified: its exact
|
||||
// posterior is far wider than message passing reports, because the level it
|
||||
// shares with its partners is pinned only by the prior.
|
||||
for k in players.iter().chain(holes.iter()) {
|
||||
let bp = h.current_skill(k).unwrap().sigma();
|
||||
let exact = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&[(k, 1.0)])
|
||||
.unwrap()
|
||||
.sigma();
|
||||
assert!(
|
||||
exact > 3.0 * bp,
|
||||
"{k}: exact marginal {exact} should be much wider than the reported \
|
||||
{bp} in an additive model"
|
||||
);
|
||||
}
|
||||
}
|
||||
+28
-24
@@ -11,7 +11,7 @@ fn add_events_bulk_via_iter() {
|
||||
.sigma(2.0)
|
||||
.beta(1.0)
|
||||
.p_draw(0.0)
|
||||
.drift(ConstantDrift(0.0))
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 30,
|
||||
epsilon: 1e-6,
|
||||
@@ -41,9 +41,9 @@ fn add_events_bulk_via_iter() {
|
||||
h.add_events(events).unwrap();
|
||||
let report = h.converge().unwrap();
|
||||
assert!(report.converged);
|
||||
assert!(h.lookup(&"a").is_some());
|
||||
assert!(h.lookup(&"b").is_some());
|
||||
assert!(h.lookup(&"c").is_some());
|
||||
assert!(h.current_skill("a").is_some());
|
||||
assert!(h.current_skill("b").is_some());
|
||||
assert!(h.current_skill("c").is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -53,7 +53,7 @@ fn add_events_draw() {
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.p_draw(0.25)
|
||||
.drift(ConstantDrift(25.0 / 300.0))
|
||||
.drift(ConstantDrift::new(25.0 / 300.0))
|
||||
.build();
|
||||
|
||||
let events: Vec<Event<i64, &'static str>> = vec![Event {
|
||||
@@ -65,7 +65,7 @@ fn add_events_draw() {
|
||||
outcome: Outcome::draw(2),
|
||||
}];
|
||||
h.add_events(events).unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -103,9 +103,9 @@ fn fluent_event_builder_basic() {
|
||||
|
||||
let report = h.converge().unwrap();
|
||||
assert!(report.converged);
|
||||
assert!(h.lookup(&"alice").is_some());
|
||||
assert!(h.lookup(&"bob").is_some());
|
||||
assert!(h.lookup(&"carol").is_some());
|
||||
assert!(h.current_skill("alice").is_some());
|
||||
assert!(h.current_skill("bob").is_some());
|
||||
assert!(h.current_skill("carol").is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -123,7 +123,7 @@ fn fluent_event_builder_winner_convenience() {
|
||||
.winner(0)
|
||||
.commit()
|
||||
.unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -141,7 +141,7 @@ fn fluent_event_builder_draw() {
|
||||
.draw()
|
||||
.commit()
|
||||
.unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -155,14 +155,14 @@ fn current_skill_and_learning_curve() {
|
||||
.build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
h.record_winner(&"a", &"b", 2).unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let a = h.current_skill(&"a").unwrap();
|
||||
assert!(a.mu() > 25.0);
|
||||
let b = h.current_skill(&"b").unwrap();
|
||||
assert!(b.mu() < 25.0);
|
||||
|
||||
let a_curve = h.learning_curve(&"a");
|
||||
let a_curve = h.learning_curve(&"a").unwrap();
|
||||
assert_eq!(a_curve.len(), 2);
|
||||
assert_eq!(a_curve[0].0, 1);
|
||||
assert_eq!(a_curve[1].0, 2);
|
||||
@@ -181,12 +181,12 @@ fn log_evidence_total_vs_subset() {
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.p_draw(0.0)
|
||||
.drift(ConstantDrift(0.0))
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
h.record_winner(&"b", &"a", 2).unwrap();
|
||||
let total = h.log_evidence();
|
||||
let a_only = h.log_evidence_for(&[&"a"]);
|
||||
let a_only = h.log_evidence_for(&[&"a"]).unwrap();
|
||||
assert!(total.is_finite());
|
||||
assert!(a_only.is_finite());
|
||||
}
|
||||
@@ -201,9 +201,9 @@ fn predict_quality_two_teams() {
|
||||
.p_draw(0.0)
|
||||
.build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let q = h.predict_quality(&[&[&"a"], &[&"b"]]);
|
||||
let q = h.quality(&[&[&"a"], &[&"b"]]).unwrap();
|
||||
assert!(q > 0.0 && q <= 1.0);
|
||||
}
|
||||
|
||||
@@ -217,12 +217,16 @@ fn predict_outcome_two_teams_sums_to_one() {
|
||||
.p_draw(0.0)
|
||||
.build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let p = h.predict_outcome(&[&[&"a"], &[&"b"]]);
|
||||
assert_eq!(p.len(), 2);
|
||||
assert!((p[0] + p[1] - 1.0).abs() < 1e-9);
|
||||
assert!(p[0] > p[1]);
|
||||
let p = h.predict_outcome(&[&[&"a"], &[&"b"]]).unwrap();
|
||||
let wins = p.win_probabilities();
|
||||
assert_eq!(wins.len(), 2);
|
||||
// With p_draw == 0 there is no draw outcome, so the two win
|
||||
// probabilities are the whole space.
|
||||
assert!((p.total() - 1.0).abs() < 1e-9, "total = {}", p.total());
|
||||
assert!((wins[0] + wins[1] - 1.0).abs() < 1e-9);
|
||||
assert!(wins[0] > wins[1]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -232,7 +236,7 @@ fn fluent_event_builder_scores() {
|
||||
.mu(25.0)
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.drift(ConstantDrift(0.0))
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.build();
|
||||
|
||||
h.event(1)
|
||||
@@ -241,7 +245,7 @@ fn fluent_event_builder_scores() {
|
||||
.scores([12.0, 4.0])
|
||||
.commit()
|
||||
.unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let a = h.current_skill(&"alice").unwrap();
|
||||
let b = h.current_skill(&"bob").unwrap();
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
//! `Member::with_prior` / `with_drift_scale` — competitor configuration.
|
||||
//!
|
||||
//! Both were previously consumed only on the branch that *creates* a
|
||||
//! competitor, so configuration supplied for a key the history already knew was
|
||||
//! dropped with no error. `with_prior` had no coverage in this directory at
|
||||
//! all, which is how that survived.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConvergenceOptions, Event, Gaussian, History, InferenceError, Member, Outcome, Team,
|
||||
};
|
||||
|
||||
const CONVERGENCE: ConvergenceOptions = ConvergenceOptions {
|
||||
max_iter: 2_000,
|
||||
epsilon: 1e-12,
|
||||
alpha: 1.0,
|
||||
};
|
||||
|
||||
fn history() -> History {
|
||||
History::builder()
|
||||
.mu(25.0)
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.p_draw(0.0)
|
||||
.convergence(CONVERGENCE)
|
||||
.build()
|
||||
}
|
||||
|
||||
/// One event, optionally configuring `a`.
|
||||
fn bout(
|
||||
a: &'static str,
|
||||
b: &'static str,
|
||||
time: i64,
|
||||
prior: Option<Gaussian>,
|
||||
scale: Option<f64>,
|
||||
) -> Event<i64, &'static str> {
|
||||
let mut member = Member::new(a);
|
||||
if let Some(p) = prior {
|
||||
member = member.with_prior(p);
|
||||
}
|
||||
if let Some(s) = scale {
|
||||
member = member.with_drift_scale(s);
|
||||
}
|
||||
|
||||
Event {
|
||||
time,
|
||||
teams: smallvec![
|
||||
Team::with_members([member]),
|
||||
Team::with_members([Member::new(b)]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}
|
||||
}
|
||||
|
||||
fn skill_of(h: &History, key: &str) -> Gaussian {
|
||||
h.current_skill(&key).expect("key in history")
|
||||
}
|
||||
|
||||
/// Baseline: the mechanism works at all on a competitor's first appearance.
|
||||
#[test]
|
||||
fn a_prior_applies_to_a_new_competitor() {
|
||||
let seeded = Gaussian::from_ms(40.0, 1.0);
|
||||
|
||||
let mut with = history();
|
||||
with.add_events(vec![bout("a", "b", 0, Some(seeded), None)])
|
||||
.unwrap();
|
||||
let _ = with.converge().unwrap();
|
||||
|
||||
let mut without = history();
|
||||
without
|
||||
.add_events(vec![bout("a", "b", 0, None, None)])
|
||||
.unwrap();
|
||||
let _ = without.converge().unwrap();
|
||||
|
||||
assert!(
|
||||
(skill_of(&with, "a").mu() - skill_of(&without, "a").mu()).abs() > 1.0,
|
||||
"a seeded prior should move the fit"
|
||||
);
|
||||
}
|
||||
|
||||
/// The defect in #10: a prior supplied for a competitor the history already
|
||||
/// knows was silently discarded, and the caller got output computed from the
|
||||
/// default prior with no indication anything had been dropped.
|
||||
#[test]
|
||||
fn a_prior_applies_to_a_competitor_the_history_already_knows() {
|
||||
let seeded = Gaussian::from_ms(40.0, 1.0);
|
||||
|
||||
let mut late = history();
|
||||
late.add_events(vec![bout("a", "b", 0, None, None)])
|
||||
.unwrap();
|
||||
// "a" now exists. Configuring it here used to do nothing whatsoever.
|
||||
late.add_events(vec![bout("a", "b", 1, Some(seeded), None)])
|
||||
.unwrap();
|
||||
let _ = late.converge().unwrap();
|
||||
|
||||
let mut never = history();
|
||||
never
|
||||
.add_events(vec![
|
||||
bout("a", "b", 0, None, None),
|
||||
bout("a", "b", 1, None, None),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = never.converge().unwrap();
|
||||
|
||||
assert!(
|
||||
(skill_of(&late, "a").mu() - skill_of(&never, "a").mu()).abs() > 1.0,
|
||||
"a late prior must not be silently dropped: {} vs {}",
|
||||
skill_of(&late, "a").mu(),
|
||||
skill_of(&never, "a").mu()
|
||||
);
|
||||
}
|
||||
|
||||
/// Configuration is competitor-scoped, not event-scoped, and `converge` refits
|
||||
/// from competitor state — so seeding late reaches the same fit as seeding from
|
||||
/// the start. This is the documented scope, asserted rather than assumed.
|
||||
#[test]
|
||||
fn a_prior_is_whole_history_scoped_not_per_event() {
|
||||
let seeded = Gaussian::from_ms(40.0, 1.0);
|
||||
|
||||
let mut late = history();
|
||||
late.add_events(vec![bout("a", "b", 0, None, None)])
|
||||
.unwrap();
|
||||
late.add_events(vec![bout("a", "b", 1, Some(seeded), None)])
|
||||
.unwrap();
|
||||
let _ = late.converge().unwrap();
|
||||
|
||||
let mut early = history();
|
||||
early
|
||||
.add_events(vec![
|
||||
bout("a", "b", 0, Some(seeded), None),
|
||||
bout("a", "b", 1, Some(seeded), None),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = early.converge().unwrap();
|
||||
|
||||
let (l, e) = (skill_of(&late, "a"), skill_of(&early, "a"));
|
||||
assert!(
|
||||
(l.mu() - e.mu()).abs() < 1e-9 && (l.sigma() - e.sigma()).abs() < 1e-9,
|
||||
"late seeding should refit the whole history: {l:?} vs {e:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn repeating_the_same_prior_is_inert() {
|
||||
let seeded = Gaussian::from_ms(40.0, 1.0);
|
||||
|
||||
let mut once = history();
|
||||
once.add_events(vec![
|
||||
bout("a", "b", 0, Some(seeded), None),
|
||||
bout("a", "b", 1, None, None),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = once.converge().unwrap();
|
||||
|
||||
let mut every_time = history();
|
||||
every_time
|
||||
.add_events(vec![
|
||||
bout("a", "b", 0, Some(seeded), None),
|
||||
bout("a", "b", 1, Some(seeded), None),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = every_time.converge().unwrap();
|
||||
|
||||
let (o, e) = (skill_of(&once, "a"), skill_of(&every_time, "a"));
|
||||
assert!(
|
||||
(o.mu() - e.mu()).abs() < 1e-12 && (o.sigma() - e.sigma()).abs() < 1e-12,
|
||||
"declaring the same prior repeatedly changed the fit: {o:?} vs {e:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// Events within a batch have no order, so two different values for one
|
||||
/// competitor have no well-defined winner. Rejecting is what keeps the answer
|
||||
/// independent of iteration order.
|
||||
#[test]
|
||||
fn a_batch_declaring_two_different_priors_is_rejected() {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.add_events(vec![
|
||||
bout("a", "b", 0, Some(Gaussian::from_ms(40.0, 1.0)), None),
|
||||
bout("a", "b", 1, Some(Gaussian::from_ms(10.0, 1.0)), None),
|
||||
])
|
||||
.expect_err("two different priors for one competitor in one batch");
|
||||
|
||||
assert!(
|
||||
matches!(
|
||||
err,
|
||||
InferenceError::ConflictingCompetitorConfig { field: "prior", .. }
|
||||
),
|
||||
"got {err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// A member setting only `drift_scale` must not also assert the default prior,
|
||||
/// or it would silently undo a prior seeded earlier. This is why the collected
|
||||
/// configuration tracks each field separately rather than a merged `Rating`.
|
||||
#[test]
|
||||
fn setting_one_field_late_leaves_the_other_alone() {
|
||||
let seeded = Gaussian::from_ms(40.0, 1.0);
|
||||
|
||||
let mut h = history();
|
||||
h.add_events(vec![bout("a", "b", 0, Some(seeded), None)])
|
||||
.unwrap();
|
||||
// Only the scale this time — the prior above must survive.
|
||||
h.add_events(vec![bout("a", "b", 1, None, Some(0.5))])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let mut both_upfront = history();
|
||||
both_upfront
|
||||
.add_events(vec![
|
||||
bout("a", "b", 0, Some(seeded), Some(0.5)),
|
||||
bout("a", "b", 1, None, None),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = both_upfront.converge().unwrap();
|
||||
|
||||
let (a, b) = (skill_of(&h, "a"), skill_of(&both_upfront, "a"));
|
||||
assert!(
|
||||
(a.mu() - b.mu()).abs() < 1e-9 && (a.sigma() - b.sigma()).abs() < 1e-9,
|
||||
"setting drift_scale late clobbered the earlier prior: {a:?} vs {b:?}"
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
//! Every public entry point that takes a magnitude, in one place.
|
||||
//!
|
||||
//! This defect class was closed three times in one session and reopened twice,
|
||||
//! because each fix validated the layer it had just touched and inferred the
|
||||
//! rest: `HistoryBuilder` first, then `Game`'s own entry points, then the
|
||||
//! constructors beneath both. A per-site fix cannot notice the site nobody
|
||||
//! thought of.
|
||||
//!
|
||||
//! So this enumerates them. `sigma`, `beta` and `gamma` all enter inference
|
||||
//! only as squares, which means a negative value does not fail — it behaves as
|
||||
//! its absolute value, bit for bit, and the sign vanishes with no diagnostic.
|
||||
//! Non-finite values poison every posterior derived from them.
|
||||
//!
|
||||
//! Adding a public constructor that takes one of these and not adding it here
|
||||
//! is the failure this file exists to make harder.
|
||||
|
||||
use std::panic::{AssertUnwindSafe, catch_unwind};
|
||||
|
||||
use trueskill_tt::{ConstantDrift, Gaussian, History, Member, Outcome, Rating};
|
||||
|
||||
/// Did the entry point refuse the value, by panic or by `Err`?
|
||||
fn refuses(f: impl FnOnce() -> bool) -> bool {
|
||||
catch_unwind(AssertUnwindSafe(f)).unwrap_or(true)
|
||||
}
|
||||
|
||||
/// One entry point, as a name and a closure that applies a value to it.
|
||||
type Case = (&'static str, Box<dyn Fn(f64) -> bool>);
|
||||
|
||||
/// Entry points that must reject a negative magnitude.
|
||||
///
|
||||
/// Each closure returns `true` if it refused by returning an error; a panic is
|
||||
/// also a refusal and is caught.
|
||||
#[test]
|
||||
fn every_magnitude_parameter_rejects_a_negative_value() {
|
||||
let cases: Vec<Case> = vec![
|
||||
(
|
||||
"Gaussian::from_ms(sigma)",
|
||||
Box::new(|v| {
|
||||
let _ = Gaussian::from_ms(25.0, v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"Rating::new(beta)",
|
||||
Box::new(|v| {
|
||||
let _ = Rating::<i64, ConstantDrift>::new(
|
||||
Gaussian::default(),
|
||||
v,
|
||||
ConstantDrift::new(0.0),
|
||||
);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"ConstantDrift::new(gamma)",
|
||||
Box::new(|v| {
|
||||
let _ = ConstantDrift::new(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"HistoryBuilder::sigma",
|
||||
Box::new(|v| {
|
||||
let _ = History::builder().sigma(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"HistoryBuilder::beta",
|
||||
Box::new(|v| {
|
||||
let _ = History::builder().beta(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"HistoryBuilder::score_sigma",
|
||||
Box::new(|v| {
|
||||
let _ = History::builder().score_sigma(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"HistoryBuilder::p_draw",
|
||||
Box::new(|v| {
|
||||
let _ = History::builder().p_draw(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"Member::with_drift_scale (at ingestion)",
|
||||
Box::new(|v| {
|
||||
let mut h = History::builder().build();
|
||||
h.add_events(vec![trueskill_tt::Event {
|
||||
time: 1i64,
|
||||
teams: smallvec::smallvec![
|
||||
trueskill_tt::Team::with_members([Member::new("a").with_drift_scale(v)]),
|
||||
trueskill_tt::Team::with_members([Member::new("b")]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}])
|
||||
.is_err()
|
||||
}),
|
||||
),
|
||||
(
|
||||
"Outcome::scores_with_noise (at ingestion)",
|
||||
Box::new(|v| {
|
||||
let mut h = History::builder().build();
|
||||
h.add_events(vec![trueskill_tt::Event {
|
||||
time: 1i64,
|
||||
teams: smallvec::smallvec![
|
||||
trueskill_tt::Team::with_members([Member::new("a")]),
|
||||
trueskill_tt::Team::with_members([Member::new("b")]),
|
||||
],
|
||||
outcome: Outcome::scores_with_noise([3.0, 1.0], v),
|
||||
}])
|
||||
.is_err()
|
||||
}),
|
||||
),
|
||||
];
|
||||
|
||||
let mut accepted = Vec::new();
|
||||
for (name, f) in &cases {
|
||||
if !refuses(|| f(-1.0)) {
|
||||
accepted.push(*name);
|
||||
}
|
||||
}
|
||||
|
||||
assert!(
|
||||
accepted.is_empty(),
|
||||
"these accepted a negative magnitude, which is squared away silently \
|
||||
rather than honoured or refused:\n {}",
|
||||
accepted.join("\n ")
|
||||
);
|
||||
}
|
||||
|
||||
/// Same set, for NaN and infinity.
|
||||
///
|
||||
/// `Gaussian::from_ms` is deliberately absent: a broken fit produces a NaN
|
||||
/// sigma legitimately and `converge` reports it as `NonFiniteResult`. Rejecting
|
||||
/// it in the constructor turned that reporting path into a panic inside
|
||||
/// inference — see the comment on `from_ms`.
|
||||
#[test]
|
||||
fn every_magnitude_parameter_rejects_a_non_finite_value() {
|
||||
let cases: Vec<Case> = vec![
|
||||
(
|
||||
"Rating::new(beta)",
|
||||
Box::new(|v| {
|
||||
let _ = Rating::<i64, ConstantDrift>::new(
|
||||
Gaussian::default(),
|
||||
v,
|
||||
ConstantDrift::new(0.0),
|
||||
);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"ConstantDrift::new(gamma)",
|
||||
Box::new(|v| {
|
||||
let _ = ConstantDrift::new(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"HistoryBuilder::sigma",
|
||||
Box::new(|v| {
|
||||
let _ = History::builder().sigma(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"HistoryBuilder::beta",
|
||||
Box::new(|v| {
|
||||
let _ = History::builder().beta(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"HistoryBuilder::mu",
|
||||
Box::new(|v| {
|
||||
let _ = History::builder().mu(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"HistoryBuilder::score_sigma",
|
||||
Box::new(|v| {
|
||||
let _ = History::builder().score_sigma(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
(
|
||||
"HistoryBuilder::p_draw",
|
||||
Box::new(|v| {
|
||||
let _ = History::builder().p_draw(v);
|
||||
false
|
||||
}),
|
||||
),
|
||||
];
|
||||
|
||||
let mut accepted = Vec::new();
|
||||
for (name, f) in &cases {
|
||||
for bad in [f64::NAN, f64::INFINITY] {
|
||||
if !refuses(|| f(bad)) {
|
||||
accepted.push(format!("{name} accepted {bad}"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
assert!(
|
||||
accepted.is_empty(),
|
||||
"these accepted a non-finite magnitude:\n {}",
|
||||
accepted.join("\n ")
|
||||
);
|
||||
}
|
||||
|
||||
/// The suite must not pass by refusing everything.
|
||||
#[test]
|
||||
fn ordinary_values_are_still_accepted() {
|
||||
let _ = Gaussian::from_ms(25.0, 8.33);
|
||||
let _ = Rating::<i64, ConstantDrift>::new(Gaussian::default(), 4.17, ConstantDrift::new(0.05));
|
||||
let _ = ConstantDrift::new(0.0833);
|
||||
let _ = History::builder()
|
||||
.mu(25.0)
|
||||
.sigma(8.33)
|
||||
.beta(4.17)
|
||||
.score_sigma(1.0)
|
||||
.p_draw(0.1);
|
||||
|
||||
// Zero beta and zero gamma are legitimate, not degenerate.
|
||||
let _ = ConstantDrift::new(0.0);
|
||||
let _ = Rating::<i64, ConstantDrift>::new(Gaussian::default(), 0.0, ConstantDrift::new(0.0));
|
||||
}
|
||||
@@ -0,0 +1,154 @@
|
||||
//! Stopping short of convergence is an error, not a flag on a success.
|
||||
//!
|
||||
//! A fit that hits `max_iter` is wrong by a little: every rating is finite,
|
||||
//! the ordering looks sensible, and nothing in the numbers says they were
|
||||
//! still moving. When that was `Ok` with `converged: false`, detecting it was
|
||||
//! opt-in and `let _ = h.converge()` was the natural way to opt out — which is
|
||||
//! how a real defect once hid in this crate's own suite.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, Event, History, InferenceError, Member, Outcome, Team,
|
||||
};
|
||||
|
||||
type H = History<i64, ConstantDrift, trueskill_tt::NullObserver, &'static str>;
|
||||
|
||||
fn duel(a: &'static str, b: &'static str, t: i64) -> Event<i64, &'static str> {
|
||||
Event {
|
||||
time: t,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(a)]),
|
||||
Team::with_members([Member::new(b)]),
|
||||
],
|
||||
outcome: Outcome::scores([3.0, 1.0]),
|
||||
}
|
||||
}
|
||||
|
||||
fn capped(max_iter: usize) -> H {
|
||||
History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.5))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build()
|
||||
}
|
||||
|
||||
fn fill(h: &mut H) {
|
||||
h.add_events((1..=6).map(|t| duel("a", "b", t)).collect::<Vec<_>>())
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hitting_the_cap_is_an_error() {
|
||||
let mut h = capped(1);
|
||||
fill(&mut h);
|
||||
let err = h.converge().unwrap_err();
|
||||
match err {
|
||||
InferenceError::NotConverged {
|
||||
iterations,
|
||||
final_step,
|
||||
epsilon,
|
||||
..
|
||||
} => {
|
||||
assert_eq!(iterations, 1);
|
||||
assert!(
|
||||
final_step.0 > epsilon || final_step.1 > epsilon,
|
||||
"{final_step:?}"
|
||||
);
|
||||
}
|
||||
other => panic!("expected NotConverged, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// The message has to name what to do about it, since the fit looks fine.
|
||||
#[test]
|
||||
fn the_error_says_how_to_fix_it() {
|
||||
let mut h = capped(1);
|
||||
fill(&mut h);
|
||||
let text = h.converge().unwrap_err().to_string();
|
||||
assert!(text.contains("did not converge in 1 iterations"), "{text}");
|
||||
assert!(text.contains("max_iter"), "{text}");
|
||||
assert!(text.contains("alpha"), "{text}");
|
||||
}
|
||||
|
||||
/// The escape hatch: a deliberately capped fit is still reachable.
|
||||
#[test]
|
||||
fn converge_partial_returns_the_short_fit() {
|
||||
let mut h = capped(1);
|
||||
fill(&mut h);
|
||||
let report = h.converge_partial().unwrap();
|
||||
assert_eq!(report.iterations, 1);
|
||||
assert!(!report.converged);
|
||||
assert!(h.current_skill(&"a").is_some());
|
||||
}
|
||||
|
||||
/// Both agree when the fit does converge, so the strict path costs nothing.
|
||||
#[test]
|
||||
fn the_two_agree_on_a_converged_fit() {
|
||||
let mut strict = capped(20_000);
|
||||
fill(&mut strict);
|
||||
let a = strict.converge().unwrap();
|
||||
|
||||
let mut partial = capped(20_000);
|
||||
fill(&mut partial);
|
||||
let b = partial.converge_partial().unwrap();
|
||||
|
||||
assert!(a.converged && b.converged);
|
||||
assert_eq!(a.iterations, b.iterations);
|
||||
assert_eq!(a.final_step, b.final_step);
|
||||
}
|
||||
|
||||
/// The default cap must be high enough that an ordinary history clears it.
|
||||
/// At the old value of 30 this history stopped short and said nothing.
|
||||
#[test]
|
||||
fn the_default_cap_clears_an_ordinary_history() {
|
||||
let mut h: History<i64, ConstantDrift, _, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.05))
|
||||
.build();
|
||||
|
||||
let mut events = Vec::new();
|
||||
for t in 0..20i64 {
|
||||
for j in 0..8usize {
|
||||
let k = (t as usize) * 8 + j;
|
||||
events.push(Event {
|
||||
time: t,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(format!("p{}", k % 100))]),
|
||||
Team::with_members([Member::new(format!("p{}", (k + 37) % 100))]),
|
||||
],
|
||||
outcome: Outcome::scores([3.0, 1.0]),
|
||||
});
|
||||
}
|
||||
}
|
||||
h.add_events(events).unwrap();
|
||||
|
||||
let report = h
|
||||
.converge()
|
||||
.expect("an ordinary history must converge by default");
|
||||
assert!(
|
||||
report.iterations > 30,
|
||||
"needed {} sweeps",
|
||||
report.iterations
|
||||
);
|
||||
assert!(report.iterations < trueskill_tt::ITERATIONS);
|
||||
}
|
||||
|
||||
/// An empty history converges trivially rather than erroring.
|
||||
#[test]
|
||||
fn an_empty_history_converges() {
|
||||
let mut h = capped(1);
|
||||
let report = h.converge().unwrap();
|
||||
assert!(report.converged);
|
||||
assert_eq!(report.iterations, 0);
|
||||
}
|
||||
@@ -0,0 +1,162 @@
|
||||
//! Determinism across *processes*, which an in-process test cannot see.
|
||||
//!
|
||||
//! Rust seeds its default hasher once per process, so every `HashMap`
|
||||
//! iteration order is fixed for a run and varies between runs. A test that
|
||||
//! compares results within one process therefore cannot detect a float sum
|
||||
//! whose order comes from a map — all its samples share one seed.
|
||||
//!
|
||||
//! That is not hypothetical. `tests/determinism.rs` compares four thread counts
|
||||
//! inside one process and passed throughout, while `posterior_of` was returning
|
||||
//! two distinct bit patterns across 40 separate runs on identical input.
|
||||
//!
|
||||
//! This re-executes the test binary and compares `f64::to_bits`.
|
||||
|
||||
use std::{env, process::Command};
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, Event, History, Member, NullObserver, Outcome, Team,
|
||||
UnknownKeys,
|
||||
};
|
||||
|
||||
/// Set in the child so it reports instead of re-spawning.
|
||||
const CHILD: &str = "TSTT_DETERMINISM_CHILD";
|
||||
|
||||
const RUNS: usize = 40;
|
||||
|
||||
type H = History<i64, ConstantDrift, NullObserver, String>;
|
||||
|
||||
fn fitted() -> H {
|
||||
let mut h: H = History::builder()
|
||||
.key_type::<String>()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.05))
|
||||
.unknown_keys(UnknownKeys::Prior)
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
|
||||
let mut events = Vec::new();
|
||||
for t in 0..12i64 {
|
||||
for k in 0..6usize {
|
||||
let a = format!("p{}", (t as usize * 6 + k) % 10);
|
||||
let b = format!("p{}", (t as usize * 6 + k + 4) % 10);
|
||||
events.push(Event {
|
||||
time: t,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(a)]),
|
||||
Team::with_members([Member::new(b)]),
|
||||
],
|
||||
outcome: Outcome::scores([3.0, 1.0]),
|
||||
});
|
||||
}
|
||||
}
|
||||
h.add_events(events).unwrap();
|
||||
assert!(h.converge().unwrap().converged);
|
||||
h
|
||||
}
|
||||
|
||||
/// Every quantity that could plausibly depend on iteration order, as bits.
|
||||
fn fingerprint() -> String {
|
||||
let h = fitted();
|
||||
|
||||
// Unknown keys with UNEQUAL but COMPARABLE coefficients, which is what
|
||||
// makes the sum order-sensitive.
|
||||
//
|
||||
// Equal terms sum order-independently and would make this pass vacuously.
|
||||
// Terms of wildly different magnitudes are no better: the small ones fall
|
||||
// below the running total's ULP and are absorbed whatever the order —
|
||||
// measured, spreading these over nine decades dropped the detection rate
|
||||
// to roughly one run in forty. Comparable sizes keep every term able to
|
||||
// change the last bits.
|
||||
let ghosts: Vec<String> = (0..24).map(|i| format!("ghost{i}")).collect();
|
||||
let mut terms: Vec<(&String, f64)> = ghosts
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, k)| (k, 1.0 + i as f64 * 0.37))
|
||||
.collect();
|
||||
let known = "p0".to_string();
|
||||
terms.push((&known, -1.0));
|
||||
|
||||
let posterior = h.joint().unwrap().posterior_of(&terms).unwrap();
|
||||
|
||||
let a = "p0".to_string();
|
||||
let b = "p1".to_string();
|
||||
let target = [(&a, 1.0), (&b, -1.0)];
|
||||
let teams: [&[&String]; 2] = [&[&a], &[&b]];
|
||||
let evr = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.expected_variance_reduction(&teams, &target)
|
||||
.unwrap();
|
||||
|
||||
let curves = h.learning_curves();
|
||||
let mut curve_bits: u64 = 0;
|
||||
let mut keys: Vec<&String> = curves.keys().collect();
|
||||
keys.sort();
|
||||
for key in keys {
|
||||
for (t, g) in &curves[key] {
|
||||
curve_bits ^= (*t as u64).rotate_left(17)
|
||||
^ g.mu().to_bits().rotate_left(31)
|
||||
^ g.sigma().to_bits();
|
||||
}
|
||||
}
|
||||
|
||||
format!(
|
||||
"post={:016x} evr={:016x} le={:016x} curves={curve_bits:016x}",
|
||||
posterior.sigma().to_bits(),
|
||||
evr.to_bits(),
|
||||
h.log_evidence().to_bits(),
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn results_are_identical_across_processes() {
|
||||
if env::var(CHILD).is_ok() {
|
||||
println!("FINGERPRINT {}", fingerprint());
|
||||
return;
|
||||
}
|
||||
|
||||
let exe = env::current_exe().expect("current exe");
|
||||
let mut seen: Vec<String> = Vec::new();
|
||||
|
||||
for run in 0..RUNS {
|
||||
let out = Command::new(&exe)
|
||||
.args([
|
||||
"results_are_identical_across_processes",
|
||||
"--exact",
|
||||
"--nocapture",
|
||||
])
|
||||
.env(CHILD, "1")
|
||||
.output()
|
||||
.expect("spawn child");
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"child {run} failed: {}",
|
||||
String::from_utf8_lossy(&out.stderr)
|
||||
);
|
||||
let stdout = String::from_utf8_lossy(&out.stdout);
|
||||
let line = stdout
|
||||
.lines()
|
||||
.find_map(|l| l.strip_prefix("FINGERPRINT "))
|
||||
.unwrap_or_else(|| panic!("child {run} printed no fingerprint:\n{stdout}"))
|
||||
.to_string();
|
||||
seen.push(line);
|
||||
}
|
||||
|
||||
let first = &seen[0];
|
||||
let differing: Vec<&String> = seen.iter().filter(|s| *s != first).collect();
|
||||
assert!(
|
||||
differing.is_empty(),
|
||||
"results differ across processes on identical input.\n {} of {RUNS} runs differed\n \
|
||||
first: {first}\n differing: {}",
|
||||
differing.len(),
|
||||
differing[0]
|
||||
);
|
||||
}
|
||||
+31
-16
@@ -17,7 +17,7 @@ fn rating() -> R {
|
||||
R::new(
|
||||
Gaussian::from_ms(25.0, 25.0 / 3.0),
|
||||
25.0 / 6.0,
|
||||
ConstantDrift(25.0 / 300.0),
|
||||
ConstantDrift::new(25.0 / 300.0),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -126,8 +126,10 @@ fn empty_history_converges_trivially() {
|
||||
/// indexed out of bounds in release, so this must run in both profiles.
|
||||
#[test]
|
||||
fn converge_on_an_empty_history_with_owned_keys() {
|
||||
let mut history: History<i64, ConstantDrift, NullObserver, String> =
|
||||
History::builder_with_key().score_sigma(5.0).build();
|
||||
let mut history: History<i64, ConstantDrift, NullObserver, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.score_sigma(5.0)
|
||||
.build();
|
||||
|
||||
let report = history.converge().unwrap();
|
||||
|
||||
@@ -158,6 +160,7 @@ fn event_builder_rejects_a_weights_length_mismatch() {
|
||||
kind: "weights",
|
||||
expected: 1,
|
||||
got: 2,
|
||||
..
|
||||
}
|
||||
),
|
||||
"expected a weights MismatchedShape, got {err:?}"
|
||||
@@ -170,8 +173,9 @@ fn event_builder_rejects_a_weights_length_mismatch() {
|
||||
fn event_builder_weights_mismatch_leaves_the_history_untouched() {
|
||||
let mut h = History::default();
|
||||
|
||||
// Two teams, so ingestion would otherwise succeed — a one-team event is
|
||||
// rejected for an unrelated reason and would pass this vacuously.
|
||||
// Two teams, so ingestion would otherwise succeed. A one-team event is
|
||||
// rejected as `NotEnoughTeams` before the weights are ever examined, so
|
||||
// building this with one team would pass vacuously.
|
||||
let _ = h
|
||||
.event(1)
|
||||
.team(["a"])
|
||||
@@ -180,7 +184,10 @@ fn event_builder_weights_mismatch_leaves_the_history_untouched() {
|
||||
.winner(0)
|
||||
.commit();
|
||||
|
||||
assert!(h.learning_curve("a").is_empty());
|
||||
// The rejected event never reached the history, so "a" was never interned.
|
||||
// `None` is the honest answer, and it is distinguishable from a competitor
|
||||
// that IS known but has no appearances yet.
|
||||
assert!(h.learning_curve("a").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -195,7 +202,7 @@ fn empty_event_stream_then_converge() {
|
||||
fn empty_history_queries_do_not_panic() {
|
||||
let h = History::default();
|
||||
assert!(h.learning_curves().is_empty());
|
||||
assert!(h.learning_curve("nobody").is_empty());
|
||||
assert!(h.learning_curve("nobody").is_none());
|
||||
assert!(h.current_skill("nobody").is_none());
|
||||
}
|
||||
|
||||
@@ -215,7 +222,7 @@ fn scored_event_rejects_non_positive_sigma() {
|
||||
.event(1)
|
||||
.team(["a"])
|
||||
.team(["b"])
|
||||
.scores_with_sigma([3.0, 1.0], f64::NAN)
|
||||
.scores_with_noise([3.0, 1.0], f64::NAN)
|
||||
.commit()
|
||||
.unwrap_err();
|
||||
assert!(matches!(
|
||||
@@ -280,8 +287,16 @@ fn log_evidence_survives_a_long_diff_chain() {
|
||||
/// `erfc` approximation; the evidence floor keeps `ln` finite.
|
||||
#[test]
|
||||
fn log_evidence_finite_for_near_certain_outcome() {
|
||||
let overwhelming = R::new(Gaussian::from_ms(5_000.0, 0.5), 1.0, ConstantDrift(0.0));
|
||||
let hopeless = R::new(Gaussian::from_ms(-5_000.0, 0.5), 1.0, ConstantDrift(0.0));
|
||||
let overwhelming = R::new(
|
||||
Gaussian::from_ms(5_000.0, 0.5),
|
||||
1.0,
|
||||
ConstantDrift::new(0.0),
|
||||
);
|
||||
let hopeless = R::new(
|
||||
Gaussian::from_ms(-5_000.0, 0.5),
|
||||
1.0,
|
||||
ConstantDrift::new(0.0),
|
||||
);
|
||||
let a = [overwhelming];
|
||||
let b = [hopeless];
|
||||
let teams: Vec<&[R]> = vec![&a, &b];
|
||||
@@ -310,7 +325,7 @@ fn empty_history_has_no_filtered_estimates() {
|
||||
|
||||
assert!(history.filtered_learning_curves().is_empty());
|
||||
|
||||
assert!(history.filtered_learning_curve("nobody").is_empty());
|
||||
assert!(history.filtered_learning_curve("nobody").is_none());
|
||||
}
|
||||
|
||||
// --- Boundary inputs (#26) ----------------------------------------------
|
||||
@@ -325,7 +340,7 @@ fn tight() -> ConvergenceOptions {
|
||||
|
||||
fn assert_curve_finite(h: &History, keys: &[&str], what: &str) {
|
||||
for key in keys {
|
||||
for (time, g) in h.learning_curve(*key) {
|
||||
for (time, g) in h.learning_curve(*key).unwrap() {
|
||||
assert!(
|
||||
g.mu().is_finite() && g.sigma().is_finite(),
|
||||
"{what}: non-finite posterior for {key} at t={time} (mu={} sigma={})",
|
||||
@@ -351,7 +366,7 @@ fn zero_weight_does_not_produce_a_non_finite_posterior() {
|
||||
.commit()
|
||||
.expect("a zero weight is accepted today; update this test if that changes");
|
||||
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
assert_curve_finite(&h, &["a", "b"], "zero weight");
|
||||
}
|
||||
@@ -368,7 +383,7 @@ fn negative_weight_does_not_produce_a_non_finite_posterior() {
|
||||
.commit()
|
||||
.expect("a negative weight is accepted today; update this test if that changes");
|
||||
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
assert_curve_finite(&h, &["a", "b"], "negative weight");
|
||||
}
|
||||
@@ -389,7 +404,7 @@ fn out_of_order_timestamps_converge_to_the_same_answer() {
|
||||
h.record_winner(&"a", &"b", time).unwrap();
|
||||
}
|
||||
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h
|
||||
}
|
||||
|
||||
@@ -416,7 +431,7 @@ fn extreme_beta_and_sigma_stay_finite() {
|
||||
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
h.record_winner(&"a", &"b", 2).unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
assert_curve_finite(&h, &["a", "b"], &format!("beta={beta} sigma={sigma}"));
|
||||
}
|
||||
|
||||
+153
-52
@@ -1,101 +1,202 @@
|
||||
//! Determinism tests: identical posteriors across RAYON_NUM_THREADS
|
||||
//! values. Only compiled with the `rayon` feature.
|
||||
//! Determinism across `RAYON_NUM_THREADS`, on a workload that actually reaches
|
||||
//! the parallel path.
|
||||
//!
|
||||
//! This test previously proved less than it appeared to. `sweep_color_groups`
|
||||
//! takes its `par_iter` branch only for colour groups of at least
|
||||
//! `RAYON_THRESHOLD` (64) events, and the old fixture built 20 slices of 10
|
||||
//! events — a colour group is a subset of one slice's events, so it could never
|
||||
//! exceed 10. The branch was unreachable, confirmed by CPU-vs-wall time:
|
||||
//! `user 0.64` on eight threads is one core.
|
||||
//!
|
||||
//! It also compared a single competitor's curve out of forty, and never
|
||||
//! compared `log_evidence`, `final_step` or `iterations`.
|
||||
//!
|
||||
//! The fixture below guarantees the parallel branch **by construction**: within
|
||||
//! a slice every event uses a disjoint pair of competitors, so greedy colouring
|
||||
//! puts all of them in colour 0, and that group is `EVENTS_PER_SLICE` long.
|
||||
//! Competitors recur across slices, so the fit still has temporal coupling and
|
||||
//! drift rather than being a set of independent duels.
|
||||
|
||||
#![cfg(feature = "rayon")]
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{ConstantDrift, ConvergenceOptions, Event, History, Member, Outcome, Team};
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, Event, Gaussian, History, Member, Outcome, Team,
|
||||
};
|
||||
|
||||
/// Build a deterministic workload using a simple LCG (no external rand crate).
|
||||
fn build_and_converge(seed: u64) -> Vec<(i64, trueskill_tt::Gaussian)> {
|
||||
let mut h = History::<i64, _, _, String>::builder_with_key()
|
||||
/// Comfortably above the crate's internal `RAYON_THRESHOLD` of 64.
|
||||
const EVENTS_PER_SLICE: usize = 96;
|
||||
const SLICES: i64 = 8;
|
||||
/// Two per event, all disjoint within a slice.
|
||||
const COMPETITORS: usize = EVENTS_PER_SLICE * 2;
|
||||
|
||||
/// Everything a thread count could plausibly perturb.
|
||||
struct Fingerprint {
|
||||
curves: Vec<(String, Vec<(i64, Gaussian)>)>,
|
||||
log_evidence: f64,
|
||||
final_step: (f64, f64),
|
||||
iterations: usize,
|
||||
}
|
||||
|
||||
fn build_and_converge() -> Fingerprint {
|
||||
let mut h = History::builder()
|
||||
.key_type::<String>()
|
||||
.mu(25.0)
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.drift(ConstantDrift(25.0 / 300.0))
|
||||
.drift(ConstantDrift::new(25.0 / 300.0))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 30,
|
||||
epsilon: 1e-6,
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-9,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
|
||||
// LCG for deterministic pseudo-random ints.
|
||||
let mut rng = seed;
|
||||
let mut next = || {
|
||||
rng = rng
|
||||
.wrapping_mul(6364136223846793005)
|
||||
.wrapping_add(1442695040888963407);
|
||||
rng
|
||||
};
|
||||
|
||||
let mut events: Vec<Event<i64, String>> = Vec::with_capacity(200);
|
||||
for ev_i in 0..200 {
|
||||
let a = (next() % 40) as usize;
|
||||
let mut b = (next() % 40) as usize;
|
||||
while b == a {
|
||||
b = (next() % 40) as usize;
|
||||
let mut events: Vec<Event<i64, String>> = Vec::new();
|
||||
for slice in 0..SLICES {
|
||||
for e in 0..EVENTS_PER_SLICE {
|
||||
// Disjoint within the slice: event `e` owns competitors 2e and
|
||||
// 2e+1. Rotating by the slice index makes the pairings differ
|
||||
// between slices, so competitors accumulate a real history.
|
||||
let a = (2 * e + slice as usize) % COMPETITORS;
|
||||
let b = (2 * e + 1 + slice as usize * 3) % COMPETITORS;
|
||||
if a == b {
|
||||
continue;
|
||||
}
|
||||
// ~10 events per slice so color groups have material parallelism.
|
||||
events.push(Event {
|
||||
time: (ev_i as i64 / 10) + 1,
|
||||
time: slice + 1,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(format!("p{a}"))]),
|
||||
Team::with_members([Member::new(format!("p{b}"))]),
|
||||
],
|
||||
outcome: Outcome::winner((next() % 2) as u32, 2),
|
||||
outcome: Outcome::winner(u32::from((e + slice as usize) % 2 == 0), 2),
|
||||
});
|
||||
}
|
||||
}
|
||||
h.add_events(events).unwrap();
|
||||
h.converge().unwrap();
|
||||
// Sample one competitor's curve for the comparison.
|
||||
h.learning_curve("p0")
|
||||
|
||||
let report = h.converge().expect("fixture must converge");
|
||||
|
||||
let mut curves: Vec<(String, Vec<(i64, Gaussian)>)> = h
|
||||
.learning_curves()
|
||||
.into_iter()
|
||||
.map(|(k, v)| (k.clone(), v))
|
||||
.collect();
|
||||
curves.sort_by(|a, b| a.0.cmp(&b.0));
|
||||
|
||||
Fingerprint {
|
||||
curves,
|
||||
log_evidence: h.log_evidence(),
|
||||
final_step: report.final_step,
|
||||
iterations: report.iterations,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn posteriors_identical_across_thread_counts() {
|
||||
let sizes = [1usize, 2, 4, 8];
|
||||
let mut results: Vec<Vec<(i64, trueskill_tt::Gaussian)>> = Vec::new();
|
||||
let mut results: Vec<Fingerprint> = Vec::new();
|
||||
|
||||
for &n in &sizes {
|
||||
let pool = rayon::ThreadPoolBuilder::new()
|
||||
.num_threads(n)
|
||||
.build()
|
||||
.expect("rayon pool build");
|
||||
let curve = pool.install(|| build_and_converge(42));
|
||||
results.push(curve);
|
||||
results.push(pool.install(build_and_converge));
|
||||
}
|
||||
|
||||
let reference = &results[0];
|
||||
for (i, curve) in results.iter().enumerate().skip(1) {
|
||||
|
||||
// Guard against the failure this test previously had: passing while
|
||||
// measuring almost nothing.
|
||||
assert!(
|
||||
reference.curves.len() > 100,
|
||||
"expected every competitor's curve, got {}",
|
||||
reference.curves.len()
|
||||
);
|
||||
|
||||
for (i, got) in results.iter().enumerate().skip(1) {
|
||||
let n = sizes[i];
|
||||
|
||||
assert_eq!(
|
||||
got.iterations, reference.iterations,
|
||||
"iterations differ at {n} threads"
|
||||
);
|
||||
assert_eq!(
|
||||
got.final_step.0.to_bits(),
|
||||
reference.final_step.0.to_bits(),
|
||||
"final_step.0 differs at {n} threads: {:?} vs {:?}",
|
||||
reference.final_step,
|
||||
got.final_step
|
||||
);
|
||||
assert_eq!(
|
||||
got.final_step.1.to_bits(),
|
||||
reference.final_step.1.to_bits(),
|
||||
"final_step.1 differs at {n} threads"
|
||||
);
|
||||
assert_eq!(
|
||||
got.log_evidence.to_bits(),
|
||||
reference.log_evidence.to_bits(),
|
||||
"log_evidence differs at {n} threads: {} vs {}",
|
||||
reference.log_evidence,
|
||||
got.log_evidence
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
got.curves.len(),
|
||||
reference.curves.len(),
|
||||
"competitor count differs at {n} threads"
|
||||
);
|
||||
|
||||
for ((ref_key, ref_curve), (key, curve)) in reference.curves.iter().zip(got.curves.iter()) {
|
||||
assert_eq!(ref_key, key, "competitor order differs at {n} threads");
|
||||
assert_eq!(
|
||||
curve.len(),
|
||||
reference.len(),
|
||||
"curve length differs at {n} threads",
|
||||
n = sizes[i],
|
||||
);
|
||||
for (j, (&(t_ref, g_ref), &(t, g))) in reference.iter().zip(curve.iter()).enumerate() {
|
||||
assert_eq!(
|
||||
t_ref,
|
||||
t,
|
||||
"time point {j} differs at {n} threads: ref={t_ref} vs got={t}",
|
||||
n = sizes[i],
|
||||
ref_curve.len(),
|
||||
"curve length differs for {key} at {n} threads"
|
||||
);
|
||||
for (&(t_ref, g_ref), &(t, g)) in ref_curve.iter().zip(curve.iter()) {
|
||||
assert_eq!(t_ref, t, "time point differs for {key} at {n} threads");
|
||||
assert_eq!(
|
||||
g_ref.mu().to_bits(),
|
||||
g.mu().to_bits(),
|
||||
"mu bits differ at {n} threads, time {t}: ref={ref_mu} got={got_mu}",
|
||||
n = sizes[i],
|
||||
ref_mu = g_ref.mu(),
|
||||
got_mu = g.mu(),
|
||||
"mu differs for {key} at t={t}, {n} threads: {} vs {}",
|
||||
g_ref.mu(),
|
||||
g.mu()
|
||||
);
|
||||
assert_eq!(
|
||||
g_ref.sigma().to_bits(),
|
||||
g.sigma().to_bits(),
|
||||
"sigma bits differ at {n} threads, time {t}: ref={ref_sigma} got={got_sigma}",
|
||||
n = sizes[i],
|
||||
ref_sigma = g_ref.sigma(),
|
||||
got_sigma = g.sigma(),
|
||||
"sigma differs for {key} at t={t}, {n} threads: {} vs {}",
|
||||
g_ref.sigma(),
|
||||
g.sigma()
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The fixture must keep reaching the parallel branch.
|
||||
///
|
||||
/// `RAYON_THRESHOLD` is private, so this pins the property that makes the
|
||||
/// branch reachable rather than the branch itself: within a slice every event
|
||||
/// uses a disjoint competitor pair, so greedy colouring puts all
|
||||
/// `EVENTS_PER_SLICE` of them in one colour group. If someone shrinks the
|
||||
/// fixture, this fails rather than the suite quietly going back to testing the
|
||||
/// sequential path.
|
||||
#[test]
|
||||
fn the_fixture_still_exceeds_the_rayon_threshold() {
|
||||
const RAYON_THRESHOLD: usize = 64;
|
||||
const {
|
||||
assert!(
|
||||
EVENTS_PER_SLICE >= RAYON_THRESHOLD,
|
||||
"a colour group holds at most EVENTS_PER_SLICE events, which must \
|
||||
reach the crate's RAYON_THRESHOLD for the parallel sweep to run"
|
||||
);
|
||||
}
|
||||
|
||||
// Measured by instrumenting `sweep_color_groups`: this fixture produces
|
||||
// one colour group of 96 events and takes the parallel branch on all 872
|
||||
// sweeps. The old fixture's 10-event slices could not reach 64 at all.
|
||||
assert_eq!(EVENTS_PER_SLICE, 96);
|
||||
}
|
||||
|
||||
+124
-29
@@ -2,7 +2,7 @@
|
||||
//!
|
||||
//! The scale multiplies the *variance* the history's `Drift` contributes for
|
||||
//! that competitor, so `scale` is in the same units as `gamma`:
|
||||
//! `ConstantDrift(g)` at `scale = s` behaves as `ConstantDrift(g * s)` would.
|
||||
//! `ConstantDrift::new(g)` at `scale = s` behaves as `ConstantDrift::new(g * s)` would.
|
||||
//! `scale = 0.0` pins a competitor still — an anchor, a rating floor, a course
|
||||
//! difficulty — while everyone around them keeps drifting.
|
||||
|
||||
@@ -53,12 +53,12 @@ fn fit(events: Vec<Event<i64, &'static str>>, gamma: f64) -> Fit {
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.p_draw(0.0)
|
||||
.drift(ConstantDrift(gamma))
|
||||
.drift(ConstantDrift::new(gamma))
|
||||
.convergence(CONVERGENCE)
|
||||
.build();
|
||||
|
||||
h.add_events(events).unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h
|
||||
}
|
||||
|
||||
@@ -160,7 +160,7 @@ fn scale_is_equivalent_to_scaling_gamma() {
|
||||
assert_eq!(t_l, t_r);
|
||||
assert!(
|
||||
(g_l.mu() - g_r.mu()).abs() < 1e-9 && (g_l.sigma() - g_r.sigma()).abs() < 1e-9,
|
||||
"ConstantDrift(0.3) at scale 0.5 must equal ConstantDrift(0.15) for {key} at \
|
||||
"ConstantDrift::new(0.3) at scale 0.5 must equal ConstantDrift::new(0.15) for {key} at \
|
||||
t={t_l}: ({}, {}) vs ({}, {})",
|
||||
g_l.mu(),
|
||||
g_l.sigma(),
|
||||
@@ -218,7 +218,7 @@ fn mixed_static_and_drifting_graph_converges() {
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.p_draw(0.0)
|
||||
.drift(ConstantDrift(25.0 / 300.0))
|
||||
.drift(ConstantDrift::new(25.0 / 300.0))
|
||||
.convergence(CONVERGENCE)
|
||||
.build();
|
||||
|
||||
@@ -259,7 +259,7 @@ fn mixed_static_and_drifting_graph_converges() {
|
||||
|
||||
fn reject(scale: f64) -> InferenceError {
|
||||
let mut h = History::builder()
|
||||
.drift(ConstantDrift(25.0 / 300.0))
|
||||
.drift(ConstantDrift::new(25.0 / 300.0))
|
||||
.build();
|
||||
|
||||
let events: Vec<Event<i64, &'static str>> = vec![Event {
|
||||
@@ -277,13 +277,11 @@ fn reject(scale: f64) -> InferenceError {
|
||||
|
||||
#[test]
|
||||
fn negative_scale_is_rejected() {
|
||||
assert_eq!(
|
||||
assert!(matches!(
|
||||
reject(-1.0),
|
||||
InferenceError::InvalidParameter {
|
||||
name: "drift_scale",
|
||||
value: -1.0
|
||||
}
|
||||
);
|
||||
InferenceError::InvalidParameter { name: "drift_scale", value, .. }
|
||||
if value == -1.0
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -341,19 +339,26 @@ fn zero_scale_pins_a_competitor_in_the_filtered_pass() {
|
||||
);
|
||||
}
|
||||
|
||||
/// `drift_scale` is competitor configuration captured at first appearance, the
|
||||
/// same as `prior` — a later `with_drift_scale` on a key the history already
|
||||
/// knows is ignored. This guards that decision rather than driving it: the
|
||||
/// behaviour falls out of where the capture happens, and the point of the test
|
||||
/// is that moving the capture would be a visible break, not a silent one.
|
||||
/// `drift_scale` is competitor configuration, and configuration supplied for a
|
||||
/// competitor the history already knows is now *applied* rather than dropped.
|
||||
///
|
||||
/// This test previously asserted the opposite. It was written as a deliberate
|
||||
/// change-detector — "moving the capture would be a visible break, not a silent
|
||||
/// one" — and that is exactly what happened: the capture moved, and the
|
||||
/// assertion inverted rather than being deleted.
|
||||
///
|
||||
/// Because configuration lives on the competitor and `converge` refits from
|
||||
/// competitor state, a late pin applies to the *whole* history, not just to
|
||||
/// events after it. So a scale set on the second batch must reach the same fit
|
||||
/// as one set from the very first event.
|
||||
#[test]
|
||||
fn drift_scale_is_ignored_after_first_appearance() {
|
||||
fn drift_scale_applies_when_set_after_first_appearance() {
|
||||
let mut late = History::builder()
|
||||
.mu(25.0)
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.p_draw(0.0)
|
||||
.drift(ConstantDrift(25.0 / 300.0))
|
||||
.drift(ConstantDrift::new(25.0 / 300.0))
|
||||
.convergence(CONVERGENCE)
|
||||
.build();
|
||||
|
||||
@@ -368,7 +373,7 @@ fn drift_scale_is_ignored_after_first_appearance() {
|
||||
}])
|
||||
.unwrap();
|
||||
|
||||
// Second batch asks for a pin. Too late: the competitor already exists.
|
||||
// Second batch asks for a pin. No longer too late.
|
||||
late.add_events(vec![Event {
|
||||
time: 1000,
|
||||
teams: smallvec![
|
||||
@@ -378,25 +383,115 @@ fn drift_scale_is_ignored_after_first_appearance() {
|
||||
outcome: Outcome::winner(1, 2),
|
||||
}])
|
||||
.unwrap();
|
||||
late.converge().unwrap();
|
||||
let _ = late.converge().unwrap();
|
||||
|
||||
let ignored = curve(&late, "anchor");
|
||||
let drifting = curve(&fit(distant_pair(None), 25.0 / 300.0), "anchor");
|
||||
let applied = curve(&late, "anchor");
|
||||
let pinned_from_the_start = curve(&fit(distant_pair(Some(0.0)), 25.0 / 300.0), "anchor");
|
||||
let never_pinned = curve(&fit(distant_pair(None), 25.0 / 300.0), "anchor");
|
||||
|
||||
for ((t_l, g_l), (t_r, g_r)) in ignored.iter().zip(drifting.iter()) {
|
||||
for ((t_l, g_l), (t_r, g_r)) in applied.iter().zip(pinned_from_the_start.iter()) {
|
||||
assert_eq!(t_l, t_r);
|
||||
assert!(
|
||||
(g_l.sigma() - g_r.sigma()).abs() < 1e-9,
|
||||
"a scale set after first appearance must be ignored, leaving the fit \
|
||||
identical to one that never set it: t={t_l}, {} vs {}",
|
||||
"a late pin should refit the whole history: t={t_l}, {} vs {}",
|
||||
g_l.sigma(),
|
||||
g_r.sigma()
|
||||
);
|
||||
}
|
||||
|
||||
let pinned = curve(&fit(distant_pair(Some(0.0)), 25.0 / 300.0), "anchor");
|
||||
// And it must actually have done something.
|
||||
assert!(
|
||||
(ignored[1].1.sigma() - pinned[1].1.sigma()).abs() > 1e-6,
|
||||
"sanity: the pinned fit must actually differ, or the assertion above is vacuous"
|
||||
applied
|
||||
.iter()
|
||||
.zip(never_pinned.iter())
|
||||
.any(|((_, a), (_, b))| (a.sigma() - b.sigma()).abs() > 1e-9),
|
||||
"the pin had no effect at all — the silent drop is back"
|
||||
);
|
||||
}
|
||||
|
||||
/// Re-declaring the same configuration must be inert. This is the shape a
|
||||
/// caller gets when the configuration is a property of the domain — "layouts
|
||||
/// are static" — so every ingestion path repeats it on every event.
|
||||
///
|
||||
/// Both histories see exactly the same events; only how many times the scale
|
||||
/// is declared differs.
|
||||
#[test]
|
||||
fn repeating_the_same_configuration_changes_nothing() {
|
||||
let events = |declare_every_time: bool| {
|
||||
let anchor = |first: bool| {
|
||||
if first || declare_every_time {
|
||||
Member::new("anchor").with_drift_scale(0.0)
|
||||
} else {
|
||||
Member::new("anchor")
|
||||
}
|
||||
};
|
||||
vec![
|
||||
Event {
|
||||
time: 0,
|
||||
teams: smallvec![
|
||||
Team::with_members([anchor(true)]),
|
||||
Team::with_members([Member::new("player")]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
},
|
||||
Event {
|
||||
time: 1000,
|
||||
teams: smallvec![
|
||||
Team::with_members([anchor(false)]),
|
||||
Team::with_members([Member::new("player")]),
|
||||
],
|
||||
outcome: Outcome::winner(1, 2),
|
||||
},
|
||||
]
|
||||
};
|
||||
|
||||
let once = curve(&fit(events(false), 25.0 / 300.0), "anchor");
|
||||
let every_time = curve(&fit(events(true), 25.0 / 300.0), "anchor");
|
||||
|
||||
for ((t_l, a), (t_r, b)) in once.iter().zip(every_time.iter()) {
|
||||
assert_eq!(t_l, t_r);
|
||||
assert!(
|
||||
(a.sigma() - b.sigma()).abs() < 1e-12,
|
||||
"t={t_l}: declaring the same scale repeatedly changed the fit, {} vs {}",
|
||||
a.sigma(),
|
||||
b.sigma()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_batch_that_contradicts_itself_is_rejected() {
|
||||
let mut h = History::builder().convergence(CONVERGENCE).build();
|
||||
|
||||
let err = h
|
||||
.add_events(vec![
|
||||
Event {
|
||||
time: 0,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("anchor").with_drift_scale(0.0)]),
|
||||
Team::with_members([Member::new("player")]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
},
|
||||
Event {
|
||||
time: 1,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("anchor").with_drift_scale(1.0)]),
|
||||
Team::with_members([Member::new("player")]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
},
|
||||
])
|
||||
.expect_err("two different scales for one competitor in one batch");
|
||||
|
||||
assert!(
|
||||
matches!(
|
||||
err,
|
||||
InferenceError::ConflictingCompetitorConfig {
|
||||
field: "drift_scale",
|
||||
..
|
||||
}
|
||||
),
|
||||
"got {err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
@@ -12,14 +12,21 @@ use trueskill_tt::{ConstantDrift, Game, GameOptions, Gaussian, Outcome, Rating};
|
||||
type R = Rating<i64, ConstantDrift>;
|
||||
|
||||
fn ts_rating(mu: f64, sigma: f64, beta: f64, gamma: f64) -> R {
|
||||
R::new(Gaussian::from_ms(mu, sigma), beta, ConstantDrift(gamma))
|
||||
R::new(
|
||||
Gaussian::from_ms(mu, sigma),
|
||||
beta,
|
||||
ConstantDrift::new(gamma),
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn game_1v1_golden_matches_historical() {
|
||||
let a = ts_rating(25.0, 25.0 / 3.0, 25.0 / 6.0, 25.0 / 300.0);
|
||||
let b = ts_rating(25.0, 25.0 / 3.0, 25.0 / 6.0, 25.0 / 300.0);
|
||||
let (a_post, b_post) = Game::<i64, _>::one_v_one(&a, &b, Outcome::winner(0, 2)).unwrap();
|
||||
let post = Game::<i64, _>::one_v_one(&a, &b, Outcome::winner(0, 2), &GameOptions::default())
|
||||
.unwrap()
|
||||
.posteriors();
|
||||
let (a_post, b_post) = (post[0][0], post[1][0]);
|
||||
// Historical golden from pre-T2 test_1vs1 (team 0 wins):
|
||||
assert_ulps_eq!(
|
||||
a_post,
|
||||
|
||||
@@ -0,0 +1,197 @@
|
||||
//! `EventBuilder::members` must reach exactly what the typed path reaches.
|
||||
//!
|
||||
//! Before this existed, `EventBuilder` could set weights and nothing else, so
|
||||
//! `prior` and `drift_scale` were expressible only through `Event`/`Team`/
|
||||
//! `Member` + `add_events`. Which ingestion route a competitor arrived through
|
||||
//! decided whether it could be configured at all.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, Event, Gaussian, History, InferenceError, Member, Outcome,
|
||||
Team,
|
||||
};
|
||||
|
||||
type H = History<i64, ConstantDrift, trueskill_tt::NullObserver, &'static str>;
|
||||
|
||||
fn history() -> H {
|
||||
History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.5))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build()
|
||||
}
|
||||
|
||||
const PRIOR: Gaussian = Gaussian::from_ms(3.0, 1.5);
|
||||
|
||||
/// The contract that makes the escape hatch worth having: same configuration,
|
||||
/// same fit, bit for bit.
|
||||
#[test]
|
||||
fn members_matches_the_typed_path_exactly() {
|
||||
let mut typed = history();
|
||||
typed
|
||||
.add_events(vec![Event {
|
||||
time: 1,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("player")]),
|
||||
Team::with_members([Member::new("layout_7")
|
||||
.with_drift_scale(0.0)
|
||||
.with_prior(PRIOR)]),
|
||||
],
|
||||
outcome: Outcome::scores([5.0, 2.0]),
|
||||
}])
|
||||
.unwrap();
|
||||
assert!(typed.converge().unwrap().converged);
|
||||
|
||||
let mut fluent = history();
|
||||
fluent
|
||||
.event(1)
|
||||
.team(["player"])
|
||||
.members([Member::new("layout_7")
|
||||
.with_drift_scale(0.0)
|
||||
.with_prior(PRIOR)])
|
||||
.scores([5.0, 2.0])
|
||||
.commit()
|
||||
.unwrap();
|
||||
assert!(fluent.converge().unwrap().converged);
|
||||
|
||||
for key in ["player", "layout_7"] {
|
||||
let a = typed.current_skill(&key).unwrap();
|
||||
let b = fluent.current_skill(&key).unwrap();
|
||||
// Exact equality, on the public moments rather than the natural
|
||||
// parameters: `mu` and `variance` are `tau/pi` and `1/pi`, so
|
||||
// bit-equal natural parameters give bit-equal moments.
|
||||
assert_eq!(a.mu(), b.mu(), "{key} mu");
|
||||
assert_eq!(a.variance(), b.variance(), "{key} variance");
|
||||
}
|
||||
}
|
||||
|
||||
/// The configuration has to actually take effect, not merely round-trip: a
|
||||
/// competitor pinned with `drift_scale = 0.0` must not move across slices,
|
||||
/// where an unpinned one does.
|
||||
///
|
||||
/// The comparison is against a control rather than against a fixed epsilon.
|
||||
/// Pinned marginals are not bit-identical across slices — each slice combines
|
||||
/// its own forward and backward messages, so the arithmetic order differs and
|
||||
/// the last bit moves. What "pinned" promises is that no drift variance
|
||||
/// accumulates, and the control is what makes that measurable.
|
||||
#[test]
|
||||
fn a_drift_scale_set_through_members_is_applied() {
|
||||
fn spread(h: &H, key: &'static str) -> f64 {
|
||||
let curve = h.learning_curve(&key).unwrap();
|
||||
assert!(curve.len() >= 2, "{key}: expected several appearances");
|
||||
let (lo, hi) = curve.iter().fold((f64::MAX, f64::MIN), |(lo, hi), (_, g)| {
|
||||
(lo.min(g.sigma()), hi.max(g.sigma()))
|
||||
});
|
||||
(hi - lo) / hi
|
||||
}
|
||||
|
||||
let mut h = history();
|
||||
for t in 1..=4 {
|
||||
h.event(t)
|
||||
.team(["player"])
|
||||
.members([Member::new("pinned").with_drift_scale(0.0)])
|
||||
.scores([5.0, 2.0])
|
||||
.commit()
|
||||
.unwrap();
|
||||
// Same shape, no pinning: the control.
|
||||
h.event(t)
|
||||
.team(["rival"])
|
||||
.team(["drifting"])
|
||||
.scores([5.0, 2.0])
|
||||
.commit()
|
||||
.unwrap();
|
||||
}
|
||||
assert!(h.converge().unwrap().converged);
|
||||
|
||||
let pinned = spread(&h, "pinned");
|
||||
let drifting = spread(&h, "drifting");
|
||||
assert!(pinned < 1e-9, "pinned competitor moved: {pinned:e}");
|
||||
assert!(
|
||||
drifting > 1e-3,
|
||||
"control did not move, so the test proves nothing: {drifting:e}"
|
||||
);
|
||||
}
|
||||
|
||||
/// `weights` still applies to a team added through `members`, and still
|
||||
/// records a mismatch rather than partially applying it.
|
||||
#[test]
|
||||
fn weights_still_guards_a_members_team() {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.event(1)
|
||||
.team(["a"])
|
||||
.members([Member::new("b"), Member::new("c")])
|
||||
.weights([1.0])
|
||||
.winner(0)
|
||||
.commit()
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(
|
||||
err,
|
||||
InferenceError::MismatchedShape {
|
||||
kind: "weights",
|
||||
expected: 2,
|
||||
got: 1,
|
||||
..
|
||||
}
|
||||
),
|
||||
"{err:?}"
|
||||
);
|
||||
assert!(h.current_skill(&"b").is_none(), "nothing may reach history");
|
||||
}
|
||||
|
||||
/// An invalid `drift_scale` surfaces from `commit`, not from a panic and not
|
||||
/// silently.
|
||||
#[test]
|
||||
fn an_invalid_drift_scale_surfaces_from_commit() {
|
||||
for bad in [-1.0, f64::NAN, f64::INFINITY] {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.event(1)
|
||||
.team(["a"])
|
||||
.members([Member::new("b").with_drift_scale(bad)])
|
||||
.winner(0)
|
||||
.commit()
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(
|
||||
err,
|
||||
InferenceError::InvalidParameter {
|
||||
name: "drift_scale",
|
||||
..
|
||||
}
|
||||
),
|
||||
"{bad}: {err:?}"
|
||||
);
|
||||
assert!(h.current_skill(&"b").is_none(), "{bad} reached the history");
|
||||
}
|
||||
}
|
||||
|
||||
/// `members` and `team` compose in either order.
|
||||
#[test]
|
||||
fn members_and_team_interleave() {
|
||||
let mut h = history();
|
||||
h.event(1)
|
||||
.members([Member::new("a").with_prior(PRIOR)])
|
||||
.team(["b"])
|
||||
.scores([3.0, 1.0])
|
||||
.commit()
|
||||
.unwrap();
|
||||
h.event(2)
|
||||
.team(["b"])
|
||||
.members([Member::new("c").with_prior(PRIOR)])
|
||||
.scores([2.0, 4.0])
|
||||
.commit()
|
||||
.unwrap();
|
||||
assert!(h.converge().unwrap().converged);
|
||||
for key in ["a", "b", "c"] {
|
||||
assert!(h.current_skill(&key).is_some(), "{key} missing");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
//! The evidence accessors span two independent axes — smoothed vs forward-only,
|
||||
//! all-keys vs key-restricted — and all four corners must exist and differ.
|
||||
//!
|
||||
//! `filtered_log_evidence_for` was the missing corner: the one a per-competitor
|
||||
//! prequential score needs.
|
||||
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, Event, History, InferenceError, Member, NullObserver, Outcome, Team,
|
||||
};
|
||||
|
||||
type H = History<i64, ConstantDrift, NullObserver, &'static str>;
|
||||
|
||||
/// Two disjoint cohorts, so a key restriction is guaranteed to leave events out.
|
||||
fn two_cohorts() -> H {
|
||||
let mut h = H::default();
|
||||
let mut events = Vec::new();
|
||||
for t in 1..=6 {
|
||||
for (x, y) in [("a", "b"), ("c", "d")] {
|
||||
events.push(Event {
|
||||
time: t,
|
||||
teams: [
|
||||
Team::with_members([Member::new(x)]),
|
||||
Team::with_members([Member::new(y)]),
|
||||
]
|
||||
.into_iter()
|
||||
.collect(),
|
||||
outcome: Outcome::winner(0, 2),
|
||||
});
|
||||
}
|
||||
}
|
||||
h.add_events(events).expect("fixture ingests");
|
||||
h.converge().expect("fixture converges");
|
||||
h
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn all_four_corners_are_distinct_quantities() {
|
||||
let h = two_cohorts();
|
||||
|
||||
let smoothed_all = h.log_evidence();
|
||||
let smoothed_ab = h.log_evidence_for(&[&"a", &"b"]).unwrap();
|
||||
let filtered_all = h.filtered_log_evidence();
|
||||
let filtered_ab = h.filtered_log_evidence_for(&[&"a", &"b"]).unwrap();
|
||||
|
||||
for (name, v) in [
|
||||
("smoothed_all", smoothed_all),
|
||||
("smoothed_ab", smoothed_ab),
|
||||
("filtered_all", filtered_all),
|
||||
("filtered_ab", filtered_ab),
|
||||
] {
|
||||
assert!(
|
||||
v.is_finite() && v <= 0.0,
|
||||
"{name} = {v} is not a log probability"
|
||||
);
|
||||
}
|
||||
|
||||
// Restricting to one cohort must drop the other cohort's events. Half the
|
||||
// events, and the two cohorts are symmetric, so it lands near half.
|
||||
assert!(
|
||||
smoothed_ab > smoothed_all,
|
||||
"restricting must drop evidence terms: {smoothed_ab} vs {smoothed_all}"
|
||||
);
|
||||
assert!(filtered_ab > filtered_all);
|
||||
|
||||
// The forward-only corner is a genuinely different quantity from the
|
||||
// smoothed one, not an alias for it.
|
||||
assert!(
|
||||
(filtered_ab - smoothed_ab).abs() > 1e-9,
|
||||
"filtered and smoothed restricted evidence coincide ({filtered_ab} vs {smoothed_ab}); \
|
||||
one of them is not computing what it claims"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restricting_to_both_cohorts_recovers_the_unrestricted_value() {
|
||||
let h = two_cohorts();
|
||||
|
||||
// Control on the filter itself: naming every competitor must restrict
|
||||
// nothing, so this catches a filter that drops events it should keep.
|
||||
let all_named = h
|
||||
.filtered_log_evidence_for(&[&"a", &"b", &"c", &"d"])
|
||||
.unwrap();
|
||||
assert!(
|
||||
(all_named - h.filtered_log_evidence()).abs() < 1e-12,
|
||||
"naming everyone changed the answer: {all_named} vs {}",
|
||||
h.filtered_log_evidence()
|
||||
);
|
||||
}
|
||||
|
||||
/// The restriction selects *events*, not competitors: naming one member of a
|
||||
/// pair that only ever plays each other selects the same events as naming both.
|
||||
#[test]
|
||||
fn naming_either_member_of_a_pair_selects_the_same_events() {
|
||||
let h = two_cohorts();
|
||||
|
||||
let ab = h.filtered_log_evidence_for(&[&"a"]).unwrap();
|
||||
let ab_pair = h.filtered_log_evidence_for(&[&"a", &"b"]).unwrap();
|
||||
assert!(
|
||||
(ab - ab_pair).abs() < 1e-12,
|
||||
"a and b only ever play each other, so naming either or both selects \
|
||||
the same events: {ab} vs {ab_pair}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unknown_key_is_an_error_here_too() {
|
||||
let h = two_cohorts();
|
||||
|
||||
let err = h
|
||||
.filtered_log_evidence_for(&[&"typo"])
|
||||
.expect_err("unknown key");
|
||||
assert!(matches!(err, InferenceError::UnknownKey { .. }), "{err:?}");
|
||||
|
||||
// Control: the same call on a known key succeeds.
|
||||
h.filtered_log_evidence_for(&[&"a"]).expect("a is known");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn current_skills_agrees_with_current_skill() {
|
||||
let h = two_cohorts();
|
||||
|
||||
let all = h.current_skills();
|
||||
assert_eq!(all.len(), 4, "four competitors played");
|
||||
|
||||
for key in ["a", "b", "c", "d"] {
|
||||
let one = h.current_skill(key).expect("played");
|
||||
let from_map = all[key];
|
||||
assert_eq!(
|
||||
(one.mu(), one.sigma()),
|
||||
(from_map.mu(), from_map.sigma()),
|
||||
"current_skills disagrees with current_skill for {key}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn current_skills_omits_a_registered_but_unplayed_competitor() {
|
||||
let mut h = two_cohorts();
|
||||
h.register(Member::new("e")).expect("e is new");
|
||||
|
||||
let all = h.current_skills();
|
||||
assert!(
|
||||
!all.contains_key("e"),
|
||||
"a competitor with no appearances has no posterior to report"
|
||||
);
|
||||
assert!(
|
||||
h.current_skill("e").is_none(),
|
||||
"control: the singular agrees"
|
||||
);
|
||||
assert_eq!(all.len(), 4);
|
||||
}
|
||||
+13
-13
@@ -47,7 +47,7 @@ fn tight() -> ConvergenceOptions {
|
||||
fn filtered_evidence_sits_between_coin_flip_and_batch() {
|
||||
let mut history = repeated_winner(5);
|
||||
|
||||
history.converge().unwrap();
|
||||
let _ = history.converge().unwrap();
|
||||
|
||||
let coin_flip = 5.0 * 0.5f64.ln();
|
||||
let batch = history.log_evidence();
|
||||
@@ -71,10 +71,10 @@ fn filtered_evidence_sits_between_coin_flip_and_batch() {
|
||||
fn filtered_first_point_is_less_certain_than_smoothed() {
|
||||
let mut history = repeated_winner(12);
|
||||
|
||||
history.converge().unwrap();
|
||||
let _ = history.converge().unwrap();
|
||||
|
||||
let smoothed = history.learning_curve("a");
|
||||
let filtered = history.filtered_learning_curve("a");
|
||||
let smoothed = history.learning_curve("a").unwrap();
|
||||
let filtered = history.filtered_learning_curve("a").unwrap();
|
||||
|
||||
assert_eq!(
|
||||
smoothed.len(),
|
||||
@@ -121,13 +121,13 @@ fn filtered_first_point_is_less_certain_than_smoothed() {
|
||||
fn filtered_curves_plural_agrees_with_singular() {
|
||||
let mut history = repeated_winner(4);
|
||||
|
||||
history.converge().unwrap();
|
||||
let _ = history.converge().unwrap();
|
||||
|
||||
let curves = history.filtered_learning_curves();
|
||||
|
||||
assert_eq!(
|
||||
curves["b"],
|
||||
history.filtered_learning_curve("b"),
|
||||
history.filtered_learning_curve("b").unwrap(),
|
||||
"the plural form must agree with the singular for the same key"
|
||||
);
|
||||
}
|
||||
@@ -180,10 +180,10 @@ fn single_slice_filtered_matches_smoothed() {
|
||||
])
|
||||
.unwrap();
|
||||
|
||||
history.converge().unwrap();
|
||||
let _ = history.converge().unwrap();
|
||||
|
||||
let smoothed = history.learning_curve("a");
|
||||
let filtered = history.filtered_learning_curve("a");
|
||||
let smoothed = history.learning_curve("a").unwrap();
|
||||
let filtered = history.filtered_learning_curve("a").unwrap();
|
||||
|
||||
assert_eq!(smoothed.len(), 1);
|
||||
assert_eq!(filtered.len(), 1);
|
||||
@@ -223,16 +223,16 @@ fn filtered_curves_do_not_depend_on_ingestion_order() {
|
||||
|
||||
let mut batched = History::builder().convergence(tight()).build();
|
||||
batched.add_events(all.clone()).unwrap();
|
||||
batched.converge().unwrap();
|
||||
let _ = batched.converge().unwrap();
|
||||
|
||||
let mut incremental = History::builder().convergence(tight()).build();
|
||||
for event in all {
|
||||
incremental.add_events([event]).unwrap();
|
||||
}
|
||||
incremental.converge().unwrap();
|
||||
let _ = incremental.converge().unwrap();
|
||||
|
||||
let from_batched = batched.filtered_learning_curve("a");
|
||||
let from_incremental = incremental.filtered_learning_curve("a");
|
||||
let from_batched = batched.filtered_learning_curve("a").unwrap();
|
||||
let from_incremental = incremental.filtered_learning_curve("a").unwrap();
|
||||
|
||||
assert_eq!(from_batched.len(), from_incremental.len());
|
||||
|
||||
|
||||
+169
-4
@@ -8,7 +8,7 @@ fn default_rating() -> R {
|
||||
R::new(
|
||||
Gaussian::from_ms(25.0, 25.0 / 3.0),
|
||||
25.0 / 6.0,
|
||||
ConstantDrift(25.0 / 300.0),
|
||||
ConstantDrift::new(25.0 / 300.0),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -32,14 +32,21 @@ fn game_ranked_1v1_golden() {
|
||||
fn game_one_v_one_shortcut() {
|
||||
let a = default_rating();
|
||||
let b = default_rating();
|
||||
let (a_post, b_post) = Game::<i64, _>::one_v_one(&a, &b, Outcome::winner(0, 2)).unwrap();
|
||||
let game =
|
||||
Game::<i64, _>::one_v_one(&a, &b, Outcome::winner(0, 2), &GameOptions::default()).unwrap();
|
||||
let post = game.posteriors();
|
||||
let (a_post, b_post) = (post[0][0], post[1][0]);
|
||||
assert!(a_post.mu() > 25.0);
|
||||
assert!(b_post.mu() < 25.0);
|
||||
|
||||
// It returns a game like every other constructor, so evidence is askable.
|
||||
// Two identical ratings make either result equally likely.
|
||||
assert!((game.log_evidence() - 0.5_f64.ln()).abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn game_ranked_rejects_bad_p_draw() {
|
||||
let a = R::new(Gaussian::default(), 1.0, ConstantDrift(0.0));
|
||||
let a = R::new(Gaussian::default(), 1.0, ConstantDrift::new(0.0));
|
||||
let err = Game::<i64, _>::ranked(
|
||||
&[&[a], &[a]],
|
||||
Outcome::winner(0, 2),
|
||||
@@ -55,7 +62,7 @@ fn game_ranked_rejects_bad_p_draw() {
|
||||
|
||||
#[test]
|
||||
fn game_ranked_rejects_mismatched_ranks() {
|
||||
let a = R::new(Gaussian::default(), 1.0, ConstantDrift(0.0));
|
||||
let a = R::new(Gaussian::default(), 1.0, ConstantDrift::new(0.0));
|
||||
let err = Game::<i64, _>::ranked(
|
||||
&[&[a], &[a]],
|
||||
Outcome::ranking([0, 1, 2]),
|
||||
@@ -95,3 +102,161 @@ fn game_log_evidence_is_finite() {
|
||||
assert!(g.log_evidence().is_finite());
|
||||
assert!(g.log_evidence() < 0.0);
|
||||
}
|
||||
|
||||
/// `one_v_one` used to hardcode `GameOptions::default()`, so a 1v1 could
|
||||
/// never set `p_draw` and a drawn 1v1 was unreachable through it.
|
||||
#[test]
|
||||
fn one_v_one_honours_the_draw_probability_it_is_given() {
|
||||
let a = default_rating();
|
||||
let b = default_rating();
|
||||
|
||||
// Default options still reject a draw, because the default p_draw is zero.
|
||||
let err = Game::<i64, _>::one_v_one(&a, &b, Outcome::draw(2), &GameOptions::default())
|
||||
.expect_err("a draw needs a positive p_draw");
|
||||
assert!(matches!(
|
||||
err,
|
||||
InferenceError::TieWithoutDrawProbability { .. }
|
||||
));
|
||||
|
||||
// With a draw probability supplied it succeeds — which was impossible
|
||||
// before the signature took options.
|
||||
let options = GameOptions {
|
||||
p_draw: 0.25,
|
||||
..GameOptions::default()
|
||||
};
|
||||
let post = Game::<i64, _>::one_v_one(&a, &b, Outcome::draw(2), &options)
|
||||
.expect("a draw is representable once p_draw is positive")
|
||||
.posteriors();
|
||||
let (a_post, b_post) = (post[0][0], post[1][0]);
|
||||
|
||||
// A symmetric draw leaves the means alone and sharpens both sides.
|
||||
assert!((a_post.mu() - b_post.mu()).abs() < 1e-9);
|
||||
assert!(a_post.sigma() < 25.0 / 3.0);
|
||||
}
|
||||
|
||||
/// Convergence options reach the 1v1 path too, not just `p_draw`.
|
||||
#[test]
|
||||
fn one_v_one_honours_convergence_options() {
|
||||
let a = default_rating();
|
||||
let b = default_rating();
|
||||
let options = GameOptions {
|
||||
convergence: ConvergenceOptions::default(),
|
||||
..GameOptions::default()
|
||||
};
|
||||
let post = Game::<i64, _>::one_v_one(&a, &b, Outcome::winner(0, 2), &options)
|
||||
.unwrap()
|
||||
.posteriors();
|
||||
assert!(post[0][0].mu() > 25.0);
|
||||
}
|
||||
|
||||
/// `Game` is a public entry point that does not pass through `History`'s
|
||||
/// ingestion chokepoint, so it needs its own boundary — and did not have one.
|
||||
///
|
||||
/// A one-team game panicked at `src/game.rs:317` with "range start index 1 out
|
||||
/// of range for slice of length 0", in release, from safe API. This is the
|
||||
/// same defect `tests/ingestion_shape.rs` covers for `History`; fixing that
|
||||
/// path left this one open, because they share no validation.
|
||||
mod malformed_games {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_one_team_ranked_game_is_an_error_not_a_panic() {
|
||||
let a = default_rating();
|
||||
let err = Game::<i64, _>::ranked(&[&[a]], Outcome::winner(0, 1), &GameOptions::default())
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::NotEnoughTeams { got: 1, .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_one_team_scored_game_is_an_error_not_a_panic() {
|
||||
let a = default_rating();
|
||||
let err = Game::<i64, _>::scored(
|
||||
&[&[a]],
|
||||
Outcome::scores([1.0]),
|
||||
&GameOptions {
|
||||
score_sigma: 1.0,
|
||||
..GameOptions::default()
|
||||
},
|
||||
)
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::NotEnoughTeams { got: 1, .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_zero_team_game_is_an_error() {
|
||||
let err =
|
||||
Game::<i64, ConstantDrift>::ranked(&[], Outcome::ranking([]), &GameOptions::default())
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::NotEnoughTeams { got: 0, .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// The quiet half: an empty team contributed no performance, so the game
|
||||
/// returned a finite posterior for its opponent as though it had won one.
|
||||
#[test]
|
||||
fn an_empty_team_is_an_error() {
|
||||
let a = default_rating();
|
||||
let err =
|
||||
Game::<i64, _>::ranked(&[&[], &[a]], Outcome::winner(0, 2), &GameOptions::default())
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::EmptyTeam { team: 0, .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_non_finite_score_is_an_error() {
|
||||
let a = default_rating();
|
||||
for bad in [f64::NAN, f64::INFINITY, f64::NEG_INFINITY] {
|
||||
let err = Game::<i64, _>::scored(
|
||||
&[&[a], &[a]],
|
||||
Outcome::scores([bad, 1.0]),
|
||||
&GameOptions {
|
||||
score_sigma: 1.0,
|
||||
..GameOptions::default()
|
||||
},
|
||||
)
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::InvalidParameter { name: "score", .. }),
|
||||
"{bad}: {err:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// `free_for_all` and `one_v_one` build their teams internally, so they
|
||||
/// must keep working — the check must not catch well-formed games.
|
||||
#[test]
|
||||
fn well_formed_games_are_untouched() {
|
||||
let a = default_rating();
|
||||
assert!(
|
||||
Game::<i64, _>::ranked(
|
||||
&[&[a], &[a]],
|
||||
Outcome::winner(0, 2),
|
||||
&GameOptions::default()
|
||||
)
|
||||
.is_ok()
|
||||
);
|
||||
assert!(
|
||||
Game::<i64, _>::free_for_all(
|
||||
&[&a, &a, &a],
|
||||
Outcome::ranking([0, 1, 2]),
|
||||
&GameOptions::default()
|
||||
)
|
||||
.is_ok()
|
||||
);
|
||||
assert!(
|
||||
Game::<i64, _>::one_v_one(&a, &a, Outcome::winner(0, 2), &GameOptions::default())
|
||||
.is_ok()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
//! Per-key queries must distinguish "I have never heard of this key" from a
|
||||
//! genuine, empty-but-real answer.
|
||||
//!
|
||||
//! Each test carries a control: the same call on a key the history *does* know,
|
||||
//! so it cannot pass merely because everything returns the same thing.
|
||||
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, Event, History, InferenceError, Member, NullObserver, Outcome, Team,
|
||||
};
|
||||
|
||||
type H = History<i64, ConstantDrift, NullObserver, &'static str>;
|
||||
|
||||
fn history() -> H {
|
||||
let mut h = H::default();
|
||||
h.add_events((1..=4).map(|t| {
|
||||
Event {
|
||||
time: t,
|
||||
teams: [
|
||||
Team::with_members([Member::new("a")]),
|
||||
Team::with_members([Member::new("b")]),
|
||||
]
|
||||
.into_iter()
|
||||
.collect(),
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}
|
||||
}))
|
||||
.expect("fixture ingests");
|
||||
h.converge().expect("fixture converges");
|
||||
h
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn learning_curve_separates_unknown_from_unplayed() {
|
||||
let mut h = history();
|
||||
|
||||
assert!(h.learning_curve("typo").is_none(), "unknown key is None");
|
||||
assert_eq!(
|
||||
h.learning_curve("a").expect("a is known").len(),
|
||||
4,
|
||||
"control: a played every round"
|
||||
);
|
||||
|
||||
// Registered but never played: known, so `Some`, and empty because there
|
||||
// are no appearances to report.
|
||||
h.register(Member::new("c")).expect("c is new");
|
||||
assert_eq!(
|
||||
h.learning_curve("c").expect("c is registered"),
|
||||
vec![],
|
||||
"registered-but-unplayed is an empty curve, not None"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn filtered_learning_curve_separates_unknown_from_unplayed() {
|
||||
let mut h = history();
|
||||
|
||||
assert!(h.filtered_learning_curve("typo").is_none());
|
||||
assert_eq!(
|
||||
h.filtered_learning_curve("a").expect("a is known").len(),
|
||||
4,
|
||||
"control"
|
||||
);
|
||||
|
||||
h.register(Member::new("c")).expect("c is new");
|
||||
assert_eq!(
|
||||
h.filtered_learning_curve("c").expect("c is registered"),
|
||||
vec![]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn log_evidence_for_rejects_unknown_keys() {
|
||||
let h = history();
|
||||
|
||||
// The defect this guards: an all-unknown target list left the internal
|
||||
// filter empty, which means "no restriction" — so the call returned the
|
||||
// whole-history evidence, a plausible number that silently invalidates the
|
||||
// leave-one-out comparison it was computed for.
|
||||
let whole = h.log_evidence();
|
||||
let err = h
|
||||
.log_evidence_for(&[&"typo"])
|
||||
.expect_err("unknown key is an error");
|
||||
assert!(
|
||||
matches!(err, InferenceError::UnknownKey { .. }),
|
||||
"expected UnknownKey, got {err:?}"
|
||||
);
|
||||
|
||||
// Control: a known key restricts, and does so to something that is not
|
||||
// simply the whole-history value.
|
||||
let restricted = h.log_evidence_for(&[&"a"]).expect("a is known");
|
||||
assert!(restricted.is_finite());
|
||||
assert!(restricted <= 0.0);
|
||||
let _ = whole;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn log_evidence_for_rejects_a_mix_of_known_and_unknown() {
|
||||
let h = history();
|
||||
|
||||
let err = h
|
||||
.log_evidence_for(&[&"a", &"typo"])
|
||||
.expect_err("one unknown key poisons the list");
|
||||
match err {
|
||||
InferenceError::UnknownKey { member, .. } => {
|
||||
assert_eq!(member, 1, "the reported position is the offending key's");
|
||||
}
|
||||
other => panic!("expected UnknownKey, got {other:?}"),
|
||||
}
|
||||
|
||||
h.log_evidence_for(&[&"a", &"b"])
|
||||
.expect("control: both known");
|
||||
}
|
||||
@@ -30,9 +30,27 @@ fn event(a: &str, b: &str, time: i64) -> Event<i64, String> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Like [`event`], but `a` carries competitor configuration.
|
||||
///
|
||||
/// `prior` and `drift_scale` configure the competitor rather than the event, so
|
||||
/// they are the part of ingestion most exposed to order: they are consumed once,
|
||||
/// where the competitor's state is written.
|
||||
fn configured_event(a: &str, b: &str, time: i64, scale: f64) -> Event<i64, String> {
|
||||
Event {
|
||||
time,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(a.to_string()).with_drift_scale(scale)]),
|
||||
Team::with_members([Member::new(b.to_string())]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}
|
||||
}
|
||||
|
||||
fn converged_skills(events: Vec<Event<i64, String>>, batched: bool) -> Vec<(String, Gaussian)> {
|
||||
let mut h: History<i64, _, _, String> =
|
||||
History::builder_with_key().convergence(tight()).build();
|
||||
let mut h: History<i64, _, _, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.convergence(tight())
|
||||
.build();
|
||||
|
||||
if batched {
|
||||
h.add_events(events).unwrap();
|
||||
@@ -145,3 +163,65 @@ fn back_dated_event_matches_batched() {
|
||||
let incremental = converged_skills(events, false);
|
||||
assert_same(&batched, &incremental, "back-dated event");
|
||||
}
|
||||
|
||||
/// The invariant this file protects was only ever checked for *unconfigured*
|
||||
/// competitors — every helper above built members with `Member::new`.
|
||||
///
|
||||
/// Configuration is the part most exposed to ordering, because it is consumed
|
||||
/// once at the point the competitor's state is written rather than replayed per
|
||||
/// event. These cover it.
|
||||
#[test]
|
||||
fn configured_competitors_are_order_independent() {
|
||||
let events = vec![
|
||||
configured_event("a", "b", 0, 0.0),
|
||||
configured_event("a", "c", 1, 0.0),
|
||||
configured_event("a", "b", 2, 0.0),
|
||||
event("b", "c", 3),
|
||||
];
|
||||
|
||||
assert_same(
|
||||
&converged_skills(events.clone(), true),
|
||||
&converged_skills(events, false),
|
||||
"configuration repeated on every appearance",
|
||||
);
|
||||
}
|
||||
|
||||
/// Configuration supplied only on a *later* event is the case that used to be
|
||||
/// silently dropped. It must now reach the same fit either way it is ingested.
|
||||
#[test]
|
||||
fn late_configuration_is_order_independent() {
|
||||
let events = vec![
|
||||
event("a", "b", 0),
|
||||
configured_event("a", "c", 1, 0.0),
|
||||
event("a", "b", 2),
|
||||
];
|
||||
|
||||
assert_same(
|
||||
&converged_skills(events.clone(), true),
|
||||
&converged_skills(events, false),
|
||||
"configuration supplied after first appearance",
|
||||
);
|
||||
}
|
||||
|
||||
/// And it must actually be doing something — an implementation that dropped
|
||||
/// configuration entirely would pass both tests above.
|
||||
#[test]
|
||||
fn configuration_changes_the_fit_however_it_is_ingested() {
|
||||
let configured = vec![
|
||||
event("a", "b", 0),
|
||||
configured_event("a", "c", 1, 0.0),
|
||||
event("a", "b", 2),
|
||||
];
|
||||
let plain = vec![event("a", "b", 0), event("a", "c", 1), event("a", "b", 2)];
|
||||
|
||||
for batched in [true, false] {
|
||||
let with = converged_skills(configured.clone(), batched);
|
||||
let without = converged_skills(plain.clone(), batched);
|
||||
assert!(
|
||||
with.iter()
|
||||
.zip(&without)
|
||||
.any(|((_, x), (_, y))| (x.sigma() - y.sigma()).abs() > 1e-9),
|
||||
"batched={batched}: configuration had no effect, so the order tests are vacuous"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
//! Malformed events must be rejected at the ingestion boundary.
|
||||
//!
|
||||
//! Every case here was reachable from safe public API in a release build. Two
|
||||
//! of them are the two shapes this crate's defects keep taking: a panic from
|
||||
//! deep inside inference, and a finite, plausible-looking posterior computed
|
||||
//! from an event that should never have been accepted.
|
||||
//!
|
||||
//! `InferenceError::NotEnoughTeams` and `EmptyTeam` already existed when these
|
||||
//! were found — they were checked on the prediction paths and nowhere else, so
|
||||
//! ingestion could still manufacture the states they describe.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{Event, History, InferenceError, Member, Outcome, Team};
|
||||
|
||||
type Ev = Event<i64, &'static str>;
|
||||
|
||||
fn history() -> History<i64, trueskill_tt::ConstantDrift, trueskill_tt::NullObserver, &'static str>
|
||||
{
|
||||
History::builder().score_sigma(1.0).build()
|
||||
}
|
||||
|
||||
fn teams(names: &[&[&'static str]]) -> smallvec::SmallVec<[Team<&'static str>; 4]> {
|
||||
names
|
||||
.iter()
|
||||
.map(|team| Team::with_members(team.iter().map(|k| Member::new(*k))))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The regression this file exists for: `run_chain` builds one diff link per
|
||||
/// adjacent pair of teams, so a one-team event left it indexing `links[1..]`
|
||||
/// on an empty vector and panicked — in release, from `History::add_events`.
|
||||
#[test]
|
||||
fn a_one_team_event_is_an_error_not_a_panic() {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.add_events(vec![Ev {
|
||||
time: 1,
|
||||
teams: teams(&[&["a"]]),
|
||||
outcome: Outcome::winner(0, 1),
|
||||
}])
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::NotEnoughTeams { got: 1, .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_zero_team_event_is_an_error() {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.add_events(vec![Ev {
|
||||
time: 1,
|
||||
teams: smallvec![],
|
||||
outcome: Outcome::ranking([]),
|
||||
}])
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::NotEnoughTeams { got: 0, .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// The quiet half. An empty team contributes no performance, so before this
|
||||
/// was rejected the event converged and handed back a finite posterior for its
|
||||
/// opponent — a plausible constant computed from nothing.
|
||||
#[test]
|
||||
fn an_empty_team_is_an_error_rather_than_a_free_win() {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.add_events(vec![Ev {
|
||||
time: 1,
|
||||
teams: teams(&[&[], &["b"]]),
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}])
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::EmptyTeam { team: 0, .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
// Nothing was recorded, so the history is still empty.
|
||||
assert!(h.current_skill(&"b").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_empty_team_is_reported_by_position() {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.add_events(vec![Ev {
|
||||
time: 1,
|
||||
teams: teams(&[&["a"], &[]]),
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}])
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::EmptyTeam { team: 1, .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// A NaN score used to ingest cleanly. `converge` reported `NonFiniteResult`,
|
||||
/// but a caller who read `current_skill` first was handed `tau: NaN` with
|
||||
/// nothing to say so.
|
||||
#[test]
|
||||
fn a_non_finite_score_is_rejected_at_ingestion() {
|
||||
for bad in [f64::NAN, f64::INFINITY, f64::NEG_INFINITY] {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.add_events(vec![Ev {
|
||||
time: 1,
|
||||
teams: teams(&[&["a"], &["b"]]),
|
||||
outcome: Outcome::scores([bad, 0.0]),
|
||||
}])
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::InvalidParameter { name: "score", .. }),
|
||||
"{bad}: {err:?}"
|
||||
);
|
||||
assert!(h.current_skill(&"a").is_none(), "{bad} was recorded anyway");
|
||||
}
|
||||
}
|
||||
|
||||
/// A non-finite weight behaved exactly as `0.0` — the member contributed
|
||||
/// nothing — while `converge` reported `converged: true` after one iteration
|
||||
/// with a step of `(0.0, 0.0)`. So a NaN arriving from a division or a parse
|
||||
/// was indistinguishable from a deliberate zero, and looked like a clean fit.
|
||||
#[test]
|
||||
fn a_non_finite_weight_is_rejected_at_ingestion() {
|
||||
for bad in [f64::NAN, f64::INFINITY, f64::NEG_INFINITY] {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.event(1)
|
||||
.team(["a"])
|
||||
.weights([bad])
|
||||
.team(["b"])
|
||||
.winner(0)
|
||||
.commit()
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::InvalidParameter { name: "weight", .. }),
|
||||
"{bad}: {err:?}"
|
||||
);
|
||||
assert!(h.current_skill(&"a").is_none(), "{bad} reached the history");
|
||||
}
|
||||
}
|
||||
|
||||
/// Zero and negative weights are expressible choices about how much a member
|
||||
/// contributes, not malformed input, and `tests/degenerate_inputs.rs` pins
|
||||
/// their behaviour deliberately. Rejecting non-finite values must not catch
|
||||
/// them too.
|
||||
#[test]
|
||||
fn zero_and_negative_weights_still_ingest() {
|
||||
for w in [0.0, -1.0, 0.5] {
|
||||
let mut h = history();
|
||||
h.event(1)
|
||||
.team(["a"])
|
||||
.weights([w])
|
||||
.team(["b"])
|
||||
.winner(0)
|
||||
.commit()
|
||||
.unwrap_or_else(|e| panic!("weight {w} should ingest: {e:?}"));
|
||||
assert!(h.current_skill(&"a").is_some(), "weight {w}");
|
||||
}
|
||||
}
|
||||
|
||||
/// The fluent builder routes through the same chokepoint, so it inherits the
|
||||
/// checks rather than needing its own.
|
||||
#[test]
|
||||
fn the_event_builder_inherits_the_shape_checks() {
|
||||
let mut h = history();
|
||||
let err = h.event(1).team(["a"]).winner(0).commit().unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::NotEnoughTeams { got: 1, .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// A well-formed event is untouched by any of this.
|
||||
#[test]
|
||||
fn a_well_formed_event_still_ingests() {
|
||||
let mut h = history();
|
||||
h.add_events(vec![Ev {
|
||||
time: 1,
|
||||
teams: teams(&[&["a"], &["b"]]),
|
||||
outcome: Outcome::scores([3.0, 1.0]),
|
||||
}])
|
||||
.unwrap();
|
||||
assert!(h.converge().unwrap().converged);
|
||||
assert!(h.current_skill(&"a").unwrap().mu() > h.current_skill(&"b").unwrap().mu());
|
||||
}
|
||||
@@ -0,0 +1,349 @@
|
||||
//! `History::joint` factorises once and answers many questions.
|
||||
//!
|
||||
//! The contract that matters is *identity*: a `Joint` must return exactly what
|
||||
//! the one-shot call returns, bit for bit. A faster path that quietly disagreed
|
||||
//! with the slow one would be worse than no fast path — a caller would get
|
||||
//! different numbers depending on how many questions they happened to ask.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, Event, History, InferenceError, Member, Outcome, Team,
|
||||
UnknownKeys,
|
||||
};
|
||||
|
||||
type H = History<i64, ConstantDrift, trueskill_tt::NullObserver, &'static str>;
|
||||
|
||||
fn duel(a: &'static str, b: &'static str, t: i64, sa: f64, sb: f64) -> Event<i64, &'static str> {
|
||||
Event {
|
||||
time: t,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(a)]),
|
||||
Team::with_members([Member::new(b)]),
|
||||
],
|
||||
outcome: Outcome::scores([sa, sb]),
|
||||
}
|
||||
}
|
||||
|
||||
fn ranked(a: &'static str, b: &'static str, t: i64) -> Event<i64, &'static str> {
|
||||
Event {
|
||||
time: t,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(a)]),
|
||||
Team::with_members([Member::new(b)]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}
|
||||
}
|
||||
|
||||
fn history(unknown: UnknownKeys) -> H {
|
||||
History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.5))
|
||||
.unknown_keys(unknown)
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build()
|
||||
}
|
||||
|
||||
/// Several slices, competitors with different last appearances, so `latest`
|
||||
/// and `at_slice` both have work to do.
|
||||
fn fitted(unknown: UnknownKeys) -> H {
|
||||
let mut h = history(unknown);
|
||||
h.add_events(vec![
|
||||
duel("a", "b", 1, 5.0, 2.0),
|
||||
duel("c", "d", 1, 3.0, 3.5),
|
||||
duel("a", "c", 2, 6.0, 1.0),
|
||||
duel("b", "d", 3, 4.0, 3.0),
|
||||
duel("a", "d", 4, 7.0, 2.0),
|
||||
duel("b", "c", 5, 2.0, 4.0),
|
||||
])
|
||||
.unwrap();
|
||||
let report = h.converge().unwrap();
|
||||
assert!(report.converged, "fixture must converge");
|
||||
h
|
||||
}
|
||||
|
||||
const PAIRS: [(&str, &str); 6] = [
|
||||
("a", "b"),
|
||||
("a", "c"),
|
||||
("a", "d"),
|
||||
("b", "c"),
|
||||
("b", "d"),
|
||||
("c", "d"),
|
||||
];
|
||||
|
||||
/// A joint reused across questions answers exactly what a fresh one per
|
||||
/// question does. That is the whole correctness claim behind caching the
|
||||
/// factorisation (#51); it used to be checked against the `History` one-shot
|
||||
/// wrappers, which were deleted in #78, so it is checked against a fresh
|
||||
/// factorisation instead — the same comparison, without the wrapper.
|
||||
#[test]
|
||||
fn a_reused_joint_answers_exactly_what_a_fresh_one_does() {
|
||||
let h = fitted(UnknownKeys::Reject);
|
||||
let joint = h.joint().unwrap();
|
||||
|
||||
for (a, b) in PAIRS {
|
||||
let terms = [(&a, 1.0), (&b, -1.0)];
|
||||
let one_shot = h.joint().unwrap().posterior_of(&terms).unwrap();
|
||||
let cached = joint.posterior_of(&terms).unwrap();
|
||||
assert_eq!(one_shot.mu(), cached.mu(), "{a} - {b}");
|
||||
assert_eq!(one_shot.variance(), cached.variance(), "{a} - {b}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_joint_agrees_at_a_pinned_time_too() {
|
||||
let h = fitted(UnknownKeys::Reject);
|
||||
let joint = h.joint().unwrap();
|
||||
|
||||
for time in 1..=5 {
|
||||
for (a, b) in PAIRS {
|
||||
let terms = [(&a, 1.0), (&b, -1.0)];
|
||||
let one_shot = h.joint().unwrap().posterior_of_at(time, &terms);
|
||||
let cached = joint.posterior_of_at(time, &terms);
|
||||
match (one_shot, cached) {
|
||||
(Ok(x), Ok(y)) => {
|
||||
assert_eq!(x.mu(), y.mu(), "t={time} {a} - {b}");
|
||||
assert_eq!(x.variance(), y.variance(), "t={time} {a} - {b}");
|
||||
}
|
||||
(Err(x), Err(y)) => assert_eq!(x, y, "t={time} {a} - {b}"),
|
||||
(x, y) => panic!("t={time} {a} - {b}: disagreed on success: {x:?} vs {y:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_joint_scores_candidate_matchups_identically() {
|
||||
let h = fitted(UnknownKeys::Reject);
|
||||
let joint = h.joint().unwrap();
|
||||
let (a, b) = ("a", "b");
|
||||
let target = [(&a, 1.0), (&b, -1.0)];
|
||||
|
||||
for (x, y) in PAIRS {
|
||||
let teams: [&[&&str]; 2] = [&[&x], &[&y]];
|
||||
let one_shot = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.expected_variance_reduction(&teams, &target)
|
||||
.unwrap();
|
||||
let cached = joint.expected_variance_reduction(&teams, &target).unwrap();
|
||||
assert_eq!(one_shot, cached, "{x} vs {y}");
|
||||
}
|
||||
}
|
||||
|
||||
/// The whole point: a competitor appears once per slice, so the joint is over
|
||||
/// appearances rather than competitors, and a caller sizing a batch needs to
|
||||
/// know which.
|
||||
#[test]
|
||||
fn variables_counts_appearances_not_competitors() {
|
||||
let h = fitted(UnknownKeys::Reject);
|
||||
let joint = h.joint().unwrap();
|
||||
// Four competitors, twelve appearances across five slices, all with
|
||||
// positive drift between them, so no two collapse.
|
||||
assert_eq!(joint.variables(), 12);
|
||||
}
|
||||
|
||||
/// How much the collapse is worth, which is the part a caller has to plan
|
||||
/// around: a drift-free competitor contributes **one** variable however long
|
||||
/// the history, so the same events at `gamma = 0` and `gamma > 0` differ by
|
||||
/// roughly the slice count in problem size — and by its cube in solve time.
|
||||
///
|
||||
/// Reported by a consumer as an 8x difference in solve time on a ~2,000-node,
|
||||
/// 76-slice model (787 ms career against 6,214 ms drifting). This pins the
|
||||
/// mechanism behind that so a change to the collapse rule cannot quietly
|
||||
/// remove it.
|
||||
#[test]
|
||||
fn drift_free_competitors_shrink_the_joint_by_the_slice_count() {
|
||||
fn variables(gamma: f64) -> usize {
|
||||
let mut h = History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(gamma))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
h.add_events(
|
||||
(1..=10)
|
||||
.map(|t| duel("a", "b", t, 5.0, 2.0))
|
||||
.collect::<Vec<_>>(),
|
||||
)
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h.joint().unwrap().variables()
|
||||
}
|
||||
|
||||
let drifting = variables(0.5);
|
||||
let career = variables(0.0);
|
||||
|
||||
// Two competitors over ten slices: twenty appearances, or two variables.
|
||||
assert_eq!(drifting, 20);
|
||||
assert_eq!(career, 2);
|
||||
assert_eq!(
|
||||
drifting / career,
|
||||
10,
|
||||
"collapse should track the slice count"
|
||||
);
|
||||
}
|
||||
|
||||
/// With `drift = 0` consecutive appearances are the same latent variable, so
|
||||
/// the joint is smaller than the appearance count.
|
||||
#[test]
|
||||
fn pinned_competitors_collapse_consecutive_appearances() {
|
||||
let mut h = History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
h.add_events(vec![
|
||||
duel("a", "b", 1, 5.0, 2.0),
|
||||
duel("a", "b", 2, 4.0, 3.0),
|
||||
duel("a", "b", 3, 6.0, 1.0),
|
||||
])
|
||||
.unwrap();
|
||||
assert!(h.converge().unwrap().converged);
|
||||
assert_eq!(h.joint().unwrap().variables(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_ranked_history_has_no_exact_joint() {
|
||||
let mut h = history(UnknownKeys::Reject);
|
||||
h.add_events(vec![duel("a", "b", 1, 5.0, 2.0), ranked("a", "b", 2)])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
assert!(matches!(
|
||||
h.joint().unwrap_err(),
|
||||
InferenceError::JointUnavailable { .. }
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_empty_history_has_no_joint() {
|
||||
let h = history(UnknownKeys::Reject);
|
||||
assert!(matches!(
|
||||
h.joint().unwrap_err(),
|
||||
InferenceError::JointUnavailable { .. }
|
||||
));
|
||||
}
|
||||
|
||||
/// Unknown keys are decided per query, not when the joint is factorised — the
|
||||
/// factorisation does not depend on the question.
|
||||
#[test]
|
||||
fn unknown_keys_are_rejected_per_query() {
|
||||
let h = fitted(UnknownKeys::Reject);
|
||||
let joint = h.joint().unwrap();
|
||||
let (a, z) = ("a", "nobody");
|
||||
assert!(matches!(
|
||||
joint.posterior_of(&[(&a, 1.0), (&z, -1.0)]).unwrap_err(),
|
||||
InferenceError::UnknownKey { .. }
|
||||
));
|
||||
// The handle is still usable afterwards.
|
||||
let b = "b";
|
||||
assert!(joint.posterior_of(&[(&a, 1.0), (&b, -1.0)]).is_ok());
|
||||
}
|
||||
|
||||
/// Under `Prior`, an unseen competitor is independent of everything in the
|
||||
/// history, and a reused joint must add the same prior variance a fresh one
|
||||
/// does.
|
||||
#[test]
|
||||
fn unseen_competitors_match_a_fresh_factorisation() {
|
||||
let h = fitted(UnknownKeys::Prior);
|
||||
let joint = h.joint().unwrap();
|
||||
let (a, z) = ("a", "nobody");
|
||||
let terms = [(&a, 1.0), (&z, -1.0)];
|
||||
let one_shot = h.joint().unwrap().posterior_of(&terms).unwrap();
|
||||
let cached = joint.posterior_of(&terms).unwrap();
|
||||
assert_eq!(one_shot.mu(), cached.mu());
|
||||
assert_eq!(one_shot.variance(), cached.variance());
|
||||
}
|
||||
|
||||
/// A drift too small to represent must collapse, not corrupt the matrix.
|
||||
///
|
||||
/// The collapse rule used to fire only at `drift <= 0.0` exactly. Anything
|
||||
/// smaller-but-positive got an explicit `1.0 / drift` precision, and at
|
||||
/// `drift = 1e-16` that entry is `1e16` — so `1e16 + 0.28` rounds back to
|
||||
/// `1e16` and the prior and contrasts are annihilated in the stored `f64`.
|
||||
///
|
||||
/// Measured before the fix, at `drift_scale = 1e-10` this returned a variance
|
||||
/// **12 000x too small** (a 111x overconfident interval) as `Ok`, with a band
|
||||
/// just above it returning a misleading `JointUnavailable`.
|
||||
#[test]
|
||||
fn a_drift_too_small_to_represent_collapses_rather_than_corrupting() {
|
||||
fn variance(scale: f64) -> f64 {
|
||||
let mut h: History<i64, ConstantDrift, _, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.5))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
let mut events = Vec::new();
|
||||
for t in 0..15i64 {
|
||||
for k in 0..4usize {
|
||||
let x = format!("p{}", (t as usize * 4 + k) % 8);
|
||||
let y = format!("p{}", (t as usize * 4 + k + 3) % 8);
|
||||
events.push(Event {
|
||||
time: t,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(x).with_drift_scale(scale)]),
|
||||
Team::with_members([Member::new(y).with_drift_scale(scale)]),
|
||||
],
|
||||
outcome: Outcome::scores([3.0, 1.0]),
|
||||
});
|
||||
}
|
||||
}
|
||||
h.add_events(events).unwrap();
|
||||
assert!(h.converge().unwrap().converged);
|
||||
let (a, b) = ("p0".to_string(), "p1".to_string());
|
||||
let joint = h
|
||||
.joint()
|
||||
.expect("a tiny drift must not make the joint unavailable");
|
||||
let g = joint.posterior_of(&[(&a, 1.0), (&b, -1.0)]).unwrap();
|
||||
g.sigma() * g.sigma()
|
||||
}
|
||||
|
||||
let collapsed = variance(0.0);
|
||||
|
||||
// Below the threshold every scale must reach the collapsed answer exactly,
|
||||
// and none may error.
|
||||
for scale in [1e-3, 1e-4, 1e-6, 1e-8, 1e-10, 1e-12] {
|
||||
let v = variance(scale);
|
||||
assert_eq!(
|
||||
v.to_bits(),
|
||||
collapsed.to_bits(),
|
||||
"drift_scale {scale:e}: {v} vs collapsed {collapsed}"
|
||||
);
|
||||
}
|
||||
|
||||
// Above it, real drift is still modelled — otherwise this test would pass
|
||||
// by collapsing everything.
|
||||
let drifting = variance(1e-2);
|
||||
assert!(
|
||||
(drifting - collapsed).abs() / collapsed > 1e-5,
|
||||
"a drift of 1e-2 must still move the answer: {drifting} vs {collapsed}"
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
//! The realistic program: keys arrive owned, queries are written with literals.
|
||||
//!
|
||||
//! Every prediction and joint query used to take `&[&[&K]]`, which at
|
||||
//! `K = String` made a string literal *impossible* — the shape required three
|
||||
//! levels of temporaries that all had to outlive the call. They are generic
|
||||
//! over the borrowed key now, so one spelling works at both key types.
|
||||
//!
|
||||
//! Both key types are exercised in every test, because the point is that the
|
||||
//! spelling is the same.
|
||||
|
||||
use trueskill_tt::{ConstantDrift, History, NullObserver};
|
||||
|
||||
type Owned = History<i64, ConstantDrift, NullObserver, String>;
|
||||
type Borrowed = History<i64, ConstantDrift, NullObserver, &'static str>;
|
||||
|
||||
fn owned() -> Owned {
|
||||
let mut h: Owned = History::builder().key_type::<String>().build();
|
||||
for t in 1..=4 {
|
||||
h.record_winner(&"alice".to_string(), &"bob".to_string(), t)
|
||||
.expect("ingests");
|
||||
}
|
||||
h.converge().expect("converges");
|
||||
h
|
||||
}
|
||||
|
||||
fn borrowed() -> Borrowed {
|
||||
let mut h = History::default();
|
||||
for t in 1..=4 {
|
||||
h.record_winner(&"alice", &"bob", t).expect("ingests");
|
||||
}
|
||||
h.converge().expect("converges");
|
||||
h
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn predictions_take_literals_at_either_key_type() {
|
||||
let teams: &[&[&str]] = &[&["alice"], &["bob"]];
|
||||
|
||||
let a = owned()
|
||||
.predict_win_probabilities(teams)
|
||||
.expect("K = String");
|
||||
let b = borrowed()
|
||||
.predict_win_probabilities(teams)
|
||||
.expect("K = &'static str");
|
||||
|
||||
assert_eq!(a, b, "the same fit through the same spelling");
|
||||
assert!(a[0] > a[1], "alice won every game");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_team_shaped_query_accepts_the_same_slice() {
|
||||
let h = owned();
|
||||
let teams: &[&[&str]] = &[&["alice"], &["bob"]];
|
||||
|
||||
h.quality(teams).expect("quality");
|
||||
let _ = h.predict_outcome(teams).expect("outcome");
|
||||
h.predict_ranking(teams, &[0, 1]).expect("ranking");
|
||||
h.expected_information_gain(teams)
|
||||
.expect("information gain");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn linear_combinations_take_bare_keys() {
|
||||
// `&[(&K, f64)]` at `K = String` meant `&[(&String, f64)]` — no literals.
|
||||
// A scored history, because the joint needs one.
|
||||
let mut h: Owned = History::builder().key_type::<String>().build();
|
||||
for t in 1..=4 {
|
||||
h.event(t)
|
||||
.team([String::from("alice")])
|
||||
.team([String::from("bob")])
|
||||
.scores([21.0, 9.0])
|
||||
.commit()
|
||||
.expect("ingests");
|
||||
}
|
||||
h.converge().expect("converges");
|
||||
|
||||
let terms: &[(&str, f64)] = &[("alice", 1.0), ("bob", -1.0)];
|
||||
let gap = h
|
||||
.joint()
|
||||
.expect("scored history has a joint")
|
||||
.posterior_of(terms)
|
||||
.expect("both keys are known");
|
||||
|
||||
assert!(gap.mu() > 0.0, "alice outscored bob every round");
|
||||
}
|
||||
|
||||
/// `lookup` is gone with `Index` (#73); the accessors that answer the same
|
||||
/// question all take a borrowed key.
|
||||
#[test]
|
||||
fn membership_queries_accept_a_borrowed_key() {
|
||||
let h = owned();
|
||||
assert!(h.current_skill("alice").is_some());
|
||||
assert!(h.rating("alice").is_some());
|
||||
assert!(h.learning_curve("alice").is_some());
|
||||
|
||||
assert!(h.current_skill("nobody").is_none());
|
||||
assert!(h.rating("nobody").is_none());
|
||||
assert!(h.learning_curve("nobody").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gamma_sets_drift_without_naming_constant_drift() {
|
||||
let mut a: Borrowed = History::builder().gamma(0.5).build();
|
||||
let mut b: Borrowed = History::builder().drift(ConstantDrift::new(0.5)).build();
|
||||
|
||||
for h in [&mut a, &mut b] {
|
||||
h.record_winner(&"x", &"y", 1).unwrap();
|
||||
h.record_winner(&"y", &"x", 100).unwrap();
|
||||
h.converge().unwrap();
|
||||
}
|
||||
|
||||
let (ga, gb) = (a.current_skill("x").unwrap(), b.current_skill("x").unwrap());
|
||||
assert_eq!((ga.mu(), ga.sigma()), (gb.mu(), gb.sigma()));
|
||||
|
||||
// Control: the shorthand is not a no-op — a different gamma differs.
|
||||
let mut c: Borrowed = History::builder().gamma(0.0).build();
|
||||
c.record_winner(&"x", &"y", 1).unwrap();
|
||||
c.record_winner(&"y", &"x", 100).unwrap();
|
||||
c.converge().unwrap();
|
||||
assert_ne!(c.current_skill("x").unwrap().sigma(), ga.sigma());
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "gamma must be finite and non-negative")]
|
||||
fn a_negative_gamma_is_rejected_rather_than_squared_away() {
|
||||
let _: Borrowed = History::builder().gamma(-0.5).build();
|
||||
}
|
||||
@@ -24,10 +24,11 @@ impl Lcg {
|
||||
}
|
||||
|
||||
fn nan_after_fit(players: usize) -> usize {
|
||||
let mut h: History<i64, ConstantDrift, NullObserver, String> = History::builder_with_key()
|
||||
let mut h: History<i64, ConstantDrift, NullObserver, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.beta(1.0)
|
||||
.sigma(6.0)
|
||||
.drift(ConstantDrift(0.1))
|
||||
.drift(ConstantDrift::new(0.1))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: ITERATIONS,
|
||||
epsilon: EPSILON,
|
||||
@@ -46,7 +47,7 @@ fn nan_after_fit(players: usize) -> usize {
|
||||
let (w, l) = if rng.coin() { (a, b) } else { (b, a) };
|
||||
h.record_winner(&ids[w], &ids[l], 0).unwrap();
|
||||
}
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
ids.iter()
|
||||
.filter(|id| {
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
//! The libm rule, enforced rather than asserted in prose.
|
||||
//!
|
||||
//! CLAUDE.md requires transcendentals to go through `libm`, not `std`:
|
||||
//!
|
||||
//! > IEEE 754 pins the basic operations and `sqrt` but says nothing about
|
||||
//! > `exp`/`log`/`erf`, and `std` delegates to the *system* math library —
|
||||
//! > measured, `f64::exp` and `libm::exp` disagree on 9.7% of inputs by one
|
||||
//! > ULP. Since inference is an iterative fixed point, one ULP can change an
|
||||
//! > iteration count.
|
||||
//!
|
||||
//! The rule was stated clearly and still violated in three production sites,
|
||||
//! one of them `hypot` on the path of every scored event — whose measured
|
||||
//! divergence, 12.1%, is *higher* than the `exp` figure the rule cites as its
|
||||
//! own justification. Prose is evidently not enough, so this is a test.
|
||||
//!
|
||||
//! Tests may use either, which the crate documents, so `#[cfg(test)]` blocks
|
||||
//! are excluded.
|
||||
|
||||
use std::{fs, path::Path};
|
||||
|
||||
/// Method-call spellings that reach the system math library.
|
||||
///
|
||||
/// `sqrt` is deliberately absent: IEEE 754 specifies it exactly, so `std` and
|
||||
/// `libm` cannot disagree. `abs`, `recip`, `powi` and `mul_add` are likewise
|
||||
/// exact or specified.
|
||||
const FORBIDDEN: &[&str] = &[
|
||||
"exp", "exp2", "exp_m1", "ln", "ln_1p", "log", "log2", "log10", "powf", "sin", "cos", "tan",
|
||||
"asin", "acos", "atan", "atan2", "sinh", "cosh", "tanh", "hypot", "cbrt", "erf", "erfc",
|
||||
];
|
||||
|
||||
/// Strip `#[cfg(test)]` items by brace matching, plus comments and string
|
||||
/// literals, so a mention in prose is not mistaken for a call.
|
||||
fn production_code(source: &str) -> String {
|
||||
let mut out = String::with_capacity(source.len());
|
||||
let bytes: Vec<char> = source.chars().collect();
|
||||
let mut i = 0;
|
||||
|
||||
while i < bytes.len() {
|
||||
let rest: String = bytes[i..].iter().take(16).collect();
|
||||
|
||||
if rest.starts_with("#[cfg(test)]") {
|
||||
// Skip to the opening brace of the guarded item, then past its
|
||||
// matching close.
|
||||
let mut j = i;
|
||||
while j < bytes.len() && bytes[j] != '{' {
|
||||
j += 1;
|
||||
}
|
||||
let mut depth = 0usize;
|
||||
while j < bytes.len() {
|
||||
match bytes[j] {
|
||||
'{' => depth += 1,
|
||||
'}' => {
|
||||
depth -= 1;
|
||||
if depth == 0 {
|
||||
j += 1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
j += 1;
|
||||
}
|
||||
i = j;
|
||||
continue;
|
||||
}
|
||||
|
||||
if rest.starts_with("//") {
|
||||
while i < bytes.len() && bytes[i] != '\n' {
|
||||
i += 1;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if rest.starts_with("/*") {
|
||||
i += 2;
|
||||
while i + 1 < bytes.len() && !(bytes[i] == '*' && bytes[i + 1] == '/') {
|
||||
i += 1;
|
||||
}
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
|
||||
if bytes[i] == '"' {
|
||||
i += 1;
|
||||
while i < bytes.len() && bytes[i] != '"' {
|
||||
if bytes[i] == '\\' {
|
||||
i += 1;
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
out.push(bytes[i]);
|
||||
i += 1;
|
||||
}
|
||||
|
||||
out
|
||||
}
|
||||
|
||||
fn rust_files(dir: &Path, out: &mut Vec<std::path::PathBuf>) {
|
||||
for entry in fs::read_dir(dir).expect("read src") {
|
||||
let path = entry.expect("dir entry").path();
|
||||
if path.is_dir() {
|
||||
rust_files(&path, out);
|
||||
} else if path.extension().is_some_and(|e| e == "rs") {
|
||||
out.push(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn production_code_never_calls_a_std_transcendental() {
|
||||
let mut files = Vec::new();
|
||||
rust_files(Path::new("src"), &mut files);
|
||||
assert!(files.len() > 10, "expected to find the crate's sources");
|
||||
|
||||
let mut offences = Vec::new();
|
||||
|
||||
for path in &files {
|
||||
let source = fs::read_to_string(path).expect("read source");
|
||||
let code = production_code(&source);
|
||||
|
||||
for (n, line) in code.lines().enumerate() {
|
||||
for name in FORBIDDEN {
|
||||
let needle = format!(".{name}(");
|
||||
if line.contains(&needle) {
|
||||
offences.push(format!("{}:{}: {}", path.display(), n + 1, line.trim()));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
assert!(
|
||||
offences.is_empty(),
|
||||
"production code must call libm, not std, for transcendentals \
|
||||
(`sqrt` is exempt — IEEE 754 specifies it):\n{}",
|
||||
offences.join("\n")
|
||||
);
|
||||
}
|
||||
|
||||
/// The stripper has to actually strip, or the test above passes vacuously.
|
||||
#[test]
|
||||
fn the_test_module_stripper_works() {
|
||||
let source = r#"
|
||||
fn production() { let _ = libm::exp(1.0); }
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
fn allowed() { let x = 1.0f64.exp(); }
|
||||
}
|
||||
|
||||
fn also_production() {}
|
||||
"#;
|
||||
let code = production_code(source);
|
||||
assert!(
|
||||
code.contains("also_production"),
|
||||
"stripped too much: {code}"
|
||||
);
|
||||
assert!(
|
||||
!code.contains(".exp()"),
|
||||
"failed to strip cfg(test): {code}"
|
||||
);
|
||||
}
|
||||
|
||||
/// And it must not strip a doc comment's worth of prose into oblivion, nor
|
||||
/// mistake prose for a call.
|
||||
#[test]
|
||||
fn prose_is_not_mistaken_for_a_call() {
|
||||
let source = "/// Uses `x.exp()` in the docs.\nfn f() { let _ = libm::exp(1.0); }\n";
|
||||
let code = production_code(source);
|
||||
assert!(!code.contains(".exp()"), "doc comment leaked: {code}");
|
||||
assert!(code.contains("libm::exp"), "stripped real code: {code}");
|
||||
}
|
||||
@@ -0,0 +1,372 @@
|
||||
//! Calibration of the crate's marginals against the EXACT posterior.
|
||||
//!
|
||||
//! A scored history is linear-Gaussian — `MarginFactor` encodes
|
||||
//! `score_a - score_b ~ N(perf_a - perf_b, score_sigma^2)` — so the true joint
|
||||
//! posterior has a closed form and the crate can be checked against ground
|
||||
//! truth rather than against intuition. That is not possible for ranked
|
||||
//! outcomes, whose truncation likelihood EP genuinely approximates.
|
||||
//!
|
||||
//! Two things are pinned here, and one is deliberately only recorded.
|
||||
//!
|
||||
//! **Pinned: on a tree the crate is exact**, means and variances both. Message
|
||||
//! passing has no approximation to make when the factor graph has no cycles, so
|
||||
//! any drift here would be a real defect.
|
||||
//!
|
||||
//! **Pinned: means are exact even with cycles.** This is the standard result
|
||||
//! for Gaussian belief propagation (Weiss & Freeman 2001) and it is what makes
|
||||
//! ratings trustworthy.
|
||||
//!
|
||||
//! **Recorded, not asserted: with cycles, marginal variances are too narrow.**
|
||||
//! Measured on the round-robin fixture below, the crate reports sigma 1.430
|
||||
//! where the exact posterior is 2.851 — a ratio of 0.502. That is the known
|
||||
//! behaviour of loopy Gaussian BP, not a bug in this crate, and it is left
|
||||
//! unasserted because fixing it is exactly what #46 proposes.
|
||||
//!
|
||||
//! Why that matters for a consumer, and why #46 cannot be implemented as "add
|
||||
//! a covariance accessor": the exact correlation between two nodes here is
|
||||
//! +0.857, so a consumer computing `sqrt(sa^2 + sb^2)` for a difference
|
||||
//! overstates its width. But the too-narrow marginals partially cancel that,
|
||||
//! leaving 1.327x rather than 2.646x. Adding true correlations to these
|
||||
//! marginals without also correcting them would give 0.765 against a true
|
||||
//! 1.524 — *overconfident*, which is the unsafe direction.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{ConstantDrift, ConvergenceOptions, Event, History, Member, Outcome, Team};
|
||||
|
||||
const N: usize = 5;
|
||||
const MU0: f64 = 0.0;
|
||||
const SIGMA0: f64 = 6.0;
|
||||
const BETA: f64 = 1.0;
|
||||
const SCORE_SIGMA: f64 = 2.0;
|
||||
|
||||
/// A STAR: every event touches c0, so the node-event graph is a tree and
|
||||
/// Gaussian BP is exact. Any discrepancy here is not caused by loops.
|
||||
fn tree_fixture() -> Vec<(usize, usize, f64)> {
|
||||
vec![(0, 1, 3.0), (0, 2, 5.0), (0, 3, 4.0), (0, 4, 6.0)]
|
||||
}
|
||||
|
||||
/// (winner, loser, score_diff)
|
||||
fn fixture() -> Vec<(usize, usize, f64)> {
|
||||
vec![
|
||||
(0, 1, 3.0),
|
||||
(0, 2, 5.0),
|
||||
(1, 2, 2.0),
|
||||
(3, 4, 1.0),
|
||||
(0, 3, 4.0),
|
||||
(1, 4, 2.5),
|
||||
(2, 3, 0.5),
|
||||
(0, 4, 6.0),
|
||||
(1, 3, 1.5),
|
||||
(2, 4, 3.0),
|
||||
]
|
||||
}
|
||||
|
||||
/// Invert a small symmetric positive-definite matrix by Gauss-Jordan.
|
||||
fn inverse(mut a: Vec<Vec<f64>>) -> Vec<Vec<f64>> {
|
||||
let n = a.len();
|
||||
let mut inv: Vec<Vec<f64>> = (0..n)
|
||||
.map(|i| (0..n).map(|j| if i == j { 1.0 } else { 0.0 }).collect())
|
||||
.collect();
|
||||
for col in 0..n {
|
||||
// partial pivot
|
||||
let mut piv = col;
|
||||
for r in col + 1..n {
|
||||
if a[r][col].abs() > a[piv][col].abs() {
|
||||
piv = r;
|
||||
}
|
||||
}
|
||||
a.swap(col, piv);
|
||||
inv.swap(col, piv);
|
||||
let d = a[col][col];
|
||||
for j in 0..n {
|
||||
a[col][j] /= d;
|
||||
inv[col][j] /= d;
|
||||
}
|
||||
for r in 0..n {
|
||||
if r == col {
|
||||
continue;
|
||||
}
|
||||
let f = a[r][col];
|
||||
for j in 0..n {
|
||||
a[r][j] -= f * a[col][j];
|
||||
inv[r][j] -= f * inv[col][j];
|
||||
}
|
||||
}
|
||||
}
|
||||
inv
|
||||
}
|
||||
|
||||
/// The exact posterior of a linear-Gaussian model:
|
||||
/// precision = prior precision + sum of a_k a_k^T / v_k.
|
||||
fn exact_for(obs: &[(usize, usize, f64)]) -> (Vec<f64>, Vec<Vec<f64>>) {
|
||||
let mut lambda = vec![vec![0.0; N]; N];
|
||||
let mut eta = [0.0; N];
|
||||
for (i, row) in lambda.iter_mut().enumerate() {
|
||||
row[i] = 1.0 / (SIGMA0 * SIGMA0);
|
||||
eta[i] = MU0 / (SIGMA0 * SIGMA0);
|
||||
}
|
||||
|
||||
// Each 1v1 observation: d ~ N(x_a - x_b, score_sigma^2 + 2 beta^2)
|
||||
let v = SCORE_SIGMA * SCORE_SIGMA + 2.0 * BETA * BETA;
|
||||
for &(a, b, d) in obs {
|
||||
let mut vec_a = [0.0; N];
|
||||
vec_a[a] = 1.0;
|
||||
vec_a[b] = -1.0;
|
||||
for i in 0..N {
|
||||
for j in 0..N {
|
||||
lambda[i][j] += vec_a[i] * vec_a[j] / v;
|
||||
}
|
||||
eta[i] += vec_a[i] * d / v;
|
||||
}
|
||||
}
|
||||
|
||||
let cov = inverse(lambda);
|
||||
let mean: Vec<f64> = (0..N)
|
||||
.map(|i| (0..N).map(|j| cov[i][j] * eta[j]).sum())
|
||||
.collect();
|
||||
(mean, cov)
|
||||
}
|
||||
|
||||
fn key(i: usize) -> &'static str {
|
||||
["c0", "c1", "c2", "c3", "c4"][i]
|
||||
}
|
||||
|
||||
/// Returns (worst mean error, worst sd ratio).
|
||||
fn fitted(
|
||||
obs: &[(usize, usize, f64)],
|
||||
) -> History<i64, ConstantDrift, trueskill_tt::NullObserver, &'static str> {
|
||||
let mut h: History<i64, _, _, &'static str> = History::builder()
|
||||
.mu(MU0)
|
||||
.sigma(SIGMA0)
|
||||
.beta(BETA)
|
||||
.score_sigma(SCORE_SIGMA)
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
|
||||
let events: Vec<Event<i64, &'static str>> = obs
|
||||
.iter()
|
||||
.copied()
|
||||
.map(|(a, b, d)| Event {
|
||||
time: 1,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(key(a))]),
|
||||
Team::with_members([Member::new(key(b))]),
|
||||
],
|
||||
outcome: Outcome::scores([d, 0.0]),
|
||||
})
|
||||
.collect();
|
||||
h.add_events(events).unwrap();
|
||||
let report = h.converge().unwrap();
|
||||
assert!(
|
||||
report.converged,
|
||||
"fixture must converge: {:?}",
|
||||
report.final_step
|
||||
);
|
||||
|
||||
h
|
||||
}
|
||||
|
||||
/// Returns (worst mean error, worst sd ratio gap).
|
||||
fn run(name: &str, obs: Vec<(usize, usize, f64)>) -> (f64, f64) {
|
||||
println!("\n########## {name} ##########");
|
||||
let h = fitted(&obs);
|
||||
let (mean, cov) = exact_for(&obs);
|
||||
|
||||
println!("\n== marginals: crate vs the exact linear-Gaussian posterior ==");
|
||||
println!(
|
||||
"{:>4} {:>12} {:>12} {:>12} {:>12} {:>8}",
|
||||
"node", "crate mu", "exact mu", "crate sd", "exact sd", "sd ratio"
|
||||
);
|
||||
for i in 0..N {
|
||||
let g = h.current_skill(&key(i)).unwrap();
|
||||
let exact_sd = cov[i][i].sqrt();
|
||||
println!(
|
||||
"{:>4} {:>12.6} {:>12.6} {:>12.6} {:>12.6} {:>8.3}",
|
||||
key(i),
|
||||
g.mu(),
|
||||
mean[i],
|
||||
g.sigma(),
|
||||
exact_sd,
|
||||
g.sigma() / exact_sd
|
||||
);
|
||||
}
|
||||
|
||||
let mut worst_mean = 0.0f64;
|
||||
let mut worst_ratio_gap = 0.0f64;
|
||||
for i in 0..N {
|
||||
let g = h.current_skill(&key(i)).unwrap();
|
||||
worst_mean = worst_mean.max((g.mu() - mean[i]).abs());
|
||||
worst_ratio_gap = worst_ratio_gap.max((g.sigma() / cov[i][i].sqrt() - 1.0).abs());
|
||||
}
|
||||
|
||||
println!("\n== what a consumer actually computes for a DIFFERENCE ==");
|
||||
println!(
|
||||
"{:>8} {:>12} {:>14} {:>14} {:>12}",
|
||||
"pair", "exact", "naive(exact)", "naive(crate)", "crate err"
|
||||
);
|
||||
for i in 0..N {
|
||||
for j in i + 1..N {
|
||||
if i != 0 && j != 1 {
|
||||
continue;
|
||||
}
|
||||
let gi = h.current_skill(&key(i)).unwrap();
|
||||
let gj = h.current_skill(&key(j)).unwrap();
|
||||
let exact_sd = (cov[i][i] + cov[j][j] - 2.0 * cov[i][j]).sqrt();
|
||||
let naive_exact = (cov[i][i] + cov[j][j]).sqrt();
|
||||
let naive_crate = (gi.sigma().powi(2) + gj.sigma().powi(2)).sqrt();
|
||||
let corr = cov[i][j] / (cov[i][i].sqrt() * cov[j][j].sqrt());
|
||||
println!(
|
||||
"{:>8} {:>12.6} {:>14.6} {:>14.6} {:>11.3}x (corr {corr:.4})",
|
||||
format!("{}-{}", key(i), key(j)),
|
||||
exact_sd,
|
||||
naive_exact,
|
||||
naive_crate,
|
||||
naive_crate / exact_sd
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
(worst_mean, worst_ratio_gap)
|
||||
}
|
||||
|
||||
/// With no cycles there is nothing for message passing to approximate.
|
||||
#[test]
|
||||
fn on_a_tree_the_marginals_are_exact() {
|
||||
let (mean_err, sd_gap) = run("TREE (star: no loops, BP is exact)", tree_fixture());
|
||||
assert!(
|
||||
mean_err < 1e-9,
|
||||
"tree means should be exact, worst error {mean_err}"
|
||||
);
|
||||
assert!(
|
||||
sd_gap < 1e-9,
|
||||
"tree sigmas should be exact, worst ratio gap {sd_gap}"
|
||||
);
|
||||
}
|
||||
|
||||
/// With cycles the means stay exact — the property ratings depend on — while
|
||||
/// the variances do not. The variance gap is measured and reported rather than
|
||||
/// asserted; see the module docs.
|
||||
#[test]
|
||||
fn with_cycles_the_means_stay_exact_but_the_variances_shrink() {
|
||||
let (mean_err, sd_gap) = run("LOOPY (round robin)", fixture());
|
||||
assert!(
|
||||
mean_err < 1e-9,
|
||||
"loopy means must still be exact, worst error {mean_err}"
|
||||
);
|
||||
assert!(
|
||||
sd_gap > 0.1,
|
||||
"the loopy variance gap is the premise of #46; if it has closed, that \
|
||||
issue and these docs need revisiting (worst ratio gap {sd_gap})"
|
||||
);
|
||||
}
|
||||
|
||||
/// The point of #46: `posterior_of` must reproduce the exact joint, including
|
||||
/// the correlation that marginals cannot express.
|
||||
#[test]
|
||||
fn posterior_of_matches_the_exact_joint() {
|
||||
for (name, obs) in [("tree", tree_fixture()), ("loopy", fixture())] {
|
||||
let h = fitted(&obs);
|
||||
let (_, cov) = exact_for(&obs);
|
||||
|
||||
println!("\n== posterior_of vs exact ({name}) ==");
|
||||
println!(
|
||||
"{:>12} {:>14} {:>14} {:>10}",
|
||||
"functional", "posterior_of", "exact", "ratio"
|
||||
);
|
||||
|
||||
for (i, j) in [(0usize, 1usize), (0, 2), (1, 3), (2, 4)] {
|
||||
let got = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&[(&key(i), 1.0), (&key(j), -1.0)])
|
||||
.expect("scored slice should have a joint");
|
||||
let exact_sd = (cov[i][i] + cov[j][j] - 2.0 * cov[i][j]).sqrt();
|
||||
println!(
|
||||
"{:>12} {:>14.6} {:>14.6} {:>10.4}",
|
||||
format!("{}-{}", key(i), key(j)),
|
||||
got.sigma(),
|
||||
exact_sd,
|
||||
got.sigma() / exact_sd
|
||||
);
|
||||
assert!(
|
||||
(got.sigma() - exact_sd).abs() / exact_sd < 1e-9,
|
||||
"{name} {}-{}: posterior_of gave {} where the exact joint is {exact_sd}",
|
||||
key(i),
|
||||
key(j),
|
||||
got.sigma()
|
||||
);
|
||||
}
|
||||
|
||||
// A single competitor: this is where the loopy marginal was 2x narrow.
|
||||
for (i, row) in cov.iter().enumerate() {
|
||||
let got = h.joint().unwrap().posterior_of(&[(&key(i), 1.0)]).unwrap();
|
||||
let exact_sd = row[i].sqrt();
|
||||
assert!(
|
||||
(got.sigma() - exact_sd).abs() / exact_sd < 1e-9,
|
||||
"{name} {}: posterior_of gave {} where exact is {exact_sd}",
|
||||
key(i),
|
||||
got.sigma()
|
||||
);
|
||||
}
|
||||
println!(" single-competitor marginals also exact");
|
||||
}
|
||||
}
|
||||
|
||||
/// Cost of the dense solve as the slice grows. Recorded, not asserted.
|
||||
#[test]
|
||||
#[ignore = "timing probe, run explicitly"]
|
||||
fn cost_scaling() {
|
||||
use std::time::Instant;
|
||||
for n in [50usize, 100, 200, 400, 800] {
|
||||
let names: Vec<String> = (0..n).map(|i| format!("c{i}")).collect();
|
||||
let mut h: History<i64, _, _, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 200,
|
||||
epsilon: 1e-8,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
let mut seed = 5u64;
|
||||
let mut rnd = move || {
|
||||
seed ^= seed << 13;
|
||||
seed ^= seed >> 7;
|
||||
seed ^= seed << 17;
|
||||
seed
|
||||
};
|
||||
let events: Vec<Event<i64, String>> = (0..n * 4)
|
||||
.map(|_| {
|
||||
let a = (rnd() as usize) % n;
|
||||
let mut b = (rnd() as usize) % n;
|
||||
if b == a {
|
||||
b = (b + 1) % n;
|
||||
}
|
||||
Event {
|
||||
time: 1,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(names[a].clone())]),
|
||||
Team::with_members([Member::new(names[b].clone())]),
|
||||
],
|
||||
outcome: Outcome::scores([1.0, 0.0]),
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
h.add_events(events).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let t = Instant::now();
|
||||
let g = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&[(&names[0], 1.0), (&names[1], -1.0)])
|
||||
.unwrap();
|
||||
println!(" n={n:>4}: {:>10.2?} sigma {:.6}", t.elapsed(), g.sigma());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,217 @@
|
||||
//! Inference must report numerical breakdown rather than call it convergence.
|
||||
//!
|
||||
//! The boundary rejects inputs that are *not numbers*, but finite inputs can
|
||||
//! still overflow during inference — `beta.powi(2)` at 1e300 is infinite, and
|
||||
//! infinity minus infinity is NaN. `NonFiniteResult` is the guard for that, and
|
||||
//! it matters because the alternative is silent: NaN fails every comparison, so
|
||||
//! a naive `step < epsilon` check reads a NaN step as *converged*.
|
||||
//!
|
||||
//! That is why the crate has `step_converged` / `step_is_finite` rather than
|
||||
//! `!tuple_gt(..)`. These tests pin the guard from outside.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, Event, Gaussian, History, InferenceError, Member, Outcome, Team,
|
||||
};
|
||||
|
||||
fn scored_fit(
|
||||
sigma: f64,
|
||||
beta: f64,
|
||||
score_sigma: f64,
|
||||
scores: [f64; 2],
|
||||
) -> Result<bool, InferenceError> {
|
||||
let mut h = History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(sigma)
|
||||
.beta(beta)
|
||||
.score_sigma(score_sigma)
|
||||
.build();
|
||||
h.add_events(vec![Event {
|
||||
time: 1i64,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("a")]),
|
||||
Team::with_members([Member::new("b")]),
|
||||
],
|
||||
outcome: Outcome::scores(scores),
|
||||
}])?;
|
||||
h.converge().map(|r| r.converged)
|
||||
}
|
||||
|
||||
/// Every one of these is built from finite, individually legal parameters. The
|
||||
/// overflow happens inside inference, which is exactly the case the boundary
|
||||
/// checks cannot catch.
|
||||
///
|
||||
/// Matched rather than merely `is_err()`: an assertion that only checks "some
|
||||
/// error" would keep passing if these started failing at the boundary for an
|
||||
/// unrelated reason, and would then be testing nothing.
|
||||
#[test]
|
||||
fn overflow_during_inference_is_reported_not_hidden() {
|
||||
let cases: [(&str, f64, f64, f64, [f64; 2]); 5] = [
|
||||
("huge sigma", 1e300, 1.0, 1.0, [3.0, 1.0]),
|
||||
("huge beta", 6.0, 1e300, 1.0, [3.0, 1.0]),
|
||||
("tiny sigma", 1e-300, 1.0, 1.0, [3.0, 1.0]),
|
||||
("tiny score_sigma", 6.0, 1.0, 1e-300, [3.0, 1.0]),
|
||||
("huge scores", 6.0, 1.0, 1.0, [1e308, -1e308]),
|
||||
];
|
||||
|
||||
for (name, sigma, beta, score_sigma, scores) in cases {
|
||||
match scored_fit(sigma, beta, score_sigma, scores) {
|
||||
Err(InferenceError::NonFiniteResult { context, step, .. }) => {
|
||||
assert_eq!(context, "History::converge", "{name}");
|
||||
assert!(
|
||||
!step.0.is_finite() || !step.1.is_finite(),
|
||||
"{name}: reported NonFiniteResult with a finite step {step:?}"
|
||||
);
|
||||
}
|
||||
other => panic!("{name}: expected NonFiniteResult, got {other:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The trap the invariant exists for: NaN fails every comparison, so a naive
|
||||
/// `step < epsilon` test reads a NaN step as converged. A breakdown must never
|
||||
/// come back as a successful fit.
|
||||
#[test]
|
||||
fn a_broken_fit_is_never_reported_as_converged() {
|
||||
let mut h = History::builder().build();
|
||||
h.add_events(vec![Event {
|
||||
time: 1i64,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("a").with_prior(Gaussian::from_ms(1e300, 1e-300))]),
|
||||
Team::with_members([Member::new("b")]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}])
|
||||
.unwrap();
|
||||
|
||||
let err = h.converge().unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::NonFiniteResult { .. }),
|
||||
"a breakdown must not be reported as convergence: {err:?}"
|
||||
);
|
||||
|
||||
// `converge_partial` must not launder it into an `Ok` either — the
|
||||
// permissive path is permissive about *stopping short*, not about NaN.
|
||||
let mut h2 = History::builder().build();
|
||||
h2.add_events(vec![Event {
|
||||
time: 1i64,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("a").with_prior(Gaussian::from_ms(1e300, 1e-300))]),
|
||||
Team::with_members([Member::new("b")]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}])
|
||||
.unwrap();
|
||||
assert!(matches!(
|
||||
h2.converge_partial().unwrap_err(),
|
||||
InferenceError::NonFiniteResult { .. }
|
||||
));
|
||||
}
|
||||
|
||||
/// The neighbouring case, so the tests above cannot pass by the fit simply
|
||||
/// always failing: ordinary extreme-but-workable parameters still converge.
|
||||
#[test]
|
||||
fn merely_extreme_parameters_still_converge() {
|
||||
assert!(scored_fit(1e6, 1.0, 1.0, [3.0, 1.0]).unwrap());
|
||||
assert!(scored_fit(1e-6, 1.0, 1.0, [3.0, 1.0]).unwrap());
|
||||
assert!(scored_fit(6.0, 1.0, 1e6, [3.0, 1.0]).unwrap());
|
||||
assert!(scored_fit(6.0, 1.0, 1.0, [1e150, -1e150]).unwrap());
|
||||
}
|
||||
|
||||
/// A NaN in one competitor must not be masked by a healthy competitor reduced
|
||||
/// after it.
|
||||
///
|
||||
/// The convergence step is a fold over a `HashMap`, so which competitor is
|
||||
/// reduced last is per-process hash order. Before the fix, `tuple_max` dropped
|
||||
/// a NaN accumulator in favour of the next finite delta and this returned
|
||||
/// `Ok(converged: true)` with a NaN posterior in **16 of 30 runs** on identical
|
||||
/// input. Deterministic now, but note this test can only ever sample one hash
|
||||
/// order per run — the ordering guarantee itself is pinned by
|
||||
/// `tuple_max_propagates_a_nan_from_any_position` in the crate's unit tests.
|
||||
#[test]
|
||||
fn a_nan_competitor_is_not_masked_by_a_healthy_one() {
|
||||
let mut h = History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.p_draw(0.1)
|
||||
.build();
|
||||
h.add_events(vec![
|
||||
Event {
|
||||
time: 1i64,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("a").with_prior(Gaussian::from_ms(0.0, 1e-200))]),
|
||||
Team::with_members([Member::new("b")]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
},
|
||||
// A healthy pair in the same slice, to be reduced alongside the NaN.
|
||||
Event {
|
||||
time: 1i64,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("c")]),
|
||||
Team::with_members([Member::new("d")]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
},
|
||||
])
|
||||
.unwrap();
|
||||
|
||||
let err = h
|
||||
.converge()
|
||||
.expect_err("a NaN fit must never be reported as converged");
|
||||
assert!(
|
||||
matches!(err, InferenceError::NonFiniteResult { .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// A tie observed with a narrow draw margin between far-apart competitors must
|
||||
/// produce a fit, not NaN skills.
|
||||
///
|
||||
/// The tie branch forms the truncated variance from `v^2 - u`, and both grow as
|
||||
/// `alpha^2` while their difference stays `O(1)`. Deep enough into the tail
|
||||
/// that subtraction had four digits left: measured, it returned `1 - w`
|
||||
/// negative and `sqrt` of it was NaN. The half-line escape hatch did not cover
|
||||
/// it, because that keys on how many window-widths from the mean the window
|
||||
/// sits and a narrow window fails that however deep it is.
|
||||
///
|
||||
/// These parameters are ordinary for a precise-scoring domain, and the
|
||||
/// neighbouring wider-margin case always worked — so this was a cliff, not
|
||||
/// "extreme inputs break".
|
||||
#[test]
|
||||
fn a_narrow_draw_margin_far_into_the_tail_still_fits() {
|
||||
for (beta, p_draw, sd, gap) in [
|
||||
(1e-2, 1e-8, 1e-2, 10.0),
|
||||
(1e-3, 1e-9, 1e-3, 1.0),
|
||||
(1e-4, 1e-12, 1e-4, 1.0),
|
||||
] {
|
||||
let mut h = History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(sd)
|
||||
.beta(beta)
|
||||
.p_draw(p_draw)
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.build();
|
||||
h.add_events(vec![Event {
|
||||
time: 1i64,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("a").with_prior(Gaussian::from_ms(0.0, sd))]),
|
||||
Team::with_members([Member::new("b").with_prior(Gaussian::from_ms(gap, sd))]),
|
||||
],
|
||||
outcome: Outcome::draw(2),
|
||||
}])
|
||||
.unwrap();
|
||||
|
||||
let report = h
|
||||
.converge()
|
||||
.unwrap_or_else(|e| panic!("beta {beta:e}, p_draw {p_draw:e}: {e:?}"));
|
||||
assert!(report.converged);
|
||||
|
||||
let skill = h.current_skill(&"a").unwrap();
|
||||
assert!(
|
||||
skill.mu().is_finite() && skill.sigma().is_finite() && skill.sigma() > 0.0,
|
||||
"beta {beta:e}, p_draw {p_draw:e}: {skill:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
//! `Observer` callbacks must actually fire.
|
||||
//!
|
||||
//! `on_slice_processed` (formerly `on_batch_processed`) was declared on the
|
||||
//! trait and never called from anywhere, so implementors wired up a callback
|
||||
//! that could not run. These tests exist so that cannot silently recur.
|
||||
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use trueskill_tt::{History, Observer};
|
||||
|
||||
/// Plain fields. `Arc<O>` implements `Observer`, so the caller shares the
|
||||
/// observer itself rather than wrapping each field in its own `Arc`.
|
||||
#[derive(Default)]
|
||||
struct Recorder {
|
||||
iterations: Mutex<Vec<usize>>,
|
||||
slices: Mutex<Vec<(i64, usize, usize)>>,
|
||||
converged: Mutex<Vec<(usize, bool)>>,
|
||||
}
|
||||
|
||||
impl Observer<i64> for Recorder {
|
||||
fn on_iteration_end(&self, iter: usize, _max_step: (f64, f64)) {
|
||||
self.iterations.lock().unwrap().push(iter);
|
||||
}
|
||||
|
||||
fn on_slice_processed(&self, time: &i64, slice_idx: usize, n_events: usize) {
|
||||
self.slices
|
||||
.lock()
|
||||
.unwrap()
|
||||
.push((*time, slice_idx, n_events));
|
||||
}
|
||||
|
||||
fn on_converged(&self, iters: usize, _final_step: (f64, f64), converged: bool) {
|
||||
self.converged.lock().unwrap().push((iters, converged));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_observer_callback_fires() {
|
||||
let recorder = Arc::new(Recorder::default());
|
||||
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
|
||||
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
h.record_winner(&"b", &"c", 2).unwrap();
|
||||
h.record_winner(&"c", &"a", 3).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
assert!(
|
||||
!recorder.iterations.lock().unwrap().is_empty(),
|
||||
"on_iteration_end never fired"
|
||||
);
|
||||
assert!(
|
||||
!recorder.converged.lock().unwrap().is_empty(),
|
||||
"on_converged never fired"
|
||||
);
|
||||
assert!(
|
||||
!recorder.slices.lock().unwrap().is_empty(),
|
||||
"on_slice_processed never fired — the defect this test exists for"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn slice_callbacks_report_the_slice_they_swept() {
|
||||
let recorder = Arc::new(Recorder::default());
|
||||
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
|
||||
|
||||
h.record_winner(&"a", &"b", 10).unwrap();
|
||||
h.record_winner(&"a", &"b", 20).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let slices = recorder.slices.lock().unwrap();
|
||||
|
||||
// Only the times actually in the history, and each with its own events.
|
||||
for &(time, idx, events) in slices.iter() {
|
||||
assert!(time == 10 || time == 20, "unexpected slice time {time}");
|
||||
assert!(idx < 2, "slice index {idx} out of range");
|
||||
assert_eq!(events, 1, "each slice holds exactly one event");
|
||||
}
|
||||
|
||||
// Both slices must be reported, not just one end of the sweep.
|
||||
assert!(
|
||||
slices.iter().any(|&(t, ..)| t == 10),
|
||||
"slice 10 never reported"
|
||||
);
|
||||
assert!(
|
||||
slices.iter().any(|&(t, ..)| t == 20),
|
||||
"slice 20 never reported"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_single_slice_history_still_reports_its_sweep() {
|
||||
let recorder = Arc::new(Recorder::default());
|
||||
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
|
||||
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let slices = recorder.slices.lock().unwrap();
|
||||
assert!(
|
||||
!slices.is_empty(),
|
||||
"the single-slice path must report its sweep too"
|
||||
);
|
||||
assert!(slices.iter().all(|&(t, idx, _)| t == 1 && idx == 0));
|
||||
}
|
||||
|
||||
/// The gap #40 closed: without `impl Observer for Arc<O>`, an observer that
|
||||
/// accumulates anything had to wrap every field in its own `Arc` and derive
|
||||
/// `Clone`, because `History` consumes the observer and never hands it back.
|
||||
#[test]
|
||||
fn a_shared_observer_reaches_the_callers_handle() {
|
||||
let recorder = Arc::new(Recorder::default());
|
||||
let mut h = History::builder().observer(Arc::clone(&recorder)).build();
|
||||
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
assert!(!recorder.iterations.lock().unwrap().is_empty());
|
||||
assert!(!recorder.slices.lock().unwrap().is_empty());
|
||||
assert!(!recorder.converged.lock().unwrap().is_empty());
|
||||
}
|
||||
|
||||
/// `?Sized` on the blanket impls means the observer can be chosen at runtime.
|
||||
#[test]
|
||||
fn a_trait_object_observer_works() {
|
||||
let boxed: Box<dyn Observer<i64>> = Box::new(Recorder::default());
|
||||
let mut h = History::builder().observer(boxed).build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let shared: Arc<dyn Observer<i64>> = Arc::new(Recorder::default());
|
||||
let mut h = History::builder().observer(Arc::clone(&shared)).build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
}
|
||||
|
||||
/// A non-shared observer can be reclaimed after convergence instead.
|
||||
#[test]
|
||||
fn into_observer_returns_the_accumulated_state() {
|
||||
let mut h = History::builder().observer(Recorder::default()).build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
// Readable in place...
|
||||
assert!(!h.observer().iterations.lock().unwrap().is_empty());
|
||||
|
||||
// ...and reclaimable by value.
|
||||
let recorder = h.into_observer();
|
||||
assert!(!recorder.slices.lock().unwrap().is_empty());
|
||||
}
|
||||
|
||||
/// Borrowing works too, for an observer that outlives the history.
|
||||
#[test]
|
||||
fn a_borrowed_observer_works() {
|
||||
let recorder = Recorder::default();
|
||||
{
|
||||
let mut h = History::builder().observer(&recorder).build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
}
|
||||
assert!(!recorder.iterations.lock().unwrap().is_empty());
|
||||
}
|
||||
@@ -0,0 +1,155 @@
|
||||
//! `predict_margin`: the predictive distribution of a scored matchup.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, Event, History, InferenceError, Member, Outcome, Team,
|
||||
UnknownKeys,
|
||||
};
|
||||
|
||||
fn builder(
|
||||
policy: UnknownKeys,
|
||||
) -> History<i64, ConstantDrift, trueskill_tt::NullObserver, &'static str> {
|
||||
History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.unknown_keys(policy)
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 5_000,
|
||||
epsilon: 1e-12,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build()
|
||||
}
|
||||
|
||||
fn round(a: &'static str, b: &'static str, sa: f64, sb: f64) -> Event<i64, &'static str> {
|
||||
Event {
|
||||
time: 1,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(a)]),
|
||||
Team::with_members([Member::new(b)]),
|
||||
],
|
||||
outcome: Outcome::scores([sa, sb]),
|
||||
}
|
||||
}
|
||||
|
||||
/// A history where "veteran" and "regular" are well observed and "novice"
|
||||
/// appears once.
|
||||
fn fitted(
|
||||
policy: UnknownKeys,
|
||||
) -> History<i64, ConstantDrift, trueskill_tt::NullObserver, &'static str> {
|
||||
let mut h = builder(policy);
|
||||
let mut events: Vec<_> = (0..40)
|
||||
.map(|t| round("veteran", "regular", 10.0 + f64::from(t % 3), 5.0))
|
||||
.collect();
|
||||
events.push(round("veteran", "novice", 10.0, 6.0));
|
||||
h.add_events(events).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h
|
||||
}
|
||||
|
||||
/// The property #48 exists for: the interval must widen when the model knows
|
||||
/// less. Their hand-fitted noise law quoted the same sigma for a competitor
|
||||
/// with forty rounds and one with none.
|
||||
#[test]
|
||||
fn the_interval_widens_as_the_model_knows_less() {
|
||||
let h = fitted(UnknownKeys::Prior);
|
||||
|
||||
let well_known = h
|
||||
.predict_margin(&[&[&"veteran"], &[&"regular"]])
|
||||
.unwrap()
|
||||
.sigma();
|
||||
let thin = h
|
||||
.predict_margin(&[&[&"veteran"], &[&"novice"]])
|
||||
.unwrap()
|
||||
.sigma();
|
||||
let unseen = h
|
||||
.predict_margin(&[&[&"veteran"], &[&"stranger"]])
|
||||
.unwrap()
|
||||
.sigma();
|
||||
|
||||
assert!(
|
||||
well_known < thin && thin < unseen,
|
||||
"margin width should grow as evidence thins: {well_known} < {thin} < {unseen}"
|
||||
);
|
||||
}
|
||||
|
||||
/// #48's second requirement: an unseen competitor is a legitimate question, not
|
||||
/// an error, and the answer should come from the prior rather than be faked.
|
||||
#[test]
|
||||
fn an_unseen_competitor_is_answered_from_the_prior() {
|
||||
let h = fitted(UnknownKeys::Prior);
|
||||
let g = h.predict_margin(&[&[&"nobody"], &[&"no_one"]]).unwrap();
|
||||
|
||||
// Two unknowns: the gap is centred on zero and carries both priors plus
|
||||
// both performance noises plus the observation noise.
|
||||
assert!(g.mu().abs() < 1e-9, "mu {}", g.mu());
|
||||
let expected = (2.0 * 36.0 + 2.0 * 1.0 + 4.0f64).sqrt();
|
||||
assert!(
|
||||
(g.sigma() - expected).abs() < 1e-9,
|
||||
"sigma {} vs expected {expected}",
|
||||
g.sigma()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reject_still_rejects() {
|
||||
let h = fitted(UnknownKeys::Reject);
|
||||
assert!(matches!(
|
||||
h.predict_margin(&[&[&"veteran"], &[&"stranger"]]),
|
||||
Err(InferenceError::UnknownKey { .. })
|
||||
));
|
||||
}
|
||||
|
||||
/// The margin is the *difference*, so it must be antisymmetric in the teams.
|
||||
#[test]
|
||||
fn swapping_the_teams_negates_the_margin() {
|
||||
let h = fitted(UnknownKeys::Prior);
|
||||
let forward = h.predict_margin(&[&[&"veteran"], &[&"regular"]]).unwrap();
|
||||
let reverse = h.predict_margin(&[&[&"regular"], &[&"veteran"]]).unwrap();
|
||||
|
||||
assert!((forward.mu() + reverse.mu()).abs() < 1e-9);
|
||||
assert!((forward.sigma() - reverse.sigma()).abs() < 1e-12);
|
||||
}
|
||||
|
||||
/// The predictive interval must be wider than the skill gap alone: it also
|
||||
/// carries per-event performance noise and the observation noise.
|
||||
#[test]
|
||||
fn the_predictive_interval_exceeds_the_skill_uncertainty() {
|
||||
let h = fitted(UnknownKeys::Prior);
|
||||
let skill_gap = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&[(&"veteran", 1.0), (&"regular", -1.0)])
|
||||
.unwrap();
|
||||
let predictive = h.predict_margin(&[&[&"veteran"], &[&"regular"]]).unwrap();
|
||||
|
||||
assert!(
|
||||
(predictive.mu() - skill_gap.mu()).abs() < 1e-12,
|
||||
"means agree"
|
||||
);
|
||||
// beta^2 twice plus score_sigma^2 = 2 + 4.
|
||||
let expected = (skill_gap.sigma().powi(2) + 6.0).sqrt();
|
||||
assert!((predictive.sigma() - expected).abs() < 1e-12);
|
||||
assert!(predictive.sigma() > skill_gap.sigma());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shape_errors_are_reported() {
|
||||
let h = fitted(UnknownKeys::Prior);
|
||||
assert!(matches!(
|
||||
h.predict_margin(&[&[&"veteran"]]),
|
||||
Err(InferenceError::MismatchedShape {
|
||||
expected: 2,
|
||||
got: 1,
|
||||
..
|
||||
})
|
||||
));
|
||||
let empty: [&&str; 0] = [];
|
||||
assert!(matches!(
|
||||
h.predict_margin(&[&[&"veteran"], &empty]),
|
||||
Err(InferenceError::EmptyTeam { team: 1, .. })
|
||||
));
|
||||
}
|
||||
@@ -0,0 +1,415 @@
|
||||
//! Prediction API: N-team outcomes, draw mass, and the error paths that used
|
||||
//! to be panics or silent wrong answers.
|
||||
|
||||
use trueskill_tt::{History, InferenceError, MAX_PREDICTED_TEAMS};
|
||||
|
||||
fn history_with(names: &[&'static str], p_draw: f64) -> History {
|
||||
let mut h = History::builder().p_draw(p_draw).build();
|
||||
// Give every competitor a recorded skill by playing a small round robin.
|
||||
for pair in names.windows(2) {
|
||||
h.record_winner(&pair[0], &pair[1], 1).unwrap();
|
||||
}
|
||||
let _ = h.converge().unwrap();
|
||||
h
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unknown_keys_are_reported_not_silently_dropped() {
|
||||
let h = history_with(&["a", "b"], 0.0);
|
||||
|
||||
let err = h
|
||||
.predict_outcome(&[&[&"a"], &[&"ghost"]])
|
||||
.expect_err("an unknown key must not yield a confident prediction");
|
||||
assert!(
|
||||
matches!(
|
||||
&err,
|
||||
InferenceError::UnknownKey { team: 1, member: 0, key, .. }
|
||||
if key == "\"ghost\""
|
||||
),
|
||||
"{err:?}"
|
||||
);
|
||||
|
||||
// Every prediction entry point, not just one.
|
||||
assert!(
|
||||
h.predict_win_probabilities(&[&[&"a"], &[&"ghost"]])
|
||||
.is_err()
|
||||
);
|
||||
assert!(h.quality(&[&[&"a"], &[&"ghost"]]).is_err());
|
||||
assert!(h.predict_ranking(&[&[&"a"], &[&"ghost"]], &[0, 1]).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_entirely_unknown_team_is_an_error() {
|
||||
let h = history_with(&["a", "b"], 0.0);
|
||||
let err = h.predict_outcome(&[&[&"a"], &[&"x", &"y"]]).unwrap_err();
|
||||
assert!(
|
||||
matches!(
|
||||
&err,
|
||||
InferenceError::UnknownKey { team: 1, member: 0, key, .. }
|
||||
if key == "\"x\""
|
||||
),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn degenerate_team_shapes_are_errors_rather_than_panics() {
|
||||
let h = history_with(&["a", "b"], 0.0);
|
||||
|
||||
assert!(matches!(
|
||||
h.predict_outcome(&[&[&"a"]]).unwrap_err(),
|
||||
InferenceError::NotEnoughTeams { got: 1, .. }
|
||||
),);
|
||||
// An empty team list cannot infer the key type — nothing in `&[]` names it.
|
||||
// The annotation is the cost of `predict_*` being generic over the borrowed
|
||||
// key, and it only bites on the degenerate call.
|
||||
let none: &[&[&str]] = &[];
|
||||
assert!(matches!(
|
||||
h.predict_outcome(none).unwrap_err(),
|
||||
InferenceError::NotEnoughTeams { got: 0, .. }
|
||||
),);
|
||||
assert!(matches!(
|
||||
h.predict_outcome(&[&[&"a"], &[]]).unwrap_err(),
|
||||
InferenceError::EmptyTeam { team: 1, .. }
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn more_than_two_teams_no_longer_panics() {
|
||||
let h = history_with(&["a", "b", "c"], 0.0);
|
||||
let p = h
|
||||
.predict_outcome(&[&[&"a"], &[&"b"], &[&"c"]])
|
||||
.expect("three teams must be supported");
|
||||
assert!((p.total() - 1.0).abs() < 1e-6, "total = {}", p.total());
|
||||
// Three teams, no draws possible: exactly the six strict orderings.
|
||||
assert_eq!(p.outcomes().len(), 6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_outcome_space_is_capped_rather_than_hanging() {
|
||||
let names: Vec<&'static str> = vec!["a", "b", "c", "d", "e", "f", "g", "h"];
|
||||
let h = history_with(&names, 0.0);
|
||||
|
||||
let teams: Vec<&[&&'static str]> = Vec::new();
|
||||
let _ = teams;
|
||||
|
||||
let too_many: Vec<Vec<&&str>> = names.iter().map(|n| vec![n]).collect();
|
||||
let refs: Vec<&[&&str]> = too_many.iter().map(Vec::as_slice).collect();
|
||||
|
||||
let err = h.predict_outcome(&refs).unwrap_err();
|
||||
assert!(matches!(
|
||||
err,
|
||||
InferenceError::TooManyTeams { got: 8, max, .. } if max == MAX_PREDICTED_TEAMS
|
||||
));
|
||||
|
||||
// The cheap paths stay available at any size.
|
||||
let wins = h.predict_win_probabilities(&refs).unwrap();
|
||||
assert_eq!(wins.len(), 8);
|
||||
assert!(
|
||||
(wins.iter().sum::<f64>() - 1.0).abs() < 1e-6,
|
||||
"win probabilities must still sum to one: {wins:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// The defect that made every draw-enabled prediction wrong: `[p, 1 - p]`
|
||||
/// allocated no mass to a draw even with `p_draw > 0`.
|
||||
#[test]
|
||||
fn a_draw_carries_probability_mass_when_p_draw_is_positive() {
|
||||
let h = history_with(&["a", "b"], 0.25);
|
||||
let p = h.predict_outcome(&[&[&"a"], &[&"b"]]).unwrap();
|
||||
|
||||
let draw = p.probability_of(&[0, 0]);
|
||||
assert!(draw > 0.0, "a draw-enabled model must give draws mass");
|
||||
assert!((p.total() - 1.0).abs() < 1e-6, "total = {}", p.total());
|
||||
|
||||
let wins = p.win_probabilities();
|
||||
assert!(
|
||||
(wins.iter().sum::<f64>() + draw - 1.0).abs() < 1e-6,
|
||||
"wins {wins:?} plus draw {draw} must be the whole space"
|
||||
);
|
||||
assert!(
|
||||
(p.shared_first_place() - draw).abs() < 1e-12,
|
||||
"a two-team draw is a shared first place"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_zero_draw_probability_admits_no_ties() {
|
||||
let h = history_with(&["a", "b"], 0.0);
|
||||
let p = h.predict_outcome(&[&[&"a"], &[&"b"]]).unwrap();
|
||||
assert_eq!(p.probability_of(&[0, 0]), 0.0);
|
||||
assert!(p.shared_first_place() < 1e-12);
|
||||
}
|
||||
|
||||
/// The two routes to a win probability run through entirely different
|
||||
/// algorithms — adaptive quadrature versus the enumerated chain recursion —
|
||||
/// so agreement between them is a real cross-check, not a tautology.
|
||||
#[test]
|
||||
fn the_cheap_and_exhaustive_paths_agree() {
|
||||
for p_draw in [0.0, 0.1] {
|
||||
let h = history_with(&["a", "b", "c"], p_draw);
|
||||
let teams: &[&[&&str]] = &[&[&"a"], &[&"b"], &[&"c"]];
|
||||
|
||||
let cheap = h.predict_win_probabilities(teams).unwrap();
|
||||
let exhaustive = h.predict_outcome(teams).unwrap().win_probabilities();
|
||||
|
||||
for (i, (a, b)) in cheap.iter().zip(&exhaustive).enumerate() {
|
||||
assert!(
|
||||
(a - b).abs() < 1e-6,
|
||||
"p_draw={p_draw} team {i}: quadrature {a} vs enumeration {b}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn predict_ranking_agrees_with_the_distribution() {
|
||||
let h = history_with(&["a", "b", "c"], 0.1);
|
||||
let teams: &[&[&&str]] = &[&[&"a"], &[&"b"], &[&"c"]];
|
||||
let dist = h.predict_outcome(teams).unwrap();
|
||||
|
||||
for (ranks, expected) in dist.outcomes() {
|
||||
let direct = h.predict_ranking(teams, ranks).unwrap();
|
||||
assert!(
|
||||
(direct - expected).abs() < 1e-9,
|
||||
"ranks {ranks:?}: {direct} vs {expected}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn predict_ranking_checks_its_shape() {
|
||||
let h = history_with(&["a", "b"], 0.0);
|
||||
let err = h
|
||||
.predict_ranking(&[&[&"a"], &[&"b"]], &[0, 1, 2])
|
||||
.unwrap_err();
|
||||
assert!(matches!(
|
||||
err,
|
||||
InferenceError::MismatchedShape {
|
||||
expected: 2,
|
||||
got: 3,
|
||||
..
|
||||
}
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_stronger_competitor_is_favoured() {
|
||||
let mut h = History::builder().build();
|
||||
for t in 1..=10 {
|
||||
h.record_winner(&"strong", &"weak", t).unwrap();
|
||||
}
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let p = h.predict_outcome(&[&[&"strong"], &[&"weak"]]).unwrap();
|
||||
let (best, _) = p.most_likely().expect("a most likely outcome");
|
||||
assert_eq!(best, &[0, 1], "the winner should be favoured");
|
||||
|
||||
let wins = p.win_probabilities();
|
||||
assert!(wins[0] > wins[1], "{wins:?}");
|
||||
}
|
||||
|
||||
/// Unequal team sizes change the draw margin, because inference derives it
|
||||
/// from the teams' betas. Prediction has to follow, or it describes a
|
||||
/// different model than the one that will be fitted.
|
||||
#[test]
|
||||
fn team_size_affects_the_prediction() {
|
||||
let mut h = History::builder().p_draw(0.2).build();
|
||||
h.event(1)
|
||||
.team(["a", "b"])
|
||||
.team(["c"])
|
||||
.winner(0)
|
||||
.commit()
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let p = h.predict_outcome(&[&[&"a", &"b"], &[&"c"]]).unwrap();
|
||||
assert!((p.total() - 1.0).abs() < 1e-6, "total = {}", p.total());
|
||||
assert!(p.probability_of(&[0, 0]) > 0.0);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Expected information gain
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// The whole point of #39: "which comparison should I run next?" is a
|
||||
/// different question from "who will win?" or "is this fair?".
|
||||
#[test]
|
||||
fn information_gain_prefers_the_uncertain_pairing() {
|
||||
let mut h = History::builder().build();
|
||||
|
||||
// "known" and "rival" have played a lot; "newcomer" has played once.
|
||||
for t in 1..=15 {
|
||||
h.record_winner(&"known", &"rival", t).unwrap();
|
||||
h.record_winner(&"rival", &"known", t + 100).unwrap();
|
||||
}
|
||||
h.record_winner(&"known", &"newcomer", 500).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let settled = h
|
||||
.expected_information_gain(&[&[&"known"], &[&"rival"]])
|
||||
.unwrap();
|
||||
let unknown = h
|
||||
.expected_information_gain(&[&[&"known"], &[&"newcomer"]])
|
||||
.unwrap();
|
||||
|
||||
assert!(
|
||||
unknown > settled,
|
||||
"pairing against the newcomer should teach more: {unknown} vs {settled}"
|
||||
);
|
||||
}
|
||||
|
||||
/// The analytic ceiling, through the `History` entry point rather than the
|
||||
/// standalone one.
|
||||
#[test]
|
||||
fn information_gain_respects_the_entropy_ceiling() {
|
||||
let h = history_with(&["a", "b", "c"], 0.0);
|
||||
|
||||
let two = h.expected_information_gain(&[&[&"a"], &[&"b"]]).unwrap();
|
||||
assert!(
|
||||
(0.0..=std::f64::consts::LN_2).contains(&two),
|
||||
"two-team EIG {two} outside [0, ln 2]"
|
||||
);
|
||||
|
||||
let three = h
|
||||
.expected_information_gain(&[&[&"a"], &[&"b"], &[&"c"]])
|
||||
.unwrap();
|
||||
assert!(
|
||||
(0.0..=6.0f64.ln()).contains(&three),
|
||||
"three-team EIG {three} outside [0, ln 6]"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn information_gain_reports_unknown_keys() {
|
||||
let h = history_with(&["a", "b"], 0.0);
|
||||
assert!(matches!(
|
||||
&h.expected_information_gain(&[&[&"a"], &[&"ghost"]])
|
||||
.unwrap_err(),
|
||||
InferenceError::UnknownKey { team: 1, member: 0, key, .. }
|
||||
if key == "\"ghost\""
|
||||
));
|
||||
}
|
||||
|
||||
/// A draw-enabled history has three outcomes to weigh rather than two, so the
|
||||
/// draw branch must actually be reachable through this path.
|
||||
#[test]
|
||||
fn information_gain_accounts_for_draws() {
|
||||
let with_draws = history_with(&["a", "b"], 0.25);
|
||||
let g = with_draws
|
||||
.expected_information_gain(&[&[&"a"], &[&"b"]])
|
||||
.unwrap();
|
||||
assert!(g > 0.0 && g <= 3.0f64.ln(), "{g}");
|
||||
|
||||
// The draw outcome carries mass, so it is genuinely being weighed.
|
||||
let dist = with_draws.predict_outcome(&[&[&"a"], &[&"b"]]).unwrap();
|
||||
assert!(dist.probability_of(&[0, 0]) > 0.0);
|
||||
}
|
||||
|
||||
/// The defect that cost a consumer a day: `UnknownKey { team: 0, member: 0 }`
|
||||
/// says nothing about *which* key is unknown, so the natural handling — log it,
|
||||
/// fall back to a neutral value — converts a total miss into a plausible
|
||||
/// constant. The key has to be in the error, and in its `Display`.
|
||||
#[test]
|
||||
fn unknown_key_names_the_key_it_could_not_find() {
|
||||
let h = history_with(&["a", "b"], 0.0);
|
||||
let err = h.predict_outcome(&[&[&"a"], &[&"never_seen"]]).unwrap_err();
|
||||
|
||||
match &err {
|
||||
InferenceError::UnknownKey { key, .. } => {
|
||||
assert!(
|
||||
key.contains("never_seen"),
|
||||
"the error should name the key, got {key}"
|
||||
);
|
||||
}
|
||||
other => panic!("expected UnknownKey, got {other:?}"),
|
||||
}
|
||||
|
||||
let rendered = err.to_string();
|
||||
assert!(
|
||||
rendered.contains("never_seen"),
|
||||
"Display should name the key: {rendered}"
|
||||
);
|
||||
assert!(
|
||||
rendered.contains("pre-filter"),
|
||||
"Display should say what to do about it: {rendered}"
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// UnknownKeys policy
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
fn history_with_policy(names: &[&'static str], policy: trueskill_tt::UnknownKeys) -> History {
|
||||
let mut h = History::builder().unknown_keys(policy).build();
|
||||
for pair in names.windows(2) {
|
||||
h.record_winner(&pair[0], &pair[1], 1).unwrap();
|
||||
}
|
||||
let _ = h.converge().unwrap();
|
||||
h
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reject_is_the_default() {
|
||||
let h = history_with(&["a", "b"], 0.0);
|
||||
assert!(matches!(
|
||||
h.predict_outcome(&[&[&"a"], &[&"ghost"]]),
|
||||
Err(InferenceError::UnknownKey { .. })
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prior_answers_instead_of_erroring() {
|
||||
let h = history_with_policy(&["a", "b"], trueskill_tt::UnknownKeys::Prior);
|
||||
let p = h
|
||||
.predict_outcome(&[&[&"a"], &[&"ghost"]])
|
||||
.expect("Prior should answer rather than reject");
|
||||
assert!((p.total() - 1.0).abs() < 1e-6);
|
||||
}
|
||||
|
||||
/// Two competitors the model has never seen are genuinely a coin flip. The
|
||||
/// point is that this is now *derived* rather than a constant a caller
|
||||
/// substitutes after swallowing an error.
|
||||
#[test]
|
||||
fn two_unknown_competitors_are_an_honest_coin_flip() {
|
||||
let h = history_with_policy(&["a", "b"], trueskill_tt::UnknownKeys::Prior);
|
||||
let wins = h
|
||||
.predict_win_probabilities(&[&[&"nobody"], &[&"no_one"]])
|
||||
.unwrap();
|
||||
assert!((wins[0] - 0.5).abs() < 1e-9, "{wins:?}");
|
||||
assert!((wins[1] - 0.5).abs() < 1e-9, "{wins:?}");
|
||||
}
|
||||
|
||||
/// The property that rules out a `Skip` mode: an unknown member must make a
|
||||
/// team *less* certain, never more. Skipping would drop the member's variance
|
||||
/// from the sum and narrow the team, which is backwards.
|
||||
#[test]
|
||||
fn an_unknown_member_widens_its_team_rather_than_narrowing_it() {
|
||||
let h = history_with_policy(&["a", "b", "c"], trueskill_tt::UnknownKeys::Prior);
|
||||
|
||||
// "a" alone against "b" — then "a" plus an unknown partner against "b".
|
||||
let solo = h.predict_win_probabilities(&[&[&"a"], &[&"b"]]).unwrap();
|
||||
let with_unknown = h
|
||||
.predict_win_probabilities(&[&[&"a", &"stranger"], &[&"b"]])
|
||||
.unwrap();
|
||||
|
||||
// Adding an unknown partner pulls the outcome toward even, because the
|
||||
// team's performance spread grew.
|
||||
assert!(
|
||||
(with_unknown[0] - 0.5).abs() < (solo[0] - 0.5).abs(),
|
||||
"an unknown partner should make the result less certain: solo {solo:?}, \
|
||||
with unknown {with_unknown:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prior_reaches_every_prediction_entry_point() {
|
||||
let h = history_with_policy(&["a", "b"], trueskill_tt::UnknownKeys::Prior);
|
||||
let teams: &[&[&&str]] = &[&[&"a"], &[&"ghost"]];
|
||||
|
||||
assert!(h.quality(teams).is_ok());
|
||||
assert!(h.predict_win_probabilities(teams).is_ok());
|
||||
assert!(h.predict_outcome(teams).is_ok());
|
||||
assert!(h.predict_ranking(teams, &[0, 1]).is_ok());
|
||||
assert!(h.expected_information_gain(teams).is_ok());
|
||||
}
|
||||
@@ -0,0 +1,162 @@
|
||||
//! Bounds that any correct implementation must satisfy, swept rather than
|
||||
//! spot-checked.
|
||||
//!
|
||||
//! The crate's docs call the `ln k` ceiling "the sharpest available test of an
|
||||
//! implementation", and record that an early prototype returned 4.77 nats. It
|
||||
//! was violated again — 3.237828 nats against `ln 2` — because the existing
|
||||
//! check sampled one fixture and the violation lives in a specific regime: a
|
||||
//! large ratio between the widest and narrowest performance sigma, where the
|
||||
//! shared prediction grid could not resolve the narrow density and returned
|
||||
//! probabilities greater than one.
|
||||
//!
|
||||
//! A single fixture cannot defend a bound like this. A sweep can.
|
||||
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, GameOptions, Gaussian, InferenceError, Rating, expected_information_gain,
|
||||
};
|
||||
|
||||
type R = Rating<i64, ConstantDrift>;
|
||||
|
||||
/// How many random matchups the ceiling sweep draws.
|
||||
///
|
||||
/// Scaled by build profile rather than fixed. Each sample runs a full inference
|
||||
/// pass per outcome, and that is about **19x** faster in release — measured,
|
||||
/// 20 000 samples take 12.1s released against 23s for 2 000 in debug. `just
|
||||
/// test` runs three debug feature combinations and one release one, so a fixed
|
||||
/// count pays the slow price three times and the fast one once, which is
|
||||
/// exactly backwards.
|
||||
///
|
||||
/// The debug run is here to prove the sweep still compiles and holds on a small
|
||||
/// sample; the release run is the one that actually searches. The violation
|
||||
/// this guards was found at a rate near 1.8%, so even the debug count expects
|
||||
/// tens of hits in the regime.
|
||||
#[cfg(debug_assertions)]
|
||||
const SAMPLES: usize = 1_000;
|
||||
#[cfg(not(debug_assertions))]
|
||||
const SAMPLES: usize = 50_000;
|
||||
|
||||
/// Deterministic LCG, so a failure is reproducible from the printed seed.
|
||||
struct Lcg(u64);
|
||||
|
||||
impl Lcg {
|
||||
fn next_f64(&mut self) -> f64 {
|
||||
self.0 = self
|
||||
.0
|
||||
.wrapping_mul(6_364_136_223_846_793_005)
|
||||
.wrapping_add(1_442_695_040_888_963_407);
|
||||
// Top 53 bits to [0, 1).
|
||||
((self.0 >> 11) as f64) / ((1u64 << 53) as f64)
|
||||
}
|
||||
|
||||
fn in_range(&mut self, lo: f64, hi: f64) -> f64 {
|
||||
lo + (hi - lo) * self.next_f64()
|
||||
}
|
||||
|
||||
/// Log-uniform, so the sweep spends its samples across magnitudes rather
|
||||
/// than crowding the top of the range — the violations live at small sigma.
|
||||
fn log_uniform(&mut self, lo: f64, hi: f64) -> f64 {
|
||||
let t = self.next_f64();
|
||||
(lo.ln() + t * (hi.ln() - lo.ln())).exp()
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn information_gain_never_exceeds_the_entropy_of_the_outcome() {
|
||||
let mut rng = Lcg(0x5eed_1234_abcd_ef01);
|
||||
let ceiling = 2.0_f64.ln();
|
||||
let mut evaluated = 0usize;
|
||||
let mut refused = 0usize;
|
||||
|
||||
for i in 0..SAMPLES {
|
||||
let mu_a = rng.in_range(-100.0, 100.0);
|
||||
let mu_b = rng.in_range(-100.0, 100.0);
|
||||
let sigma_a = rng.log_uniform(1e-4, 1e2);
|
||||
let sigma_b = rng.log_uniform(1e-4, 1e2);
|
||||
let beta = rng.log_uniform(1e-4, 1e1);
|
||||
|
||||
let a = R::new(
|
||||
Gaussian::from_ms(mu_a, sigma_a),
|
||||
beta,
|
||||
ConstantDrift::new(0.0),
|
||||
);
|
||||
let b = R::new(
|
||||
Gaussian::from_ms(mu_b, sigma_b),
|
||||
beta,
|
||||
ConstantDrift::new(0.0),
|
||||
);
|
||||
let options = GameOptions {
|
||||
p_draw: 0.0,
|
||||
..GameOptions::default()
|
||||
};
|
||||
|
||||
match expected_information_gain(&[&[a], &[b]], &options) {
|
||||
Ok(gain) => {
|
||||
evaluated += 1;
|
||||
assert!(
|
||||
gain.is_finite(),
|
||||
"sample {i}: non-finite gain {gain} \
|
||||
(mu {mu_a}, {mu_b}; sigma {sigma_a:e}, {sigma_b:e}; beta {beta:e})"
|
||||
);
|
||||
assert!(
|
||||
gain >= 0.0,
|
||||
"sample {i}: negative gain {gain} \
|
||||
(mu {mu_a}, {mu_b}; sigma {sigma_a:e}, {sigma_b:e}; beta {beta:e})"
|
||||
);
|
||||
assert!(
|
||||
gain <= ceiling + 1e-9,
|
||||
"sample {i}: gain {gain} exceeds ln 2 = {ceiling} \
|
||||
(mu {mu_a}, {mu_b}; sigma {sigma_a:e}, {sigma_b:e}; beta {beta:e})"
|
||||
);
|
||||
}
|
||||
// Refusing to answer is acceptable; answering wrongly is not.
|
||||
Err(InferenceError::GridTooCoarse { .. }) => refused += 1,
|
||||
Err(e) => panic!("sample {i}: unexpected error {e:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
// The sweep must actually exercise the function, not pass by refusing
|
||||
// everything.
|
||||
assert!(
|
||||
evaluated * 2 > SAMPLES,
|
||||
"only {evaluated} of {SAMPLES} samples were evaluated ({refused} refused); \
|
||||
the sweep is no longer testing anything"
|
||||
);
|
||||
// And it must still reach the regime where the ceiling was violated —
|
||||
// large sigma ratios, which is exactly where the grid now refuses. Without
|
||||
// this the sweep could drift into only-easy inputs and stop being a guard.
|
||||
assert!(
|
||||
refused > 0,
|
||||
"no sample reached the coarse-grid regime; the sweep no longer covers \
|
||||
the case that produced 3.24 nats"
|
||||
);
|
||||
}
|
||||
|
||||
/// The regime that produced 3.237828 nats, pinned exactly.
|
||||
#[test]
|
||||
fn the_known_ceiling_violation_no_longer_answers_wrongly() {
|
||||
let a = R::new(
|
||||
Gaussian::from_ms(9.577_887_112_129_012, 0.000_132_507_526_585_134_38),
|
||||
0.000_307_235_559_013_096_2,
|
||||
ConstantDrift::new(0.0),
|
||||
);
|
||||
let b = R::new(
|
||||
Gaussian::from_ms(-14.114_932_828_525_696, 91.586_690_140_921_16),
|
||||
0.000_307_235_559_013_096_2,
|
||||
ConstantDrift::new(0.0),
|
||||
);
|
||||
let options = GameOptions {
|
||||
p_draw: 0.0,
|
||||
..GameOptions::default()
|
||||
};
|
||||
|
||||
match expected_information_gain(&[&[a], &[b]], &options) {
|
||||
Ok(gain) => assert!(
|
||||
gain <= 2.0_f64.ln() + 1e-9,
|
||||
"returned {gain}, over the ln 2 ceiling"
|
||||
),
|
||||
Err(InferenceError::GridTooCoarse { needed, max, .. }) => {
|
||||
assert!(needed > max, "needed {needed} should exceed max {max}");
|
||||
}
|
||||
Err(e) => panic!("unexpected error {e:?}"),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,152 @@
|
||||
//! No prediction path may answer from a fit it cannot answer from.
|
||||
//!
|
||||
//! `converge` grew a `NonFiniteResult` guard; nothing stopped a caller from
|
||||
//! ignoring that error and predicting anyway. The three failures that produced
|
||||
//! were each differently wrong: `Ok(NaN)`, a panic out of a `Result`-returning
|
||||
//! method, and `Ok([0.0, 0.0])` — finite, plausible, summing to zero against a
|
||||
//! doc that promises one.
|
||||
//!
|
||||
//! Every test here has a healthy control, so none can pass by everything
|
||||
//! returning `Err`.
|
||||
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, Event, Gaussian, History, InferenceError, Member, NullObserver, Outcome, Team,
|
||||
};
|
||||
|
||||
type H = History<i64, ConstantDrift, NullObserver, &'static str>;
|
||||
|
||||
fn build(beta: f64, prior: Option<Gaussian>, outcome: Outcome) -> H {
|
||||
let mut h: H = History::builder()
|
||||
.beta(beta)
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.build();
|
||||
|
||||
let member = |k: &'static str| match prior {
|
||||
Some(p) => Member::new(k).with_prior(p),
|
||||
None => Member::new(k),
|
||||
};
|
||||
|
||||
let _ = h.add_events(vec![Event {
|
||||
time: 1,
|
||||
teams: [
|
||||
Team::with_members([member("a")]),
|
||||
Team::with_members([member("b")]),
|
||||
]
|
||||
.into_iter()
|
||||
.collect(),
|
||||
outcome,
|
||||
}]);
|
||||
h
|
||||
}
|
||||
|
||||
/// Point-mass priors with `beta(0.0)` on a *ranked* event: `converge` reports
|
||||
/// `NonFiniteResult` and the stored posteriors are `pi: NaN, tau: NaN`.
|
||||
fn nan_poisoned() -> H {
|
||||
let mut h = build(
|
||||
0.0,
|
||||
Some(Gaussian::from_ms(0.0, 0.0)),
|
||||
Outcome::winner(0, 2),
|
||||
);
|
||||
let err = h.converge().expect_err("this fixture must not converge");
|
||||
assert!(
|
||||
matches!(err, InferenceError::NonFiniteResult { .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
h
|
||||
}
|
||||
|
||||
/// The same degenerate parameters on a *scored* event, where inference
|
||||
/// converges cleanly and leaves legitimate point-mass posteriors behind. The
|
||||
/// fit is fine; it is prediction that has nothing to work with.
|
||||
fn degenerate_but_converged() -> H {
|
||||
let mut h = build(
|
||||
0.0,
|
||||
Some(Gaussian::from_ms(0.0, 0.0)),
|
||||
Outcome::scores([1.0, 0.0]),
|
||||
);
|
||||
h.converge().expect("this fixture converges");
|
||||
h
|
||||
}
|
||||
|
||||
fn healthy() -> H {
|
||||
let mut h = build(1.0, None, Outcome::winner(0, 2));
|
||||
h.converge().expect("control converges");
|
||||
h
|
||||
}
|
||||
|
||||
macro_rules! all_predictions {
|
||||
($h:ident, $f:expr) => {{
|
||||
let teams: &[&[&&'static str]] = &[&[&"a"], &[&"b"]];
|
||||
let f = $f;
|
||||
f("quality", $h.quality(teams).map(|_| ()));
|
||||
f(
|
||||
"predict_win_probabilities",
|
||||
$h.predict_win_probabilities(teams).map(|_| ()),
|
||||
);
|
||||
f("predict_outcome", $h.predict_outcome(teams).map(|_| ()));
|
||||
f(
|
||||
"predict_ranking",
|
||||
$h.predict_ranking(teams, &[0, 1]).map(|_| ()),
|
||||
);
|
||||
f(
|
||||
"expected_information_gain",
|
||||
$h.expected_information_gain(teams).map(|_| ()),
|
||||
);
|
||||
}};
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_nan_poisoned_fit_is_refused_by_every_prediction_path() {
|
||||
let h = nan_poisoned();
|
||||
|
||||
all_predictions!(h, |name: &str, r: Result<(), InferenceError>| {
|
||||
match r {
|
||||
Err(InferenceError::NonFiniteResult { .. }) => {}
|
||||
other => panic!("{name} answered from a NaN fit: {other:?}"),
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn degenerate_performances_are_refused_rather_than_answered_wrongly() {
|
||||
let h = degenerate_but_converged();
|
||||
|
||||
// The fit itself is sound — the posteriors are point masses, not NaN.
|
||||
let skill = h.current_skill("a").expect("a played");
|
||||
assert_eq!(skill.sigma(), 0.0);
|
||||
assert!(skill.mu().is_finite());
|
||||
|
||||
// `quality` previously PANICKED here, out of a method that returns
|
||||
// `Result`: the contrast covariance is exactly singular when beta is zero
|
||||
// and every skill is a point mass.
|
||||
all_predictions!(h, |name: &str, r: Result<(), InferenceError>| {
|
||||
match r {
|
||||
Err(InferenceError::InvalidParameter { .. }) => {}
|
||||
other => panic!("{name} predicted from a degenerate fit: {other:?}"),
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_control_history_answers_every_prediction() {
|
||||
let h = healthy();
|
||||
|
||||
all_predictions!(h, |name: &str, r: Result<(), InferenceError>| {
|
||||
assert!(r.is_ok(), "{name} failed on a healthy history: {r:?}");
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn win_probabilities_sum_to_one_on_the_control() {
|
||||
// The promise the `Ok([0.0, 0.0])` case broke. Asserted on the control so
|
||||
// the guard above cannot be "fixed" by making every path error.
|
||||
let h = healthy();
|
||||
let p = h
|
||||
.predict_win_probabilities(&[&[&"a"], &[&"b"]])
|
||||
.expect("control predicts");
|
||||
let total: f64 = p.iter().sum();
|
||||
assert!(
|
||||
(total - 1.0).abs() < 1e-6,
|
||||
"win probabilities sum to {total}"
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
# Seeds for failure cases proptest has generated in the past. It is
|
||||
# automatically read and these particular cases re-run before any
|
||||
# novel cases are generated.
|
||||
#
|
||||
# It is recommended to check this file in to source control so that
|
||||
# everyone who runs the test benefits from these saved cases.
|
||||
cc 8859be600e638573980f78622b8fcd8b4553ca34a9a041746c417f7e4293f89c # shrinks to games = [(0, 1), (0, 1), (4, 6), (2, 0), (0, 1), (0, 6), (0, 1), (2, 0), (6, 4), (0, 1), (0, 2), (6, 4), (1, 0), (4, 0), (0, 2)]
|
||||
+29
-8
@@ -26,7 +26,10 @@ const KEYS: [&str; 8] = ["a", "b", "c", "d", "e", "f", "g", "h"];
|
||||
fn history_from(games: &[(usize, usize)]) -> History {
|
||||
let mut h = History::builder()
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 200,
|
||||
// 200 was not enough: the batched side stopped at the cap with a
|
||||
// step of 3.4e-9, so this test was comparing two truncated fits and
|
||||
// attributing the gap to ingestion order.
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-10,
|
||||
..ConvergenceOptions::default()
|
||||
})
|
||||
@@ -61,10 +64,15 @@ proptest! {
|
||||
fn converged_posteriors_are_always_finite(games in pairs()) {
|
||||
let mut h = history_from(&games);
|
||||
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
for key in KEYS {
|
||||
for (time, g) in h.learning_curve(key) {
|
||||
// A generated schedule need not touch every key, and an unplayed
|
||||
// key is `None` rather than an empty curve.
|
||||
let Some(curve) = h.learning_curve(key) else {
|
||||
continue;
|
||||
};
|
||||
for (time, g) in curve {
|
||||
assert_finite(g, &format!("{key} at t={time}"));
|
||||
}
|
||||
}
|
||||
@@ -79,7 +87,7 @@ proptest! {
|
||||
fn log_evidence_is_a_finite_log_probability(games in pairs()) {
|
||||
let mut h = history_from(&games);
|
||||
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let batch = h.log_evidence();
|
||||
let filtered = h.filtered_log_evidence();
|
||||
@@ -98,7 +106,7 @@ proptest! {
|
||||
|
||||
let before = h.filtered_log_evidence();
|
||||
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let after = h.filtered_log_evidence();
|
||||
|
||||
@@ -114,14 +122,21 @@ proptest! {
|
||||
fn ingestion_order_does_not_change_the_answer(games in pairs()) {
|
||||
let batched = {
|
||||
let mut h = history_from(&games);
|
||||
h.converge().unwrap();
|
||||
let report = h.converge().unwrap();
|
||||
prop_assert!(
|
||||
report.converged,
|
||||
"batched side stopped at {} iterations with step {:?}; comparing \
|
||||
two fits that have not converged measures truncation, not order",
|
||||
report.iterations,
|
||||
report.final_step
|
||||
);
|
||||
h
|
||||
};
|
||||
|
||||
let incremental = {
|
||||
let mut h = History::builder()
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 200,
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-10,
|
||||
..ConvergenceOptions::default()
|
||||
})
|
||||
@@ -139,7 +154,13 @@ proptest! {
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
h.converge().unwrap();
|
||||
let report = h.converge().unwrap();
|
||||
prop_assert!(
|
||||
report.converged,
|
||||
"incremental side stopped at {} iterations with step {:?}",
|
||||
report.iterations,
|
||||
report.final_step
|
||||
);
|
||||
h
|
||||
};
|
||||
|
||||
|
||||
+102
-9
@@ -1,4 +1,4 @@
|
||||
//! `quality()` beyond two rating groups.
|
||||
//! `quality()` beyond two teams.
|
||||
//!
|
||||
//! The historical golden (two equal singletons) is asserted in
|
||||
//! `src/lib.rs::tests::test_quality`. These cover the N-group generalisation,
|
||||
@@ -82,14 +82,14 @@ fn uneven_group_sizes_work() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "at least 2 rating groups")]
|
||||
#[should_panic(expected = "at least 2 teams")]
|
||||
fn single_group_panics_with_clear_message() {
|
||||
let r = rating(25.0, 3.0);
|
||||
let _ = quality(&[&[r]], BETA);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "at least 2 rating groups")]
|
||||
#[should_panic(expected = "at least 2 teams")]
|
||||
fn zero_groups_panics_with_clear_message() {
|
||||
let _ = quality(&[], BETA);
|
||||
}
|
||||
@@ -108,12 +108,105 @@ fn history_predict_quality_supports_three_teams() {
|
||||
let mut h = History::default();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
h.record_winner(&"b", &"c", 2).unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let q = h.predict_quality(&[&[&"a"], &[&"b"], &[&"c"]]);
|
||||
assert!(
|
||||
q.is_finite(),
|
||||
"3-team predict_quality must be finite, got {q}"
|
||||
);
|
||||
let q = h.quality(&[&[&"a"], &[&"b"], &[&"c"]]).unwrap();
|
||||
assert!(q.is_finite(), "3-team quality must be finite, got {q}");
|
||||
assert!((0.0..=1.0).contains(&q), "out of range: {q}");
|
||||
}
|
||||
|
||||
/// `quality()` for N identical teams has a closed form, which pins the N-group
|
||||
/// determinant path across the whole range rather than at a single golden.
|
||||
///
|
||||
/// For two identical single-player teams the standard result is
|
||||
/// `sqrt(2b^2 / (2b^2 + s1^2 + s2^2))`. With the conventional parameters
|
||||
/// (`sigma = 25/3`, `beta = 25/6`) that ratio is exactly `1/5`, and the N-group
|
||||
/// generalisation is `(1/5)^((n-1)/2)` — one factor per adjacent pair.
|
||||
///
|
||||
/// The n=3 and n=5 values this produces (0.200 and 0.040) are also what the
|
||||
/// `trueskill` Python package returns for the same configuration, so this
|
||||
/// doubles as the cross-implementation check the README asked for.
|
||||
#[test]
|
||||
fn quality_of_identical_teams_follows_its_closed_form() {
|
||||
let g = Gaussian::from_ms(25.0, 25.0 / 3.0);
|
||||
let beta = 25.0 / 6.0;
|
||||
|
||||
for n in 2..=10usize {
|
||||
let groups: Vec<Vec<Gaussian>> = (0..n).map(|_| vec![g]).collect();
|
||||
let refs: Vec<&[Gaussian]> = groups.iter().map(Vec::as_slice).collect();
|
||||
|
||||
let got = quality(&refs, beta);
|
||||
let expected = 0.2f64.powf((n - 1) as f64 / 2.0);
|
||||
|
||||
assert!(
|
||||
(got - expected).abs() / expected < 1e-9,
|
||||
"n={n}: quality {got}, closed form {expected}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Spot-check against the two values the `trueskill` Python package is known
|
||||
/// to produce for this configuration, stated as literals so a future change to
|
||||
/// the closed-form reasoning above cannot quietly take these with it.
|
||||
#[test]
|
||||
fn quality_matches_the_reference_implementation() {
|
||||
let g = Gaussian::from_ms(25.0, 25.0 / 3.0);
|
||||
let beta = 25.0 / 6.0;
|
||||
|
||||
let three: Vec<Vec<Gaussian>> = (0..3).map(|_| vec![g]).collect();
|
||||
let refs: Vec<&[Gaussian]> = three.iter().map(Vec::as_slice).collect();
|
||||
assert!((quality(&refs, beta) - 0.200).abs() < 1e-9);
|
||||
|
||||
let five: Vec<Vec<Gaussian>> = (0..5).map(|_| vec![g]).collect();
|
||||
let refs: Vec<&[Gaussian]> = five.iter().map(Vec::as_slice).collect();
|
||||
assert!((quality(&refs, beta) - 0.040).abs() < 1e-9);
|
||||
}
|
||||
|
||||
/// `quality()` used to compute `det(ata) / det(middle)` in linear space. Both
|
||||
/// are products of `k - 1` diagonal entries, so they leave `f64`'s range long
|
||||
/// before their ratio does — and the ratio is the only thing the answer needs.
|
||||
///
|
||||
/// Measured before the fix: at the crate defaults 150 groups was correct, 200
|
||||
/// returned `0`, and 250 returned `NaN` where the truth is `9.51e-88`. With a
|
||||
/// small beta it bit sooner — `sigma = beta = 1e-3` returned `NaN` at 60 groups
|
||||
/// against a true `1.32e-9`, a value that is entirely ordinary.
|
||||
///
|
||||
/// For `k` single-member groups with equal means the answer has a closed form,
|
||||
/// `(beta / sqrt(beta^2 + sigma^2))^(k-1)`, so this checks against arithmetic
|
||||
/// rather than against a recorded output.
|
||||
#[test]
|
||||
fn quality_matches_its_closed_form_past_the_overflow_point() {
|
||||
for (sigma, beta) in [(25.0 / 3.0, 25.0 / 6.0), (1e-3, 1e-3), (50.0, 25.0 / 6.0)] {
|
||||
let rating = vec![Gaussian::from_ms(25.0, sigma)];
|
||||
for k in [2usize, 50, 60, 150, 200, 250, 300] {
|
||||
let groups: Vec<&[Gaussian]> = (0..k).map(|_| rating.as_slice()).collect();
|
||||
let got = quality(&groups, beta);
|
||||
let expected = (beta / (beta * beta + sigma * sigma).sqrt()).powi(k as i32 - 1);
|
||||
|
||||
assert!(
|
||||
got.is_finite(),
|
||||
"sigma {sigma}, beta {beta}, {k} groups: got {got}"
|
||||
);
|
||||
// Subnormal results have no relative precision left to check.
|
||||
if expected > f64::MIN_POSITIVE {
|
||||
let rel = ((got - expected) / expected).abs();
|
||||
assert!(
|
||||
rel < 1e-11,
|
||||
"sigma {sigma}, beta {beta}, {k} groups: got {got:e}, \
|
||||
closed form {expected:e}, rel {rel:e}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The overflow was in the intermediates, never in the answer: every value
|
||||
/// above is an ordinary float. This pins the specific case that returned `NaN`
|
||||
/// where the true answer is nine orders of magnitude inside the normal range.
|
||||
#[test]
|
||||
fn a_small_beta_does_not_overflow_at_sixty_groups() {
|
||||
let rating = vec![Gaussian::from_ms(25.0, 1e-3)];
|
||||
let groups: Vec<&[Gaussian]> = (0..60).map(|_| rating.as_slice()).collect();
|
||||
let got = quality(&groups, 1e-3);
|
||||
assert!((got - 1.317_089e-9).abs() / 1.317_089e-9 < 1e-6, "{got:e}");
|
||||
}
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
//! Converging, appending, and converging again must reach the same fixed point
|
||||
//! as converging once over the whole event set.
|
||||
//!
|
||||
//! `tests/ingestion_equivalence.rs` covers a different question: it varies how
|
||||
//! events are *batched* but converges only at the end. This file converges
|
||||
//! between batches, which is the path a caller takes when it fits, serves for a
|
||||
//! while, then ingests more.
|
||||
//!
|
||||
//! The property matters beyond ergonomics. It says `converge` reaches a fixed
|
||||
//! point determined by the events, ratings and configuration alone — not by the
|
||||
//! message state it started from. That is what makes a restored snapshot safe:
|
||||
//! an inexact one cannot corrupt the answer, only cost an extra sweep. See #45.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{ConvergenceOptions, Event, Gaussian, History, Member, Outcome, Team};
|
||||
|
||||
fn tight() -> ConvergenceOptions {
|
||||
ConvergenceOptions {
|
||||
max_iter: 5_000,
|
||||
epsilon: 1e-12,
|
||||
alpha: 1.0,
|
||||
}
|
||||
}
|
||||
|
||||
fn ev(a: &str, b: &str, time: i64) -> Event<i64, String> {
|
||||
Event {
|
||||
time,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(a.to_string())]),
|
||||
Team::with_members([Member::new(b.to_string())]),
|
||||
],
|
||||
outcome: Outcome::winner(0, 2),
|
||||
}
|
||||
}
|
||||
|
||||
/// Ingest each chunk in turn, converging fully after every one.
|
||||
fn fit_in_chunks(chunks: Vec<Events>) -> Vec<(String, Gaussian)> {
|
||||
let mut h: History<i64, _, _, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.convergence(tight())
|
||||
.build();
|
||||
|
||||
for chunk in chunks {
|
||||
h.add_events(chunk).unwrap();
|
||||
let report = h.converge().unwrap();
|
||||
assert!(
|
||||
report.converged,
|
||||
"a chunk failed to converge, so any comparison would be measuring \
|
||||
truncation rather than the fixed point; final step {:?}",
|
||||
report.final_step
|
||||
);
|
||||
}
|
||||
|
||||
let mut skills: Vec<(String, Gaussian)> = h
|
||||
.learning_curves()
|
||||
.into_iter()
|
||||
.map(|(k, curve)| (k, curve.last().unwrap().1))
|
||||
.collect();
|
||||
skills.sort_by(|a, b| a.0.cmp(&b.0));
|
||||
skills
|
||||
}
|
||||
|
||||
fn assert_same(a: &[(String, Gaussian)], b: &[(String, Gaussian)], what: &str) {
|
||||
assert_eq!(a.len(), b.len(), "{what}: competitor count differs");
|
||||
for ((ka, ga), (kb, gb)) in a.iter().zip(b) {
|
||||
assert_eq!(ka, kb, "{what}: key order differs");
|
||||
// Measured: 6.2e-13 for a later append, 8.9e-11 for an interleaved one.
|
||||
// The bar is well clear of both but far under anything that would let a
|
||||
// genuine divergence through.
|
||||
assert!(
|
||||
(ga.mu() - gb.mu()).abs() < 1e-8 && (ga.sigma() - gb.sigma()).abs() < 1e-8,
|
||||
"{what}: {ka} differs — one-shot mu={} sigma={}, chunked mu={} sigma={}",
|
||||
ga.mu(),
|
||||
ga.sigma(),
|
||||
gb.mu(),
|
||||
gb.sigma()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
type Events = Vec<Event<i64, String>>;
|
||||
|
||||
/// Two chunks of events: the first at times 0..20, the second at 100..120.
|
||||
fn fixture() -> (Events, Events) {
|
||||
let names = ["a", "b", "c", "d", "e"];
|
||||
let mut seed = 7u64;
|
||||
let mut rnd = move || {
|
||||
seed ^= seed << 13;
|
||||
seed ^= seed >> 7;
|
||||
seed ^= seed << 17;
|
||||
seed
|
||||
};
|
||||
|
||||
let (mut early, mut late) = (Vec::new(), Vec::new());
|
||||
for t in 0..40i64 {
|
||||
let i = (rnd() % 5) as usize;
|
||||
let mut j = (rnd() % 5) as usize;
|
||||
if j == i {
|
||||
j = (j + 1) % 5;
|
||||
}
|
||||
if t < 20 {
|
||||
early.push(ev(names[i], names[j], t));
|
||||
} else {
|
||||
late.push(ev(names[i], names[j], 100 + t));
|
||||
}
|
||||
}
|
||||
(early, late)
|
||||
}
|
||||
|
||||
/// The ordinary case: new events are strictly later than everything fitted.
|
||||
#[test]
|
||||
fn appending_later_events_matches_a_single_fit() {
|
||||
let (early, late) = fixture();
|
||||
let all: Vec<_> = early.iter().cloned().chain(late.iter().cloned()).collect();
|
||||
|
||||
assert_same(
|
||||
&fit_in_chunks(vec![all]),
|
||||
&fit_in_chunks(vec![early, late]),
|
||||
"append strictly later",
|
||||
);
|
||||
}
|
||||
|
||||
/// The case the design question suspected might be weaker: appended events
|
||||
/// interleave with slices that are already fitted, so the append legitimately
|
||||
/// revises the past. It is not weaker — Through Time revises the past on every
|
||||
/// converge regardless, so there is nothing special about doing it in two steps.
|
||||
#[test]
|
||||
fn appending_interleaved_events_matches_a_single_fit() {
|
||||
let (early, late) = fixture();
|
||||
let all: Vec<_> = early.iter().cloned().chain(late.iter().cloned()).collect();
|
||||
|
||||
// Split by parity so the second chunk is back-dated into the first's range.
|
||||
let first: Vec<_> = all.iter().step_by(2).cloned().collect();
|
||||
let second: Vec<_> = all.iter().skip(1).step_by(2).cloned().collect();
|
||||
let together: Vec<_> = first
|
||||
.iter()
|
||||
.cloned()
|
||||
.chain(second.iter().cloned())
|
||||
.collect();
|
||||
|
||||
assert_same(
|
||||
&fit_in_chunks(vec![together]),
|
||||
&fit_in_chunks(vec![first, second]),
|
||||
"append interleaved",
|
||||
);
|
||||
}
|
||||
|
||||
/// Converging an already-converged history is a no-op, which is what makes a
|
||||
/// restored snapshot worth having: the work is skipped rather than redone.
|
||||
#[test]
|
||||
fn re_converging_an_unchanged_history_costs_one_iteration() {
|
||||
let (early, late) = fixture();
|
||||
let all: Vec<_> = early.into_iter().chain(late).collect();
|
||||
|
||||
let mut h: History<i64, _, _, String> = History::builder()
|
||||
.key_type::<String>()
|
||||
.convergence(tight())
|
||||
.build();
|
||||
h.add_events(all).unwrap();
|
||||
let first = h.converge().unwrap();
|
||||
assert!(first.converged);
|
||||
|
||||
let again = h.converge().unwrap();
|
||||
assert_eq!(
|
||||
again.iterations, 1,
|
||||
"a converged history should settle immediately, not re-grind"
|
||||
);
|
||||
assert!(again.converged);
|
||||
}
|
||||
+24
-16
@@ -6,7 +6,7 @@ fn record_winner_builds_history() {
|
||||
.mu(25.0)
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.drift(ConstantDrift(25.0 / 300.0))
|
||||
.drift(ConstantDrift::new(25.0 / 300.0))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 30,
|
||||
epsilon: 1e-6,
|
||||
@@ -15,26 +15,34 @@ fn record_winner_builds_history() {
|
||||
.build();
|
||||
|
||||
h.record_winner(&"alice", &"bob", 1).unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let a_idx = h.lookup(&"alice").unwrap();
|
||||
let b_idx = h.lookup(&"bob").unwrap();
|
||||
|
||||
assert_ne!(a_idx, b_idx);
|
||||
// `lookup` returned an `Index` that nothing public accepted, so the
|
||||
// observable claim is the one worth making: two distinct competitors, each
|
||||
// with their own posterior, and the winner ahead.
|
||||
assert_eq!(h.competitor_count(), 2);
|
||||
let alice = h.current_skill("alice").expect("alice played");
|
||||
let bob = h.current_skill("bob").expect("bob played");
|
||||
assert!(alice.mu() > bob.mu());
|
||||
}
|
||||
|
||||
/// The same key names the same competitor across events, which is what
|
||||
/// interning bought and the only part of it a caller can observe.
|
||||
#[test]
|
||||
fn intern_is_idempotent() {
|
||||
fn a_repeated_key_is_one_competitor() {
|
||||
let mut h: History = History::builder().build();
|
||||
let a1 = h.intern(&"alice");
|
||||
let a2 = h.intern(&"alice");
|
||||
assert_eq!(a1, a2);
|
||||
h.record_winner(&"alice", &"bob", 1).unwrap();
|
||||
h.record_winner(&"alice", &"carol", 2).unwrap();
|
||||
|
||||
assert_eq!(h.competitor_count(), 3);
|
||||
assert_eq!(h.learning_curve("alice").expect("known").len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lookup_returns_none_for_missing() {
|
||||
fn an_unknown_key_is_unknown() {
|
||||
let h: History = History::builder().build();
|
||||
assert!(h.lookup(&"nobody").is_none());
|
||||
assert!(h.current_skill("nobody").is_none());
|
||||
assert!(h.learning_curve("nobody").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -43,13 +51,13 @@ fn record_draw_with_p_draw_set() {
|
||||
.mu(25.0)
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(25.0 / 6.0)
|
||||
.drift(ConstantDrift(25.0 / 300.0))
|
||||
.drift(ConstantDrift::new(25.0 / 300.0))
|
||||
.p_draw(0.25)
|
||||
.build();
|
||||
|
||||
h.record_draw(&"alice", &"bob", 1).unwrap();
|
||||
h.converge().unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
assert!(h.lookup(&"alice").is_some());
|
||||
assert!(h.lookup(&"bob").is_some());
|
||||
assert!(h.current_skill("alice").is_some());
|
||||
assert!(h.current_skill("bob").is_some());
|
||||
}
|
||||
|
||||
@@ -0,0 +1,338 @@
|
||||
//! Configuring a competitor before anything is observed about them.
|
||||
//!
|
||||
//! The configuration a competitor needs is usually a property of the domain —
|
||||
//! "every layout is static" — not of whichever event happens to mention them
|
||||
//! first. Stating it per-event meant every ingestion path had to remember it,
|
||||
//! and two of the four paths could not state it at all.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, Event, Gaussian, History, InferenceError, Member, Outcome,
|
||||
Team,
|
||||
};
|
||||
|
||||
type H = History<i64, ConstantDrift, trueskill_tt::NullObserver, &'static str>;
|
||||
|
||||
const PINNED: Gaussian = Gaussian::from_ms(2.0, 0.5);
|
||||
|
||||
fn history() -> H {
|
||||
History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.5))
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build()
|
||||
}
|
||||
|
||||
fn duel(
|
||||
a: &'static str,
|
||||
b: &'static str,
|
||||
t: i64,
|
||||
m: Option<Member<&'static str>>,
|
||||
) -> Event<i64, &'static str> {
|
||||
Event {
|
||||
time: t,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(a)]),
|
||||
Team::with_members([m.unwrap_or_else(|| Member::new(b))]),
|
||||
],
|
||||
outcome: Outcome::scores([5.0, 2.0]),
|
||||
}
|
||||
}
|
||||
|
||||
fn skills(h: &H) -> Vec<(&'static str, Gaussian)> {
|
||||
["player", "layout"]
|
||||
.into_iter()
|
||||
.map(|k| (k, h.current_skill(&k).unwrap()))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The headline contract.
|
||||
#[test]
|
||||
fn registering_matches_configuring_on_the_first_event() {
|
||||
let configured = {
|
||||
let mut h = history();
|
||||
h.add_events(vec![
|
||||
duel(
|
||||
"player",
|
||||
"layout",
|
||||
1,
|
||||
Some(
|
||||
Member::new("layout")
|
||||
.with_drift_scale(0.0)
|
||||
.with_prior(PINNED),
|
||||
),
|
||||
),
|
||||
duel("player", "layout", 2, None),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h
|
||||
};
|
||||
|
||||
let registered = {
|
||||
let mut h = history();
|
||||
h.register(
|
||||
Member::new("layout")
|
||||
.with_drift_scale(0.0)
|
||||
.with_prior(PINNED),
|
||||
)
|
||||
.unwrap();
|
||||
h.add_events(vec![
|
||||
duel("player", "layout", 1, None),
|
||||
duel("player", "layout", 2, None),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h
|
||||
};
|
||||
|
||||
for ((k, a), (_, b)) in skills(&configured).into_iter().zip(skills(®istered)) {
|
||||
assert_eq!(a.mu(), b.mu(), "{k} mu");
|
||||
assert_eq!(a.variance(), b.variance(), "{k} variance");
|
||||
}
|
||||
}
|
||||
|
||||
/// The case `EventBuilder` and the typed path cannot reach: a competitor whose
|
||||
/// first appearance arrives through the two-argument convenience route.
|
||||
#[test]
|
||||
fn registration_reaches_a_competitor_first_seen_through_record_winner() {
|
||||
let mut h = history();
|
||||
h.register(
|
||||
Member::new("layout")
|
||||
.with_drift_scale(0.0)
|
||||
.with_prior(PINNED),
|
||||
)
|
||||
.unwrap();
|
||||
h.record_winner(&"player", &"layout", 1).unwrap();
|
||||
h.record_winner(&"player", &"layout", 2).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let rating = h.rating(&"layout").unwrap();
|
||||
assert_eq!(rating.drift_scale(), 0.0);
|
||||
assert_eq!(rating.prior().mu(), PINNED.mu());
|
||||
|
||||
// Pinned means pinned: no drift across the two slices.
|
||||
let curve = h.learning_curve(&"layout").unwrap();
|
||||
assert!(curve.len() >= 2);
|
||||
let widest = curve
|
||||
.iter()
|
||||
.map(|(_, g)| g.sigma())
|
||||
.fold(f64::MIN, f64::max);
|
||||
let narrowest = curve
|
||||
.iter()
|
||||
.map(|(_, g)| g.sigma())
|
||||
.fold(f64::MAX, f64::min);
|
||||
assert!(
|
||||
(widest - narrowest) / widest < 1e-9,
|
||||
"{narrowest} .. {widest}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn registering_a_known_competitor_is_an_error() {
|
||||
let mut h = history();
|
||||
h.record_winner(&"player", &"layout", 1).unwrap();
|
||||
let err = h.register(Member::new("layout")).unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::AlreadyRegistered { .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn registering_twice_is_an_error() {
|
||||
let mut h = history();
|
||||
h.register(Member::new("layout").with_drift_scale(0.0))
|
||||
.unwrap();
|
||||
let err = h
|
||||
.register(Member::new("layout").with_drift_scale(1.0))
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::AlreadyRegistered { .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
// The first registration stands.
|
||||
assert_eq!(h.rating(&"layout").unwrap().drift_scale(), 0.0);
|
||||
}
|
||||
|
||||
/// `weight` is per-event and meaningless here, so it is rejected rather than
|
||||
/// dropped — dropping it silently is the defect class this whole area keeps
|
||||
/// producing.
|
||||
#[test]
|
||||
fn a_weight_on_a_registration_is_rejected() {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.register(Member::new("layout").with_weight(0.5))
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::InvalidParameter { name: "weight", .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_invalid_drift_scale_on_a_registration_is_rejected() {
|
||||
for bad in [-1.0, f64::NAN, f64::INFINITY] {
|
||||
let mut h = history();
|
||||
let err = h
|
||||
.register(Member::new("layout").with_drift_scale(bad))
|
||||
.unwrap_err();
|
||||
assert!(
|
||||
matches!(
|
||||
err,
|
||||
InferenceError::InvalidParameter {
|
||||
name: "drift_scale",
|
||||
..
|
||||
}
|
||||
),
|
||||
"{bad}: {err:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Registration makes the fit independent of the order events arrive in,
|
||||
/// which is what the per-event shape could not guarantee.
|
||||
#[test]
|
||||
fn registration_makes_the_fit_order_independent() {
|
||||
let build = |reversed: bool| {
|
||||
let mut h = history();
|
||||
h.register(
|
||||
Member::new("layout")
|
||||
.with_drift_scale(0.0)
|
||||
.with_prior(PINNED),
|
||||
)
|
||||
.unwrap();
|
||||
let mut events = vec![
|
||||
duel("player", "layout", 1, None),
|
||||
duel("player", "layout", 2, None),
|
||||
duel("player", "layout", 3, None),
|
||||
];
|
||||
if reversed {
|
||||
events.reverse();
|
||||
}
|
||||
h.add_events(events).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h
|
||||
};
|
||||
|
||||
let forward = build(false);
|
||||
let backward = build(true);
|
||||
for ((k, a), (_, b)) in skills(&forward).into_iter().zip(skills(&backward)) {
|
||||
assert_eq!(a.mu(), b.mu(), "{k} mu");
|
||||
assert_eq!(a.variance(), b.variance(), "{k} variance");
|
||||
}
|
||||
}
|
||||
|
||||
/// `rating` is the read-back that made a configuration mistake detectable from
|
||||
/// outside the crate at all. Every other accessor reports what inference
|
||||
/// inferred; this reports what it was told.
|
||||
#[test]
|
||||
fn rating_reads_back_what_was_stored() {
|
||||
let mut h = history();
|
||||
assert!(h.rating(&"nobody").is_none());
|
||||
|
||||
h.register(
|
||||
Member::new("layout")
|
||||
.with_drift_scale(0.25)
|
||||
.with_prior(PINNED),
|
||||
)
|
||||
.unwrap();
|
||||
let r = h.rating(&"layout").unwrap();
|
||||
assert_eq!(r.drift_scale(), 0.25);
|
||||
assert_eq!(r.prior().mu(), PINNED.mu());
|
||||
assert_eq!(r.prior().variance(), PINNED.variance());
|
||||
|
||||
// A competitor created by an event reports the history defaults.
|
||||
h.record_winner(&"player", &"layout", 1).unwrap();
|
||||
assert_eq!(h.rating(&"player").unwrap().drift_scale(), 1.0);
|
||||
}
|
||||
|
||||
/// The decision this issue turned on: two different values for one competitor
|
||||
/// are an error whether they arrive in one batch or two.
|
||||
///
|
||||
/// Last-write-wins across batches cut against the invariant
|
||||
/// `tests/ingestion_equivalence.rs` protects — the same contradictory events
|
||||
/// errored when batched and succeeded, order-dependently, one at a time.
|
||||
mod conflicting_configuration {
|
||||
use super::*;
|
||||
|
||||
fn seed(scale: f64) -> Event<i64, &'static str> {
|
||||
duel(
|
||||
"player",
|
||||
"layout",
|
||||
1,
|
||||
Some(Member::new("layout").with_drift_scale(scale)),
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn within_one_batch_is_an_error() {
|
||||
let mut h = history();
|
||||
let err = h.add_events(vec![seed(0.0), seed(1.0)]).unwrap_err();
|
||||
assert!(
|
||||
matches!(
|
||||
err,
|
||||
InferenceError::ConflictingCompetitorConfig {
|
||||
field: "drift_scale",
|
||||
..
|
||||
}
|
||||
),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn across_two_batches_is_also_an_error() {
|
||||
let mut h = history();
|
||||
h.add_events(vec![seed(0.0)]).unwrap();
|
||||
let err = h.add_events(vec![seed(1.0)]).unwrap_err();
|
||||
assert!(
|
||||
matches!(
|
||||
err,
|
||||
InferenceError::ConflictingCompetitorConfig {
|
||||
field: "drift_scale",
|
||||
..
|
||||
}
|
||||
),
|
||||
"{err:?}"
|
||||
);
|
||||
// Rejected before anything mutates: the first declaration stands.
|
||||
assert_eq!(h.rating(&"layout").unwrap().drift_scale(), 0.0);
|
||||
}
|
||||
|
||||
/// Repeating the *same* value stays inert, which is the expected shape
|
||||
/// when the configuration is a property of the domain.
|
||||
#[test]
|
||||
fn repeating_the_same_value_is_inert() {
|
||||
let mut h = history();
|
||||
h.add_events(vec![seed(0.0)]).unwrap();
|
||||
h.add_events(vec![seed(0.0)]).unwrap();
|
||||
assert_eq!(h.rating(&"layout").unwrap().drift_scale(), 0.0);
|
||||
}
|
||||
|
||||
/// A registration and a later event that agree are fine; one that
|
||||
/// disagrees is the same error.
|
||||
#[test]
|
||||
fn a_registration_conflicts_with_a_later_event() {
|
||||
let mut h = history();
|
||||
h.register(Member::new("layout").with_drift_scale(0.0))
|
||||
.unwrap();
|
||||
h.add_events(vec![seed(0.0)]).unwrap();
|
||||
|
||||
let mut h2 = history();
|
||||
h2.register(Member::new("layout").with_drift_scale(0.0))
|
||||
.unwrap();
|
||||
let err = h2.add_events(vec![seed(1.0)]).unwrap_err();
|
||||
assert!(
|
||||
matches!(err, InferenceError::ConflictingCompetitorConfig { .. }),
|
||||
"{err:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
+3
-3
@@ -9,7 +9,7 @@ fn scored_two_team_one_event_pulls_winner_up() {
|
||||
.mu(0.0)
|
||||
.sigma(2.0)
|
||||
.beta(1.0)
|
||||
.drift(ConstantDrift(0.0))
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.score_sigma(1.0)
|
||||
.build();
|
||||
|
||||
@@ -46,7 +46,7 @@ fn scored_zero_margin_treats_as_tie() {
|
||||
.mu(0.0)
|
||||
.sigma(2.0)
|
||||
.beta(1.0)
|
||||
.drift(ConstantDrift(0.0))
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.score_sigma(1.0)
|
||||
.build();
|
||||
|
||||
@@ -88,7 +88,7 @@ fn scored_three_team_partial_order() {
|
||||
.mu(0.0)
|
||||
.sigma(2.0)
|
||||
.beta(1.0)
|
||||
.drift(ConstantDrift(0.0))
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.score_sigma(1.0)
|
||||
.build();
|
||||
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
//! The `Time` generic, exercised end to end.
|
||||
//!
|
||||
//! `History<T: Time, ..>` has always been generic over the time axis, `Untimed`
|
||||
//! has always been exported, and `Drift<T>` is generic specifically so that
|
||||
//! "seasonal or calendar-aware drift is expressible without going through
|
||||
//! `i64`". None of it was reachable: every construction route pinned `T = i64`,
|
||||
//! `HistoryBuilder`'s fields are private, and its `Default` existed only for the
|
||||
//! `i64` instantiation.
|
||||
//!
|
||||
//! Nothing in the repository constructed a non-`i64` history, which is why that
|
||||
//! went unnoticed. This file is the guard against it recurring — it is as much
|
||||
//! about the generic being *exercised* as about any single assertion.
|
||||
|
||||
use trueskill_tt::{ConstantDrift, Drift, History, HistoryBuilder, Time, Untimed};
|
||||
|
||||
/// A domain time type: a season number. Exactly what the `Time` trait exists
|
||||
/// to support, and what a consumer with `chrono` dates would write.
|
||||
#[derive(Copy, Clone, Debug, PartialEq, Eq, PartialOrd, Ord)]
|
||||
struct Season(u16);
|
||||
|
||||
impl Time for Season {
|
||||
fn elapsed_to(&self, later: &Self) -> i64 {
|
||||
i64::from(later.0.saturating_sub(self.0))
|
||||
}
|
||||
}
|
||||
|
||||
/// Drift that only accumulates between seasons, not within one — the
|
||||
/// calendar-aware case the trait's own docs cite.
|
||||
#[derive(Copy, Clone, Debug)]
|
||||
struct SeasonalDrift {
|
||||
per_season: f64,
|
||||
}
|
||||
|
||||
impl Drift<Season> for SeasonalDrift {
|
||||
fn variance_delta(&self, from: &Season, to: &Season) -> f64 {
|
||||
self.variance_for_elapsed(from.elapsed_to(to))
|
||||
}
|
||||
|
||||
fn variance_for_elapsed(&self, elapsed: i64) -> f64 {
|
||||
elapsed.max(0) as f64 * self.per_season * self.per_season
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_untimed_history_fits_through_the_builder() {
|
||||
let mut h = History::builder().time_type::<Untimed>().build();
|
||||
for _ in 0..5 {
|
||||
h.record_winner(&"alice", &"bob", Untimed).unwrap();
|
||||
}
|
||||
assert!(h.converge().unwrap().converged);
|
||||
|
||||
let alice = h.current_skill(&"alice").unwrap();
|
||||
let bob = h.current_skill(&"bob").unwrap();
|
||||
assert!(alice.mu() > bob.mu(), "{alice:?} vs {bob:?}");
|
||||
assert!(alice.sigma().is_finite() && alice.sigma() > 0.0);
|
||||
}
|
||||
|
||||
/// `Untimed::elapsed_to` is always 0, so no drift accumulates however many
|
||||
/// events there are. That is the property the type exists for, and it had never
|
||||
/// been checked.
|
||||
#[test]
|
||||
fn untimed_accumulates_no_drift() {
|
||||
fn final_sigma<T: Time + Copy>(time: T, drift: ConstantDrift) -> f64 {
|
||||
let mut h = History::builder().time_type::<T>().drift(drift).build();
|
||||
for _ in 0..8 {
|
||||
h.record_winner(&"a", &"b", time).unwrap();
|
||||
}
|
||||
let _ = h.converge().unwrap();
|
||||
h.current_skill(&"a").unwrap().sigma()
|
||||
}
|
||||
|
||||
// Under Untimed the drift setting cannot matter, because elapsed is always 0.
|
||||
let none = final_sigma(Untimed, ConstantDrift::new(0.0));
|
||||
let large = final_sigma(Untimed, ConstantDrift::new(5.0));
|
||||
assert_eq!(
|
||||
none.to_bits(),
|
||||
large.to_bits(),
|
||||
"Untimed must ignore drift entirely: {none} vs {large}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_custom_time_type_and_a_custom_drift_work_together() {
|
||||
let mut h = History::builder()
|
||||
.time_type::<Season>()
|
||||
.drift(SeasonalDrift { per_season: 0.5 })
|
||||
.build();
|
||||
|
||||
for season in 1..=4u16 {
|
||||
for _ in 0..3 {
|
||||
h.record_winner(&"veteran", &"rookie", Season(season))
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
assert!(h.converge().unwrap().converged);
|
||||
|
||||
let curve = h.learning_curve(&"veteran").unwrap();
|
||||
assert_eq!(curve.len(), 4, "one point per season: {curve:?}");
|
||||
for (season, g) in &curve {
|
||||
assert!(
|
||||
g.mu().is_finite() && g.sigma() > 0.0,
|
||||
"season {season:?}: {g:?}"
|
||||
);
|
||||
}
|
||||
// Times come back as the domain type, not as an integer.
|
||||
assert_eq!(curve[0].0, Season(1));
|
||||
assert_eq!(curve[3].0, Season(4));
|
||||
}
|
||||
|
||||
/// Seasonal drift must actually widen a gap across seasons — otherwise the
|
||||
/// custom `Drift` is being ignored and the test above would pass regardless.
|
||||
#[test]
|
||||
fn a_custom_drift_is_actually_consulted() {
|
||||
fn sigma_with(per_season: f64) -> f64 {
|
||||
let mut h = History::builder()
|
||||
.time_type::<Season>()
|
||||
.drift(SeasonalDrift { per_season })
|
||||
.build();
|
||||
for season in 1..=6u16 {
|
||||
h.record_winner(&"a", &"b", Season(season)).unwrap();
|
||||
}
|
||||
let _ = h.converge().unwrap();
|
||||
h.current_skill(&"a").unwrap().sigma()
|
||||
}
|
||||
|
||||
let still = sigma_with(0.0);
|
||||
let drifting = sigma_with(2.0);
|
||||
assert!(
|
||||
drifting > still * 1.05,
|
||||
"a drifting fit must be less certain: {drifting} vs {still}"
|
||||
);
|
||||
}
|
||||
|
||||
/// The other axis: a custom key type, through the same mechanism.
|
||||
#[test]
|
||||
fn key_type_replaces_builder_with_key() {
|
||||
let mut h = History::builder().key_type::<String>().build();
|
||||
h.record_winner(&"alice".to_string(), &"bob".to_string(), 1)
|
||||
.unwrap();
|
||||
assert!(h.converge().unwrap().converged);
|
||||
assert!(h.current_skill("alice").is_some());
|
||||
}
|
||||
|
||||
/// Both axes at once, via the explicit constructor rather than the setters.
|
||||
#[test]
|
||||
fn new_constructs_on_any_axis_directly() {
|
||||
let mut h = HistoryBuilder::<Season, _, _, String>::new().build();
|
||||
h.record_winner(&"a".to_string(), &"b".to_string(), Season(7))
|
||||
.unwrap();
|
||||
assert!(h.converge().unwrap().converged);
|
||||
assert_eq!(h.learning_curve("a").unwrap()[0].0, Season(7));
|
||||
}
|
||||
@@ -0,0 +1,337 @@
|
||||
//! The joint must span slices, because Through Time reads each competitor at
|
||||
//! their own last appearance.
|
||||
//!
|
||||
//! The exact posterior of a multi-slice scored history is still Gaussian: the
|
||||
//! prior, the drift between appearances, and the scored likelihoods are all
|
||||
//! Gaussian. So it can be written out by hand and compared against, which is
|
||||
//! the check a single-slice fixture cannot make.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, Event, History, Member, Outcome, Team, UnknownKeys,
|
||||
};
|
||||
|
||||
const SIGMA0: f64 = 6.0;
|
||||
const BETA: f64 = 1.0;
|
||||
const SCORE_SIGMA: f64 = 2.0;
|
||||
const GAMMA: f64 = 0.5;
|
||||
|
||||
type H = History<i64, ConstantDrift, trueskill_tt::NullObserver, &'static str>;
|
||||
|
||||
fn history(gamma: f64) -> H {
|
||||
History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(SIGMA0)
|
||||
.beta(BETA)
|
||||
.score_sigma(SCORE_SIGMA)
|
||||
.drift(ConstantDrift::new(gamma))
|
||||
.unknown_keys(UnknownKeys::Reject)
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build()
|
||||
}
|
||||
|
||||
fn duel(a: &'static str, b: &'static str, t: i64, sa: f64, sb: f64) -> Event<i64, &'static str> {
|
||||
Event {
|
||||
time: t,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(a)]),
|
||||
Team::with_members([Member::new(b)]),
|
||||
],
|
||||
outcome: Outcome::scores([sa, sb]),
|
||||
}
|
||||
}
|
||||
|
||||
fn inverse(mut a: Vec<Vec<f64>>) -> Vec<Vec<f64>> {
|
||||
let n = a.len();
|
||||
let mut inv: Vec<Vec<f64>> = (0..n)
|
||||
.map(|i| (0..n).map(|j| f64::from(u8::from(i == j))).collect())
|
||||
.collect();
|
||||
for col in 0..n {
|
||||
let mut piv = col;
|
||||
for r in col + 1..n {
|
||||
if a[r][col].abs() > a[piv][col].abs() {
|
||||
piv = r;
|
||||
}
|
||||
}
|
||||
a.swap(col, piv);
|
||||
inv.swap(col, piv);
|
||||
let d = a[col][col];
|
||||
for j in 0..n {
|
||||
a[col][j] /= d;
|
||||
inv[col][j] /= d;
|
||||
}
|
||||
for r in 0..n {
|
||||
if r == col {
|
||||
continue;
|
||||
}
|
||||
let f = a[r][col];
|
||||
for j in 0..n {
|
||||
a[r][j] -= f * a[col][j];
|
||||
inv[r][j] -= f * inv[col][j];
|
||||
}
|
||||
}
|
||||
}
|
||||
inv
|
||||
}
|
||||
|
||||
/// Two competitors, two slices ten units apart, one duel in each.
|
||||
///
|
||||
/// The exact precision is written out explicitly here rather than obtained
|
||||
/// from the crate, so this is an independent check rather than a restatement.
|
||||
/// Variables are `[a0, b0, a1, b1]`.
|
||||
#[test]
|
||||
fn a_two_slice_joint_matches_the_exact_posterior() {
|
||||
let mut h = history(GAMMA);
|
||||
h.add_events(vec![
|
||||
duel("a", "b", 0, 5.0, 2.0),
|
||||
duel("a", "b", 10, 4.0, 3.0),
|
||||
])
|
||||
.unwrap();
|
||||
let report = h.converge().unwrap();
|
||||
assert!(report.converged, "{:?}", report.final_step);
|
||||
|
||||
let prior_prec = 1.0 / (SIGMA0 * SIGMA0);
|
||||
let drift_prec = 1.0 / (10.0 * GAMMA * GAMMA);
|
||||
let obs_prec = 1.0 / (SCORE_SIGMA * SCORE_SIGMA + 2.0 * BETA * BETA);
|
||||
|
||||
let mut lambda = vec![vec![0.0; 4]; 4];
|
||||
// priors on the first appearances
|
||||
lambda[0][0] += prior_prec;
|
||||
lambda[1][1] += prior_prec;
|
||||
// drift a0-a1 and b0-b1
|
||||
for (p, q) in [(0usize, 2usize), (1, 3)] {
|
||||
lambda[p][p] += drift_prec;
|
||||
lambda[q][q] += drift_prec;
|
||||
lambda[p][q] -= drift_prec;
|
||||
lambda[q][p] -= drift_prec;
|
||||
}
|
||||
// one duel per slice: contrast (+1, -1) on that slice's variables
|
||||
for (p, q) in [(0usize, 1usize), (2, 3)] {
|
||||
lambda[p][p] += obs_prec;
|
||||
lambda[q][q] += obs_prec;
|
||||
lambda[p][q] -= obs_prec;
|
||||
lambda[q][p] -= obs_prec;
|
||||
}
|
||||
let cov = inverse(lambda);
|
||||
|
||||
// The crate reads each competitor at their latest appearance: a1, b1.
|
||||
let exact_gap = (cov[2][2] + cov[3][3] - 2.0 * cov[2][3]).sqrt();
|
||||
let got = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&[(&"a", 1.0), (&"b", -1.0)])
|
||||
.unwrap();
|
||||
assert!(
|
||||
(got.sigma() - exact_gap).abs() / exact_gap < 1e-9,
|
||||
"difference: got {} exact {exact_gap}",
|
||||
got.sigma()
|
||||
);
|
||||
|
||||
let exact_single = cov[2][2].sqrt();
|
||||
let got_single = h.joint().unwrap().posterior_of(&[(&"a", 1.0)]).unwrap();
|
||||
assert!(
|
||||
(got_single.sigma() - exact_single).abs() / exact_single < 1e-9,
|
||||
"single node: got {} exact {exact_single}",
|
||||
got_single.sigma()
|
||||
);
|
||||
}
|
||||
|
||||
/// The case that motivated this: competitors read at *different* slices, with
|
||||
/// the last slice holding only one of them. Under the old latest-slice joint
|
||||
/// this was `UnknownKey`.
|
||||
#[test]
|
||||
fn competitors_last_seen_in_different_slices_are_comparable() {
|
||||
let mut h = history(GAMMA);
|
||||
h.add_events(vec![
|
||||
duel("a", "b", 0, 5.0, 2.0),
|
||||
duel("a", "c", 10, 4.0, 3.0),
|
||||
// the final slice holds one duel that does not involve b at all
|
||||
duel("a", "c", 20, 6.0, 1.0),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
// b last appeared at time 0; a and c at time 20. All three must resolve.
|
||||
for (x, y) in [("a", "b"), ("b", "c"), ("a", "c")] {
|
||||
let g = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&[(&x, 1.0), (&y, -1.0)])
|
||||
.unwrap_or_else(|e| panic!("{x} - {y} should resolve across slices: {e}"));
|
||||
assert!(g.sigma() > 0.0 && g.sigma().is_finite());
|
||||
}
|
||||
}
|
||||
|
||||
/// The mean must agree with what message passing reports, which is exact even
|
||||
/// with cycles. Only the second moment needs the joint.
|
||||
#[test]
|
||||
fn means_agree_with_the_marginals() {
|
||||
let mut h = history(GAMMA);
|
||||
h.add_events(vec![
|
||||
duel("a", "b", 0, 5.0, 2.0),
|
||||
duel("b", "c", 5, 3.0, 1.0),
|
||||
duel("a", "c", 10, 4.0, 2.0),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
for k in ["a", "b", "c"] {
|
||||
let marginal = h.current_skill(&k).unwrap().mu();
|
||||
let joint = h.joint().unwrap().posterior_of(&[(&k, 1.0)]).unwrap().mu();
|
||||
assert!(
|
||||
(marginal - joint).abs() < 1e-9,
|
||||
"{k}: marginal {marginal}, joint {joint}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// With zero drift a competitor has one latent skill however many slices it
|
||||
/// appears in, so spreading the same events over time must not change the
|
||||
/// answer. This exercises the appearance-merging path.
|
||||
#[test]
|
||||
fn zero_drift_makes_slice_layout_irrelevant() {
|
||||
let spread = {
|
||||
let mut h = history(0.0);
|
||||
h.add_events(vec![
|
||||
duel("a", "b", 0, 5.0, 2.0),
|
||||
duel("a", "b", 10, 4.0, 3.0),
|
||||
duel("a", "b", 20, 6.0, 1.0),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&[(&"a", 1.0), (&"b", -1.0)])
|
||||
.unwrap()
|
||||
};
|
||||
let together = {
|
||||
let mut h = history(0.0);
|
||||
h.add_events(vec![
|
||||
duel("a", "b", 0, 5.0, 2.0),
|
||||
duel("a", "b", 0, 4.0, 3.0),
|
||||
duel("a", "b", 0, 6.0, 1.0),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&[(&"a", 1.0), (&"b", -1.0)])
|
||||
.unwrap()
|
||||
};
|
||||
|
||||
assert!(
|
||||
(spread.sigma() - together.sigma()).abs() < 1e-9,
|
||||
"zero drift: spread {} vs together {}",
|
||||
spread.sigma(),
|
||||
together.sigma()
|
||||
);
|
||||
}
|
||||
|
||||
/// More drift means less is carried forward from old evidence, so a comparison
|
||||
/// against a competitor last seen long ago must widen.
|
||||
#[test]
|
||||
fn drift_widens_a_comparison_across_time() {
|
||||
let mut previous = 0.0;
|
||||
for gamma in [0.0f64, 0.1, 0.5, 2.0] {
|
||||
let mut h = history(gamma);
|
||||
h.add_events(vec![
|
||||
duel("a", "b", 0, 5.0, 2.0),
|
||||
duel("a", "c", 100, 4.0, 3.0),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
// b was last seen at time 0; a at time 100.
|
||||
let g = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&[(&"a", 1.0), (&"b", -1.0)])
|
||||
.unwrap();
|
||||
assert!(
|
||||
g.sigma() > previous,
|
||||
"gamma={gamma}: sigma {} did not exceed {previous}",
|
||||
g.sigma()
|
||||
);
|
||||
previous = g.sigma();
|
||||
}
|
||||
}
|
||||
|
||||
/// `posterior_of_at` pins the reading to a moment, where `posterior_of` takes
|
||||
/// each competitor wherever they were last seen.
|
||||
#[test]
|
||||
fn posterior_of_at_reads_as_of_a_time() {
|
||||
let mut h = history(GAMMA);
|
||||
h.add_events(vec![
|
||||
duel("a", "b", 0, 5.0, 2.0),
|
||||
duel("a", "b", 10, 4.0, 3.0),
|
||||
duel("a", "b", 20, 6.0, 1.0),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let early = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of_at(0, &[(&"a", 1.0), (&"b", -1.0)])
|
||||
.unwrap();
|
||||
let late = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of_at(20, &[(&"a", 1.0), (&"b", -1.0)])
|
||||
.unwrap();
|
||||
let latest = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&[(&"a", 1.0), (&"b", -1.0)])
|
||||
.unwrap();
|
||||
|
||||
// Asking as of the final slice is the same as asking for the latest.
|
||||
assert!((late.mu() - latest.mu()).abs() < 1e-9);
|
||||
assert!((late.sigma() - latest.sigma()).abs() < 1e-9);
|
||||
|
||||
// Reading at time 0 is a different quantity, and the smoothed estimate
|
||||
// there is informed by everything that came after.
|
||||
assert!(
|
||||
(early.mu() - late.mu()).abs() > 1e-6,
|
||||
"as-of-0 and as-of-20 should differ: {} vs {}",
|
||||
early.mu(),
|
||||
late.mu()
|
||||
);
|
||||
|
||||
// A time before any event has nothing to read.
|
||||
assert!(
|
||||
h.joint()
|
||||
.unwrap()
|
||||
.posterior_of_at(-1, &[(&"a", 1.0)])
|
||||
.is_err()
|
||||
);
|
||||
}
|
||||
|
||||
/// Times between slices resolve to the latest appearance at or before them.
|
||||
#[test]
|
||||
fn a_time_between_slices_reads_the_previous_appearance() {
|
||||
let mut h = history(GAMMA);
|
||||
h.add_events(vec![
|
||||
duel("a", "b", 0, 5.0, 2.0),
|
||||
duel("a", "b", 100, 4.0, 3.0),
|
||||
])
|
||||
.unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
|
||||
let at_zero = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of_at(0, &[(&"a", 1.0)])
|
||||
.unwrap();
|
||||
let between = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of_at(50, &[(&"a", 1.0)])
|
||||
.unwrap();
|
||||
assert!((at_zero.mu() - between.mu()).abs() < 1e-12);
|
||||
assert!((at_zero.sigma() - between.sigma()).abs() < 1e-12);
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
//! The traits a consumer needs on the public types, pinned so they cannot be
|
||||
//! removed by accident.
|
||||
//!
|
||||
//! This is written from a consumer's position — deriving `Debug` on a struct
|
||||
//! that *holds* a `History` — because that is the thing that failed. Asserting
|
||||
//! `History: Debug` in isolation would not have caught the generic-bound half:
|
||||
//! `Rating` derives `PartialEq`, but that is only usable if `D: PartialEq`, and
|
||||
//! the crate's own only `Drift` impl did not satisfy it.
|
||||
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, ConvergenceReport, Event, GameOptions, Gaussian, History,
|
||||
HistoryBuilder, InferenceError, Member, Outcome, Rating, Team,
|
||||
};
|
||||
|
||||
/// The reported failure, verbatim: a consumer holding a history in app state.
|
||||
#[derive(Debug)]
|
||||
#[allow(
|
||||
dead_code,
|
||||
reason = "held only so `derive(Debug)` has something to render"
|
||||
)]
|
||||
struct App {
|
||||
history: History,
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_struct_holding_a_history_can_derive_debug() {
|
||||
let app = App {
|
||||
history: History::default(),
|
||||
};
|
||||
|
||||
let rendered = format!("{app:?}");
|
||||
|
||||
// Summarising, not a dump of every skill store — the same choice `Joint`'s
|
||||
// manual `Debug` makes about its n² factorisation.
|
||||
assert!(rendered.contains("competitors"), "{rendered}");
|
||||
assert!(rendered.contains("time_slices"), "{rendered}");
|
||||
assert!(
|
||||
!rendered.contains("SkillStore"),
|
||||
"History's Debug should summarise, not dump: {rendered}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn history_builder_is_debug_and_clone() {
|
||||
let b: HistoryBuilder<i64, ConstantDrift, _, &'static str> = History::builder();
|
||||
let cloned = b.clone();
|
||||
assert!(!format!("{cloned:?}").is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn config_and_input_value_types_are_comparable() {
|
||||
assert_eq!(ConstantDrift::new(0.1), ConstantDrift::new(0.1));
|
||||
assert_ne!(ConstantDrift::new(0.1), ConstantDrift::new(0.2));
|
||||
|
||||
assert_eq!(ConvergenceOptions::default(), ConvergenceOptions::default());
|
||||
assert_eq!(GameOptions::default(), GameOptions::default());
|
||||
|
||||
// `Rating: PartialEq` is only reachable through `D: PartialEq`.
|
||||
assert_eq!(Rating::<i64, ConstantDrift>::default(), Rating::default());
|
||||
assert_ne!(
|
||||
Rating::default(),
|
||||
Rating::<i64, ConstantDrift>::default().with_drift_scale(2.0)
|
||||
);
|
||||
|
||||
assert_eq!(Member::new("a"), Member::new("a"));
|
||||
assert_ne!(Member::new("a"), Member::new("b"));
|
||||
assert_eq!(
|
||||
Team::with_members([Member::new("a")]),
|
||||
Team::with_members([Member::new("a")])
|
||||
);
|
||||
|
||||
let event = || Event {
|
||||
time: 1,
|
||||
teams: [
|
||||
Team::with_members([Member::new("a")]),
|
||||
Team::with_members([Member::new("b")]),
|
||||
]
|
||||
.into_iter()
|
||||
.collect(),
|
||||
outcome: Outcome::winner(0, 2),
|
||||
};
|
||||
assert_eq!(event(), event());
|
||||
|
||||
assert_eq!(Gaussian::default(), Gaussian::default());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_history_is_send_and_sync_and_default() {
|
||||
fn assert_send_sync<X: Send + Sync>() {}
|
||||
assert_send_sync::<History>();
|
||||
assert_send_sync::<InferenceError>();
|
||||
|
||||
let mut h = History::default();
|
||||
let report: ConvergenceReport = h.converge().expect("an empty history converges");
|
||||
assert_eq!(report, report.clone());
|
||||
}
|
||||
@@ -0,0 +1,382 @@
|
||||
//! Input validation must hold in **release**, where `debug_assert!` is gone.
|
||||
//!
|
||||
//! The engine guards itself with `debug_assert!`, which documents invariants
|
||||
//! but vanishes in the profile users actually ship. Anything reachable from the
|
||||
//! public API has to be rejected with an `InferenceError` instead, at the
|
||||
//! boundary, rather than becoming NaN or an out-of-bounds panic deep inside
|
||||
//! `run_chain`.
|
||||
//!
|
||||
//! `GameOptions` and `ConvergenceOptions` both have public fields, so the
|
||||
//! eager asserts on `HistoryBuilder` do not cover the `Game` constructors —
|
||||
//! a caller can build the options struct directly.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, Event, Game, GameOptions, Gaussian, History, InferenceError,
|
||||
Member, Outcome, Rating, Team,
|
||||
};
|
||||
|
||||
type R = Rating<i64, ConstantDrift>;
|
||||
|
||||
fn rating() -> R {
|
||||
R::new(
|
||||
Gaussian::from_ms(25.0, 25.0 / 3.0),
|
||||
25.0 / 6.0,
|
||||
ConstantDrift::new(0.0),
|
||||
)
|
||||
}
|
||||
|
||||
fn options_with_alpha(alpha: f64) -> GameOptions {
|
||||
GameOptions {
|
||||
convergence: ConvergenceOptions {
|
||||
alpha,
|
||||
..ConvergenceOptions::default()
|
||||
},
|
||||
..GameOptions::default()
|
||||
}
|
||||
}
|
||||
|
||||
/// `alpha == 0.0` leaves every EP update unapplied, so inference silently
|
||||
/// returns the priors — the worst possible failure, since the output looks
|
||||
/// entirely reasonable.
|
||||
#[test]
|
||||
fn ranked_rejects_a_zero_damping_factor() {
|
||||
let (a, b) = (rating(), rating());
|
||||
let err = Game::<i64, _>::ranked(
|
||||
&[&[a], &[b]],
|
||||
Outcome::winner(0, 2),
|
||||
&options_with_alpha(0.0),
|
||||
)
|
||||
.expect_err("alpha = 0 must be rejected");
|
||||
assert!(
|
||||
matches!(err, InferenceError::InvalidParameter { name: "alpha", .. }),
|
||||
"got {err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ranked_rejects_an_out_of_range_damping_factor() {
|
||||
let (a, b) = (rating(), rating());
|
||||
for alpha in [-0.5, 1.5, f64::NAN] {
|
||||
let err = Game::<i64, _>::ranked(
|
||||
&[&[a], &[b]],
|
||||
Outcome::winner(0, 2),
|
||||
&options_with_alpha(alpha),
|
||||
)
|
||||
.expect_err("alpha out of (0, 1] must be rejected");
|
||||
assert!(
|
||||
matches!(err, InferenceError::InvalidParameter { name: "alpha", .. }),
|
||||
"alpha={alpha}: got {err:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scored_rejects_a_bad_damping_factor() {
|
||||
let (a, b) = (rating(), rating());
|
||||
let err = Game::<i64, _>::scored(
|
||||
&[&[a], &[b]],
|
||||
Outcome::scores([21.0, 9.0]),
|
||||
&options_with_alpha(0.0),
|
||||
)
|
||||
.expect_err("alpha = 0 must be rejected");
|
||||
assert!(
|
||||
matches!(err, InferenceError::InvalidParameter { name: "alpha", .. }),
|
||||
"got {err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// Already covered by `Game::ranked`, asserted here so the release-mode
|
||||
/// guarantee is stated in one place.
|
||||
#[test]
|
||||
fn ranked_rejects_an_out_of_range_draw_probability() {
|
||||
let (a, b) = (rating(), rating());
|
||||
for p_draw in [-0.5, 1.0, 1.5] {
|
||||
let options = GameOptions {
|
||||
p_draw,
|
||||
..GameOptions::default()
|
||||
};
|
||||
assert!(
|
||||
Game::<i64, _>::ranked(&[&[a], &[b]], Outcome::winner(0, 2), &options).is_err(),
|
||||
"p_draw={p_draw} must be rejected"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scored_rejects_a_non_positive_noise() {
|
||||
let (a, b) = (rating(), rating());
|
||||
for score_sigma in [0.0, -1.0, f64::NAN] {
|
||||
let options = GameOptions {
|
||||
score_sigma,
|
||||
..GameOptions::default()
|
||||
};
|
||||
assert!(
|
||||
Game::<i64, _>::scored(&[&[a], &[b]], Outcome::scores([21.0, 9.0]), &options).is_err(),
|
||||
"score_sigma={score_sigma} must be rejected"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// A tie with no draw probability makes the truncation margin zero and the
|
||||
/// two-sided update evaluate 0/0. Ingestion must refuse it.
|
||||
#[test]
|
||||
fn ingestion_rejects_a_tie_without_a_draw_probability() {
|
||||
let mut h = History::builder().p_draw(0.0).build();
|
||||
let err = h
|
||||
.add_events(vec![Event {
|
||||
time: 0,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("a")]),
|
||||
Team::with_members([Member::new("b")]),
|
||||
],
|
||||
outcome: Outcome::draw(2),
|
||||
}])
|
||||
.expect_err("a tie with p_draw = 0 must be rejected");
|
||||
assert!(
|
||||
matches!(err, InferenceError::TieWithoutDrawProbability { .. }),
|
||||
"got {err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// `Outcome::scores_with_noise` documents that a non-positive sigma is
|
||||
/// accepted at construction and rejected at ingestion.
|
||||
#[test]
|
||||
fn ingestion_rejects_a_non_positive_per_event_score_sigma() {
|
||||
for sigma in [0.0, -1.0, f64::NAN] {
|
||||
let mut h = History::builder().build();
|
||||
let err = h
|
||||
.add_events(vec![Event {
|
||||
time: 0,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new("a")]),
|
||||
Team::with_members([Member::new("b")]),
|
||||
],
|
||||
outcome: Outcome::scores_with_noise([21.0, 9.0], sigma),
|
||||
}])
|
||||
.expect_err("a non-positive per-event sigma must be rejected");
|
||||
assert!(
|
||||
matches!(err, InferenceError::InvalidParameter { .. }),
|
||||
"sigma={sigma}: got {err:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-team weights must match that team's membership. The top-level length
|
||||
/// checks in ingestion do not cover the inner dimension.
|
||||
#[test]
|
||||
fn ingestion_rejects_weights_that_do_not_match_their_team() {
|
||||
let mut h = History::builder().build();
|
||||
let mut team = Team::with_members([Member::new("a"), Member::new("b")]);
|
||||
team.members[0].weight = 1.0;
|
||||
|
||||
let err = h
|
||||
.event(0)
|
||||
.team(["a", "b"])
|
||||
.team(["c"])
|
||||
// Three weights for a two-member team.
|
||||
.weights([1.0, 1.0, 1.0])
|
||||
.winner(0)
|
||||
.commit()
|
||||
.expect_err("a weight/member length mismatch must be rejected");
|
||||
assert!(
|
||||
matches!(err, InferenceError::MismatchedShape { .. }),
|
||||
"got {err:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// `mu`, `sigma` and `beta` were the last unvalidated setters on
|
||||
/// `HistoryBuilder`, next to `p_draw`, `score_sigma` and `convergence`, which
|
||||
/// all assert eagerly.
|
||||
///
|
||||
/// Two of the rejected values are the quiet kind. A negative `sigma` or `beta`
|
||||
/// enters inference only as its square, so it produced bit-identical results
|
||||
/// to the positive value — the sign was dropped without comment.
|
||||
mod builder_parameters {
|
||||
use trueskill_tt::History;
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "mu must be finite")]
|
||||
fn a_non_finite_mu_is_rejected() {
|
||||
let _ = History::builder().mu(f64::NAN);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "sigma must be finite and positive")]
|
||||
fn a_zero_sigma_is_rejected() {
|
||||
let _ = History::builder().sigma(0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "sigma must be finite and positive")]
|
||||
fn a_negative_sigma_is_rejected() {
|
||||
let _ = History::builder().sigma(-8.33);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "sigma must be finite and positive")]
|
||||
fn an_infinite_sigma_is_rejected() {
|
||||
let _ = History::builder().sigma(f64::INFINITY);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "beta must be finite and non-negative")]
|
||||
fn a_negative_beta_is_rejected() {
|
||||
let _ = History::builder().beta(-4.17);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "beta must be finite and non-negative")]
|
||||
fn a_non_finite_beta_is_rejected() {
|
||||
let _ = History::builder().beta(f64::NAN);
|
||||
}
|
||||
|
||||
/// Zero beta is deliberately allowed: performance is then exactly skill.
|
||||
/// It has to reach a different fit than a positive beta, or "allowed"
|
||||
/// would just mean "not checked".
|
||||
#[test]
|
||||
fn a_zero_beta_is_allowed_and_changes_the_fit() {
|
||||
let fit = |beta: f64| {
|
||||
let mut h = History::builder()
|
||||
.mu(25.0)
|
||||
.sigma(25.0 / 3.0)
|
||||
.beta(beta)
|
||||
.build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h.current_skill(&"a").unwrap()
|
||||
};
|
||||
let zero = fit(0.0);
|
||||
let positive = fit(25.0 / 6.0);
|
||||
// `variance` rather than `pi`: the natural parameters are the crate's
|
||||
// internal representation and no longer public. It is the same
|
||||
// quantity inverted, so a finite positive precision is a finite
|
||||
// positive variance.
|
||||
assert!(zero.variance().is_finite() && zero.variance() > 0.0);
|
||||
assert!(
|
||||
(zero.variance() - positive.variance()).abs() > 1e-6,
|
||||
"zero beta must not merely be ignored: {zero:?} vs {positive:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// The constructors below `HistoryBuilder`, which 0.8.0's validation did not
|
||||
/// reach.
|
||||
///
|
||||
/// `sigma`, `beta` and `gamma` all enter inference only as squares, so a
|
||||
/// negative value behaves as its absolute value and the sign vanishes without
|
||||
/// comment. Measured before these guards: `from_ms(25.0, -8.33)` and
|
||||
/// `Rating::new(_, -4.17, _)` returned results bit identical to their positive
|
||||
/// counterparts, and `Rating::new(_, NaN, _)` reached `Game::ranked`, which
|
||||
/// returned `Ok` carrying `Gaussian { pi: NaN, tau: NaN }`.
|
||||
mod constructor_parameters {
|
||||
use trueskill_tt::{ConstantDrift, Gaussian, History, InferenceError, Rating};
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "sigma must not be negative")]
|
||||
fn a_negative_sigma_is_rejected_by_from_ms() {
|
||||
let _ = Gaussian::from_ms(25.0, -8.33);
|
||||
}
|
||||
|
||||
/// NaN must pass, and that is deliberate: a broken fit produces a NaN
|
||||
/// sigma and `converge` reports it as `NonFiniteResult`. Rejecting it here
|
||||
/// would turn reporting into a panic inside inference.
|
||||
#[test]
|
||||
fn a_nan_sigma_passes_through_from_ms() {
|
||||
let g = Gaussian::from_ms(25.0, f64::NAN);
|
||||
// `sigma()` is NaN exactly when the precision is: it guards `pi <= 0`
|
||||
// (reporting `inf`) and `pi == inf` (reporting `0.0`), so NaN survives
|
||||
// only from a NaN precision.
|
||||
assert!(g.sigma().is_nan());
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "beta must be finite and non-negative")]
|
||||
fn a_negative_beta_is_rejected_by_rating_new() {
|
||||
let _ =
|
||||
Rating::<i64, ConstantDrift>::new(Gaussian::default(), -4.17, ConstantDrift::new(0.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "beta must be finite and non-negative")]
|
||||
fn a_nan_beta_is_rejected_by_rating_new() {
|
||||
let _ = Rating::<i64, ConstantDrift>::new(
|
||||
Gaussian::default(),
|
||||
f64::NAN,
|
||||
ConstantDrift::new(0.0),
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_zero_beta_is_accepted_by_rating_new() {
|
||||
let _ =
|
||||
Rating::<i64, ConstantDrift>::new(Gaussian::default(), 0.0, ConstantDrift::new(0.0));
|
||||
}
|
||||
|
||||
/// `ConstantDrift` rejects at construction now that its field is private.
|
||||
#[test]
|
||||
#[should_panic(expected = "gamma must be finite and non-negative")]
|
||||
fn a_negative_gamma_is_rejected_by_constant_drift_new() {
|
||||
let _ = ConstantDrift::new(-0.0833);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[should_panic(expected = "gamma must be finite and non-negative")]
|
||||
fn a_non_finite_gamma_is_rejected_by_constant_drift_new() {
|
||||
let _ = ConstantDrift::new(f64::NAN);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gamma_reads_back_what_was_given() {
|
||||
assert_eq!(ConstantDrift::new(0.25).gamma(), 0.25);
|
||||
assert_eq!(ConstantDrift::new(0.0).gamma(), 0.0);
|
||||
}
|
||||
|
||||
/// `HistoryBuilder::drift` is generic and cannot inspect an arbitrary
|
||||
/// `Drift`, so the check on the variance each competitor accumulates is
|
||||
/// still needed — it is the only thing standing between a custom
|
||||
/// implementation and a NaN fit. `ConstantDrift` can no longer reach it,
|
||||
/// so this uses an implementation that can.
|
||||
#[test]
|
||||
fn a_custom_drift_returning_a_bad_variance_is_rejected_at_convergence() {
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
struct BadDrift(f64);
|
||||
|
||||
impl trueskill_tt::Drift<i64> for BadDrift {
|
||||
fn variance_delta(&self, _from: &i64, _to: &i64) -> f64 {
|
||||
self.0
|
||||
}
|
||||
fn variance_for_elapsed(&self, _elapsed: i64) -> f64 {
|
||||
self.0
|
||||
}
|
||||
}
|
||||
|
||||
for bad in [f64::NAN, f64::INFINITY, -1.0] {
|
||||
let mut h = History::builder().drift(BadDrift(bad)).build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
h.record_winner(&"a", &"b", 5).unwrap();
|
||||
let err = h.converge().unwrap_err();
|
||||
assert!(
|
||||
matches!(
|
||||
err,
|
||||
InferenceError::InvalidParameter {
|
||||
name: "drift variance",
|
||||
..
|
||||
}
|
||||
),
|
||||
"drift {bad}: {err:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// An ordinary drift is untouched.
|
||||
#[test]
|
||||
fn an_ordinary_drift_still_converges() {
|
||||
let mut h = History::builder()
|
||||
.drift(ConstantDrift::new(25.0 / 300.0))
|
||||
.build();
|
||||
h.record_winner(&"a", &"b", 1).unwrap();
|
||||
h.record_winner(&"a", &"b", 5).unwrap();
|
||||
assert!(h.converge().unwrap().converged);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
//! `expected_variance_reduction`: which matchup best sharpens a given question.
|
||||
|
||||
use smallvec::smallvec;
|
||||
use trueskill_tt::{
|
||||
ConstantDrift, ConvergenceOptions, Event, History, InferenceError, Member, Outcome, Team,
|
||||
UnknownKeys,
|
||||
};
|
||||
|
||||
type H = History<i64, ConstantDrift, trueskill_tt::NullObserver, &'static str>;
|
||||
|
||||
fn round(a: &'static str, b: &'static str, sa: f64, sb: f64) -> Event<i64, &'static str> {
|
||||
Event {
|
||||
time: 1,
|
||||
teams: smallvec![
|
||||
Team::with_members([Member::new(a)]),
|
||||
Team::with_members([Member::new(b)]),
|
||||
],
|
||||
outcome: Outcome::scores([sa, sb]),
|
||||
}
|
||||
}
|
||||
|
||||
fn base() -> Vec<Event<i64, &'static str>> {
|
||||
vec![
|
||||
round("a", "b", 5.0, 2.0),
|
||||
round("a", "c", 6.0, 1.0),
|
||||
round("b", "c", 4.0, 3.0),
|
||||
round("c", "d", 2.0, 1.0),
|
||||
round("a", "d", 7.0, 2.0),
|
||||
]
|
||||
}
|
||||
|
||||
fn fit(extra: Option<Event<i64, &'static str>>, policy: UnknownKeys) -> H {
|
||||
let mut h: History<i64, _, _, &'static str> = History::builder()
|
||||
.mu(0.0)
|
||||
.sigma(6.0)
|
||||
.beta(1.0)
|
||||
.score_sigma(2.0)
|
||||
.drift(ConstantDrift::new(0.0))
|
||||
.unknown_keys(policy)
|
||||
.convergence(ConvergenceOptions {
|
||||
max_iter: 20_000,
|
||||
epsilon: 1e-13,
|
||||
alpha: 1.0,
|
||||
})
|
||||
.build();
|
||||
let mut ev = base();
|
||||
if let Some(e) = extra {
|
||||
ev.push(e);
|
||||
}
|
||||
h.add_events(ev).unwrap();
|
||||
let _ = h.converge().unwrap();
|
||||
h
|
||||
}
|
||||
|
||||
/// The closed form must equal what actually happens if the matchup is played.
|
||||
/// This is the assertion that makes the whole call trustworthy: a wrong
|
||||
/// acquisition function returns plausible numbers and quietly picks worse
|
||||
/// matchups forever.
|
||||
#[test]
|
||||
fn the_closed_form_matches_an_actual_refit() {
|
||||
let h = fit(None, UnknownKeys::Reject);
|
||||
let target: Vec<(&&str, f64)> = vec![(&"a", 1.0), (&"b", -1.0)];
|
||||
let before = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&target)
|
||||
.unwrap()
|
||||
.sigma()
|
||||
.powi(2);
|
||||
|
||||
for (x, y) in [("a", "b"), ("c", "d"), ("a", "c"), ("b", "d")] {
|
||||
let predicted = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.expected_variance_reduction(&[&[&x], &[&y]], &target)
|
||||
.unwrap();
|
||||
|
||||
let after = fit(Some(round(x, y, 3.0, 1.0)), UnknownKeys::Reject);
|
||||
let actual = before
|
||||
- after
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&target)
|
||||
.unwrap()
|
||||
.sigma()
|
||||
.powi(2);
|
||||
|
||||
assert!(
|
||||
(predicted - actual).abs() / actual.abs() < 1e-9,
|
||||
"{x} vs {y}: predicted {predicted}, actual {actual}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// The reduction cannot depend on the score, because for a Gaussian likelihood
|
||||
/// the posterior variance update is data-independent. This is why the call
|
||||
/// needs no expectation despite its name.
|
||||
#[test]
|
||||
fn the_outcome_does_not_change_the_reduction() {
|
||||
let target: Vec<(&&str, f64)> = vec![(&"a", 1.0), (&"b", -1.0)];
|
||||
let h = fit(None, UnknownKeys::Reject);
|
||||
let before = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&target)
|
||||
.unwrap()
|
||||
.sigma()
|
||||
.powi(2);
|
||||
|
||||
let mut seen = Vec::new();
|
||||
for (sa, sb) in [(3.0, 1.0), (100.0, -50.0), (0.0, 0.0)] {
|
||||
let after = fit(Some(round("c", "d", sa, sb)), UnknownKeys::Reject);
|
||||
seen.push(
|
||||
before
|
||||
- after
|
||||
.joint()
|
||||
.unwrap()
|
||||
.posterior_of(&target)
|
||||
.unwrap()
|
||||
.sigma()
|
||||
.powi(2),
|
||||
);
|
||||
}
|
||||
for w in seen.windows(2) {
|
||||
assert!(
|
||||
(w[0] - w[1]).abs() < 1e-12,
|
||||
"variance reduction moved with the observed score: {seen:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// The point of the call: it must rank candidate matchups usefully. Playing the
|
||||
/// pair you are trying to separate helps most; an unrelated pair helps least.
|
||||
#[test]
|
||||
fn it_ranks_candidates_by_how_much_they_answer_the_question() {
|
||||
let h = fit(None, UnknownKeys::Reject);
|
||||
let target: Vec<(&&str, f64)> = vec![(&"a", 1.0), (&"b", -1.0)];
|
||||
|
||||
let direct = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.expected_variance_reduction(&[&[&"a"], &[&"b"]], &target)
|
||||
.unwrap();
|
||||
let unrelated = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.expected_variance_reduction(&[&[&"c"], &[&"d"]], &target)
|
||||
.unwrap();
|
||||
|
||||
assert!(direct > 0.0 && unrelated > 0.0);
|
||||
assert!(
|
||||
direct > 5.0 * unrelated,
|
||||
"playing the target pair should dominate: {direct} vs {unrelated}"
|
||||
);
|
||||
}
|
||||
|
||||
/// A matchup between two competitors nobody has seen still teaches something
|
||||
/// about them, but nothing about a target that does not involve them.
|
||||
#[test]
|
||||
fn an_unrelated_unseen_matchup_teaches_nothing_about_the_target() {
|
||||
let h = fit(None, UnknownKeys::Prior);
|
||||
let target: Vec<(&&str, f64)> = vec![(&"a", 1.0), (&"b", -1.0)];
|
||||
|
||||
let reduction = h
|
||||
.joint()
|
||||
.unwrap()
|
||||
.expected_variance_reduction(&[&[&"stranger"], &[&"nobody"]], &target)
|
||||
.unwrap();
|
||||
assert!(
|
||||
reduction.abs() < 1e-12,
|
||||
"an unseen pair shares nothing with the target: {reduction}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shape_errors_are_reported() {
|
||||
let h = fit(None, UnknownKeys::Reject);
|
||||
let target: Vec<(&&str, f64)> = vec![(&"a", 1.0), (&"b", -1.0)];
|
||||
|
||||
assert!(matches!(
|
||||
h.joint()
|
||||
.unwrap()
|
||||
.expected_variance_reduction(&[&[&"a"]], &target),
|
||||
Err(InferenceError::MismatchedShape {
|
||||
expected: 2,
|
||||
got: 1,
|
||||
..
|
||||
})
|
||||
));
|
||||
assert!(matches!(
|
||||
h.joint()
|
||||
.unwrap()
|
||||
.expected_variance_reduction(&[&[&"a"], &[&"ghost"]], &target),
|
||||
Err(InferenceError::UnknownKey { .. })
|
||||
));
|
||||
}
|
||||
Reference in New Issue
Block a user