13a395fdc91502f2e1c807500eca35bbbc4f4dd1
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  logaritmiskandClaude Opus 5
|
13a395fdc9 |
refactor!: scores_with_noise, and History::quality
Two names that described the wrong thing.
`scores_with_sigma(scores, sigma)` reads as "these scores have prior
sigma 2.0". The quantity is observation noise on the score *margin*, in
the units of the scores, and it is spelled `score_sigma` at every config
site — `HistoryBuilder::score_sigma`, `GameOptions::score_sigma`,
`EventKind::Scored { score_sigma }` — so this was the one place the
crate used a third meaning of "sigma" for it. Its own doc had to
disambiguate itself: "`sigma` overrides `HistoryBuilder::score_sigma`".
`scores_with_noise(scores, score_sigma)` on both `Outcome` and
`EventBuilder`.
`predict_quality` predicts nothing. Its own doc says it answers "is this
matchup *fair*", not "what will happen", and the `predict_*` family is
otherwise exactly the methods returning a probability or a distribution
over outcomes. `History::quality` also makes the free/method pair
consistent: free `quality` pairs with `History::quality` the way free
`expected_information_gain` already pairs with
`History::expected_information_gain`. The rule that was already being
followed and never stated — a free function scores a hypothetical from
explicit parameters, the same-named method asks it against the fit — is
now written on the method.
Closes #75. Refs #78 (part 4).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
|
||
|
logaritmiskandClaude Opus 5
|
92ae5fca17 |
feat!: prediction and joint queries take borrowed keys
`&[&[&K]]` was the worst shape in the API. At `K = String` — the
realistic case, where names arrive owned from a database or CSV — a
string literal was *impossible*, and asking "who wins" cost six lines
and four allocations of temporaries that all had to outlive the call:
let ta = vec![a.to_string()];
let ra: Vec<&String> = ta.iter().collect();
...
self.history.predict_win_probabilities(&teams)
All seven `predict_*` / `expected_*` methods, `posterior_of`,
`posterior_of_at` and the `Joint` mirrors are now generic over the
borrowed key, the same way `current_skill` and `learning_curve` already
were. `member_skills` and `resolve_terms` only ever did two things with
a key — `keys.get` and `format!("{key:?}")` — and neither needed `K`.
h.predict_win_probabilities(&[&["alice"], &["bob"]]) // K = String
h.predict_win_probabilities(&[&["alice"], &["bob"]]) // K = &'static str
h.posterior_of(&[("alice", 1.0), ("bob", -1.0)])
One spelling for both key types, and `K: Debug` becomes `Q: Debug`, so a
key type no longer has to be `Debug` to run a prediction. The old
`&[&[&"a"]]` spelling still compiles at the default key type, where `Q`
infers to `&str` and the two shapes coincide.
The one cost: `predict_outcome(&[])` can no longer infer `Q` — nothing
in an empty slice names it. It needs an annotation, and only on that
degenerate call.
`lookup` carried `ToOwned<Owned = K>`, copy-pasted from `intern`, which
genuinely needs it to create the entry. `lookup` never creates, and its
five neighbours all accept `h.f("alice")` already. Dropping the bound
strictly widens what compiles.
`HistoryBuilder::gamma` is shorthand for `.drift(ConstantDrift::new(g))`.
Drift is the most-tuned parameter after `sigma` and `GAMMA` is a public
constant, but setting it meant first discovering `ConstantDrift`, a type
a caller has no other reason to name. On the `ConstantDrift` builder
only, since `gamma` is that model's parameter rather than something
every `Drift` has, and rejecting a negative value for the same reason as
`sigma` and `beta`: it enters squared.
Refs #72 (items 2 and 3, plus the gamma shorthand; the type-parameter
reorder in item 1 is still open).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
|
||
|
logaritmiskandClaude Opus 5
|
85c4d0d87d |
fix!: correct eight wrong # Errors sections and seal the error variants
Documentation (#78). Every item below was measured against the code rather than read: - `expected_information_gain` and `predict_ranking` had `# Errors` immediately followed by `# Preconditions`, with the error list stranded at the bottom of the latter — rustdoc rendered a BLANK Errors section on both. The heading now sits with its content. - `predict_outcome`, `predict_ranking` and the free `expected_information_gain` all omitted `GridTooCoarse`. - `predict_margin` claimed `JointUnavailable` "if the LATEST slice holds ranked events". Measured with an early ranked slice and a late scored one: it fails. The condition is *any* slice. - `add_events` documented three errors and can return five more; it also claimed a weights `MismatchedShape` that is unreachable through it, since weights arrive one-per-`Member`. That check belongs to `EventBuilder::weights`, and the doc now says so. - `converge` and `converge_partial` both omitted the drift-variance `InvalidParameter`. `History` gains a hand-written `Debug` (#76). Summarising, not exhaustive — a derived one would print every competitor's skill at every slice. It exists because without it a consumer cannot `#[derive(Debug)]` on any struct holding a `History`, which is how both known consumers store one. `#[non_exhaustive]` on all 17 `InferenceError` struct variants and on `Outcome::Scored` (#74). The enum carried the attribute; no variant did, so adding a field to any of them — and downstream construction of any of them — were both in the public contract. This crate added two variants in two days. The options structs are deliberately NOT sealed. `ConvergenceOptions` and `GameOptions` are constructed by struct literal at 65 sites of which only 8 use `..default()`, and specifying all three convergence fields is a natural complete statement rather than a partial one. That is a real trade-off rather than an oversight, and it is left as a decision on #74. Also spells `UnknownKeys::Reject` explicitly at both sites that wildcarded it. `#[non_exhaustive]` on your own enum gives no exhaustiveness safety net if you then match `_`. Sealing the variants pushed ten test sites from constructing errors to `matches!`, which is the better assertion anyway — an `assert_eq!` against a constructed error breaks whenever a field is added, which is the exact fragility the attribute exists to prevent. BREAKING CHANGE: `InferenceError`'s struct variants and `Outcome::Scored` are `#[non_exhaustive]` — downstream patterns need `..` and downstream construction is no longer possible. Refs #78, #76, #74 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ |
||
|
logaritmiskandClaude Opus 5
|
71554fd944 |
feat: add UnknownKeys::Prior, and explain why there is no Skip
#44's third ask was an opt-in mode so a caller with partially-known teams need not pre-filter. The requested shape was `Skip` — drop unknown members. Measured, that is the wrong mode to build. A team's performance is the *sum* of its members, so dropping one drops its variance too. On a two-member team with one unknown: SKIP (drop the member) : performance sigma 2.37 PRIOR (member at prior) : performance sigma 6.53 (2.76x wider) Skipping makes the model *more* certain because it knows *less*, which is backwards. `Prior` is also the answer the model already gives for a competitor it knows about but has no evidence for — measured, such a competitor sits at sigma 4.99 against the prior's 6.0 — so it corresponds to a state the model can actually be in. Skipping does not. So the enum is `Reject` (default, unchanged) and `Prior`, and it is `#[non_exhaustive]` in case a real use for skipping turns up later. Placed on `HistoryBuilder` rather than per-call. Neither consumer wants it to vary between queries: one scores thousands of candidate matchups in a loop, the other's headline feature is predicting a competitor nobody has faced. That makes it a property of how the model is being used, and keeps five prediction signatures unchanged. This also gives #48 the semantics it asked for — "I have never seen this competitor, here is the prior-informed answer" — which it needs for predicting a course nobody has played. `an_unknown_member_widens_its_team_rather_than_narrowing_it` pins the property that ruled `Skip` out, so a future convenience cannot quietly reintroduce it. Closes #44. Refs #48 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ |
||
|
logaritmiskandClaude Opus 5
|
c12bc830a5 |
feat!: name the unknown key, expose tail probabilities, flag short fits
Three issues from two downstream consumers, all small, all sharing a theme: the crate had the information and would not hand it over. #44 — `UnknownKey { team: 0, member: 0 }` did not say which key. A consumer upgrading 0.1.2 -> 0.4.1 had every one of 5591 predictions return this error, fell back to a neutral 0.5, and lost its entire metadata model for a day. Nothing crashed and nothing logged; it was found by sweeping an unrelated parameter and noticing the output did not move. The 0.4.0 change that made unknown keys an error was right — the error was just too anonymous to act on. It now carries the key's `Debug` rendering, and its `Display` says what to do about it. The precondition is documented on every prediction entry point, which the reporter said would alone have saved the day. #43 — `cdf` was `pub(crate)`, so a consumer asking "is this competitor below the cutoff" approximated it with a `mu + z * sigma` band and had no way to say what confidence any `z` bought. Adds `Gaussian::probability_below` / `probability_above`. The second is separate on purpose: `1 - cdf` collapses to exactly zero past ~8.3 sigma, and a stopping rule is evaluated precisely there. Both route through the survival function added in 0.4.1, so this is visibility rather than new numerics. #50 — `ConvergenceReport` was not `#[must_use]`, so the one signal that a fit stopped short was trivially discarded. It now is, and that immediately found 78 sites doing exactly that — including this crate's own ATP example, which was capped at 10 sweeps when the history needs 30. The example now reads the report and says so. `ITERATIONS = 30` is documented as the floor it is, with the three measurements to hand: 400 events over 100 competitors already stops there at ~7e-3 against a 1e-6 tolerance, the ATP example needs 30 at a much looser one, and a consumer's 2000-node model needs 76 to 161. BREAKING CHANGE: `InferenceError::UnknownKey` gains a `key` field, and the prediction methods now require `K: Debug` in order to fill it. Closes #43, #50. Refs #44 — its third ask, an opt-in `UnknownKeys::Skip` mode, is a live API question and deliberately not answered here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ |
||
|
logaritmiskandClaude Opus 5
|
3c2f9ac64c |
feat: add expected information gain for active matchup selection
`quality()` answers "is this matchup fair". Callers picking which
comparison to run next need "is this matchup informative", and the two
coincide only for two evenly matched competitors. Without a principled
alternative, downstream code was reaching for hand-rolled heuristics
like `quality * sigma_a^2 * sigma_b^2`, which double-counts uncertainty:
the two factors are not independent.
Adds `expected_information_gain`, the outcome-weighted divergence
between current beliefs and the beliefs each result would produce:
EIG = SUM P(outcome) * KL(posterior_after(outcome) || prior)
Available standalone over `Rating`s, and as
`History::expected_information_gain` using current skills and the
history's own beta, drift and p_draw — so the outcomes it weighs are the
ones that would actually be fitted.
This is the mutual information between the outcome and the skills, which
gives an analytic ceiling: gain cannot exceed the entropy of the thing
being observed, so at most `ln k` nats for k outcomes. That bound is the
sharpest test available, because an acquisition function is unusually
exposed to returning finite, plausible, monotone numbers while being
wrong — it would simply select slightly worse matchups forever. A
prototype of this returned 4.77 nats from a sign error while passing
every monotonicity check; `never_exceeds_the_entropy_of_the_outcome`
catches that class unconditionally.
Measured against the ceiling the values are meaningful rather than
vacuous: 0.382 nats for an even matchup between diffuse priors against
an 0.693 ceiling, falling to 0.013 for a lopsided one and 0.000 for a
hopeless one.
`disagrees_with_the_quality_times_variance_heuristic` pins down that
this is not a monotone transform of the heuristic it replaces — the two
rank a lopsided matchup and a confident even one in opposite orders — so
a later "simplification" cannot quietly revert to it.
Cost is one inference pass per possible outcome, documented on the
public API alongside the shortlist-then-score pattern, so callers do not
discover it in production.
Also folds the duplicated key-gathering in `predict_quality` and
`performances` into one validated `member_skills`.
Refs #39
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
|
||
|
logaritmiskandClaude Opus 5
|
bb2a845882 |
feat!: N-team outcome prediction with draw mass, replacing the 2-team panic
`predict_outcome` asserted `teams.len() == 2` and returned `[p, 1 - p]`, allocating no probability to a draw even with `p_draw > 0`. For a draw-enabled model the numbers were simply wrong, at any team count. It now returns `Result<Prediction, InferenceError>` and supports N teams. Two algorithms, both deterministic: - Who finishes first. Performances are independent Gaussians, so this separates into a one-dimensional integral per team rather than a multivariate orthant probability. Adaptive Gauss-Kronrod evaluates it to ~1e-15, matching the exact two-team closed form. - A specific finishing order. The factor graph only constrains rank-adjacent teams, so a full order is a chain of local constraints, not a general orthant integral. That chain collapses into a sequential recursion over cumulative integrals: O(teams * grid) per order. Fixed-node Gauss-Hermite is the obvious tool for the first and is a trap: when a rival's sigma is small the CDF product becomes a step narrower than the node spacing, and the nodes step over it. Measured 4.4e-4 off the closed form on a mildly skewed matchup and 1.7e-2 on a small-sigma one, while still returning something that looks like a probability. Adaptive refinement is what makes that case safe, and `win_probabilities_survive_a_rival_with_a_tiny_sigma` pins it down. The acceptance test is an identity rather than a golden: the outcome space is exhaustive and disjoint, so the probabilities sum to one. Any drift is integration error and nothing else. Gauss-Hermite failed it at 4.4e-4; this holds to ~1e-9. Also from #21: unknown keys are now reported rather than dropped, so a team of strangers can no longer produce a confident-looking prediction. `predict_quality` returns `Result` for the same reason. BREAKING CHANGE: `predict_outcome` returns `Result<Prediction, _>` instead of `Vec<f64>`; `predict_quality` returns `Result<f64, _>`. Refs #21, #39 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ |