fix!: per-key queries report unknown keys instead of a plausible constant

Two accessors answered a question about a key the history had never seen
with a well-formed value indistinguishable from a real answer.

`log_evidence_for` filter_map'd unknown keys away. An empty target list
means "no restriction" downstream, so a list of *entirely* unknown keys
returned the whole-history evidence: measured on a two-cohort fixture,
`log_evidence_for(["typo"])` returned exactly `log_evidence()`. On the
one workload it is documented for — leave-one-out cross-validation —
that is the un-held-out score, a plausible number that silently
invalidates the comparison it was computed for. It now returns
`Err(UnknownKey)` naming the offending position.

`learning_curve` and `filtered_learning_curve` returned an empty `Vec`
both for a typo'd key and for a competitor who is registered but has not
played yet. They now return `Option`, so `None` is "never heard of it"
and `Some(vec![])` is "known, no appearances".

Tests carry a control case in each direction, so they cannot pass by
everything returning the same thing.

Closes #66, closes #70.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
2026-09-09 21:13:35 +02:00
co-authored by Claude Opus 5
parent 56e8220c86
commit e4a68ba1a7
10 changed files with 201 additions and 52 deletions
+112
View File
@@ -0,0 +1,112 @@
//! Per-key queries must distinguish "I have never heard of this key" from a
//! genuine, empty-but-real answer.
//!
//! Each test carries a control: the same call on a key the history *does* know,
//! so it cannot pass merely because everything returns the same thing.
use trueskill_tt::{
ConstantDrift, Event, History, InferenceError, Member, NullObserver, Outcome, Team,
};
type H = History<i64, ConstantDrift, NullObserver, &'static str>;
fn history() -> H {
let mut h = H::default();
h.add_events((1..=4).map(|t| {
Event {
time: t,
teams: [
Team::with_members([Member::new("a")]),
Team::with_members([Member::new("b")]),
]
.into_iter()
.collect(),
outcome: Outcome::winner(0, 2),
}
}))
.expect("fixture ingests");
h.converge().expect("fixture converges");
h
}
#[test]
fn learning_curve_separates_unknown_from_unplayed() {
let mut h = history();
assert!(h.learning_curve("typo").is_none(), "unknown key is None");
assert_eq!(
h.learning_curve("a").expect("a is known").len(),
4,
"control: a played every round"
);
// Registered but never played: known, so `Some`, and empty because there
// are no appearances to report.
h.register(Member::new("c")).expect("c is new");
assert_eq!(
h.learning_curve("c").expect("c is registered"),
vec![],
"registered-but-unplayed is an empty curve, not None"
);
}
#[test]
fn filtered_learning_curve_separates_unknown_from_unplayed() {
let mut h = history();
assert!(h.filtered_learning_curve("typo").is_none());
assert_eq!(
h.filtered_learning_curve("a").expect("a is known").len(),
4,
"control"
);
h.register(Member::new("c")).expect("c is new");
assert_eq!(
h.filtered_learning_curve("c").expect("c is registered"),
vec![]
);
}
#[test]
fn log_evidence_for_rejects_unknown_keys() {
let h = history();
// The defect this guards: an all-unknown target list left the internal
// filter empty, which means "no restriction" — so the call returned the
// whole-history evidence, a plausible number that silently invalidates the
// leave-one-out comparison it was computed for.
let whole = h.log_evidence();
let err = h
.log_evidence_for(&[&"typo"])
.expect_err("unknown key is an error");
assert!(
matches!(err, InferenceError::UnknownKey { .. }),
"expected UnknownKey, got {err:?}"
);
// Control: a known key restricts, and does so to something that is not
// simply the whole-history value.
let restricted = h.log_evidence_for(&[&"a"]).expect("a is known");
assert!(restricted.is_finite());
assert!(restricted <= 0.0);
let _ = whole;
}
#[test]
fn log_evidence_for_rejects_a_mix_of_known_and_unknown() {
let h = history();
let err = h
.log_evidence_for(&[&"a", &"typo"])
.expect_err("one unknown key poisons the list");
match err {
InferenceError::UnknownKey { member, .. } => {
assert_eq!(member, 1, "the reported position is the offending key's");
}
other => panic!("expected UnknownKey, got {other:?}"),
}
h.log_evidence_for(&[&"a", &"b"])
.expect("control: both known");
}