feat: add UnknownKeys::Prior, and explain why there is no Skip
#44's third ask was an opt-in mode so a caller with partially-known teams need not pre-filter. The requested shape was `Skip` — drop unknown members. Measured, that is the wrong mode to build. A team's performance is the *sum* of its members, so dropping one drops its variance too. On a two-member team with one unknown: SKIP (drop the member) : performance sigma 2.37 PRIOR (member at prior) : performance sigma 6.53 (2.76x wider) Skipping makes the model *more* certain because it knows *less*, which is backwards. `Prior` is also the answer the model already gives for a competitor it knows about but has no evidence for — measured, such a competitor sits at sigma 4.99 against the prior's 6.0 — so it corresponds to a state the model can actually be in. Skipping does not. So the enum is `Reject` (default, unchanged) and `Prior`, and it is `#[non_exhaustive]` in case a real use for skipping turns up later. Placed on `HistoryBuilder` rather than per-call. Neither consumer wants it to vary between queries: one scores thousands of candidate matchups in a loop, the other's headline feature is predicting a competitor nobody has faced. That makes it a property of how the model is being used, and keeps five prediction signatures unchanged. This also gives #48 the semantics it asked for — "I have never seen this competitor, here is the prior-informed answer" — which it needs for predicting a course nobody has played. `an_unknown_member_widens_its_team_rather_than_narrowing_it` pins the property that ruled `Skip` out, so a future convenience cannot quietly reintroduce it. Closes #44. Refs #48 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
+40
-9
@@ -33,6 +33,7 @@ pub struct HistoryBuilder<
|
||||
score_sigma: f64,
|
||||
convergence: ConvergenceOptions,
|
||||
observer: O,
|
||||
unknown_keys: crate::UnknownKeys,
|
||||
_time: PhantomData<T>,
|
||||
_key: PhantomData<K>,
|
||||
}
|
||||
@@ -63,6 +64,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> HistoryBuilder<
|
||||
score_sigma: self.score_sigma,
|
||||
convergence: self.convergence,
|
||||
observer: self.observer,
|
||||
unknown_keys: self.unknown_keys,
|
||||
_time: self._time,
|
||||
_key: self._key,
|
||||
}
|
||||
@@ -100,6 +102,20 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> HistoryBuilder<
|
||||
self
|
||||
}
|
||||
|
||||
/// How predictions treat a key the history has never seen.
|
||||
///
|
||||
/// Defaults to [`UnknownKeys::Reject`](crate::UnknownKeys::Reject), which
|
||||
/// errors. Set [`UnknownKeys::Prior`](crate::UnknownKeys::Prior) to have an
|
||||
/// unknown competitor answered from the configured prior instead, which
|
||||
/// makes "predict a matchup involving someone new" a first-class question.
|
||||
///
|
||||
/// This affects predictions only. Ingestion always creates a competitor for
|
||||
/// a key it has not seen, because that is what an event *is*.
|
||||
pub fn unknown_keys(mut self, unknown_keys: crate::UnknownKeys) -> Self {
|
||||
self.unknown_keys = unknown_keys;
|
||||
self
|
||||
}
|
||||
|
||||
/// Convergence tolerance, iteration cap, and EP damping.
|
||||
///
|
||||
/// # Panics
|
||||
@@ -132,6 +148,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> HistoryBuilder<
|
||||
score_sigma: self.score_sigma,
|
||||
convergence: self.convergence,
|
||||
observer,
|
||||
unknown_keys: self.unknown_keys,
|
||||
_time: self._time,
|
||||
_key: self._key,
|
||||
}
|
||||
@@ -151,6 +168,7 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> HistoryBuilder<
|
||||
score_sigma: self.score_sigma,
|
||||
convergence: self.convergence,
|
||||
observer: self.observer,
|
||||
unknown_keys: self.unknown_keys,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -166,6 +184,7 @@ impl Default for HistoryBuilder<i64, ConstantDrift, NullObserver, &'static str>
|
||||
score_sigma: 1.0,
|
||||
convergence: ConvergenceOptions::default(),
|
||||
observer: NullObserver,
|
||||
unknown_keys: crate::UnknownKeys::default(),
|
||||
_time: PhantomData,
|
||||
_key: PhantomData,
|
||||
}
|
||||
@@ -233,6 +252,7 @@ pub struct History<
|
||||
score_sigma: f64,
|
||||
convergence: ConvergenceOptions,
|
||||
observer: O,
|
||||
unknown_keys: crate::UnknownKeys,
|
||||
}
|
||||
|
||||
impl Default for History<i64, ConstantDrift, NullObserver, &'static str> {
|
||||
@@ -261,6 +281,7 @@ impl<K: Eq + Hash + Clone> History<i64, ConstantDrift, NullObserver, K> {
|
||||
score_sigma: 1.0,
|
||||
convergence: ConvergenceOptions::default(),
|
||||
observer: NullObserver,
|
||||
unknown_keys: crate::UnknownKeys::default(),
|
||||
_time: PhantomData,
|
||||
_key: PhantomData,
|
||||
}
|
||||
@@ -626,19 +647,29 @@ impl<T: Time, D: Drift<T>, O: Observer<T>, K: Eq + Hash + Clone> History<T, D, O
|
||||
|
||||
let mut members = Vec::with_capacity(team.len());
|
||||
for (member_idx, key) in team.iter().enumerate() {
|
||||
let unknown = InferenceError::UnknownKey {
|
||||
team: team_idx,
|
||||
member: member_idx,
|
||||
key: format!("{key:?}"),
|
||||
};
|
||||
let index = self.keys.get(*key).ok_or(unknown.clone())?;
|
||||
members.push(
|
||||
// A key can be missing two ways — never interned, or interned
|
||||
// with no recorded skill — and both mean the same thing to a
|
||||
// caller, so they take the same branch.
|
||||
let skill = self.keys.get(*key).and_then(|index| {
|
||||
self.time_slices
|
||||
.iter()
|
||||
.rev()
|
||||
.find_map(|ts| ts.skills.get(index).map(|s| s.posterior()))
|
||||
.ok_or(unknown)?,
|
||||
);
|
||||
});
|
||||
|
||||
members.push(match skill {
|
||||
Some(skill) => skill,
|
||||
None => match self.unknown_keys {
|
||||
crate::UnknownKeys::Prior => Gaussian::from_ms(self.mu, self.sigma),
|
||||
_ => {
|
||||
return Err(InferenceError::UnknownKey {
|
||||
team: team_idx,
|
||||
member: member_idx,
|
||||
key: format!("{key:?}"),
|
||||
});
|
||||
}
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
gathered.push(members);
|
||||
|
||||
Reference in New Issue
Block a user