feat: complete the evidence matrix and add current_skills
Three of four corners of the evidence matrix existed. The missing one was forward-only *and* key-restricted — which is exactly what per-competitor prequential scoring needs, the intersection of the two workloads `log_evidence_for` and `filtered_log_evidence` are each documented for. `filtered_log_evidence_for` fills it. It is not `log_evidence_internal(true, targets)`: that path selects `skill.forward` as the prior, which stops being a filtering quantity once `iteration` has run a backward sweep. It goes through `filtered_pass` like its unrestricted sibling, with the restriction applied to which events are *scored*, never to which are *run* — so it is a held-out score under the real history, not a score under a counterfactual one where nobody else played. Key resolution for both `*_for` accessors now shares `resolve_targets`, so they cannot drift apart on how an unknown key is reported. `current_skills` is the plural of `current_skill`. Building a leaderboard previously meant materialising every competitor's full smoothed curve via `learning_curves` and reading the last point of each. Tests carry controls in both directions: naming every competitor must recover the unrestricted value (catching a filter that drops too much), and the restricted forward-only value must differ from the restricted smoothed one (catching an alias). Refs #70. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
+21
-1
@@ -615,10 +615,18 @@ impl<T: Time> TimeSlice<T> {
|
||||
/// configured prior. The sweep runs on a scratch copy, so the real slice
|
||||
/// is untouched — which is what makes the filtered estimates independent
|
||||
/// of whether `History::converge` has run.
|
||||
/// One forward-only step for this slice.
|
||||
///
|
||||
/// `targets` restricts only the *evidence sum*, to events in which at
|
||||
/// least one target competitor appears; an empty set means no restriction.
|
||||
/// The forward messages are always built from every event in the slice —
|
||||
/// restricting those instead would answer a different question (a history
|
||||
/// in which the other events never happened), not a held-out one.
|
||||
pub(crate) fn filtered_step<D: Drift<T>>(
|
||||
&self,
|
||||
incoming: &HashMap<Index, Gaussian>,
|
||||
competitors: &CompetitorStore<T, D>,
|
||||
targets: &std::collections::HashSet<Index>,
|
||||
) -> FilteredStep {
|
||||
let mut scratch = TimeSlice {
|
||||
events: self.events.clone(),
|
||||
@@ -674,7 +682,19 @@ impl<T: Time> TimeSlice<T> {
|
||||
scratch.iterate_to_convergence(competitors);
|
||||
|
||||
FilteredStep {
|
||||
log_evidence: scratch.events.iter().map(|event| event.log_evidence).sum(),
|
||||
log_evidence: scratch
|
||||
.events
|
||||
.iter()
|
||||
.filter(|event| {
|
||||
targets.is_empty()
|
||||
|| event
|
||||
.teams
|
||||
.iter()
|
||||
.flat_map(|team| &team.items)
|
||||
.any(|item| targets.contains(&item.competitor))
|
||||
})
|
||||
.map(|event| event.log_evidence)
|
||||
.sum(),
|
||||
posteriors: scratch
|
||||
.skills
|
||||
.iter()
|
||||
|
||||
Reference in New Issue
Block a user