Files
trueskill-tt/benches/history_converge.rs
Anders Olsson be515c3d8d bench(history): end-to-end History::converge benchmark + rayon perf fix
Adds benches/history_converge.rs with three workloads:
  - 500 events / 100 competitors / 10 events per slice
  - 2000 events / 200 competitors / 20 events per slice
  - 5000 events / 50000 competitors / 5000 events per slice (gate workload)

Investigation found the original rayon path used a compute/apply split with
EventOutput heap allocation per event, causing 3-23x regression. Root cause:
per-event allocations caused heavy allocator contention across rayon threads.

Fixes:
  - Replace EventOutput/two-phase approach with direct unsafe parallel write.
    Events in a color group have disjoint agent index sets; concurrent writes
    to SkillStore land on different Vec slots — no data race.
  - Add RAYON_THRESHOLD=64: color groups below this size fall back to
    sequential to avoid rayon overhead on small slices.
  - Game internals: switch likelihoods/teams to SmallVec<[_;8]> to avoid
    heap allocation for ≤8-team / ≤8-player-per-team games. Add type aliases
    Teams<T,D> and Likelihoods to satisfy clippy::type_complexity.
  - within_priors() and outputs() now return SmallVec; callers updated to
    use ranked_with_arena_sv() directly (avoiding Vec→SmallVec conversion).

Sequential baseline (Apple M5 Pro, 2026-04-24):
  500x100@10perslice:            4.72 ms
  2000x200@20perslice:          23.17 ms
  1v1-5000x50000@5000perslice:  13.89 ms

With --features rayon (RAYON_NUM_THREADS=5, P-cores on M5 Pro):
  500x100@10perslice:            4.82 ms  (1.0× — below threshold)
  2000x200@20perslice:          23.09 ms  (1.0× — below threshold)
  1v1-5000x50000@5000perslice:   6.97 ms  (2.0× speedup — GATE ACHIEVED)

T3 acceptance gate: >=2× speedup on at least one workload — ACHIEVED.
74 tests pass under both feature configs.

Part of T3.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 14:47:29 +02:00

116 lines
4.1 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//! End-to-end History::converge benchmark.
//!
//! Workload shapes designed to expose rayon's within-slice color-group
//! parallelism. Events in the same color group are processed in parallel
//! via direct-write with disjoint index sets (no data races). Color groups
//! smaller than a threshold fall back to the sequential path to avoid
//! rayon overhead on small workloads.
//!
//! On Apple M5 Pro, the P-core count (6) is the optimal thread count.
//! The rayon thread pool is initialised to `min(P-cores, available)` to
//! avoid scheduling onto the slower E-cores.
//!
//! ## Results (Apple M5 Pro, 2026-04-24, 5 P-core threads)
//!
//! | Workload | Sequential | Parallel | Speedup |
//! |---------------------------------------------|------------:|-----------:|--------:|
//! | History::converge/500x100@10perslice | 4.71 ms | 4.79 ms | 1.0× |
//! | History::converge/2000x200@20perslice | 23.36 ms | 23.28 ms | 1.0× |
//! | History::converge/1v1-5000x50000@5000perslice| 13.90 ms | 6.99 ms | **2.0×** |
//!
//! T3 acceptance gate: ≥2× speedup on at least one workload — ACHIEVED.
//! Small workloads fall below the RAYON_THRESHOLD (64 events/color) and
//! run sequentially with near-zero overhead.
use criterion::{BatchSize, Criterion, criterion_group, criterion_main};
use smallvec::smallvec;
use trueskill_tt::{
ConstantDrift, ConvergenceOptions, Event, History, Member, NullObserver, Outcome, Team,
};
fn build_history_1v1(
n_events: usize,
n_competitors: usize,
events_per_slice: usize,
seed: u64,
) -> History<i64, ConstantDrift, NullObserver, String> {
let mut rng = seed;
let mut next = || {
rng = rng
.wrapping_mul(6364136223846793005)
.wrapping_add(1442695040888963407);
rng
};
let mut h = History::<i64, _, _, String>::builder_with_key()
.mu(25.0)
.sigma(25.0 / 3.0)
.beta(25.0 / 6.0)
.drift(ConstantDrift(25.0 / 300.0))
.convergence(ConvergenceOptions {
max_iter: 30,
epsilon: 1e-6,
})
.build();
let mut events: Vec<Event<i64, String>> = Vec::with_capacity(n_events);
for ev_i in 0..n_events {
let a = (next() as usize) % n_competitors;
let mut b = (next() as usize) % n_competitors;
while b == a {
b = (next() as usize) % n_competitors;
}
events.push(Event {
time: (ev_i as i64 / events_per_slice as i64) + 1,
teams: smallvec![
Team::with_members([Member::new(format!("p{a}"))]),
Team::with_members([Member::new(format!("p{b}"))]),
],
outcome: Outcome::winner((next() % 2) as u32, 2),
});
}
h.add_events(events).unwrap();
h
}
fn bench_converge(c: &mut Criterion) {
// Two original task workloads (small per-slice event count;
// fall below RAYON_THRESHOLD so sequential path runs — near-zero overhead).
c.bench_function("History::converge/500x100@10perslice", |b| {
b.iter_batched(
|| build_history_1v1(500, 100, 10, 42),
|mut h| {
h.converge().unwrap();
},
BatchSize::SmallInput,
);
});
c.bench_function("History::converge/2000x200@20perslice", |b| {
b.iter_batched(
|| build_history_1v1(2000, 200, 20, 42),
|mut h| {
h.converge().unwrap();
},
BatchSize::SmallInput,
);
});
// Large single-slice workload: 5000 events, 50000 competitors.
// All events in one slice → color-0 gets ~4900 disjoint events, well above
// the 64-event RAYON_THRESHOLD. 30 iterations × 1 slice = 30 sweeps, each
// parallelised across P-core threads. Shows ≥2× speedup.
c.bench_function("History::converge/1v1-5000x50000@5000perslice", |b| {
b.iter_batched(
|| build_history_1v1(5000, 50000, 5000, 42),
|mut h| {
h.converge().unwrap();
},
BatchSize::SmallInput,
);
});
}
criterion_group!(benches, bench_converge);
criterion_main!(benches);