test: scale the ceiling sweep by build profile
Each sample runs a full inference pass per outcome, and that is about 19x faster in release: 20 000 samples take 12.1s released against 23s for 2 000 in debug. `just test` runs three debug feature combinations and one release one, so a fixed sample count pays the slow price three times and the fast one once — exactly backwards. Scaling by `cfg!(debug_assertions)` puts the search where it is cheap: debug 1 000 samples 11.7s release 50 000 samples 31.6s Across the whole `just test` that is 67s against 70s before, for 25x the samples. The debug run proves the sweep compiles and holds; the release run is the one that actually searches. Not moving the suite to release-only, which was the alternative considered. `debug_assert!` is compiled out in release, and this crate documents that as load-bearing — several defects have hidden there — so dropping the debug runs would trade one class of coverage for another rather than adding any. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
@@ -17,6 +17,24 @@ use trueskill_tt::{
|
|||||||
|
|
||||||
type R = Rating<i64, ConstantDrift>;
|
type R = Rating<i64, ConstantDrift>;
|
||||||
|
|
||||||
|
/// How many random matchups the ceiling sweep draws.
|
||||||
|
///
|
||||||
|
/// Scaled by build profile rather than fixed. Each sample runs a full inference
|
||||||
|
/// pass per outcome, and that is about **19x** faster in release — measured,
|
||||||
|
/// 20 000 samples take 12.1s released against 23s for 2 000 in debug. `just
|
||||||
|
/// test` runs three debug feature combinations and one release one, so a fixed
|
||||||
|
/// count pays the slow price three times and the fast one once, which is
|
||||||
|
/// exactly backwards.
|
||||||
|
///
|
||||||
|
/// The debug run is here to prove the sweep still compiles and holds on a small
|
||||||
|
/// sample; the release run is the one that actually searches. The violation
|
||||||
|
/// this guards was found at a rate near 1.8%, so even the debug count expects
|
||||||
|
/// tens of hits in the regime.
|
||||||
|
#[cfg(debug_assertions)]
|
||||||
|
const SAMPLES: usize = 1_000;
|
||||||
|
#[cfg(not(debug_assertions))]
|
||||||
|
const SAMPLES: usize = 50_000;
|
||||||
|
|
||||||
/// Deterministic LCG, so a failure is reproducible from the printed seed.
|
/// Deterministic LCG, so a failure is reproducible from the printed seed.
|
||||||
struct Lcg(u64);
|
struct Lcg(u64);
|
||||||
|
|
||||||
@@ -49,7 +67,7 @@ fn information_gain_never_exceeds_the_entropy_of_the_outcome() {
|
|||||||
let mut evaluated = 0usize;
|
let mut evaluated = 0usize;
|
||||||
let mut refused = 0usize;
|
let mut refused = 0usize;
|
||||||
|
|
||||||
for i in 0..2_000 {
|
for i in 0..SAMPLES {
|
||||||
let mu_a = rng.in_range(-100.0, 100.0);
|
let mu_a = rng.in_range(-100.0, 100.0);
|
||||||
let mu_b = rng.in_range(-100.0, 100.0);
|
let mu_b = rng.in_range(-100.0, 100.0);
|
||||||
let sigma_a = rng.log_uniform(1e-4, 1e2);
|
let sigma_a = rng.log_uniform(1e-4, 1e2);
|
||||||
@@ -91,8 +109,8 @@ fn information_gain_never_exceeds_the_entropy_of_the_outcome() {
|
|||||||
// The sweep must actually exercise the function, not pass by refusing
|
// The sweep must actually exercise the function, not pass by refusing
|
||||||
// everything.
|
// everything.
|
||||||
assert!(
|
assert!(
|
||||||
evaluated > 1_000,
|
evaluated * 2 > SAMPLES,
|
||||||
"only {evaluated} of 2000 samples were evaluated ({refused} refused); \
|
"only {evaluated} of {SAMPLES} samples were evaluated ({refused} refused); \
|
||||||
the sweep is no longer testing anything"
|
the sweep is no longer testing anything"
|
||||||
);
|
);
|
||||||
// And it must still reach the regime where the ceiling was violated —
|
// And it must still reach the regime where the ceiling was violated —
|
||||||
|
|||||||
Reference in New Issue
Block a user