test: scale the ceiling sweep by build profile
Each sample runs a full inference pass per outcome, and that is about 19x faster in release: 20 000 samples take 12.1s released against 23s for 2 000 in debug. `just test` runs three debug feature combinations and one release one, so a fixed sample count pays the slow price three times and the fast one once — exactly backwards. Scaling by `cfg!(debug_assertions)` puts the search where it is cheap: debug 1 000 samples 11.7s release 50 000 samples 31.6s Across the whole `just test` that is 67s against 70s before, for 25x the samples. The debug run proves the sweep compiles and holds; the release run is the one that actually searches. Not moving the suite to release-only, which was the alternative considered. `debug_assert!` is compiled out in release, and this crate documents that as load-bearing — several defects have hidden there — so dropping the debug runs would trade one class of coverage for another rather than adding any. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
This commit is contained in:
@@ -17,6 +17,24 @@ use trueskill_tt::{
|
||||
|
||||
type R = Rating<i64, ConstantDrift>;
|
||||
|
||||
/// How many random matchups the ceiling sweep draws.
|
||||
///
|
||||
/// Scaled by build profile rather than fixed. Each sample runs a full inference
|
||||
/// pass per outcome, and that is about **19x** faster in release — measured,
|
||||
/// 20 000 samples take 12.1s released against 23s for 2 000 in debug. `just
|
||||
/// test` runs three debug feature combinations and one release one, so a fixed
|
||||
/// count pays the slow price three times and the fast one once, which is
|
||||
/// exactly backwards.
|
||||
///
|
||||
/// The debug run is here to prove the sweep still compiles and holds on a small
|
||||
/// sample; the release run is the one that actually searches. The violation
|
||||
/// this guards was found at a rate near 1.8%, so even the debug count expects
|
||||
/// tens of hits in the regime.
|
||||
#[cfg(debug_assertions)]
|
||||
const SAMPLES: usize = 1_000;
|
||||
#[cfg(not(debug_assertions))]
|
||||
const SAMPLES: usize = 50_000;
|
||||
|
||||
/// Deterministic LCG, so a failure is reproducible from the printed seed.
|
||||
struct Lcg(u64);
|
||||
|
||||
@@ -49,7 +67,7 @@ fn information_gain_never_exceeds_the_entropy_of_the_outcome() {
|
||||
let mut evaluated = 0usize;
|
||||
let mut refused = 0usize;
|
||||
|
||||
for i in 0..2_000 {
|
||||
for i in 0..SAMPLES {
|
||||
let mu_a = rng.in_range(-100.0, 100.0);
|
||||
let mu_b = rng.in_range(-100.0, 100.0);
|
||||
let sigma_a = rng.log_uniform(1e-4, 1e2);
|
||||
@@ -91,8 +109,8 @@ fn information_gain_never_exceeds_the_entropy_of_the_outcome() {
|
||||
// The sweep must actually exercise the function, not pass by refusing
|
||||
// everything.
|
||||
assert!(
|
||||
evaluated > 1_000,
|
||||
"only {evaluated} of 2000 samples were evaluated ({refused} refused); \
|
||||
evaluated * 2 > SAMPLES,
|
||||
"only {evaluated} of {SAMPLES} samples were evaluated ({refused} refused); \
|
||||
the sweep is no longer testing anything"
|
||||
);
|
||||
// And it must still reach the regime where the ceiling was violated —
|
||||
|
||||
Reference in New Issue
Block a user