`Gaussian` publicly implemented `Mul`, `Div`, `Add` and `Sub`. They were
the EP product, cavity and variance-space convolutions, and every one of
them lies to a reader who takes the operator at face value:
a = N(10, 2) b = N(4, 3) c = N(1, 1)
a * b N(8.15, 1.66) not 40
a - b sigma GREW, 2 -> sqrt(4 + 9)
a * N(1, 0) mu = NaN "multiply by one"
a / c pi = -0.75 mu() prints a confident 0
The last is this crate's signature defect on a public operator. `Div` is
the cavity and can legitimately leave a negative precision, which is not
a distribution — and `mu()`/`sigma()` guard `pi <= 0` and report `0.0`
and `inf`, so it comes back as a plausible number with no panic, no
`Debug` marker and nothing to test against.
The four impls are now `pub(crate)` inherent methods that say what they
do: `ep_product`, `cavity`, `convolve`, `convolve_diff`, plus `scale`
for the one operation that genuinely is arithmetic. Nothing in a user's
workflow needed operator syntax; inference did, and it still has it.
`pi()` and `tau()` follow. Storing natural parameters is a performance
decision — it makes message passing two adds — not a contract. The
public surface is now exactly: `from_ms`, `from_mv`, `mu`, `sigma`,
`variance`, `probability_below`, `probability_above`. `from_mv` and
`variance` are promoted from `pub(crate)`; they are the honest pair for
callers who already hold a variance and should not pay a round trip
through the square root.
Four integration tests asserted bit-identity on `(pi, tau)`. They assert
it on `(mu, variance)` instead — still `assert_eq!`, still exact, and
`1/pi` and `tau/pi` are deterministic, so bit-equal natural parameters
give bit-equal moments. `a_nan_sigma_passes_through_from_ms` drops its
`|| g.pi().is_nan()` half: `sigma()` substitutes for `pi <= 0` and
`pi == inf`, so NaN survives to it only from a NaN precision.
`benches/gaussian.rs` is deleted. It timed two f64 additions through the
public operators, and keeping those public solely to feed it is the same
thing #73 objected to when a benchmark was dictating five public types.
The paths it covered are exercised by `batch` and `history_converge`
through the real call chain.
Closes #71.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hcFjNDmHXZF8URGLku5zZ
202 lines
7.2 KiB
Rust
202 lines
7.2 KiB
Rust
use crate::{
|
||
N_INF,
|
||
factor::{VarId, VarStore},
|
||
gaussian::Gaussian,
|
||
ln_pdf,
|
||
};
|
||
|
||
/// Gaussian observation factor on a diff variable.
|
||
///
|
||
/// Encodes the soft evidence `m_obs ~ N(diff, sigma²)`. The outgoing message
|
||
/// to `diff` is the constant `N(m_obs, sigma²)`, so this factor converges in a
|
||
/// single propagation: subsequent calls return a zero delta.
|
||
#[derive(Debug)]
|
||
pub struct MarginFactor {
|
||
pub diff: VarId,
|
||
pub m_obs: f64,
|
||
pub sigma: f64,
|
||
pub(crate) msg: Gaussian,
|
||
pub(crate) log_evidence_cached: Option<f64>,
|
||
}
|
||
|
||
impl MarginFactor {
|
||
#[must_use]
|
||
pub fn new(diff: VarId, m_obs: f64, sigma: f64) -> Self {
|
||
debug_assert!(sigma > 0.0, "score sigma must be positive");
|
||
Self {
|
||
diff,
|
||
m_obs,
|
||
sigma,
|
||
msg: N_INF,
|
||
log_evidence_cached: None,
|
||
}
|
||
}
|
||
}
|
||
|
||
impl MarginFactor {
|
||
/// Propagate this factor's message, optionally damping the update in
|
||
/// natural-parameter space. `alpha = 1.0` matches `Factor::propagate`
|
||
/// exactly; `alpha < 1.0` writes `α·new_msg + (1−α)·old_msg`.
|
||
pub(crate) fn propagate_with_alpha(&mut self, vars: &mut VarStore, alpha: f64) -> (f64, f64) {
|
||
let marginal = vars.get(self.diff);
|
||
let cavity = marginal.cavity(self.msg);
|
||
|
||
if self.log_evidence_cached.is_none() {
|
||
self.log_evidence_cached = Some(cavity_log_evidence(cavity, self.m_obs, self.sigma));
|
||
}
|
||
|
||
let new_msg = Gaussian::from_ms(self.m_obs, self.sigma);
|
||
let damped = self.msg.damp_natural(new_msg, alpha);
|
||
let old_msg = self.msg;
|
||
self.msg = damped;
|
||
vars.set(self.diff, cavity.ep_product(damped));
|
||
|
||
old_msg.delta(damped)
|
||
}
|
||
}
|
||
|
||
/// Undamped wrappers, used by this module's tests. Inference drives these
|
||
/// factors through `propagate_with_alpha` and reads the cached log evidence
|
||
/// directly, so these are not on any production path.
|
||
#[cfg(test)]
|
||
impl MarginFactor {
|
||
pub(crate) fn propagate(&mut self, vars: &mut VarStore) -> (f64, f64) {
|
||
self.propagate_with_alpha(vars, 1.0)
|
||
}
|
||
|
||
pub(crate) fn log_evidence(&self) -> f64 {
|
||
self.log_evidence_cached.unwrap_or(0.0)
|
||
}
|
||
}
|
||
|
||
/// `ln` of the observed margin's density under the cavity.
|
||
///
|
||
/// Computed in log space rather than as `pdf(..).ln()`. The density underflows
|
||
/// to zero past about 38 sigma of separation, and clamping that to
|
||
/// `f64::MIN_POSITIVE` reported -708 nats however far out the observation
|
||
/// actually was — 4292 nats adrift at 100 sigma, and unbounded beyond. A score
|
||
/// far from what the model expected is exactly the observation a log-evidence
|
||
/// figure exists to notice.
|
||
fn cavity_log_evidence(cavity: Gaussian, m_obs: f64, sigma: f64) -> f64 {
|
||
// `hypot`, not `sqrt(a^2 + b^2)`: squaring overflows to infinity above a
|
||
// sigma of ~1.3e154 and flushes to zero below ~1.5e-154, and `Gaussian`'s
|
||
// constructors are public so a caller can reach both.
|
||
let combined_sigma = libm::hypot(cavity.sigma(), sigma);
|
||
let value = ln_pdf(m_obs, cavity.mu(), combined_sigma);
|
||
|
||
// A degenerate cavity (infinite sigma) is the only way to reach a
|
||
// non-finite result; fall back to the old floor rather than emit -inf.
|
||
if value.is_finite() {
|
||
value
|
||
} else {
|
||
libm::log(f64::MIN_POSITIVE)
|
||
}
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
|
||
#[test]
|
||
fn first_propagate_writes_tilted_marginal() {
|
||
let mut vars = VarStore::new();
|
||
let diff = vars.alloc(Gaussian::from_ms(0.0, 6.0));
|
||
let mut f = MarginFactor::new(diff, 5.0, 1.0);
|
||
|
||
f.propagate(&mut vars);
|
||
|
||
let result = vars.get(diff);
|
||
// pi = 1/36 + 1 ≈ 1.027778; tau = 0 + 5 = 5
|
||
// mu = 5 / 1.027778 ≈ 4.864865; sigma = 1/sqrt(1.027778) ≈ 0.986394
|
||
assert!((result.mu() - 4.864864864864865).abs() < 1e-12);
|
||
assert!((result.sigma() - 0.986393923832144).abs() < 1e-12);
|
||
}
|
||
|
||
#[test]
|
||
fn converges_in_one_step() {
|
||
let mut vars = VarStore::new();
|
||
let diff = vars.alloc(Gaussian::from_ms(0.0, 6.0));
|
||
let mut f = MarginFactor::new(diff, 5.0, 1.0);
|
||
|
||
f.propagate(&mut vars);
|
||
let (dmu, dsig) = f.propagate(&mut vars);
|
||
assert!(
|
||
dmu < 1e-12,
|
||
"expected ~0 delta on second propagate, got {dmu}"
|
||
);
|
||
assert!(dsig < 1e-12);
|
||
}
|
||
|
||
#[test]
|
||
fn evidence_cached_on_first_propagate() {
|
||
let mut vars = VarStore::new();
|
||
let diff = vars.alloc(Gaussian::from_ms(0.0, 6.0));
|
||
let mut f = MarginFactor::new(diff, 5.0, 1.0);
|
||
assert!(f.log_evidence_cached.is_none());
|
||
|
||
f.propagate(&mut vars);
|
||
let z = f.log_evidence_cached.unwrap();
|
||
// ln pdf(5, 0, sqrt(37)) = ln(0.046783...)
|
||
assert!((z.exp() - 0.04678300292616668).abs() < 1e-10);
|
||
|
||
// Subsequent propagations don't change it.
|
||
f.propagate(&mut vars);
|
||
assert_eq!(f.log_evidence_cached.unwrap(), z);
|
||
}
|
||
|
||
#[test]
|
||
fn log_evidence_matches_cached_ln() {
|
||
let mut vars = VarStore::new();
|
||
let diff = vars.alloc(Gaussian::from_ms(0.0, 6.0));
|
||
let mut f = MarginFactor::new(diff, 5.0, 1.0);
|
||
f.propagate(&mut vars);
|
||
let logz = f.log_evidence();
|
||
assert!((logz - (-3.062235327364623)).abs() < 1e-10);
|
||
}
|
||
|
||
#[test]
|
||
fn propagate_with_alpha_one_matches_undamped_propagate() {
|
||
let mut vars_a = VarStore::new();
|
||
let diff_a = vars_a.alloc(Gaussian::from_ms(0.0, 6.0));
|
||
let mut f_a = MarginFactor::new(diff_a, 5.0, 1.0);
|
||
let delta_a = f_a.propagate(&mut vars_a);
|
||
let result_a = vars_a.get(diff_a);
|
||
|
||
let mut vars_b = VarStore::new();
|
||
let diff_b = vars_b.alloc(Gaussian::from_ms(0.0, 6.0));
|
||
let mut f_b = MarginFactor::new(diff_b, 5.0, 1.0);
|
||
let delta_b = f_b.propagate_with_alpha(&mut vars_b, 1.0);
|
||
let result_b = vars_b.get(diff_b);
|
||
|
||
assert_eq!(result_a.pi(), result_b.pi());
|
||
assert_eq!(result_a.tau(), result_b.tau());
|
||
assert_eq!(delta_a, delta_b);
|
||
assert_eq!(f_a.msg.pi(), f_b.msg.pi());
|
||
assert_eq!(f_a.msg.tau(), f_b.msg.tau());
|
||
}
|
||
|
||
#[test]
|
||
fn propagate_with_alpha_half_blends_msg_in_natural_params() {
|
||
// Run undamped to capture (initial_msg, undamped_new_msg).
|
||
let mut vars_full = VarStore::new();
|
||
let diff_full = vars_full.alloc(Gaussian::from_ms(0.0, 6.0));
|
||
let mut f_full = MarginFactor::new(diff_full, 5.0, 1.0);
|
||
let initial_msg_pi = f_full.msg.pi();
|
||
let initial_msg_tau = f_full.msg.tau();
|
||
f_full.propagate(&mut vars_full);
|
||
let undamped_msg_pi = f_full.msg.pi();
|
||
let undamped_msg_tau = f_full.msg.tau();
|
||
|
||
// Run damped at α = 0.5 from the same initial state.
|
||
let mut vars_half = VarStore::new();
|
||
let diff_half = vars_half.alloc(Gaussian::from_ms(0.0, 6.0));
|
||
let mut f_half = MarginFactor::new(diff_half, 5.0, 1.0);
|
||
f_half.propagate_with_alpha(&mut vars_half, 0.5);
|
||
|
||
let expected_pi = 0.5 * undamped_msg_pi + 0.5 * initial_msg_pi;
|
||
let expected_tau = 0.5 * undamped_msg_tau + 0.5 * initial_msg_tau;
|
||
assert!((f_half.msg.pi() - expected_pi).abs() < 1e-12);
|
||
assert!((f_half.msg.tau() - expected_tau).abs() < 1e-12);
|
||
}
|
||
}
|