You offered this in #47 — "caching the factorisation would make repeat queries O(n^2)" — conditional on someone measuring first. Measured, and filing it as its own issue so it is not buried in a closed thread, because the case is broader than ustat.
Measured
Four consecutive posterior_of calls against the same unchanged history, ustat's hole-grain model (~2,000 nodes over 76 slices), release build:
solve 0: 5960 ms
solve 1: 5935 ms
solve 2: 5942 ms
solve 3: 5931 ms
Flat. Nothing is reused between calls, so each query re-pays the factorisation.
For comparison, the round-grain model (~250 nodes, same 76 slices) is 4.4 ms a solve, so the growth is steep in the appearance count, as expected for a dense O(n^3) solve.
Why it is worth doing even though ustat can work around it
We are adding a background job system, so a nine-minute batch is no longer disqualifying for us specifically. I want to be clear that I am not asking for this to unblock us.
The reason to do it anyway is that the cost profile makes a whole class of query impractical, and it is the class the API invites. posterior_of takes a linear functional, which naturally suggests asking for many of them: every pair in a standings table, every cell in a grid, every candidate in an active-learning sweep. Our per-hole grid is 90 cells — about nine minutes at the measured rate, against roughly one factorisation plus 90 cheap back-solves if it were cached, which should be seconds.
expected_variance_reduction sharpens this. Its whole purpose is to score candidate matchups, which means many queries against one unchanged history — the exact pattern that pays the full solve every time today. A consumer scoring 50 candidates pays 50 factorisations to answer one question.
Shape
The history is immutable between converge() calls, so the factorisation is a pure function of the fit. A cache keyed on the converged state, invalidated whenever inference runs, would not change any public signature. &self on posterior_of makes interior mutability necessary; if that is unwelcome, an explicit handle would work too:
letjoint=fit.joint()?;// factorise once
joint.posterior_of(&terms)?;// O(n^2) per query
That form has the advantage of making the cost model visible in the type rather than implicit, which for a call this expensive may be the better trade.
Sparsity is presumably the larger win after that — a time-expanded joint is banded by construction, since a competitor's appearances only couple to their neighbours in time and to co-appearing competitors — but that is a bigger change and caching is worth having first.
No urgency from here. Filing it so the measurement is on record rather than in a comment thread.
You offered this in #47 — "caching the factorisation would make repeat queries O(n^2)" — conditional on someone measuring first. Measured, and filing it as its own issue so it is not buried in a closed thread, because the case is broader than ustat.
## Measured
Four consecutive `posterior_of` calls against the **same unchanged history**, ustat's hole-grain model (~2,000 nodes over 76 slices), release build:
```
solve 0: 5960 ms
solve 1: 5935 ms
solve 2: 5942 ms
solve 3: 5931 ms
```
Flat. Nothing is reused between calls, so each query re-pays the factorisation.
For comparison, the round-grain model (~250 nodes, same 76 slices) is **4.4 ms** a solve, so the growth is steep in the appearance count, as expected for a dense O(n^3) solve.
## Why it is worth doing even though ustat can work around it
We are adding a background job system, so a nine-minute batch is no longer disqualifying for us specifically. I want to be clear that I am **not** asking for this to unblock us.
The reason to do it anyway is that the cost profile makes a whole class of query impractical, and it is the class the API invites. `posterior_of` takes a linear functional, which naturally suggests asking for *many* of them: every pair in a standings table, every cell in a grid, every candidate in an active-learning sweep. Our per-hole grid is 90 cells — about **nine minutes** at the measured rate, against roughly one factorisation plus 90 cheap back-solves if it were cached, which should be seconds.
`expected_variance_reduction` sharpens this. Its whole purpose is to score *candidate matchups*, which means many queries against one unchanged history — the exact pattern that pays the full solve every time today. A consumer scoring 50 candidates pays 50 factorisations to answer one question.
## Shape
The history is immutable between `converge()` calls, so the factorisation is a pure function of the fit. A cache keyed on the converged state, invalidated whenever inference runs, would not change any public signature. `&self` on `posterior_of` makes interior mutability necessary; if that is unwelcome, an explicit handle would work too:
```rust
let joint = fit.joint()?; // factorise once
joint.posterior_of(&terms)?; // O(n^2) per query
```
That form has the advantage of making the cost model visible in the type rather than implicit, which for a call this expensive may be the better trade.
Sparsity is presumably the larger win after that — a time-expanded joint is banded by construction, since a competitor's appearances only couple to their neighbours in time and to co-appearing competitors — but that is a bigger change and caching is worth having first.
No urgency from here. Filing it so the measurement is on record rather than in a comment thread.
Correction to the figure above, and a finding that is probably more useful than the original request.
I said our 90-cell grid is "about nine minutes". That is wrong. It took the 6 s rate from the drifting fit and applied it to a grid computed from the career fit — a different history, which the collapse rule makes far cheaper.
Measured, same 2,000-node hole model, same 76 slices, same machine:
career (gamma_skill = 0) one solve 787 ms
drifting (gamma_skill = .15) one solve 6214 ms
Roughly 8x, and the cause is your own collapse rule. "Consecutive appearances with no drift between them collapse to the same variable" means a drift-free competitor contributes one variable rather than one per slice. So the career fit's joint is a fraction of the drifting fit's, and cost tracks effective appearances rather than slices × competitors.
So the honest numbers for us are ~71 s for the grid, not nine minutes. Still too slow for a boot path, fine for a background job, and cheap if cached — but an order of magnitude less alarming than I claimed, and I would rather that be on the record than let the case rest on a number I got wrong.
What is worth taking from it
The cost model deserves a doc line, because the variance is large and non-obvious. "Dense solve, O(n^3) in the slice's competitor count" was accurate for 0.5.0's single-slice joint; for the time-expanded one the driver is effective appearances, and drift_scale = 0 collapses a competitor to a single variable however long the history. A consumer choosing between a drifting and a drift-free configuration is also choosing an 8x difference in query cost, and nothing currently says so.
That also suggests a cheap optimisation ahead of caching: the collapse already exists, so anything that widens what counts as "no drift between these appearances" — a competitor who did not appear in the intervening slices, say — shrinks the matrix directly.
The case for caching is unchanged in kind, only in magnitude. The expected_variance_reduction argument does not depend on our grid at all: scoring N candidate matchups against one unchanged history pays N factorisations to answer one question, and that is the pattern the call exists to serve.
Sorry for the bad number. The measurement script is a one-off in our tree; if a benchmark in yours would be useful I can describe what it does.
**Correction to the figure above, and a finding that is probably more useful than the original request.**
I said our 90-cell grid is "about nine minutes". That is wrong. It took the 6 s rate from the *drifting* fit and applied it to a grid computed from the **career** fit — a different history, which the collapse rule makes far cheaper.
Measured, same 2,000-node hole model, same 76 slices, same machine:
```
career (gamma_skill = 0) one solve 787 ms
drifting (gamma_skill = .15) one solve 6214 ms
```
**Roughly 8x, and the cause is your own collapse rule.** "Consecutive appearances with no drift between them collapse to the same variable" means a drift-free competitor contributes **one** variable rather than one per slice. So the career fit's joint is a fraction of the drifting fit's, and cost tracks *effective* appearances rather than slices × competitors.
So the honest numbers for us are ~71 s for the grid, not nine minutes. Still too slow for a boot path, fine for a background job, and cheap if cached — but an order of magnitude less alarming than I claimed, and I would rather that be on the record than let the case rest on a number I got wrong.
## What is worth taking from it
**The cost model deserves a doc line, because the variance is large and non-obvious.** "Dense solve, O(n^3) in the slice's competitor count" was accurate for 0.5.0's single-slice joint; for the time-expanded one the driver is effective appearances, and `drift_scale = 0` collapses a competitor to a single variable however long the history. A consumer choosing between a drifting and a drift-free configuration is also choosing an 8x difference in query cost, and nothing currently says so.
That also suggests a cheap optimisation ahead of caching: the collapse already exists, so anything that widens what counts as "no drift between these appearances" — a competitor who did not appear in the intervening slices, say — shrinks the matrix directly.
**The case for caching is unchanged in kind, only in magnitude.** The `expected_variance_reduction` argument does not depend on our grid at all: scoring N candidate matchups against one unchanged history pays N factorisations to answer one question, and that is the pattern the call exists to serve.
Sorry for the bad number. The measurement script is a one-off in our tree; if a benchmark in yours would be useful I can describe what it does.
Implemented, on feat/joint-handle. Not merged or released yet.
Thank you for the flat four-solve trace — that is the measurement that makes this a bug report rather than a feature request, and it is what made the shape obvious.
What it is
Your second form, for your reason. History::joint() returns a handle:
posterior_of, posterior_of_at and expected_variance_reduction all exist on it, and the one-shot calls on History now delegate to it — same cost as before, and the two paths cannot drift apart because there is only one implementation.
I went with the handle over interior mutability for a reason beyond the visible cost model you gave. The factor is n^2 floats: 31 MB at my fixture's 1,976 appearances, and about 128 MB at your ~4,000. A cache on &self parks that in the History for its lifetime, and a consumer who called posterior_of once in a request handler would be surprised by it. The handle's borrow also does the invalidation for free — the borrow checker forbids add_events or converge while it is alive, so there is no window where a cached factorisation could describe a fit that no longer exists.
joint.variables() reports the count the cost scales in, which is appearances, not competitors.
Measured
Synthetic history shaped like your hole-grain model, 1,976 appearances, 90 queries:
one-shot, 90 queries
68.4 s
joint: factorise
745 ms
joint: 90 queries
93 ms
total
838 ms
81.6x, with the answers bit-identical — asserted, not eyeballed. Criterion at 480 appearances puts the per-query figure at 9.04 ms one-shot against 48 us cached, 187x.
Your 90-cell grid should go from nine minutes to roughly your 5.9 s plus a few hundred milliseconds.
A second thing fell out of it
Splitting factor from solve made it obvious that no caller ever wanted A^-1 c. Every question is a bilinear form, and
c^T A^-1 a = c^T L^-T L^-1 a = (L^-1 c) . (L^-1 a)
so one forward substitution per contrast answers everything and the back substitution was wasted work — that is where half of the per-query time went. It also removes a failure mode: a variance computed as c . (A^-1 c) is a difference of products that can round to a small negative number, where |L^-1 c|^2 is a sum of squares and cannot.
On sparsity
You were right, and it is worse than "presumably". I instrumented the matrix: at n=1976 it has 7,504 nonzeros — 0.19% density. We allocate 31 MB, fill 99.81% of it with zeros, and factorise the lot. Filed as #52 with the measurement and a crates.io shortlist; the promising lead is an AMD-ordering-only crate that would let us keep our own Cholesky for two dependencies and no unsafe, rather than a full sparse solver.
Bandwidth is 207 on that fixture, but I do not think banding is the fix — one competitor absent for many slices creates a drift link spanning the matrix and the band swallows everything between. Fill-reducing ordering is the robust version.
Noted that you are not blocked and that this was filed for the record. It was still worth doing on your argument rather than your need: the API takes a linear functional, which invites asking for many, and expected_variance_reduction exists specifically to score a field of candidates against one unchanged fit. Charging a factorisation per candidate made the call's own purpose impractical.
Implemented, on `feat/joint-handle`. Not merged or released yet.
Thank you for the flat four-solve trace — that is the measurement that makes this a bug report rather than a feature request, and it is what made the shape obvious.
## What it is
Your second form, for your reason. `History::joint()` returns a handle:
```rust
let joint = h.joint()?;
for (a, b) in pairs {
let gap = joint.posterior_of(&[(&a, 1.0), (&b, -1.0)])?;
}
```
`posterior_of`, `posterior_of_at` and `expected_variance_reduction` all exist on it, and the one-shot calls on `History` now delegate to it — same cost as before, and the two paths cannot drift apart because there is only one implementation.
I went with the handle over interior mutability for a reason beyond the visible cost model you gave. The factor is `n^2` floats: 31 MB at my fixture's 1,976 appearances, and about **128 MB at your ~4,000**. A cache on `&self` parks that in the `History` for its lifetime, and a consumer who called `posterior_of` once in a request handler would be surprised by it. The handle's borrow also does the invalidation for free — the borrow checker forbids `add_events` or `converge` while it is alive, so there is no window where a cached factorisation could describe a fit that no longer exists.
`joint.variables()` reports the count the cost scales in, which is appearances, not competitors.
## Measured
Synthetic history shaped like your hole-grain model, 1,976 appearances, 90 queries:
| | |
|---|---|
| one-shot, 90 queries | **68.4 s** |
| joint: factorise | 745 ms |
| joint: 90 queries | 93 ms |
| **total** | **838 ms** |
**81.6x**, with the answers bit-identical — asserted, not eyeballed. Criterion at 480 appearances puts the per-query figure at 9.04 ms one-shot against **48 us** cached, 187x.
Your 90-cell grid should go from nine minutes to roughly your 5.9 s plus a few hundred milliseconds.
## A second thing fell out of it
Splitting factor from solve made it obvious that no caller ever wanted `A^-1 c`. Every question is a bilinear form, and
```
c^T A^-1 a = c^T L^-T L^-1 a = (L^-1 c) . (L^-1 a)
```
so one forward substitution per contrast answers everything and the back substitution was wasted work — that is where half of the per-query time went. It also removes a failure mode: a variance computed as `c . (A^-1 c)` is a difference of products that can round to a small negative number, where `|L^-1 c|^2` is a sum of squares and cannot.
## On sparsity
You were right, and it is worse than "presumably". I instrumented the matrix: at n=1976 it has **7,504 nonzeros — 0.19% density**. We allocate 31 MB, fill 99.81% of it with zeros, and factorise the lot. Filed as #52 with the measurement and a crates.io shortlist; the promising lead is an AMD-ordering-only crate that would let us keep our own Cholesky for two dependencies and no unsafe, rather than a full sparse solver.
Bandwidth is 207 on that fixture, but I do not think banding is the fix — one competitor absent for many slices creates a drift link spanning the matrix and the band swallows everything between. Fill-reducing ordering is the robust version.
Noted that you are not blocked and that this was filed for the record. It was still worth doing on your argument rather than your need: the API takes a linear functional, which invites asking for many, and `expected_variance_reduction` exists specifically to score a field of candidates against one unchanged fit. Charging a factorisation per candidate made the call's own purpose impractical.
Minor rather than patch: History::joint, Joint and Joint::variables are new public surface. Nothing was removed and no existing signature changed, so posterior_of, posterior_of_at and expected_variance_reduction keep working exactly as they did — they now delegate to the handle internally.
One change worth knowing about before you diff numbers against 0.6.0: the per-query path is now a single forward substitution rather than a full solve, so the last bits of a variance can differ. Same quantity, computed as |L^-1 c|^2 instead of c . (A^-1 c). Every golden in the suite still passes at its existing tolerance, and the joint's answers are asserted bit-identical to the one-shot ones, but if you have anything pinned to a full-precision variance from 0.6.0 that is where a difference would come from. The new form is the better-conditioned one — it is a sum of squares, so it cannot round negative.
Released in **0.7.0**, on the registry now.
Minor rather than patch: `History::joint`, `Joint` and `Joint::variables` are new public surface. Nothing was removed and no existing signature changed, so `posterior_of`, `posterior_of_at` and `expected_variance_reduction` keep working exactly as they did — they now delegate to the handle internally.
One change worth knowing about before you diff numbers against 0.6.0: the per-query path is now a single forward substitution rather than a full solve, so the last bits of a variance can differ. Same quantity, computed as `|L^-1 c|^2` instead of `c . (A^-1 c)`. Every golden in the suite still passes at its existing tolerance, and the joint's answers are asserted bit-identical to the one-shot ones, but if you have anything pinned to a full-precision variance from 0.6.0 that is where a difference would come from. The new form is the better-conditioned one — it is a sum of squares, so it cannot round negative.
Sparsity is #52.
Shipped in 0.7.0 and wired up here. Measured on the real export, round-grain connectivity — 11 competitors, 55 pairs:
379 variables
factorise 4.3 ms
55 queries 1.6 ms (~29 us each)
------------------------------
total 5.9 ms against 55 x 4.3 = 237 ms
40x, and /api/connectivity is byte-identical either way. That is ~237 ms out of our boot for a two-line change.
The handle-over-cache choice was right, and for a reason I did not anticipate when I suggested it: the borrow makes correctness structural rather than something to get right. There is no window where a stale factorisation could describe a refitted history, because the borrow checker will not compile one. A cache keyed on converged state would have needed that argument made in prose and tested for.
variables() is a good addition too — 379 for our round grain against ~2,000 for the hole grain tells a caller which side of the affordability line they are on before they pay anything.
One more datum for the cost model
Following the correction above, the drift-dependence is larger than I first reported and worth stating in one place:
fit
gamma_skill
one solve
career
0
787 ms
drifting
0.15
6214 ms
Same model, same 76 slices, same ~2,000 nodes — 8x, entirely from the collapse rule turning a drift-free competitor's 76 appearances into one variable. With the handle our 90-cell grid goes from ~71 s to roughly a second.
So this is fully solved from our side. We will not be using it at the hole grain, but for the honest reason rather than the cost one: the exact per-hole posterior clears nothing at our sample size (0 of 18 cells at 2 sigma, single node and contrast alike), so the affordability was never what stood between us and that claim. Detail in our #114 if it is of interest — the short version is that your advice to express it as a difference was right in principle and buys 2.4% here, because Hole, HolePlayer and DayForm already absorb the levels a contrast would cancel.
Thank you for turning this round so quickly, and for the benchmark numbers in the commit message — they matched ours closely enough that I trusted the change before measuring it.
Shipped in 0.7.0 and wired up here. Measured on the real export, round-grain connectivity — 11 competitors, 55 pairs:
```
379 variables
factorise 4.3 ms
55 queries 1.6 ms (~29 us each)
------------------------------
total 5.9 ms against 55 x 4.3 = 237 ms
```
**40x**, and `/api/connectivity` is byte-identical either way. That is ~237 ms out of our boot for a two-line change.
The handle-over-cache choice was right, and for a reason I did not anticipate when I suggested it: the borrow makes correctness structural rather than something to get right. There is no window where a stale factorisation could describe a refitted history, because the borrow checker will not compile one. A cache keyed on converged state would have needed that argument made in prose and tested for.
`variables()` is a good addition too — 379 for our round grain against ~2,000 for the hole grain tells a caller which side of the affordability line they are on before they pay anything.
## One more datum for the cost model
Following the correction above, the drift-dependence is larger than I first reported and worth stating in one place:
| fit | `gamma_skill` | one solve |
|---|---|---|
| career | 0 | **787 ms** |
| drifting | 0.15 | **6214 ms** |
Same model, same 76 slices, same ~2,000 nodes — 8x, entirely from the collapse rule turning a drift-free competitor's 76 appearances into one variable. With the handle our 90-cell grid goes from ~71 s to roughly a second.
So this is fully solved from our side. We will not be using it at the hole grain, but for the honest reason rather than the cost one: the exact per-hole posterior clears nothing at our sample size (0 of 18 cells at 2 sigma, single node and contrast alike), so the affordability was never what stood between us and that claim. Detail in our #114 if it is of interest — the short version is that your advice to express it as a difference was right in principle and buys 2.4% here, because `Hole`, `HolePlayer` and `DayForm` already absorb the levels a contrast would cancel.
Thank you for turning this round so quickly, and for the benchmark numbers in the commit message — they matched ours closely enough that I trusted the change before measuring it.
Thank you for the 40x measurement, and for correcting your own number.
First, an apology of the same kind. You posted the correction at 05:45. I replied at 05:55 and confidently told you your grid would go "from nine minutes to roughly 5.9 s" — quoting back a figure you had retracted ten minutes earlier, because I answered without reading the thread first. You went to the trouble of correcting a number that made your own case look stronger, and I overwrote it. Noted, and worth more care on my side.
The doc line — done
You were right that this deserved stating, and the mechanism is worse than "the variance is large": choosing a drift configuration is choosing a query cost, and it was invisible.
Documented on Joint, Joint::variables and posterior_of, with your 787 ms / 6214 ms measurement quoted as the illustration. variables() is now named as the number that decides affordability, since it can be read before committing to a batch — which is the use you found for it independently.
Pinned in tests/joint_handle.rs as well: ten slices, two competitors, 20 variables drifting against 2 at gamma = 0. Exactly the slice count, so a future change to the collapse rule cannot quietly remove the property the docs now promise.
Your suggested optimisation is already in place
anything that widens what counts as "no drift between these appearances" — a competitor who did not appear in the intervening slices, say — shrinks the matrix directly
That one is already true, and it is worth knowing because it bounds what is left. The joint is built from slice.appearances(), which yields only actual appearances — an absence is not an appearance. A competitor seen in the first and last of a hundred slices contributes two variables and one drift link spanning the gap, not a hundred. So on that axis the matrix is already as small as the model permits, and the remaining win is the one in #52: it is 0.19% dense and we factorise it densely.
On the hole grain
Understood, and the reason you gave is the more interesting one. If the exact per-hole posterior clears nothing at your sample size — 0 of 18 cells at 2 sigma, single node and contrast alike — then affordability was never the binding constraint, and making it fast would have bought a faster way to publish nothing. That is a better outcome for this issue than a performance win would have been.
The 2.4% from expressing it as a difference is a useful negative result too: it says Hole, HolePlayer and DayForm are already absorbing the levels a contrast would cancel, so the advice was right in principle and nearly inert in your parameterisation. Worth me remembering before repeating it.
Yes to the benchmark description if it is cheap to write down — benches/joint.rs currently uses a synthetic history, and a real-shaped one would be a better regression target for #52.
Thank you for the 40x measurement, and for correcting your own number.
**First, an apology of the same kind.** You posted the correction at 05:45. I replied at 05:55 and confidently told you your grid would go "from nine minutes to roughly 5.9 s" — quoting back a figure you had retracted ten minutes earlier, because I answered without reading the thread first. You went to the trouble of correcting a number that made your own case look stronger, and I overwrote it. Noted, and worth more care on my side.
## The doc line — done
You were right that this deserved stating, and the mechanism is worse than "the variance is large": choosing a drift configuration *is* choosing a query cost, and it was invisible.
Documented on `Joint`, `Joint::variables` and `posterior_of`, with your 787 ms / 6214 ms measurement quoted as the illustration. `variables()` is now named as the number that decides affordability, since it can be read before committing to a batch — which is the use you found for it independently.
Pinned in `tests/joint_handle.rs` as well: ten slices, two competitors, 20 variables drifting against 2 at `gamma = 0`. Exactly the slice count, so a future change to the collapse rule cannot quietly remove the property the docs now promise.
## Your suggested optimisation is already in place
> anything that widens what counts as "no drift between these appearances" — a competitor who did not appear in the intervening slices, say — shrinks the matrix directly
That one is already true, and it is worth knowing because it bounds what is left. The joint is built from `slice.appearances()`, which yields only *actual* appearances — an absence is not an appearance. A competitor seen in the first and last of a hundred slices contributes **two** variables and one drift link spanning the gap, not a hundred. So on that axis the matrix is already as small as the model permits, and the remaining win is the one in #52: it is 0.19% dense and we factorise it densely.
## On the hole grain
Understood, and the reason you gave is the more interesting one. If the exact per-hole posterior clears nothing at your sample size — 0 of 18 cells at 2 sigma, single node and contrast alike — then affordability was never the binding constraint, and making it fast would have bought a faster way to publish nothing. That is a better outcome for this issue than a performance win would have been.
The 2.4% from expressing it as a difference is a useful negative result too: it says `Hole`, `HolePlayer` and `DayForm` are already absorbing the levels a contrast would cancel, so the advice was right in principle and nearly inert in your parameterisation. Worth me remembering before repeating it.
Yes to the benchmark description if it is cheap to write down — `benches/joint.rs` currently uses a synthetic history, and a real-shaped one would be a better regression target for #52.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
You offered this in #47 — "caching the factorisation would make repeat queries O(n^2)" — conditional on someone measuring first. Measured, and filing it as its own issue so it is not buried in a closed thread, because the case is broader than ustat.
Measured
Four consecutive
posterior_ofcalls against the same unchanged history, ustat's hole-grain model (~2,000 nodes over 76 slices), release build:Flat. Nothing is reused between calls, so each query re-pays the factorisation.
For comparison, the round-grain model (~250 nodes, same 76 slices) is 4.4 ms a solve, so the growth is steep in the appearance count, as expected for a dense O(n^3) solve.
Why it is worth doing even though ustat can work around it
We are adding a background job system, so a nine-minute batch is no longer disqualifying for us specifically. I want to be clear that I am not asking for this to unblock us.
The reason to do it anyway is that the cost profile makes a whole class of query impractical, and it is the class the API invites.
posterior_oftakes a linear functional, which naturally suggests asking for many of them: every pair in a standings table, every cell in a grid, every candidate in an active-learning sweep. Our per-hole grid is 90 cells — about nine minutes at the measured rate, against roughly one factorisation plus 90 cheap back-solves if it were cached, which should be seconds.expected_variance_reductionsharpens this. Its whole purpose is to score candidate matchups, which means many queries against one unchanged history — the exact pattern that pays the full solve every time today. A consumer scoring 50 candidates pays 50 factorisations to answer one question.Shape
The history is immutable between
converge()calls, so the factorisation is a pure function of the fit. A cache keyed on the converged state, invalidated whenever inference runs, would not change any public signature.&selfonposterior_ofmakes interior mutability necessary; if that is unwelcome, an explicit handle would work too:That form has the advantage of making the cost model visible in the type rather than implicit, which for a call this expensive may be the better trade.
Sparsity is presumably the larger win after that — a time-expanded joint is banded by construction, since a competitor's appearances only couple to their neighbours in time and to co-appearing competitors — but that is a bigger change and caching is worth having first.
No urgency from here. Filing it so the measurement is on record rather than in a comment thread.
Correction to the figure above, and a finding that is probably more useful than the original request.
I said our 90-cell grid is "about nine minutes". That is wrong. It took the 6 s rate from the drifting fit and applied it to a grid computed from the career fit — a different history, which the collapse rule makes far cheaper.
Measured, same 2,000-node hole model, same 76 slices, same machine:
Roughly 8x, and the cause is your own collapse rule. "Consecutive appearances with no drift between them collapse to the same variable" means a drift-free competitor contributes one variable rather than one per slice. So the career fit's joint is a fraction of the drifting fit's, and cost tracks effective appearances rather than slices × competitors.
So the honest numbers for us are ~71 s for the grid, not nine minutes. Still too slow for a boot path, fine for a background job, and cheap if cached — but an order of magnitude less alarming than I claimed, and I would rather that be on the record than let the case rest on a number I got wrong.
What is worth taking from it
The cost model deserves a doc line, because the variance is large and non-obvious. "Dense solve, O(n^3) in the slice's competitor count" was accurate for 0.5.0's single-slice joint; for the time-expanded one the driver is effective appearances, and
drift_scale = 0collapses a competitor to a single variable however long the history. A consumer choosing between a drifting and a drift-free configuration is also choosing an 8x difference in query cost, and nothing currently says so.That also suggests a cheap optimisation ahead of caching: the collapse already exists, so anything that widens what counts as "no drift between these appearances" — a competitor who did not appear in the intervening slices, say — shrinks the matrix directly.
The case for caching is unchanged in kind, only in magnitude. The
expected_variance_reductionargument does not depend on our grid at all: scoring N candidate matchups against one unchanged history pays N factorisations to answer one question, and that is the pattern the call exists to serve.Sorry for the bad number. The measurement script is a one-off in our tree; if a benchmark in yours would be useful I can describe what it does.
Implemented, on
feat/joint-handle. Not merged or released yet.Thank you for the flat four-solve trace — that is the measurement that makes this a bug report rather than a feature request, and it is what made the shape obvious.
What it is
Your second form, for your reason.
History::joint()returns a handle:posterior_of,posterior_of_atandexpected_variance_reductionall exist on it, and the one-shot calls onHistorynow delegate to it — same cost as before, and the two paths cannot drift apart because there is only one implementation.I went with the handle over interior mutability for a reason beyond the visible cost model you gave. The factor is
n^2floats: 31 MB at my fixture's 1,976 appearances, and about 128 MB at your ~4,000. A cache on&selfparks that in theHistoryfor its lifetime, and a consumer who calledposterior_ofonce in a request handler would be surprised by it. The handle's borrow also does the invalidation for free — the borrow checker forbidsadd_eventsorconvergewhile it is alive, so there is no window where a cached factorisation could describe a fit that no longer exists.joint.variables()reports the count the cost scales in, which is appearances, not competitors.Measured
Synthetic history shaped like your hole-grain model, 1,976 appearances, 90 queries:
81.6x, with the answers bit-identical — asserted, not eyeballed. Criterion at 480 appearances puts the per-query figure at 9.04 ms one-shot against 48 us cached, 187x.
Your 90-cell grid should go from nine minutes to roughly your 5.9 s plus a few hundred milliseconds.
A second thing fell out of it
Splitting factor from solve made it obvious that no caller ever wanted
A^-1 c. Every question is a bilinear form, andso one forward substitution per contrast answers everything and the back substitution was wasted work — that is where half of the per-query time went. It also removes a failure mode: a variance computed as
c . (A^-1 c)is a difference of products that can round to a small negative number, where|L^-1 c|^2is a sum of squares and cannot.On sparsity
You were right, and it is worse than "presumably". I instrumented the matrix: at n=1976 it has 7,504 nonzeros — 0.19% density. We allocate 31 MB, fill 99.81% of it with zeros, and factorise the lot. Filed as #52 with the measurement and a crates.io shortlist; the promising lead is an AMD-ordering-only crate that would let us keep our own Cholesky for two dependencies and no unsafe, rather than a full sparse solver.
Bandwidth is 207 on that fixture, but I do not think banding is the fix — one competitor absent for many slices creates a drift link spanning the matrix and the band swallows everything between. Fill-reducing ordering is the robust version.
Noted that you are not blocked and that this was filed for the record. It was still worth doing on your argument rather than your need: the API takes a linear functional, which invites asking for many, and
expected_variance_reductionexists specifically to score a field of candidates against one unchanged fit. Charging a factorisation per candidate made the call's own purpose impractical.Released in 0.7.0, on the registry now.
Minor rather than patch:
History::joint,JointandJoint::variablesare new public surface. Nothing was removed and no existing signature changed, soposterior_of,posterior_of_atandexpected_variance_reductionkeep working exactly as they did — they now delegate to the handle internally.One change worth knowing about before you diff numbers against 0.6.0: the per-query path is now a single forward substitution rather than a full solve, so the last bits of a variance can differ. Same quantity, computed as
|L^-1 c|^2instead ofc . (A^-1 c). Every golden in the suite still passes at its existing tolerance, and the joint's answers are asserted bit-identical to the one-shot ones, but if you have anything pinned to a full-precision variance from 0.6.0 that is where a difference would come from. The new form is the better-conditioned one — it is a sum of squares, so it cannot round negative.Sparsity is #52.
Shipped in 0.7.0 and wired up here. Measured on the real export, round-grain connectivity — 11 competitors, 55 pairs:
40x, and
/api/connectivityis byte-identical either way. That is ~237 ms out of our boot for a two-line change.The handle-over-cache choice was right, and for a reason I did not anticipate when I suggested it: the borrow makes correctness structural rather than something to get right. There is no window where a stale factorisation could describe a refitted history, because the borrow checker will not compile one. A cache keyed on converged state would have needed that argument made in prose and tested for.
variables()is a good addition too — 379 for our round grain against ~2,000 for the hole grain tells a caller which side of the affordability line they are on before they pay anything.One more datum for the cost model
Following the correction above, the drift-dependence is larger than I first reported and worth stating in one place:
gamma_skillSame model, same 76 slices, same ~2,000 nodes — 8x, entirely from the collapse rule turning a drift-free competitor's 76 appearances into one variable. With the handle our 90-cell grid goes from ~71 s to roughly a second.
So this is fully solved from our side. We will not be using it at the hole grain, but for the honest reason rather than the cost one: the exact per-hole posterior clears nothing at our sample size (0 of 18 cells at 2 sigma, single node and contrast alike), so the affordability was never what stood between us and that claim. Detail in our #114 if it is of interest — the short version is that your advice to express it as a difference was right in principle and buys 2.4% here, because
Hole,HolePlayerandDayFormalready absorb the levels a contrast would cancel.Thank you for turning this round so quickly, and for the benchmark numbers in the commit message — they matched ours closely enough that I trusted the change before measuring it.
Thank you for the 40x measurement, and for correcting your own number.
First, an apology of the same kind. You posted the correction at 05:45. I replied at 05:55 and confidently told you your grid would go "from nine minutes to roughly 5.9 s" — quoting back a figure you had retracted ten minutes earlier, because I answered without reading the thread first. You went to the trouble of correcting a number that made your own case look stronger, and I overwrote it. Noted, and worth more care on my side.
The doc line — done
You were right that this deserved stating, and the mechanism is worse than "the variance is large": choosing a drift configuration is choosing a query cost, and it was invisible.
Documented on
Joint,Joint::variablesandposterior_of, with your 787 ms / 6214 ms measurement quoted as the illustration.variables()is now named as the number that decides affordability, since it can be read before committing to a batch — which is the use you found for it independently.Pinned in
tests/joint_handle.rsas well: ten slices, two competitors, 20 variables drifting against 2 atgamma = 0. Exactly the slice count, so a future change to the collapse rule cannot quietly remove the property the docs now promise.Your suggested optimisation is already in place
That one is already true, and it is worth knowing because it bounds what is left. The joint is built from
slice.appearances(), which yields only actual appearances — an absence is not an appearance. A competitor seen in the first and last of a hundred slices contributes two variables and one drift link spanning the gap, not a hundred. So on that axis the matrix is already as small as the model permits, and the remaining win is the one in #52: it is 0.19% dense and we factorise it densely.On the hole grain
Understood, and the reason you gave is the more interesting one. If the exact per-hole posterior clears nothing at your sample size — 0 of 18 cells at 2 sigma, single node and contrast alike — then affordability was never the binding constraint, and making it fast would have bought a faster way to publish nothing. That is a better outcome for this issue than a performance win would have been.
The 2.4% from expressing it as a difference is a useful negative result too: it says
Hole,HolePlayerandDayFormare already absorbing the levels a contrast would cancel, so the advice was right in principle and nearly inert in your parameterisation. Worth me remembering before repeating it.Yes to the benchmark description if it is cheap to write down —
benches/joint.rscurrently uses a synthetic history, and a real-shaped one would be a better regression target for #52.