Make QuasisepSolver.condition respect parallel=True and use a rank-J conditional covariance - #282
Open
dfm wants to merge 2 commits into
Open
Make QuasisepSolver.condition respect parallel=True and use a rank-J conditional covariance#282dfm wants to merge 2 commits into
dfm wants to merge 2 commits into
Conversation
Fixes #280. The quasiseparable conditional covariance was built with three sequential scans inside qsm_mul, and the conditioned GaussianProcess did not inherit the parallel flag, so its Cholesky was sequential too. This adds associative-scan versions of the two qsm_mul recurrences, threads a parallel kwarg through qsm_mul and SquareQSM.gram, and forwards parallel to the conditioned GP when its covariance is a SymmQSM. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTMk4JdA8mBFozHWnxqUqy
Adversarial review of the previous commit found that forwarding parallel=True
to the conditioned GP exposed a numerical weakness: the conditional covariance
built as M - (L^{-1} M)^T (L^{-1} M) has 4J generators that encode the
difference of two nearly equal matrices, and cholesky_parallel loses many
digits (or produces NaN) on that representation.
When conditioning with the GP's own kernel, the conditional covariance is
N - N K^{-1} N, which has rank J and is far better conditioned for both the
sequential and the parallel Cholesky. To use it, kernel=None is now passed
through to Solver.condition/condition_diag as the same-kernel signal, and
both built-in solvers store their kernel (and QuasisepSolver its noise).
Also: QuasisepSolver.covariance respects parallel; qsm_mul no longer
mislabels the product of two different SymmQSMs as symmetric; tests cover
Block transitions, symmetric operands, Banded noise, chained conditioning,
and the conditioned GP's log_probability/sample in both modes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTMk4JdA8mBFozHWnxqUqy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #280.
Summary
parallel=TrueonQuasisepSolverdid not reachcondition. Conditioning at the training coordinates ran four sequential scans: three insideqsm_mul(one forL^{-1} @ M, two for the.gram()), and one in the Cholesky of the conditionedGaussianProcess, which never inherited theparallelflag. The issue's timings on GPU line up with exactly that count.This PR fixes it in two steps.
1. Parallel
qsm_mul, and forward the flagThe two
lax.scanrecurrences inqsm_mulare affine maps with a two-sided linear part,phi -> a @ phi @ b.T + B. Those compose without any Kronecker product by carrying triples(a, b, B):so both scans now have
associative_scanversions, andqsm_mulandSquareQSM.gramtake a staticparallelkeyword.gp.conditionforwardsparallelto the conditioned GP when the parent solver is aQuasisepSolverand the covariance came back as aSymmQSM, andQuasisepSolver.covariancerespects the flag as well.2. A rank-J representation for the same-kernel conditional
An adversarial review of step 1 found a real regression: forwarding
parallelmeant the conditioned GP's covariance was factorized by the existingcholesky_parallel, which lost several digits on it and produced NaN samples and-inflog probabilities in ordinary configurations (e.g. Matern32, N=1000,diag=1e-2, default conditioning jitter).The cause is the representation, not the condition number. The conditional at the training points was built as
M - (L^{-1} M)^T (L^{-1} M), which has 4J generators that encode the difference of two nearly equal matrices. When conditioning with the GP's own kernel, the same matrix isN - N @ K^{-1} @ NwithNthe noise, which has rank J and needs onlysymm_inv(which already has a parallel version). On that representation the parallel Cholesky error drops from NaN to ~1e-8, and even the sequential Cholesky improves by four to six orders of magnitude:To use this,
kernel=Noneis now passed through toSolver.conditionandSolver.condition_diagas the "same kernel" signal (the same convention #272 adopts), both built-in solvers storeself.kernel, andQuasisepSolverstores itsnoise. The cross-kernel path keeps the product form, now with parallel scans.Along the way this also fixes a pre-existing bug where
qsm_mullabelled the product of two differentSymmQSMs as symmetric and dropped the upper triangle.Verification
qsm_mulparity with dense and Block transition matrices, non-symmetric products of symmetric operands, Banded noise conditioning, chained conditioning,predict(return_var=True), and the conditioned GP'slog_probabilityandsamplein both modes under x64 for Matern32, Cosine, and a sum kernel. The Cosine case fails on step 1 alone.jax.make_jaxprshows zeroscanprimitives on aparallel=TrueGP forlog_probability,condition, the conditioned GP'slog_probability,predict(return_var=True), andcovariance.diagfrom 1 to 1e-4, three conditioning jitters): zero NaN factors or samples in every cell, in both modes.conditiondropped from 3128 to 188 bytes per data point (about 1.9 GB instead of 31 GB at N=1e7 in float64).condition0.5 s vs 90 s sequential, before the rank-J change removed the dominant term.Notes
Solversubclasses:condition/condition_diagmay now receivekernel=Noneand must storeself.kernel. Add O(J^2)-per-test-point predictive mean/variance from the QSM Cholesky #272 makes the same change, so the two converge semantically but will conflict textually insolver.pyandgp.condition.kernel=argument, and predictions at newX_test(viaGeneralQSM).cholesky_parallelalso remains fragile on badly represented matrices in general; probably worth its own issue.🤖 Generated with Claude Code
https://claude.ai/code/session_01JTMk4JdA8mBFozHWnxqUqy