Calibrate against several LC setups at once (v4.3.0)

[RobbinBouwmeester]

What

Calibration ranks all 6,543 heads of the multitask model and keeps exactly one (_best_correlating_head), so a gradient that sits between trained setups is described by the closest single setup and the other 6,542 columns are thrown away.

MultiHeadRidgeCalibration keeps that same ranking, calibrates the 80 best heads individually with SplineTransformerCalibration, and fits a ridge regression from those calibrated estimates onto the observed retention times. Each head contributes an estimate already in the reference's unit; the ridge decides how much to trust each one.

This is now the default for multitask models (updated after review): calibrate and predict_and_calibrate use MultiHeadRidgeCalibration whenever the model predicts for more than one LC setup, and keep SplineTransformerCalibration for single-task models. The previous behaviour is one argument away:

from deeplc import predict_and_calibrate
from deeplc.calibration import SplineTransformerCalibration

rt = predict_and_calibrate(psms, psm_list_reference=reference,
                           calibration=SplineTransformerCalibration())

Calibration.uses_all_heads (new, False everywhere else) is what tells calibrate/predict_and_calibrate to hand over the whole (n, n_heads) matrix instead of one column.

Result on the eight held-out PRIDE setups (Figure 1b corpus)

Reference and test peptides are disjoint; relative MAE is MAE / observed gradient span. Measured through the public API, not a notebook reimplementation.

setup	reference n	current	multi-head	gain
PXD079655	725	0.0221	0.0184	17 %
PXD079795	300	0.0135	0.0118	12 %
PXD079927	2,000	0.0115	0.0100	13 %
PXD080314	675	0.0884	0.0817	7.5 %
PXD080826	2,000	0.0092	0.0065	29 %
PXD081880	230	0.0092	0.0057	38 %
PXD081924	1,103	0.0107	0.0077	28 %
PXD082486	1,430	0.1929	0.1915	0.7 %

8 of 8 improved, median 0.01248 → 0.01090 (13 %). PXD082486 pools several fractions into one run, so its retention times are not a single gradient and nothing helps there; PXD080314 is the wheat-gluten out-of-domain case.

Cost

Not slower than the current path: median calibration fit 2.3 s → 1.0 s, because _best_correlating_head loops in Python over 6,543 columns with np.corrcoef per column while the new ranking is vectorised. (That suggests a separate small win: vectorising the existing helper.) Prediction is unchanged — the full matrix is computed either way.

How 80 was chosen

A sweep over k = 1, 2, 3, 5, 10, 20, 40, 80, 160, 320, 640, 1280, 2000 on the same corpus: median relative MAE 0.01248 (k=1) → 0.01137 (10) → 0.01090 (80, flat to 320) → 0.01125 (2000). Also compared, and rejected:

• unweighted average instead of ridge: best 0.01198 at k=5, and worse than one head beyond k≈40, since equal weights give a head correlating 0.7 the same say as one at 0.999;
• LASSO: matches the ridge (0.01093) while keeping 8-15 non-zero heads, so the same information, no advantage;
• ranking after calibration (rank by MAE once each head is calibrated): reaches 0.01078, marginally better, but it costs ~2,000 extra spline fits per calibration and the ranking criterion is in-sample. Not worth the cost, so the production Pearson ranking is kept.

The class also never fits more weights than half the reference supports, which matters for the 230-peptidoform case.

Verification

• pytest tests: 141 passed, including 12 new tests in tests/test_multihead_calibration.py (beats a single head when the target mixes two setups, records the best head, guards on unfitted use and mismatched head counts, 1-D input, weight cap, empty input, and the two core integration paths).
• ruff check / ruff format --check: clean.
• Version 4.3.0, CHANGELOG and docs/source/models.rst updated.

🤖 Generated with Claude Code

[RalfG]

Does this explicitly need to be part of the public API? As part of deeplc.calibration, it would be. But I don't think an explicit mention in __all__ is needed. It's not consistent with the other items in __all__.

[RobbinBouwmeester]

Removed: the top-level import and the all entry are gone; the class stays public as deeplc.calibration.MultiHeadRidgeCalibration (commit 8b7e325).

[RalfG]

We should probably compile some instructions for LLMs in how to document code. This initial paragraph can be quite confusing. I expected an description of this class, and instead got one of the previous MT calibration method 😅 Generally, it could also be a bit more concise.

[RalfG]

In general, it's also clear that Claude writes comments, docstrings, and PR messages from the context of the conversation that led to the work.

I had a short chat with Claude about it to come up with some instructions 😅

Documentation should be self-contained, not conversation-dependent.

Comments, docstrings, and PR descriptions must be understandable to a reader with no access to the conversation or reasoning that produced the code. Do not assume the reader knows what was discussed, tried, or rejected beforehand.

Comments and docstrings should describe the current state and behavior of the code: what it does and how it is used. They should not read as a log of the development process. Historical context (why a previous approach was replaced, what motivated a design choice) belongs in commit messages, not in comments or docstrings, unless it is strictly necessary to understand the current implementation, in which case it should be brief.

PR descriptions may include more of this process-oriented information: the reasoning behind the change, alternatives considered, and relevant historical context. This is the appropriate place for a log of the development process, distinct from the code documentation itself.

[RobbinBouwmeester]

Rewritten so the docstring describes only this class and its current behaviour (commit 8b7e325). Your documentation instructions are now in CLAUDE.md at the repo root, so they apply to every future session in this repo.

[RobbinBouwmeester]

Added as CLAUDE.md at the repo root (commit 8b7e325), using your wording: self-contained docs, current-state docstrings, history in commit messages, benchmarks in PR descriptions and the changelog, inline comments only where the code is not self-explanatory. Claude Code picks that file up automatically.

One more change since your review, on Robbin's request: multi-head calibration is now the DEFAULT for multitask models (commit fcef741). Single-task models keep the spline, and passing any Calibration instance overrides it. PR description updated accordingly.

[RalfG]

I'd also advocate for instructing Claude to only use inline comments to describe code when not obvious from the code itself. To me, this line feels quite self-explanatory and does not require three lines of comments to explain itself :P

[RobbinBouwmeester]

Trimmed to one line (commit 8b7e325). The inline-comment rule is also in the new CLAUDE.md.

[RalfG]

Could also be left out? Benchmarking results can go in the PR message, but do not really need to be part of a docstring.

[RobbinBouwmeester]

Removed from the docstring; the numbers stay in the changelog and the PR description (commit 8b7e325).

[RalfG]

Should not be a lazy import; move to head of module.

[RobbinBouwmeester]

Moved to the module head (commit 8b7e325).