Relocate Delta's selection policy into cost-model priors; plumb the model through the builders#8700
Relocate Delta's selection policy into cost-model priors; plumb the model through the builders#8700connortsui20 wants to merge 5 commits into
Conversation
cfa1afe to
a380163
Compare
56f8b12 to
f4617b8
Compare
SizeCost gains a prior table keyed by SchemeId: SchemePrior carries a multiplier applied to the scheme's raw ratio signal (values below 1.0 demand the scheme win by a margin) and a floor the effective ratio must strictly exceed. Priors apply in candidate pricing and, through it, in the SkipThreshold best-case path, so scheme-side and model-side views of a handicapped scheme stay consistent. Schemes without an entry are priced off their raw ratio exactly as before, and SizeCost::default() registers no priors, so behavior is unchanged. Priors are policy, not measurement: this is the home for judgments like the Delta 'tax', which moves here from the scheme in the next commit. SizeCost gives up Copy for the table; it remains Clone + Default. Unit tests pin the prior arithmetic bit-for-bit against the historical scheme-side expression (raw * multiplier, floor comparison included) in both the pricing and threshold paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
DeltaScheme now reports its raw estimated ratio (full_width / delta_bits) and asks the threshold about its raw best case; the 'delta tax' multiplier (0.95) and minimum-win floor (1.25) are registered as Delta's SchemePrior on the btrblocks default cost model, which BtrBlocksCompressorBuilder::build() now attaches. The model applies the identical arithmetic in the identical order (raw * multiplier, floor comparison), so scores and selections are bit-identical: the golden suite — including the unstable variant where Delta wins int_monotone_jitter — shows zero snapshot churn. The ALL_SCHEMES 'prefer above delta' registration comment moves to default_cost_model, where the policy now lives and is documented as policy, not measurement. DeltaScheme::new(min_ratio) is deprecated: the floor belongs on the model (SizeCost::with_scheme_prior). The constructor keeps working — a configured floor still gates via the historical post-penalty comparison — but it can only tighten policy, since the model's prior floor also applies. Note for embedders: assembling CascadingCompressor directly with DeltaScheme now requires attaching a prior-bearing model (as the builder does) to retain the handicap; the scheme alone no longer penalizes itself. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
The builder now carries the cost model and attaches it to the CascadingCompressor at build(), defaulting to the btrblocks default model (SizeCost with Delta's prior) as before, so default behavior is unchanged (zero golden snapshot churn). The vortex_compressor::cost module is re-exported from vortex-btrblocks alongside the existing framework re-exports so builder users can name the model types. A unit test proves the plumbing is live rather than inert: two fixed-ratio marker schemes are selected differently under the default model and under an inverted model injected through the builder. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
CompressorConfig::BtrBlocks holds the whole BtrBlocksCompressorBuilder,
so a cost model configured via with_cost_model now rides through
WriteStrategyBuilder::build() into both the data compressor (which
clones the builder and excludes IntDictScheme) and the stats/values
compressor, with no strategy code changes needed; this change documents
that on with_btrblocks_builder and proves it with an integration test:
on the same highly compressible column, a strategy carrying an inverted
model writes a strictly larger file than the default strategy.
No Python surface is added yet, deviating deliberately from the
NEXT-STEPS strawman ('Python preset stubs'): a Python surface with only
the default model behind it is confusable dead API; it lands with the
first real preset in C1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
DeltaScheme's public docs linked to the private builder module, tripping rustdoc's private_intra_doc_links warning; refer to the default cost model in prose instead. (Belongs with the R4 delta-prior change; kept as a follow-up to avoid rewriting pushed history.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
f4617b8 to
d7a3b3a
Compare
a380163 to
b81955f
Compare
Merging this PR will improve performance by 11.86%
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
Tracking Issue: #8701
Rationale for this change
Delta's selection policy — the 0.95 "delta tax" and the 1.25 minimum-win floor — is an execution-speed judgment expressed as ratio fudge factors inside the scheme. This PR moves that policy onto the cost model as data (a per-scheme prior), and makes the model configurable end to end: through
BtrBlocksCompressorBuilderand, via the file writer's strategy, into both the data and stats compressors (#7697).Stacked on #8699. Behavior-preserving by construction: the model applies the identical arithmetic in the identical order, so scores are bit-identical — zero golden snapshot churn, including the variant where Delta wins corpus entries.
What changes are included in this PR?
SizeCostgains per-scheme priors (SchemePrior { multiplier, min_ratio });DeltaSchemereports raw ratios, and its prior is registered on the btrblocks default cost model, where the policy is documented as policy, not measurement.BtrBlocksCompressorBuilder::with_cost_model; a configured model rides throughWriteStrategyBuilderinto both writer compressors. Liveness tests prove the plumbing drives selection (an inverted model flips the winner, and writes a strictly larger file).What APIs are changed? Are there any user-facing changes?
Additive:
SchemePrior,SizeCost::with_scheme_prior,BtrBlocksCompressorBuilder::with_cost_model, and acostre-export fromvortex-btrblocks. Deprecated:DeltaScheme::new(min_ratio)— the floor belongs on the model; the constructor keeps working and can only tighten policy. Embedders assemblingCascadingCompressordirectly withDeltaSchemeshould attach a prior-bearing model (as the builder does) to retain the handicap. Default compressor output is byte-identical.