feat: unify native/ASAP experiments and plan SQL workloads jointly - #695
Open
zzylol wants to merge 2 commits into
Open
feat: unify native/ASAP experiments and plan SQL workloads jointly#695zzylol wants to merge 2 commits into
zzylol wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Measure existing ASAP execution against native Prometheus, ClickHouse, and VictoriaMetrics through one reproducible experiment, and expose cross-query SQL alternatives to Planner.
What and how
Before this PR
The shared fixture compared only native databases. SQL publication selected each query independently, so a grouped sum and total sum could not expose a cross-query rollup alternative. Separate replay tools did not provide unified lifecycle accounting.
After this PR
One experiment exercises all three native/ASAP pairs and retains actual plans, routes, values, and costs. Related SQL queries share a selection context; rollup candidates become visible without forcing the cost model to select them.
The live smoke compiles two sum queries, installs one SQL materialization for their workload, loads data in two batches, and queries a later SQL window. A third uninstalled window verifies exact fallback.
Verification
git diff --checkpassed.96504c7dand Planner3be523fa, using an isolated build copy with CI-pinned sketchlib and local Collector. Immutable native images: Prometheussha256:5ce7540c3c00ef4ab0c9d2c995c6a5b9c421f44b4a115d97a2c7af3b1c21cbb0; ClickHousesha256:fa394da808cc53f76d0344429421d6c422a6ee85fe7450135c0e3cff4df9bcbb; VictoriaMetricssha256:26dfc4ab90cf6fedc8115269adc22d300bcd5eab82ea1ebed3c8e22b49886ce1.Limits and follow-up
This tiny debug-build smoke establishes functional integration and did not demonstrate a positive performance benefit. Native containers have individual resource limits; host backend processes do not have equivalent aggregate cgroup limits.
The existing SQL backfill guard rejects a later second job after first-seen metadata appears. The runner preserves that guard and performs one complete finite SQL backfill after loading all batches. SQL continuous summary maintenance and its break-even remain explicitly unavailable. Other maintenance results describe finite input batches, not sustained load. Functional smoke planning costs are declared seeds, not measured calibration or a planner-optimality claim.