Context
A query's WindowImplementationCandidate list (PlanningQuery.window_implementations) is built from several sources and then rewritten in several places. This issue records how that works today and lists open questions, with the goal of simplifying the interface.
Where candidates come from
| # |
Source |
Code |
| S1 |
Snapshot implementation.window_candidates: HashMap<PromQL, Vec<WindowImplementationCandidate>> (provider-priced, never rewritten) |
control_plane/src/physical/compiler.rs:226, :547, :621 |
| S2 |
HTTP PhysicalPlanQueryRequest.window_implementations (compile-and-publish / cost-manifests), passed through as-is |
control_plane/src/main.rs:578, :959 |
| S3 |
derived_window_candidates (one production caller: planning_request), used when S1 has no entry for the query |
control_plane/src/physical/compiler.rs:2435 |
S3 today, for each range-selector window W (or the lookback if the query has none), with evaluation interval E:
E < W && W % E == 0: Sliding, slide = E, two candidates: Pane{E} and FullWindow
- otherwise: Tumbling, slide = W, one candidate:
Pane{W}
- Cost:
derived_window_cost scales the snapshot's single implementation_cost template.
Rewrites and filters
| # |
What |
Code |
| R1 |
After logical selection, for S3 queries only: candidates at derived-maintenance windows are collapsed to one Tumbling Pane{W} (…-derived-cohort) and repriced |
compiler.rs:684-727 |
| R2 |
Hybrid mode drops states with no candidate at their window; each state keeps only candidates at its own window |
compiler.rs:1082, :1188 |
| R3 |
Pane sharing makes shared-pane-<fp>, priced from window_implementations.first(), S3 queries only |
physical/pane_reuse.rs:80 |
Consumption: validate_window_implementations → ControlPlaneCostModel::with_window_implementation_costs → select_lifecycle → Planner plan_summary_maintenance_lifecycles → selected_window_implementation_id.
Questions
- Can S1 be deleted? No production caller sets
implementation.window_candidates, and no checked-in snapshot does either. Its only users are tests: compiler.rs (supplied_window_candidates_are_not_replaced_by_the_derivation, ~:5482), and data_plane/tests/asapquery_compatibility_process_e2e.rs:789 (run_shared_dashboard(multi_pane=true)), which injects a 5s pane with a hand-set cost to force the choice. Deleting it removes window_candidates, the key check at :547, the contains_key branches at :621/:673/:741, and makes synthesized_window_queries always cover every query. What should replace provider-priced pane quotes in those tests? Should the backend keep a way to accept measured per-pane costs, or is the derived formula the intended source of truth?
- Should S2 derive candidates too? The HTTP path is the only entry for
DistributedCollectors (the snapshot path requires BackendLocalRemoteWrite). Its only in-repo caller is data_plane/tests/backend_process_e2e.rs. Should it keep caller-supplied candidates, or share S3?
- Intermediate pane sizes. Proposal: also offer pane
p with E < p < W, p % E == 0, W % p == 0. But WindowMaterializationLayout::validate (crates/asap_types/src/aggregation_config.rs:50) requires the pane to divide both window and slide. With slide = E, a flat Pane{p} with p > E is rejected, and readouts at every E tick could not align with pane ends anyway. Options:
- a)
HierarchicalRollup { base_pane_secs: E, levels_secs: [p] }. The compiler accepts it only as Extension("backend.exact-hierarchical-rollup.v1") (compiler.rs:2561), and the data plane has no implementation today.
- b) Finer panes
p | gcd(E, W), p < E. These never beat Pane{E} for one query, but could enable sharing across queries with different E.
- c) Change the slide to
p. That changes evaluation alignment, so off-tick evaluations fall back to exact.
Which is intended?
- Tumbling fallback when
W % E != 0 or E > W. Example: W = 300s, E = 45s produces Tumbling Pane{300}, slide 300. Evaluations every 45s rarely line up with pane ends, so most reads should fail the serving pane-alignment check and go to exact. Should S3 use Pane{gcd(E, W)} with slide gcd(E, W) instead?
- R1 / R3 only apply to S3 queries. If S1 goes away, is the "synthesized" distinction still needed at all?
Context
A query's
WindowImplementationCandidatelist (PlanningQuery.window_implementations) is built from several sources and then rewritten in several places. This issue records how that works today and lists open questions, with the goal of simplifying the interface.Where candidates come from
implementation.window_candidates: HashMap<PromQL, Vec<WindowImplementationCandidate>>(provider-priced, never rewritten)control_plane/src/physical/compiler.rs:226,:547,:621PhysicalPlanQueryRequest.window_implementations(compile-and-publish / cost-manifests), passed through as-iscontrol_plane/src/main.rs:578,:959derived_window_candidates(one production caller:planning_request), used when S1 has no entry for the querycontrol_plane/src/physical/compiler.rs:2435S3 today, for each range-selector window W (or the lookback if the query has none), with evaluation interval E:
E < W && W % E == 0: Sliding, slide = E, two candidates:Pane{E}andFullWindowPane{W}derived_window_costscales the snapshot's singleimplementation_costtemplate.Rewrites and filters
Tumbling Pane{W}(…-derived-cohort) and repricedcompiler.rs:684-727compiler.rs:1082,:1188shared-pane-<fp>, priced fromwindow_implementations.first(), S3 queries onlyphysical/pane_reuse.rs:80Consumption:
validate_window_implementations→ControlPlaneCostModel::with_window_implementation_costs→select_lifecycle→ Plannerplan_summary_maintenance_lifecycles→selected_window_implementation_id.Questions
implementation.window_candidates, and no checked-in snapshot does either. Its only users are tests:compiler.rs(supplied_window_candidates_are_not_replaced_by_the_derivation, ~:5482), anddata_plane/tests/asapquery_compatibility_process_e2e.rs:789(run_shared_dashboard(multi_pane=true)), which injects a 5s pane with a hand-set cost to force the choice. Deleting it removeswindow_candidates, the key check at:547, thecontains_keybranches at:621/:673/:741, and makessynthesized_window_queriesalways cover every query. What should replace provider-priced pane quotes in those tests? Should the backend keep a way to accept measured per-pane costs, or is the derived formula the intended source of truth?DistributedCollectors(the snapshot path requiresBackendLocalRemoteWrite). Its only in-repo caller isdata_plane/tests/backend_process_e2e.rs. Should it keep caller-supplied candidates, or share S3?pwithE < p < W,p % E == 0,W % p == 0. ButWindowMaterializationLayout::validate(crates/asap_types/src/aggregation_config.rs:50) requires the pane to divide both window and slide. With slide = E, a flatPane{p}withp > Eis rejected, and readouts at every E tick could not align with pane ends anyway. Options:HierarchicalRollup { base_pane_secs: E, levels_secs: [p] }. The compiler accepts it only asExtension("backend.exact-hierarchical-rollup.v1")(compiler.rs:2561), and the data plane has no implementation today.p | gcd(E, W),p < E. These never beatPane{E}for one query, but could enable sharing across queries with different E.p. That changes evaluation alignment, so off-tick evaluations fall back to exact.Which is intended?
W % E != 0orE > W. Example: W = 300s, E = 45s producesTumbling Pane{300}, slide 300. Evaluations every 45s rarely line up with pane ends, so most reads should fail the serving pane-alignment check and go to exact. Should S3 usePane{gcd(E, W)}with slidegcd(E, W)instead?