Skip to content

feat: use scenario-matched empirical costs in the optimizer - #725

Draft
milindsrivastava1997 wants to merge 18 commits into
mainfrom
feat/profiled-atomic-cost-loader
Draft

feat: use scenario-matched empirical costs in the optimizer#725
milindsrivastava1997 wants to merge 18 commits into
mainfrom
feat/profiled-atomic-cost-loader

Conversation

@milindsrivastava1997

Copy link
Copy Markdown
Contributor

Goal

Turn benchmarked sketch costs into a dataset-conditioned cost-based planning workflow for ASAPQuery, starting with KLL quantile_over_time on the Google task-usage trace.

What has been done

  • Load versioned, workload-profiled atomic cost documents from sketch-bench.
  • Enforce accuracy and atomic CPU latency SLAs in the optimizer, with structured infeasibility errors.
  • Make query cost fanout depend on the cardinality of grouping-label tuples (N_G).
  • Add the Dataset Wrangler: it materializes a trace scenario, inventory, and reproducible manifest.
  • Require an atomic-cost profile selector containing both workload and scenario identity; mismatched benchmark data is rejected.
  • Record and preserve the first provenance-gated Google KLL planning run and its intermediate artifacts.

Current scope is deliberately research-first: CPU-only optimization, no filter selectivity, KLL as the first workload, and E2E retained as selected-plan validation rather than the optimizer sweep mechanism.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant