Follow-up to #23696, per @adriangb's decomposition proposal (PR 2).
RowFilterGenerator::build() currently redoes split_conjunction + FilterCandidateBuilder::build + reassign_expr_columns for every row group. On any multi-row-group scan with a pushdown predicate, that per-RG tree walk and column resolution is wasted work.
Move it to once per file: prebuild the row-filter candidate list when the file is opened, then instantiate a RowFilter per row group cheaply from the prebuilt list.
Proposed shape (extracted from #23696, decoupled from fully_matched):
- add
PrebuiltRowFilterCandidate / prebuild_row_filter_candidates (once per file: tree walk + candidate construction) and row_filter_from_prebuilt (per RG: cheap bind against the fresh array readers);
- delete
RowFilterGenerator;
- reimplement the existing public
build_row_filter on top of the two new functions (its signature must stay — parquet_nested_filter_pushdown / parquet_struct_filter_pushdown benches depend on it).
Parity to preserve: same conjunct order (sort_unstable_by_key(required_bytes) when reorder_predicates), same metric wiring (every predicate shares pushdown_rows_pruned, only the last predicate counts pushdown_rows_matched). One intended behavioural delta: reassign_expr_columns errors surface once at open time instead of being swallowed per-build by log::debug!.
Needs nothing from fully_matched.
Follow-up to #23696, per @adriangb's decomposition proposal (PR 2).
RowFilterGenerator::build()currently redoessplit_conjunction+FilterCandidateBuilder::build+reassign_expr_columnsfor every row group. On any multi-row-group scan with a pushdown predicate, that per-RG tree walk and column resolution is wasted work.Move it to once per file: prebuild the row-filter candidate list when the file is opened, then instantiate a
RowFilterper row group cheaply from the prebuilt list.Proposed shape (extracted from #23696, decoupled from
fully_matched):PrebuiltRowFilterCandidate/prebuild_row_filter_candidates(once per file: tree walk + candidate construction) androw_filter_from_prebuilt(per RG: cheap bind against the fresh array readers);RowFilterGenerator;build_row_filteron top of the two new functions (its signature must stay —parquet_nested_filter_pushdown/parquet_struct_filter_pushdownbenches depend on it).Parity to preserve: same conjunct order (
sort_unstable_by_key(required_bytes)whenreorder_predicates), same metric wiring (every predicate sharespushdown_rows_pruned, only the last predicate countspushdown_rows_matched). One intended behavioural delta:reassign_expr_columnserrors surface once at open time instead of being swallowed per-build bylog::debug!.Needs nothing from
fully_matched.