Skip to content

bench: scalar functions over dictionary-encoded columns - #24586

Open
radmirnovii wants to merge 1 commit into
apache:mainfrom
radmirnovii:dict-peel-bench
Open

bench: scalar functions over dictionary-encoded columns#24586
radmirnovii wants to merge 1 commit into
apache:mainfrom
radmirnovii:dict-peel-bench

Conversation

@radmirnovii

Copy link
Copy Markdown

Which issue does this PR close?

Refs #20935 and #19458.

Rationale for this change

Scalar functions over dictionary-encoded columns have no benchmark coverage:
neither the hand-written dictionary arms (e.g. reverse's, from #23930) nor
the cast away from the encoding that every function without preservation pays
(e.g. encode). This lands the baseline first, so follow-up work generalizing
the arms shows its delta on stable benchmark ids.

What changes are included in this PR?

One criterion benchmark:

  • reverse over a dictionary of its own per batch (cold) and one shared
    across batches the way a Parquet column chunk delivers them (warm) — today
    the two cost the same, since an arm cannot reuse anything across batches;
  • cast_away: the dictionary cast to its value type and the function called
    once per row — what any function without encoding preservation pays;
  • flat: the same rows with no encoding anywhere, as a bound.

encode cannot be called over a dictionary today, so it has no
dictionary-typed groups yet.

Are these changes tested?

cargo bench -p datafusion-physical-expr --bench scalar_function_dictionary
runs clean.

Are there any user-facing changes?

No.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NysGeXTG5opiJKBApsAe5H

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-expr Changes to the physical-expr crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant