Compare primitive values with RowFn - #9587
Draft
connortsui20 wants to merge 1 commit into
Draft
Conversation
Merging this PR will regress 28 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | mul_u64_nonnull_neon |
15.3 µs | 20.9 µs | -27.14% |
| ❌ | WallTime | lt_i64_nullable_avx512 |
2.9 µs | 3.6 µs | -20.13% |
| ❌ | WallTime | eq_i64_constant_avx512 |
2.1 µs | 2.6 µs | -19.2% |
| ❌ | WallTime | lt_i64_nullable_avx2 |
2.9 µs | 3.6 µs | -18.06% |
| ❌ | Simulation | int_gt[16] |
97.3 µs | 117.3 µs | -17.06% |
| ❌ | WallTime | eq_i64_constant_avx2 |
2.2 µs | 2.7 µs | -16% |
| ❌ | Simulation | int_gt[1024] |
102.8 µs | 122.1 µs | -15.76% |
| ❌ | Simulation | or_chain[16] |
1.1 ms | 1.4 ms | -15.5% |
| ❌ | Simulation | or_chain[1024] |
1.2 ms | 1.4 ms | -14.67% |
| ❌ | Simulation | take[duplicates/repeated/primitive/nonnull/chunks=16/indices=1000] |
201.4 µs | 235.3 µs | -14.42% |
| ❌ | Simulation | float_gt[16] |
239.8 µs | 278.2 µs | -13.83% |
| ❌ | Simulation | float_gt[1024] |
263.6 µs | 305.9 µs | -13.82% |
| ❌ | WallTime | multiply_shapes_neon[(16384, PerRowPerRow)] |
17.4 µs | 20.1 µs | -13.31% |
| ❌ | WallTime | compare_int_nullable_avx512 |
3.6 µs | 4.2 µs | -13.19% |
| ❌ | WallTime | mul_i64_nonnull_neon |
17.4 µs | 20 µs | -13.18% |
| ❌ | WallTime | mul_i8_nonnull_avx2 |
11.3 µs | 13 µs | -13.17% |
| ❌ | Simulation | is_not_null_pred[16] |
195.4 µs | 225 µs | -13.14% |
| ❌ | Simulation | case_when_all_true[1000] |
102.3 µs | 116.2 µs | -11.95% |
| ❌ | Simulation | case_when_all_false[1000] |
100.1 µs | 113.6 µs | -11.83% |
| ❌ | Simulation | is_not_null_pred[1024] |
207.2 µs | 234.7 µs | -11.7% |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ct/primitive-comparison-simd (57d3d96) with develop (3a1838c)3
Footnotes
-
106 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
4 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩
-
No successful run was found on
ct/row-fn-canonical-validity(f17be3d) during the generation of this report, sodevelop(3a1838c) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
from
August 24, 2026 21:34
3e39406 to
931772a
Compare
connortsui20
changed the base branch from
develop
to
ct/compare-bench-cpu-features
August 24, 2026 21:34
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
from
August 24, 2026 21:44
931772a to
c0148bd
Compare
connortsui20
changed the base branch from
ct/compare-bench-cpu-features
to
ct/cpu-features-numeric-benches
August 24, 2026 21:44
connortsui20
changed the title
connortsui20
added a commit
that referenced
this pull request
Aug 24, 2026
## Summary The measurement half of #9587, split out so the numbers land before the kernels do. Bottom of a three-PR stack: this PR, then #9599, then #9587. Tagging these with `#[cpu_features]` first means the walltime legs record the portable lane kernel's throughput on `avx2`, `avx512`, and `neon` metal as a baseline series. The kernel PR then reports against it rather than introducing both the benchmark and the thing it measures in one diff. ## Changes - Adds the primitive comparison cases a hand-written SIMD kernel would have to beat: constant on the left, `u8`, `u64`, and `f32`. - Tags those four with `#[cpu_features]`, so each walltime leg measures them under its own build flags instead of in simulation. The existing cases are untouched and keep their simulation series. - `bench_compare` now carries an `ItemsCount`, so the report reads as throughput rather than a time that only means something next to another run over the same array length. - Adds module docs recording why these four are tagged and the rest are not. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
from
August 24, 2026 23:30
c0148bd to
46ad4bb
Compare
connortsui20
added a commit
that referenced
this pull request
Aug 25, 2026
…9599) ## Summary Sweeps `#[cpu_features]` across the microbenchmarks that clearly earn it: the binary numeric arithmetic and comparison kernels. Those are portable lane loops — the source is identical on every target and the vector width the compiler picks comes from the build flags — which is the case the attribute exists for. Measuring them in simulation under one fixed `+avx2` build hides the only variable that matters. Middle of a three-PR stack: #9598, then this PR, then #9587. It carries no kernel changes of its own — everything here is a benchmark attribute — so it can be reordered or rebased onto `develop` without touching the other two. ## Changes Tagged: - `binary_ops`: the primitive arithmetic cases (`add_*`, `subtract_*`, `multiply_*`, `mul_*`, `div_i64_*`, `sub_i64_constant`, and the three `*_shapes` matrices) and the two primitive comparison cases (`eq_i64_constant`, `lt_i64_nullable`). - `compare`: `compare_int`, `compare_int_nullable`, `compare_int_constant`, `compare_int_eq`, `compare_float`. - `scalar_subtract`. - `lane_kernels`: `lanezip_checked_add_u32` and its `arrow_checked_add_u32` baseline. The baseline is tagged too — comparing the two is only meaningful under the same build flags. Left in simulation, with the reasoning recorded in each file's module docs: - Decimal arithmetic and comparison: `i128` widening and per-lane rescaling, not something a wider vector register decides. - Boolean `and`/`or`: already word-at-a-time over a bitmap. - String and struct comparison: dominated by view chasing and per-field dispatch. - Casts in `lane_kernels`: vectorization-sensitive, but out of scope here. Also left alone: the `between` benchmarks in `vortex-fastlanes` (`new_raw_prim_test_between` is a raw-primitive comparison kernel and does qualify) and the bit-packed comparison matrices. Both are `types =`/`consts =` parameterized, which `#[cpu_features]` has no coverage for yet, and both would fan out to dozens of walltime series. Worth a follow-up rather than a guess in this PR. Note that tagging moves a benchmark out of the sharded simulation job, so these series restart on the walltime legs instead of continuing their simulation history. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
2 times, most recently
from
August 27, 2026 19:42
1163162 to
4edd7e4
Compare
connortsui20
changed the title
connortsui20
changed the base branch from
develop
to
ct/row-fn-bool-output-deferred
August 27, 2026 19:43
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
2 times, most recently
from
August 27, 2026 21:22
72e2bc8 to
ebda2b0
Compare
connortsui20
changed the base branch from
ct/row-fn-bool-output-deferred
to
ct/row-fn-constant-decode
August 27, 2026 21:23
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
from
August 27, 2026 21:32
ebda2b0 to
139c28f
Compare
connortsui20
added
the
action/bench-sql
github-actions
Bot
removed
the
action/bench-sql
Contributor
Polar Signals Profiling ResultsLatest Run
Previous Runs (10)
Powered by Polar Signals Cloud |
Contributor
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (0.990x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.010x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.978x ➖, 2↑ 0↓)
duckdb / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.001x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.037x ➖, 0↑ 2↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.007x ➖, 1↑ 1↓)
duckdb / parquet / ns (0.992x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: Clickbench Sorted on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.123x ❌, 0↑ 7↓)
datafusion / parquet / ns (1.016x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.015x ➖, 1↑ 4↓)
duckdb / parquet / ns (1.002x ➖, 1↑ 1↓)
File Size Changes (100 files changed, +0.1% overall, 59↑ 41↓)
Totals:
|
Contributor
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.003x ➖, 1↑ 2↓)
datafusion / parquet / ns (1.008x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed / ns (1.001x ➖, 2↑ 4↓)
duckdb / parquet / ns (0.997x ➖, 4↑ 6↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.993x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.002x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.994x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.993x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: Statistical and Population Genetics 📖Commits: PR How to read Verdict and Engines
duckdb / vortex-file-compressed / ns (0.965x ➖, 3↑ 1↓)
duckdb / parquet / ns (1.000x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.004x ➖, 0↑ 1↓)
datafusion / parquet / ns (1.002x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (1.008x ➖, 0↑ 2↓)
duckdb / parquet / ns (1.003x ➖, 0↑ 1↓)
No file size changes detected. |
Contributor
Benchmarks: FineWeb S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.961x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.028x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed / ns (0.978x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.198x ➖, 0↑ 2↓)
|
Contributor
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.943x ➖, 1↑ 0↓)
datafusion / parquet / ns (0.930x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed / ns (0.995x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.964x ➖, 0↑ 0↓)
|
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
from
August 28, 2026 18:22
139c28f to
f003d6b
Compare
connortsui20
changed the base branch from
ct/row-fn-constant-decode
to
ct/row-fn-canonical-validity
August 28, 2026 18:24
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
from
August 28, 2026 18:41
f003d6b to
7821b1b
Compare
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
from
August 28, 2026 19:19
7821b1b to
f556fdf
Compare
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
from
August 28, 2026 19:40
f556fdf to
3a9fc30
Compare
connortsui20
added
the
action/bench-sql
github-actions
Bot
removed
the
action/bench-sql
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
from
August 29, 2026 00:37
3a9fc30 to
8bfca37
Compare
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
force-pushed
the
ct/primitive-comparison-simd
branch
from
August 29, 2026 02:00
8bfca37 to
57d3d96
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves primitive comparisons onto
RowFnand removes the handwritten primitive lane-kernel fallback.Changes
PrimitiveComparedispatches every primitive type, comparison operator, and constant orientation through multiversionedRowVisitor::visit_bool. Cross-generated optimized x86-64 IR confirms vectorized row comparisons, but does not establish end-to-end benchmark parity.CodSpeed compares this layer directly against #9694. It does not include the lower-layer gains from direct constant decoding or canonical validity attachment.