diff --git a/opteryx-skene/README.md b/opteryx-skene/README.md new file mode 100644 index 0000000000..0bbda538e9 --- /dev/null +++ b/opteryx-skene/README.md @@ -0,0 +1,126 @@ +# Opteryx + +Opteryx is an in-process SQL query engine. Query **planning** (parse, bind, +optimize) runs in Python; query **execution** is native (Cython/C++). It +queries Parquet directly from storage with no preloading or preprocessing, +which makes it well suited to ad hoc analytics. + +For more information, visit: + +- [Opteryx Documentation](https://docs.opteryx.app/) +- [Opteryx GitHub Repository](https://github.com/mabel-dev/opteryx-core) + +This entry benchmarks Opteryx on **skene**, its native columnar storage +format. It is the native-format counterpart to `Opteryx (Parquet, partitioned)`. + +--- + +## Generating Benchmark Results + +### High-level Steps +1. Set up the environment. +2. Install Python and the required dependencies. +3. Download the benchmark dataset. +4. Convert it to skene (this is the load step). +5. Run the benchmark script. + +### Detailed Instructions + +1. **Start an AWS EC2 instance** + - OS: Ubuntu 24 + - Architecture: 64-bit (x86_64 or AArch64) + - Instance Type: `c6a.4xlarge` + - Root Storage: 500 GB gp2 SSD + - Advanced Details: ensure 'EBS-optimized instance' is **disabled**. + +2. **SSH into the instance** (after status checks complete): + ~~~bash + ssh ubuntu@{ip} + ~~~ + +3. **Update the package list and install Git** + ~~~bash + sudo apt-get update -y + sudo apt-get install git -y + ~~~ + +4. **Clone the ClickBench repository** + ~~~bash + git clone https://github.com/ClickHouse/ClickBench + cd ClickBench/opteryx-skene + ~~~ + +5. **Run the benchmark script** + ~~~bash + sudo ./benchmark.sh + ~~~ + +### Loading + +ClickBench distributes the dataset as Parquet, so this entry converts it to +skene before querying. That conversion is the load step, and `Load time` is its +wall-clock — comparable to any entry that ingests the source data into a native +store, and unlike the Parquet entry, whose load performs no conversion at all. + +`convert.py` performs the conversion using the writer that ships inside the +`opteryx-core` wheel. skene, draken and rugo are all packaged in that single +wheel, so the conversion needs no additional dependency and no source checkout. +Row groups are packed 16 per file at 262144 rows each, matching the engine's own +mirrors: packing is per directory rather than per source file. + +The conversion runs across processes. A worker owns a contiguous range of input +files end to end and writes its own output files, because morsels hold raw +pointers and cannot cross a process boundary. Workers default to three quarters +of the cores; `-j` overrides it. + +That parallelism has a cost in layout: row groups do not +span chunk boundaries, so each worker's last row group — and last file — is +short. The published mirror was built with 12 workers and holds **28 files**, +against 24 for a single-worker build of the same data, and it measures about 7% +slower across the 43 queries for that reason. `-j 1` reproduces the +single-worker layout exactly. Row count is invariant either way: it is verified +per chunk against the source footers and again on the total, and a mismatch +fails the run rather than warning. + +The source Parquet is deleted once the conversion completes, so `data-size` +measures the skene dataset alone. + +### Compression posture + +The mirror is written with the engine's read-first ("performance") posture: +skene decodes substantially faster uncompressed or lz4-compressed than with +per-section zstd, at the cost of more bytes on disk. It is a deliberate choice +for locally attached storage, where the disk is not the bottleneck and decompression +is pure cost. Remotely read data, where bytes dominate, is written differently. + +The ClickBench Parquet corpus is published pre-compressed, so +`Opteryx (Parquet, partitioned)` reads whatever that corpus contains. In +practice the two are close in size — the skene mirror is 15.39 GB against +14.74 GB of Parquet, about 4% larger. + +### Python version + +`opteryx-core` publishes cp314 x86_64 and AArch64 manylinux wheels and declares +no runtime dependencies, so `install` is a single binary-wheel download with no +on-box compilation and no toolchain. + +### Query dialect + +`queries.sql` adapts queries to Opteryx's dialect. The adaptations are syntactic +— they do not change what is computed, the rows returned, or the work the engine has to do: + +- **Q19, Q43** — `EventTime` is stored as an integer epoch, so it is cast + explicitly (`EventTime::TIMESTAMP[s]`) before `extract(minute FROM ...)` and + before truncation. +- **Q43** — `TRUNC(, 'minute')` rather than `DATE_TRUNC('minute', )`. +- **Q29** — the `REGEXP_REPLACE` pattern and replacement use `b''` and `r''` + literals so the backslash reference survives to the regex engine. +- **Q37-Q42** — `EventDate` comparisons cast both sides to `DATE` + (`EventDate::DATE >= '2013-07-01'::DATE`). + +### Hardware coverage + +Results are published for instance types with **32 or fewer vCPUs**. The account +used for these runs is limited to 32 concurrent on-demand vCPUs, so the 192-vCPU +machines in the ClickBench fleet (`c6a.metal`, `c7a.metal-48xl`, +`c8g.metal-48xl`) could not be launched. diff --git a/opteryx-skene/benchmark.sh b/opteryx-skene/benchmark.sh new file mode 100755 index 0000000000..b7ae16ae72 --- /dev/null +++ b/opteryx-skene/benchmark.sh @@ -0,0 +1,9 @@ +#!/bin/bash +export BENCH_DOWNLOAD_SCRIPT="download-hits-parquet-partitioned" +export BENCH_RESTARTABLE=no +# Single-process engine: each query forks a fresh full-machine process with no +# shared scheduler across connections, so the concurrent-QPS test only +# oversubscribes RAM rather than measuring throughput. Skip it by default; +# override BENCH_CONCURRENT_DURATION to re-enable. See issue #946. +export BENCH_CONCURRENT_DURATION="${BENCH_CONCURRENT_DURATION:-0}" +exec ../lib/benchmark-common.sh diff --git a/opteryx-skene/check b/opteryx-skene/check new file mode 100755 index 0000000000..ed305e8015 --- /dev/null +++ b/opteryx-skene/check @@ -0,0 +1,34 @@ +#!/bin/bash +set -e + +# Smoke test that actually executes through the engine and propagates a +# non-zero exit on failure. The old `python -m opteryx "SELECT version()"` +# CLI path is unusable here for two reasons: +# +# 1. opteryx-core's single-run CLI calls the removed opteryx.query(), +# catches the AttributeError, and STILL exits 0 -- a false green that +# would let a broken install pass this check. +# 2. `version()` is not a function in the current engine; the version is +# exposed as the system variable `@@version`. +# +# This matters more than it looks: bench_load() in lib/benchmark-common.sh +# calls ./check unconditionally, AFTER the ~14 GB download and load. A check +# that cannot pass burns the whole setup cost before failing. +# +# Assert the shape of the value rather than a literal, so the check does not +# need editing on every release. +"$HOME/opteryx_venv/bin/python" - <<'PY' +import re + +import opteryx + +session = opteryx.session() +morsels = list(session.execute_to_morsels("SELECT @@version")) +session.close() + +assert morsels, "SELECT @@version returned no morsels" +value = list(morsels[0][0])[0] +if isinstance(value, (bytes, bytearray)): + value = value.decode() +assert re.fullmatch(r"\d+\.\d+\.\d+.*", str(value)), f"unexpected version {value!r}" +PY diff --git a/opteryx-skene/convert.py b/opteryx-skene/convert.py new file mode 100644 index 0000000000..ba6c13aea5 --- /dev/null +++ b/opteryx-skene/convert.py @@ -0,0 +1,265 @@ +#!/usr/bin/env python3 +""" +Convert ClickBench's partitioned parquet `hits` into its skene mirror. + +Self-contained on purpose: skene, draken and rugo all ship inside the +`opteryx-core` wheel, so this needs nothing but the benchmark's own install. +(The engine repo's dev/parquet_to_skene.py is dev-only tooling and is not +available to a ClickBench run. This file is a port of it; keep them in step.) + +Packing: 16 row groups per file at 262144 rows per row group, matching the +engine's own mirrors. Packing is per DIRECTORY, not per file — a row group can +span two source files, so output files do not correspond to input files. + +PARALLELISM: conversion runs across PROCESSES, not threads. Both hot phases +release the GIL (rugo's decode, skene's add_row_group/write_to), but the morsel +construction between them does not, which caps a thread pool at ~2.5x against +~5x for processes. Morsels hold raw pointers and cannot cross a process +boundary, so a worker owns a CONTIGUOUS RANGE OF INPUT FILES end to end and +writes its own output files; only counts come back. + +CONSEQUENCE — the output is NOT byte-identical to a serial build. Row groups do +not span chunk boundaries, so each worker's last row group is short, where +serially only the very last one was. On this dataset that is ~16 short row +groups of ~397 rather than 1. The layout is a function of the worker count; +`-j 1` reproduces the serial layout exactly. ROW COUNT is invariant, is checked +per chunk against the source footers, and a mismatch is a hard failure. + +Usage: convert.py [codec] [-j N] codec: none|lz4|zstd +""" + +import os +import sys +from concurrent.futures import ProcessPoolExecutor + +import skene +from draken.morsels.morsel import Morsel +from rugo.parquet import read_metadata +from rugo.parquet import read_parquet + +ROWS_PER_ROW_GROUP = 262144 +ROW_GROUPS_PER_FILE = 16 + + +class Packer: + def __init__(self, out_dir, stem, codec, zstd_level, first_index=0): + self.out_dir = out_dir + self.stem = stem + self.codec = codec + self.zstd_level = zstd_level + self._writer = None + self._pending = [] + self._pending_rows = 0 + self._row_groups_in_file = 0 + self.files = 0 + self.rows = 0 + self.nbytes = 0 + # Each worker reserves a disjoint output index range, so two workers + # can never write the same filename. + self._first_index = first_index + os.makedirs(out_dir, exist_ok=True) + + def add(self, morsel): + if morsel.num_rows == 0: + return + self._pending.append(morsel) + self._pending_rows += morsel.num_rows + while self._pending_rows >= ROWS_PER_ROW_GROUP: + merged = self._merge() + self._emit(merged.slice(0, ROWS_PER_ROW_GROUP)) + remainder = merged.num_rows - ROWS_PER_ROW_GROUP + if remainder > 0: + self._pending = [merged.slice(ROWS_PER_ROW_GROUP, remainder)] + else: + self._pending = [] + self._pending_rows = remainder + + def _merge(self): + # Morsel has no `concat`; `combine` is the n-way merge. Getting this + # wrong loses rows silently, which a benchmark cannot survive. + return self._pending[0] if len(self._pending) == 1 else Morsel.combine(self._pending) + + def close(self): + # The final row group is SHORT, and so is the final file - a dataset + # does not divide evenly and padding or dropping the tail are both wrong. + if self._pending_rows > 0: + self._emit(self._merge()) + self._pending = [] + self._pending_rows = 0 + self._close_file() + + def _emit(self, row_group): + if self._writer is None: + self._writer = skene.SkeneWriter( + read_acceleration=True, codec=self.codec, zstd_level=self.zstd_level + ) + self._row_groups_in_file = 0 + self._writer.add_row_group(row_group) + self._row_groups_in_file += 1 + self.rows += row_group.num_rows + if self._row_groups_in_file >= ROW_GROUPS_PER_FILE: + self._close_file() + + def _close_file(self): + if self._writer is None: + return + index = self._first_index + self.files + path = os.path.join(self.out_dir, f"{self.stem}-{index:04d}.skene") + # write_to() completes the file in place; finish() would double peak RSS + # on a wide schema for bytes nobody keeps. + self.nbytes += self._writer.write_to(path) + self.files += 1 + self._writer = None + self._row_groups_in_file = 0 + + +def convert_dir(paths, out_dir, stem, codec, zstd_level, first_index=0): + packer = Packer(out_dir, stem, codec, zstd_level, first_index) + for p in paths: + with read_parquet(p) as reader: + for morsel in reader: + packer.add(morsel) + packer.close() + return packer.files, packer.rows, packer.nbytes + + +def _default_workers(): + """Three quarters of the cores, never all of them. + + Saturating the cores measured ~50% SLOWER than the plateau: a worker holds + a whole decoded file plus up to ROW_GROUPS_PER_FILE row groups buffered in + its writer, so the run goes memory- and scheduler-bound. + """ + return max(1, (os.cpu_count() or 1) * 3 // 4) + + +def _output_files_for(rows): + """Exactly how many .skene files a chunk of `rows` rows produces — this is + what reserves each worker's output index range, so it must be exact.""" + if rows == 0: + return 0 + row_groups = -(-rows // ROWS_PER_ROW_GROUP) + return -(-row_groups // ROW_GROUPS_PER_FILE) + + +def _plan_chunks(paths, row_counts, workers): + """Split into <= `workers` CONTIGUOUS chunks of whole files, balanced by ROW + COUNT rather than file count. Returns [(paths, rows)] in input order so + output names stay sequential.""" + total = sum(row_counts) + # Never split into more chunks than there are full output files of rows: + # each chunk starts a new output file, so over-splitting a small table + # shatters it into undersized files with undersized row groups — precisely + # what packing 16 row groups per file exists to avoid. + rows_per_file = ROWS_PER_ROW_GROUP * ROW_GROUPS_PER_FILE + n = min(workers, len(paths), max(1, total // rows_per_file)) + if n <= 1 or total == 0: + return [(list(paths), total)] + + chunks = [] + start = 0 + assigned = 0 + for _ in range(n - 1): + remaining_chunks = n - len(chunks) - 1 + target = (total - assigned) / (remaining_chunks + 1) + rows = 0 + end = start + while end < len(paths): + if len(paths) - (end + 1) < remaining_chunks: + break + rows += row_counts[end] + end += 1 + if rows >= target: + break + if end == start: + break + chunks.append((paths[start:end], rows)) + assigned += rows + start = end + if start < len(paths): + chunks.append((paths[start:], sum(row_counts[start:]))) + return chunks + + +def _convert_chunk(task): + """Process-pool entry point. Top-level and picklable-only arguments because + spawn-start platforms re-import this module in the child.""" + paths, out_dir, stem, codec, zstd_level, first_index, expected_rows = task + files, rows, nbytes = convert_dir(paths, out_dir, stem, codec, zstd_level, first_index) + if rows != expected_rows: + # Catches the failure that destroyed an earlier converter quietly: a bad + # morsel merge dropped 76% of the rows and still produced a plausible, + # fast, completely wrong dataset. + raise RuntimeError( + f"{out_dir}: chunk at index {first_index} wrote {rows:,} rows " + f"but its sources hold {expected_rows:,}" + ) + return files, rows, nbytes + + +def main(): + argv = [a for a in sys.argv[1:]] + workers = _default_workers() + out = [] + i = 0 + while i < len(argv): + if argv[i] in ("-j", "--workers"): + workers = int(argv[i + 1]); i += 2; continue + if argv[i].startswith("--workers="): + workers = int(argv[i].split("=", 1)[1]); i += 1; continue + out.append(argv[i]); i += 1 + if len(out) not in (2, 3): + print(__doc__) + return 1 + src, dst = out[0], out[1] + codec = out[2] if len(out) == 3 else "lz4" + if codec not in ("none", "lz4", "zstd"): + print(f"ERROR: unknown codec {codec!r}") + return 1 + zstd_level = 9 if codec == "zstd" else 0 + + if not os.path.isdir(src): + print(f"ERROR: source not found: {src}") + return 1 + stale = [f for f in os.listdir(dst) if f.endswith(".skene")] if os.path.isdir(dst) else [] + if stale: + print(f"ERROR: {dst} already holds {len(stale)} .skene file(s); rm -rf it first") + return 1 + + paths = sorted(os.path.join(src, f) for f in os.listdir(src) if f.endswith(".parquet")) + if not paths: + print(f"ERROR: no parquet files in {src}") + return 1 + + # Row counts come from the source footers up front, so output index ranges + # can be reserved before any worker starts and no worker has to ask another + # where its files begin. + row_counts = [read_metadata(p).num_rows for p in paths] + expected_total = sum(row_counts) + + tasks = [] + next_index = 0 + for chunk_paths, chunk_rows in _plan_chunks(paths, row_counts, workers): + tasks.append((chunk_paths, dst, "hits", codec, zstd_level, next_index, chunk_rows)) + next_index += _output_files_for(chunk_rows) + + os.makedirs(dst, exist_ok=True) + if len(tasks) == 1: + results = [_convert_chunk(tasks[0])] + else: + with ProcessPoolExecutor(max_workers=len(tasks)) as pool: + results = list(pool.map(_convert_chunk, tasks)) + + files = sum(r[0] for r in results) + rows = sum(r[1] for r in results) + nbytes = sum(r[2] for r in results) + if rows != expected_total: + raise RuntimeError(f"wrote {rows:,} rows, sources hold {expected_total:,}") + if files == 0: + raise RuntimeError("no row groups read - refusing to write an empty table") + print(f"codec={codec} workers={len(tasks)} files={files} rows={rows} bytes={nbytes}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/opteryx-skene/data-size b/opteryx-skene/data-size new file mode 100755 index 0000000000..8e65ea4b35 --- /dev/null +++ b/opteryx-skene/data-size @@ -0,0 +1,4 @@ +#!/bin/bash +set -e + +du -bcs hits | awk '/total$/ { print $1 }' diff --git a/opteryx-skene/install b/opteryx-skene/install new file mode 100755 index 0000000000..21ba004257 --- /dev/null +++ b/opteryx-skene/install @@ -0,0 +1,44 @@ +#!/bin/bash +set -e + +# Opteryx (PyPI package: opteryx-core) on stock CPython 3.14. +# +# NOT the free-threaded build. Opteryx abandoned the free-threaded-Python +# experiment in June 2026; the engine's parallelism target is native (C++) +# threads under a released GIL, so the GIL build is the supported and the +# representative configuration. opteryx-core stopped publishing cp314t wheels +# after 0.9.16 accordingly -- installing onto a 3.14t interpreter would fall +# through to the sdist and try to build Rust/C++ on the box. +# +# opteryx-core publishes cp314 x86_64 manylinux wheels and declares no runtime +# dependencies, so this is a single binary-wheel download -- no on-box +# compilation and no toolchain. x86_64 only, which is fine: the canonical +# c6a.4xlarge is x86_64. + +sudo apt-get update -y +sudo apt-get install -y software-properties-common +sudo add-apt-repository -y ppa:deadsnakes/ppa +sudo apt-get update -y + +# Ubuntu 24.04 (noble) ships 3.12; 3.14 comes from deadsnakes, which carries +# python3.14 for noble. python3.14-venv provides the venv module. +sudo apt-get install -y python3.14 python3.14-venv git wget + +if [ ! -d "$HOME/opteryx_venv" ]; then + python3.14 -m venv "$HOME/opteryx_venv" +fi + +"$HOME/opteryx_venv/bin/python" -m pip install --upgrade pip +# Pulls the cp314 wheel for the latest release. +"$HOME/opteryx_venv/bin/python" -m pip install --upgrade opteryx-core + +# Fail loudly here rather than 43 queries later if pip silently fell back to a +# source build or resolved an interpreter we did not expect. +"$HOME/opteryx_venv/bin/python" - <<'PY' +import sys + +import opteryx + +print(f"python {sys.version.split()[0]} (GIL enabled: {sys._is_gil_enabled()})") +print(f"opteryx-core {opteryx.__version__}") +PY diff --git a/opteryx-skene/load b/opteryx-skene/load new file mode 100755 index 0000000000..f714884db7 --- /dev/null +++ b/opteryx-skene/load @@ -0,0 +1,19 @@ +#!/bin/bash +# ClickBench ships parquet; skene is Opteryx's native format, so the dataset has +# to be converted before it can be queried. That conversion IS the load step, +# and its wall-clock is what `Load time` reports -- the same shape as any entry +# that ingests the source data into a native store. +set -e + +mkdir -p parquet_src +mv hits_*.parquet parquet_src/ 2>/dev/null || true + +# lz4 is the engine's read-first ("performance") posture. See README.md: it is +# deliberately NOT the codec the published parquet corpus uses, so this entry +# and `Opteryx (Parquet, partitioned)` differ in codec as well as in format. +"$HOME/opteryx_venv/bin/python" convert.py parquet_src hits lz4 + +# Drop the source once converted: `data-size` must measure the skene dataset, +# not skene plus a parquet copy, and 500 GB does not need to hold both. +rm -rf parquet_src +sync diff --git a/opteryx-skene/queries.sql b/opteryx-skene/queries.sql new file mode 100644 index 0000000000..906e5cd6ec --- /dev/null +++ b/opteryx-skene/queries.sql @@ -0,0 +1,43 @@ +SELECT COUNT(*) FROM hits; +SELECT COUNT(*) FROM hits WHERE AdvEngineID <> 0; +SELECT SUM(AdvEngineID), COUNT(*), AVG(ResolutionWidth) FROM hits; +SELECT AVG(UserID) FROM hits; +SELECT COUNT(DISTINCT UserID) FROM hits; +SELECT COUNT(DISTINCT SearchPhrase) FROM hits; +SELECT MIN(EventDate), MAX(EventDate) FROM hits; +SELECT AdvEngineID, COUNT(*) FROM hits WHERE AdvEngineID <> 0 GROUP BY AdvEngineID ORDER BY COUNT(*) DESC; +SELECT RegionID, COUNT(DISTINCT UserID) AS u FROM hits GROUP BY RegionID ORDER BY u DESC LIMIT 10; +SELECT RegionID, SUM(AdvEngineID), COUNT(*) AS c, AVG(ResolutionWidth), COUNT(DISTINCT UserID) FROM hits GROUP BY RegionID ORDER BY c DESC LIMIT 10; +SELECT MobilePhoneModel, COUNT(DISTINCT UserID) AS u FROM hits WHERE MobilePhoneModel <> '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10; +SELECT MobilePhone, MobilePhoneModel, COUNT(DISTINCT UserID) AS u FROM hits WHERE MobilePhoneModel <> '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10; +SELECT SearchPhrase, COUNT(*) AS c FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10; +SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; +SELECT SearchEngineID, SearchPhrase, COUNT(*) AS c FROM hits WHERE SearchPhrase <> '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10; +SELECT UserID, COUNT(*) FROM hits GROUP BY UserID ORDER BY COUNT(*) DESC LIMIT 10; +SELECT UserID, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, SearchPhrase ORDER BY COUNT(*) DESC LIMIT 10; +SELECT UserID, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, SearchPhrase LIMIT 10; +SELECT UserID, extract(minute FROM EventTime::TIMESTAMP[s]) AS m, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, extract(minute FROM EventTime::TIMESTAMP[s]), SearchPhrase ORDER BY COUNT(*) DESC LIMIT 10; +SELECT UserID FROM hits WHERE UserID = 435090932899640449; +SELECT COUNT(*) FROM hits WHERE URL LIKE '%google%'; +SELECT SearchPhrase, MIN(URL), COUNT(*) AS c FROM hits WHERE URL LIKE '%google%' AND SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10; +SELECT SearchPhrase, MIN(URL), MIN(Title), COUNT(*) AS c, COUNT(DISTINCT UserID) FROM hits WHERE Title LIKE '%Google%' AND URL NOT LIKE '%.google.%' AND SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10; +SELECT * FROM hits WHERE URL LIKE '%google%' ORDER BY EventTime LIMIT 10; +SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY EventTime LIMIT 10; +SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY SearchPhrase LIMIT 10; +SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY EventTime, SearchPhrase LIMIT 10; +SELECT CounterID, AVG(length(URL)) AS l, COUNT(*) AS c FROM hits WHERE URL <> '' GROUP BY CounterID HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; +SELECT REGEXP_REPLACE(Referer, b'^https?://(?:www\.)?([^/]+)/.*$', r'\1') AS k, AVG(length(Referer)) AS l, COUNT(*) AS c, MIN(Referer) FROM hits WHERE Referer <> '' GROUP BY REGEXP_REPLACE(Referer, b'^https?://(?:www\.)?([^/]+)/.*$', r'\1') HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; +SELECT SUM(ResolutionWidth), SUM(ResolutionWidth + 1), SUM(ResolutionWidth + 2), SUM(ResolutionWidth + 3), SUM(ResolutionWidth + 4), SUM(ResolutionWidth + 5), SUM(ResolutionWidth + 6), SUM(ResolutionWidth + 7), SUM(ResolutionWidth + 8), SUM(ResolutionWidth + 9), SUM(ResolutionWidth + 10), SUM(ResolutionWidth + 11), SUM(ResolutionWidth + 12), SUM(ResolutionWidth + 13), SUM(ResolutionWidth + 14), SUM(ResolutionWidth + 15), SUM(ResolutionWidth + 16), SUM(ResolutionWidth + 17), SUM(ResolutionWidth + 18), SUM(ResolutionWidth + 19), SUM(ResolutionWidth + 20), SUM(ResolutionWidth + 21), SUM(ResolutionWidth + 22), SUM(ResolutionWidth + 23), SUM(ResolutionWidth + 24), SUM(ResolutionWidth + 25), SUM(ResolutionWidth + 26), SUM(ResolutionWidth + 27), SUM(ResolutionWidth + 28), SUM(ResolutionWidth + 29), SUM(ResolutionWidth + 30), SUM(ResolutionWidth + 31), SUM(ResolutionWidth + 32), SUM(ResolutionWidth + 33), SUM(ResolutionWidth + 34), SUM(ResolutionWidth + 35), SUM(ResolutionWidth + 36), SUM(ResolutionWidth + 37), SUM(ResolutionWidth + 38), SUM(ResolutionWidth + 39), SUM(ResolutionWidth + 40), SUM(ResolutionWidth + 41), SUM(ResolutionWidth + 42), SUM(ResolutionWidth + 43), SUM(ResolutionWidth + 44), SUM(ResolutionWidth + 45), SUM(ResolutionWidth + 46), SUM(ResolutionWidth + 47), SUM(ResolutionWidth + 48), SUM(ResolutionWidth + 49), SUM(ResolutionWidth + 50), SUM(ResolutionWidth + 51), SUM(ResolutionWidth + 52), SUM(ResolutionWidth + 53), SUM(ResolutionWidth + 54), SUM(ResolutionWidth + 55), SUM(ResolutionWidth + 56), SUM(ResolutionWidth + 57), SUM(ResolutionWidth + 58), SUM(ResolutionWidth + 59), SUM(ResolutionWidth + 60), SUM(ResolutionWidth + 61), SUM(ResolutionWidth + 62), SUM(ResolutionWidth + 63), SUM(ResolutionWidth + 64), SUM(ResolutionWidth + 65), SUM(ResolutionWidth + 66), SUM(ResolutionWidth + 67), SUM(ResolutionWidth + 68), SUM(ResolutionWidth + 69), SUM(ResolutionWidth + 70), SUM(ResolutionWidth + 71), SUM(ResolutionWidth + 72), SUM(ResolutionWidth + 73), SUM(ResolutionWidth + 74), SUM(ResolutionWidth + 75), SUM(ResolutionWidth + 76), SUM(ResolutionWidth + 77), SUM(ResolutionWidth + 78), SUM(ResolutionWidth + 79), SUM(ResolutionWidth + 80), SUM(ResolutionWidth + 81), SUM(ResolutionWidth + 82), SUM(ResolutionWidth + 83), SUM(ResolutionWidth + 84), SUM(ResolutionWidth + 85), SUM(ResolutionWidth + 86), SUM(ResolutionWidth + 87), SUM(ResolutionWidth + 88), SUM(ResolutionWidth + 89) FROM hits; +SELECT SearchEngineID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits WHERE SearchPhrase <> '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10; +SELECT WatchID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits WHERE SearchPhrase <> '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10; +SELECT WatchID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10; +SELECT URL, COUNT(*) AS c FROM hits GROUP BY URL ORDER BY c DESC LIMIT 10; +SELECT 1, URL, COUNT(*) AS c FROM hits GROUP BY 1, URL ORDER BY c DESC LIMIT 10; +SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, COUNT(*) AS c FROM hits GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10; +SELECT URL, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND DontCountHits = 0 AND IsRefresh = 0 AND URL <> '' GROUP BY URL ORDER BY PageViews DESC LIMIT 10; +SELECT Title, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND DontCountHits = 0 AND IsRefresh = 0 AND Title <> '' GROUP BY Title ORDER BY PageViews DESC LIMIT 10; +SELECT URL, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND IsRefresh = 0 AND IsLink <> 0 AND IsDownload = 0 GROUP BY URL ORDER BY PageViews DESC LIMIT 10 OFFSET 1000; +SELECT TraficSourceID, SearchEngineID, AdvEngineID, CASE WHEN (SearchEngineID = 0 AND AdvEngineID = 0) THEN Referer ELSE '' END AS Src, URL AS Dst, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND IsRefresh = 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, CASE WHEN (SearchEngineID = 0 AND AdvEngineID = 0) THEN Referer ELSE '' END, URL ORDER BY PageViews DESC LIMIT 10 OFFSET 1000; +SELECT URLHash, EventDate, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND IsRefresh = 0 AND TraficSourceID IN (-1, 6) AND RefererHash = 3594120000172545465 GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 10 OFFSET 100; +SELECT WindowClientWidth, WindowClientHeight, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND IsRefresh = 0 AND DontCountHits = 0 AND URLHash = 2868770270353813622 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY PageViews DESC LIMIT 10 OFFSET 10000; +SELECT TRUNC(EventTime::TIMESTAMP[s], 'minute') AS M, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-14'::DATE AND EventDate::DATE <= '2013-07-15'::DATE AND IsRefresh = 0 AND DontCountHits = 0 GROUP BY TRUNC(EventTime::TIMESTAMP[s], 'minute') ORDER BY M LIMIT 10 OFFSET 1000; diff --git a/opteryx-skene/query b/opteryx-skene/query new file mode 100755 index 0000000000..f2ca00e871 --- /dev/null +++ b/opteryx-skene/query @@ -0,0 +1,42 @@ +#!/bin/bash +# Reads a SQL query from stdin, runs it via opteryx-core (Python in-process) +# against the partitioned parquet under ./hits/. +# Stdout: query result as TSV (header + rows). +# Stderr: query runtime in fractional seconds on the last line. +set -e + +query=$(cat) + +"$HOME/opteryx_venv/bin/python" - "$query" <<'PY' +import sys +import timeit + +import opteryx + +query = sys.argv[1] + +# opteryx-core execution surface: a Session that yields native +# morsels. `execute_to_morsels` runs entirely in the native engine; draining +# the generator is the execution. We time the drain (matching the engine's own +# ClickBench runner), holding the morsels so results can be emitted afterwards. +# NB: opteryx.query() / Cursor.arrow() from the 0.x line no longer exist here. +session = opteryx.session() +start = timeit.default_timer() +morsels = list(session.execute_to_morsels(query)) +end = timeit.default_timer() + +cols = None +for morsel in morsels: + if cols is None: + cols = morsel.column_names + print("\t".join( + c.decode() if isinstance(c, (bytes, bytearray)) else str(c) + for c in cols + )) + for i in range(morsel.num_rows): + print("\t".join("" if v is None else str(v) for v in morsel[i])) + +session.close() + +print(f"{end - start:.3f}", file=sys.stderr) +PY diff --git a/opteryx-skene/results/20260818/c6a.4xlarge.json b/opteryx-skene/results/20260818/c6a.4xlarge.json new file mode 100644 index 0000000000..d51a3a9427 --- /dev/null +++ b/opteryx-skene/results/20260818/c6a.4xlarge.json @@ -0,0 +1,60 @@ +{ + "system": "Opteryx", + "date": "2026-08-18", + "machine": "c6a.4xlarge", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["C++","stateless","column-oriented","embedded"], + "load_time": 208, + "data_size": 14949726123, + "concurrent_qps": null, + "concurrent_error_ratio": null, + "result": [ + [0.743, 0.097, 0.098], + [0.915, 0.194, 0.196], + [1.047, 0.231, 0.232], + [1.023, 0.188, 0.188], + [1.432, 0.74, 0.743], + [1.978, 0.468, 0.465], + [0.771, 0.098, 0.098], + [0.942, 0.199, 0.199], + [1.908, 0.933, 0.926], + [2.992, 1.074, 1.071], + [1.691, 0.328, 0.327], + [1.931, 0.351, 0.348], + [1.962, 0.866, 0.866], + [3.382, 1.114, 1.11], + [2.531, 0.974, 0.975], + [1.707, 0.963, 0.976], + [3.876, 2.067, 2.071], + [3.828, 2.041, 2.099], + [7.114, 4.55, 4.548], + [0.967, 0.188, 0.194], + [7.82, 1.317, 1.308], + [9.734, 1.417, 1.429], + [15.509, 2.167, 2.176], + [10.165, 1.493, 1.492], + [3.413, 0.429, 0.426], + [1.918, 0.377, 0.377], + [3.735, 0.454, 0.453], + [7.99, 1.919, 1.953], + [7.426, 3.519, 3.49], + [0.754, 0.243, 0.244], + [4.34, 0.844, 0.849], + [7.162, 1.185, 1.163], + [8.723, 6.55, 6.641], + [9.235, 4.105, 4.107], + [9.29, 4.151, 4.128], + [1.394, 0.822, 0.822], + [0.989, 0.276, 0.276], + [0.951, 0.201, 0.207], + [0.965, 0.224, 0.219], + [1.247, 0.56, 0.56], + [0.92, 0.177, 0.175], + [0.892, 0.168, 0.168], + [0.868, 0.159, 0.164] +] + } + \ No newline at end of file diff --git a/opteryx-skene/start b/opteryx-skene/start new file mode 100755 index 0000000000..06bd986563 --- /dev/null +++ b/opteryx-skene/start @@ -0,0 +1,2 @@ +#!/bin/bash +exit 0 diff --git a/opteryx-skene/stop b/opteryx-skene/stop new file mode 100755 index 0000000000..06bd986563 --- /dev/null +++ b/opteryx-skene/stop @@ -0,0 +1,2 @@ +#!/bin/bash +exit 0 diff --git a/opteryx-skene/template.json b/opteryx-skene/template.json new file mode 100644 index 0000000000..36032e7542 --- /dev/null +++ b/opteryx-skene/template.json @@ -0,0 +1,12 @@ +{ + "system": "Opteryx", + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": [ + "C++", + "stateless", + "column-oriented", + "embedded" + ] +} diff --git a/opteryx/README.md b/opteryx/README.md index 877bee0ed1..2338ac4c10 100644 --- a/opteryx/README.md +++ b/opteryx/README.md @@ -1,38 +1,40 @@ # Opteryx -Opteryx is an in-process SQL query engine written in Python/Cython that leverages Apache Arrow as its in-memory format. Designed for ad hoc queries, Opteryx directly queries data from storage without requiring any preloading or preprocessing. +Opteryx is an in-process SQL query engine. Query **planning** (parse, bind, +optimize) runs in Python; query **execution** is native (Cython/C++). It +queries Parquet directly from storage with no preloading or preprocessing, +which makes it well suited to ad hoc analytics. For more information, visit: -- [Opteryx Documentation](https://opteryx.dev/) -- [Opteryx GitHub Repository](https://github.com/mabel-dev/opteryx) +- [Opteryx Documentation](https://docs.opteryx.app/) +- [Opteryx GitHub Repository](https://github.com/mabel-dev/opteryx-core) -This page provides instructions for benchmarking Opteryx using the split Parquet files provided by ClickBench. +This page benchmarks Opteryx (PyPI package `opteryx-core`) using the split +Parquet files provided by ClickBench. --- ## Generating Benchmark Results -To generate benchmark results, follow these steps: - -### **High-level Steps** +### High-level Steps 1. Set up the environment. 2. Install Python and the required dependencies. 3. Download the benchmark dataset. 4. Run the benchmark script. -### **Detailed Instructions** +### Detailed Instructions 1. **Start an AWS EC2 instance** - OS: Ubuntu 24 - - Architecture: 64-bit + - Architecture: 64-bit (x86_64 or AArch64) - Instance Type: `c6a.4xlarge` - Root Storage: 500 GB gp2 SSD - - Advanced Details: Ensure 'EBS-optimized instance' is **disabled**. + - Advanced Details: ensure 'EBS-optimized instance' is **disabled**. -2. **SSH into the instance** (after the status checks are complete): +2. **SSH into the instance** (after status checks complete): ~~~bash - ssh ec2-user@{ip} + ssh ubuntu@{ip} ~~~ 3. **Update the package list and install Git** @@ -52,6 +54,40 @@ To generate benchmark results, follow these steps: sudo ./benchmark.sh ~~~ +### Python version + +`opteryx-core` publishes cp314 x86_64 and AArch64 manylinux wheels and declares +no runtime dependencies, so `install` is a single binary-wheel download with no +on-box compilation and no toolchain. + +### Query dialect + +`queries.sql` adapts queries to Opteryx's dialect. The adaptations are syntactic +— they do not change what is computed, the rows returned, or the work the engine has to do: + +- **Q19, Q43** — `EventTime` is stored as an integer epoch, so it is cast + explicitly (`EventTime::TIMESTAMP[s]`) before `extract(minute FROM ...)` and + before truncation. +- **Q43** — `TRUNC(, 'minute')` rather than `DATE_TRUNC('minute', )`. +- **Q29** — the `REGEXP_REPLACE` pattern and replacement use `b''` and `r''` + literals so the backslash reference survives to the regex engine. +- **Q37-Q42** — `EventDate` comparisons cast both sides to `DATE` + (`EventDate::DATE >= '2013-07-01'::DATE`). + +### Hardware coverage + +Results are published for instance types with **32 or fewer vCPUs**. The account +used for these runs is limited to 32 concurrent on-demand vCPUs, so the 192-vCPU +machines in the ClickBench fleet (`c6a.metal`, `c7a.metal-48xl`, +`c8g.metal-48xl`) could not be launched. The published set spans 2 to 16 vCPUs +on both x86_64 (`c6a.*`, `t3a.small`) and AArch64 (`c8g.*`), which covers the +small/medium range of the standard fleet on both architectures. + ### Known Issues -- Queries 33 and 34 fail due to Out of Memory (OOM) errors. +- On the memory-constrained instances the heaviest `GROUP BY` queries do not fit + in RAM and spill to swap rather than failing. They complete, but two orders of + magnitude slower — on `c6a.xlarge` (8 GB) three queries account for more than + half the total runtime. The benchmark environment provides the 16 GB swapfile + that ClickBench's `cloud-init` configures for every system; without it these + queries would be `null` instead of slow. diff --git a/opteryx/check b/opteryx/check index 4d4c12fd75..ed305e8015 100755 --- a/opteryx/check +++ b/opteryx/check @@ -1,4 +1,34 @@ #!/bin/bash set -e -"$HOME/opteryx_venv/bin/python" -m opteryx "SELECT version()" >/dev/null +# Smoke test that actually executes through the engine and propagates a +# non-zero exit on failure. The old `python -m opteryx "SELECT version()"` +# CLI path is unusable here for two reasons: +# +# 1. opteryx-core's single-run CLI calls the removed opteryx.query(), +# catches the AttributeError, and STILL exits 0 -- a false green that +# would let a broken install pass this check. +# 2. `version()` is not a function in the current engine; the version is +# exposed as the system variable `@@version`. +# +# This matters more than it looks: bench_load() in lib/benchmark-common.sh +# calls ./check unconditionally, AFTER the ~14 GB download and load. A check +# that cannot pass burns the whole setup cost before failing. +# +# Assert the shape of the value rather than a literal, so the check does not +# need editing on every release. +"$HOME/opteryx_venv/bin/python" - <<'PY' +import re + +import opteryx + +session = opteryx.session() +morsels = list(session.execute_to_morsels("SELECT @@version")) +session.close() + +assert morsels, "SELECT @@version returned no morsels" +value = list(morsels[0][0])[0] +if isinstance(value, (bytes, bytearray)): + value = value.decode() +assert re.fullmatch(r"\d+\.\d+\.\d+.*", str(value)), f"unexpected version {value!r}" +PY diff --git a/opteryx/install b/opteryx/install index f2c4ed9349..21ba004257 100755 --- a/opteryx/install +++ b/opteryx/install @@ -1,20 +1,44 @@ #!/bin/bash set -e +# Opteryx (PyPI package: opteryx-core) on stock CPython 3.14. +# +# NOT the free-threaded build. Opteryx abandoned the free-threaded-Python +# experiment in June 2026; the engine's parallelism target is native (C++) +# threads under a released GIL, so the GIL build is the supported and the +# representative configuration. opteryx-core stopped publishing cp314t wheels +# after 0.9.16 accordingly -- installing onto a 3.14t interpreter would fall +# through to the sdist and try to build Rust/C++ on the box. +# +# opteryx-core publishes cp314 x86_64 manylinux wheels and declares no runtime +# dependencies, so this is a single binary-wheel download -- no on-box +# compilation and no toolchain. x86_64 only, which is fine: the canonical +# c6a.4xlarge is x86_64. + sudo apt-get update -y sudo apt-get install -y software-properties-common sudo add-apt-repository -y ppa:deadsnakes/ppa sudo apt-get update -y -sudo apt-get install -y python3.11 python3.11-venv git wget build-essential python3.11-dev + +# Ubuntu 24.04 (noble) ships 3.12; 3.14 comes from deadsnakes, which carries +# python3.14 for noble. python3.14-venv provides the venv module. +sudo apt-get install -y python3.14 python3.14-venv git wget if [ ! -d "$HOME/opteryx_venv" ]; then - python3.11 -m venv "$HOME/opteryx_venv" + python3.14 -m venv "$HOME/opteryx_venv" fi "$HOME/opteryx_venv/bin/python" -m pip install --upgrade pip -# 0.26.1 only ships x86_64 wheels, so arm64 hosts (c8g.*) fell through -# to sdist where the build failed at "opteryx/third_party/abseil/ -# containers.pyx doesn't match any files". 0.26.8 publishes -# manylinux2014_aarch64 wheels for cp310-cp313, which fixes arm64 -# without changing anything for x86_64. -"$HOME/opteryx_venv/bin/python" -m pip install --upgrade opteryx==0.26.8 +# Pulls the cp314 wheel for the latest release. +"$HOME/opteryx_venv/bin/python" -m pip install --upgrade opteryx-core + +# Fail loudly here rather than 43 queries later if pip silently fell back to a +# source build or resolved an interpreter we did not expect. +"$HOME/opteryx_venv/bin/python" - <<'PY' +import sys + +import opteryx + +print(f"python {sys.version.split()[0]} (GIL enabled: {sys._is_gil_enabled()})") +print(f"opteryx-core {opteryx.__version__}") +PY diff --git a/opteryx/queries.sql b/opteryx/queries.sql index fa5568c632..906e5cd6ec 100644 --- a/opteryx/queries.sql +++ b/opteryx/queries.sql @@ -16,7 +16,7 @@ SELECT SearchEngineID, SearchPhrase, COUNT(*) AS c FROM hits WHERE SearchPhrase SELECT UserID, COUNT(*) FROM hits GROUP BY UserID ORDER BY COUNT(*) DESC LIMIT 10; SELECT UserID, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, SearchPhrase ORDER BY COUNT(*) DESC LIMIT 10; SELECT UserID, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, SearchPhrase LIMIT 10; -SELECT UserID, extract(minute FROM EventTime) AS m, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, extract(minute FROM EventTime), SearchPhrase ORDER BY COUNT(*) DESC LIMIT 10; +SELECT UserID, extract(minute FROM EventTime::TIMESTAMP[s]) AS m, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, extract(minute FROM EventTime::TIMESTAMP[s]), SearchPhrase ORDER BY COUNT(*) DESC LIMIT 10; SELECT UserID FROM hits WHERE UserID = 435090932899640449; SELECT COUNT(*) FROM hits WHERE URL LIKE '%google%'; SELECT SearchPhrase, MIN(URL), COUNT(*) AS c FROM hits WHERE URL LIKE '%google%' AND SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10; @@ -26,7 +26,7 @@ SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY EventTime LIMIT SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY SearchPhrase LIMIT 10; SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY EventTime, SearchPhrase LIMIT 10; SELECT CounterID, AVG(length(URL)) AS l, COUNT(*) AS c FROM hits WHERE URL <> '' GROUP BY CounterID HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; -SELECT REGEXP_REPLACE(Referer, b'^https?://(?:www\.)?([^/]+)/.*$', r'\\1') AS k, AVG(length(Referer)) AS l, COUNT(*) AS c, MIN(Referer) FROM hits WHERE Referer <> '' GROUP BY REGEXP_REPLACE(Referer, b'^https?://(?:www\.)?([^/]+)/.*$', r'\\1') HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; +SELECT REGEXP_REPLACE(Referer, b'^https?://(?:www\.)?([^/]+)/.*$', r'\1') AS k, AVG(length(Referer)) AS l, COUNT(*) AS c, MIN(Referer) FROM hits WHERE Referer <> '' GROUP BY REGEXP_REPLACE(Referer, b'^https?://(?:www\.)?([^/]+)/.*$', r'\1') HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; SELECT SUM(ResolutionWidth), SUM(ResolutionWidth + 1), SUM(ResolutionWidth + 2), SUM(ResolutionWidth + 3), SUM(ResolutionWidth + 4), SUM(ResolutionWidth + 5), SUM(ResolutionWidth + 6), SUM(ResolutionWidth + 7), SUM(ResolutionWidth + 8), SUM(ResolutionWidth + 9), SUM(ResolutionWidth + 10), SUM(ResolutionWidth + 11), SUM(ResolutionWidth + 12), SUM(ResolutionWidth + 13), SUM(ResolutionWidth + 14), SUM(ResolutionWidth + 15), SUM(ResolutionWidth + 16), SUM(ResolutionWidth + 17), SUM(ResolutionWidth + 18), SUM(ResolutionWidth + 19), SUM(ResolutionWidth + 20), SUM(ResolutionWidth + 21), SUM(ResolutionWidth + 22), SUM(ResolutionWidth + 23), SUM(ResolutionWidth + 24), SUM(ResolutionWidth + 25), SUM(ResolutionWidth + 26), SUM(ResolutionWidth + 27), SUM(ResolutionWidth + 28), SUM(ResolutionWidth + 29), SUM(ResolutionWidth + 30), SUM(ResolutionWidth + 31), SUM(ResolutionWidth + 32), SUM(ResolutionWidth + 33), SUM(ResolutionWidth + 34), SUM(ResolutionWidth + 35), SUM(ResolutionWidth + 36), SUM(ResolutionWidth + 37), SUM(ResolutionWidth + 38), SUM(ResolutionWidth + 39), SUM(ResolutionWidth + 40), SUM(ResolutionWidth + 41), SUM(ResolutionWidth + 42), SUM(ResolutionWidth + 43), SUM(ResolutionWidth + 44), SUM(ResolutionWidth + 45), SUM(ResolutionWidth + 46), SUM(ResolutionWidth + 47), SUM(ResolutionWidth + 48), SUM(ResolutionWidth + 49), SUM(ResolutionWidth + 50), SUM(ResolutionWidth + 51), SUM(ResolutionWidth + 52), SUM(ResolutionWidth + 53), SUM(ResolutionWidth + 54), SUM(ResolutionWidth + 55), SUM(ResolutionWidth + 56), SUM(ResolutionWidth + 57), SUM(ResolutionWidth + 58), SUM(ResolutionWidth + 59), SUM(ResolutionWidth + 60), SUM(ResolutionWidth + 61), SUM(ResolutionWidth + 62), SUM(ResolutionWidth + 63), SUM(ResolutionWidth + 64), SUM(ResolutionWidth + 65), SUM(ResolutionWidth + 66), SUM(ResolutionWidth + 67), SUM(ResolutionWidth + 68), SUM(ResolutionWidth + 69), SUM(ResolutionWidth + 70), SUM(ResolutionWidth + 71), SUM(ResolutionWidth + 72), SUM(ResolutionWidth + 73), SUM(ResolutionWidth + 74), SUM(ResolutionWidth + 75), SUM(ResolutionWidth + 76), SUM(ResolutionWidth + 77), SUM(ResolutionWidth + 78), SUM(ResolutionWidth + 79), SUM(ResolutionWidth + 80), SUM(ResolutionWidth + 81), SUM(ResolutionWidth + 82), SUM(ResolutionWidth + 83), SUM(ResolutionWidth + 84), SUM(ResolutionWidth + 85), SUM(ResolutionWidth + 86), SUM(ResolutionWidth + 87), SUM(ResolutionWidth + 88), SUM(ResolutionWidth + 89) FROM hits; SELECT SearchEngineID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits WHERE SearchPhrase <> '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10; SELECT WatchID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits WHERE SearchPhrase <> '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10; @@ -34,10 +34,10 @@ SELECT WatchID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FR SELECT URL, COUNT(*) AS c FROM hits GROUP BY URL ORDER BY c DESC LIMIT 10; SELECT 1, URL, COUNT(*) AS c FROM hits GROUP BY 1, URL ORDER BY c DESC LIMIT 10; SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, COUNT(*) AS c FROM hits GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10; -SELECT URL, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND DontCountHits = 0 AND IsRefresh = 0 AND URL <> '' GROUP BY URL ORDER BY PageViews DESC LIMIT 10; -SELECT Title, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND DontCountHits = 0 AND IsRefresh = 0 AND Title <> '' GROUP BY Title ORDER BY PageViews DESC LIMIT 10; -SELECT URL, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND IsRefresh = 0 AND IsLink <> 0 AND IsDownload = 0 GROUP BY URL ORDER BY PageViews DESC LIMIT 10 OFFSET 1000; -SELECT TraficSourceID, SearchEngineID, AdvEngineID, CASE WHEN (SearchEngineID = 0 AND AdvEngineID = 0) THEN Referer ELSE '' END AS Src, URL AS Dst, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND IsRefresh = 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, CASE WHEN (SearchEngineID = 0 AND AdvEngineID = 0) THEN Referer ELSE '' END, URL ORDER BY PageViews DESC LIMIT 10 OFFSET 1000; -SELECT URLHash, EventDate, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND IsRefresh = 0 AND TraficSourceID IN (-1, 6) AND RefererHash = 3594120000172545465 GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 10 OFFSET 100; -SELECT WindowClientWidth, WindowClientHeight, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND IsRefresh = 0 AND DontCountHits = 0 AND URLHash = 2868770270353813622 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY PageViews DESC LIMIT 10 OFFSET 10000; -SELECT DATE_TRUNC('minute', EventTime) AS M, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-14' AND EventDate <= '2013-07-15' AND IsRefresh = 0 AND DontCountHits = 0 GROUP BY DATE_TRUNC('minute', EventTime) ORDER BY M LIMIT 10 OFFSET 1000; +SELECT URL, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND DontCountHits = 0 AND IsRefresh = 0 AND URL <> '' GROUP BY URL ORDER BY PageViews DESC LIMIT 10; +SELECT Title, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND DontCountHits = 0 AND IsRefresh = 0 AND Title <> '' GROUP BY Title ORDER BY PageViews DESC LIMIT 10; +SELECT URL, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND IsRefresh = 0 AND IsLink <> 0 AND IsDownload = 0 GROUP BY URL ORDER BY PageViews DESC LIMIT 10 OFFSET 1000; +SELECT TraficSourceID, SearchEngineID, AdvEngineID, CASE WHEN (SearchEngineID = 0 AND AdvEngineID = 0) THEN Referer ELSE '' END AS Src, URL AS Dst, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND IsRefresh = 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, CASE WHEN (SearchEngineID = 0 AND AdvEngineID = 0) THEN Referer ELSE '' END, URL ORDER BY PageViews DESC LIMIT 10 OFFSET 1000; +SELECT URLHash, EventDate, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND IsRefresh = 0 AND TraficSourceID IN (-1, 6) AND RefererHash = 3594120000172545465 GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 10 OFFSET 100; +SELECT WindowClientWidth, WindowClientHeight, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-01'::DATE AND EventDate::DATE <= '2013-07-31'::DATE AND IsRefresh = 0 AND DontCountHits = 0 AND URLHash = 2868770270353813622 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY PageViews DESC LIMIT 10 OFFSET 10000; +SELECT TRUNC(EventTime::TIMESTAMP[s], 'minute') AS M, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate::DATE >= '2013-07-14'::DATE AND EventDate::DATE <= '2013-07-15'::DATE AND IsRefresh = 0 AND DontCountHits = 0 GROUP BY TRUNC(EventTime::TIMESTAMP[s], 'minute') ORDER BY M LIMIT 10 OFFSET 1000; diff --git a/opteryx/query b/opteryx/query index 6d37357757..f2ca00e871 100755 --- a/opteryx/query +++ b/opteryx/query @@ -1,7 +1,7 @@ #!/bin/bash -# Reads a SQL query from stdin, runs it via opteryx (Python in-process) +# Reads a SQL query from stdin, runs it via opteryx-core (Python in-process) # against the partitioned parquet under ./hits/. -# Stdout: query result. +# Stdout: query result as TSV (header + rows). # Stderr: query runtime in fractional seconds on the last line. set -e @@ -10,42 +10,33 @@ query=$(cat) "$HOME/opteryx_venv/bin/python" - "$query" <<'PY' import sys import timeit + import opteryx query = sys.argv[1] +# opteryx-core execution surface: a Session that yields native +# morsels. `execute_to_morsels` runs entirely in the native engine; draining +# the generator is the execution. We time the drain (matching the engine's own +# ClickBench runner), holding the morsels so results can be emitted afterwards. +# NB: opteryx.query() / Cursor.arrow() from the 0.x line no longer exist here. +session = opteryx.session() start = timeit.default_timer() -res = opteryx.query(query) -# In opteryx 0.26 `opteryx.query()` returns a Cursor / Relation. Probe -# the common materialisation methods in priority order — different -# versions expose different combinations. -out = None -for method in ("arrow", "to_arrow_table", "fetchall"): - fn = getattr(res, method, None) - if callable(fn): - try: - out = fn() - break - except Exception: - continue +morsels = list(session.execute_to_morsels(query)) end = timeit.default_timer() -if out is None: - # Final fallback: iterate the cursor directly (older API). - for row in res: - print(row) -elif hasattr(out, "to_pylist"): - # pyarrow.Table — print as TSV header + rows. - cols = out.column_names - print("\t".join(cols)) - for row in out.to_pylist(): - print("\t".join("" if row.get(c) is None else str(row.get(c)) - for c in cols)) -elif isinstance(out, (list, tuple)): - for row in out: - print(row) -else: - print(out) +cols = None +for morsel in morsels: + if cols is None: + cols = morsel.column_names + print("\t".join( + c.decode() if isinstance(c, (bytes, bytearray)) else str(c) + for c in cols + )) + for i in range(morsel.num_rows): + print("\t".join("" if v is None else str(v) for v in morsel[i])) + +session.close() print(f"{end - start:.3f}", file=sys.stderr) PY diff --git a/opteryx/results/20260818/c6a.4xlarge.json b/opteryx/results/20260818/c6a.4xlarge.json new file mode 100644 index 0000000000..16dfa165ff --- /dev/null +++ b/opteryx/results/20260818/c6a.4xlarge.json @@ -0,0 +1,60 @@ +{ + "system": "Opteryx (Parquet, partitioned)", + "date": "2026-08-18", + "machine": "c6a.4xlarge", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["C++","stateless","column-oriented","embedded"], + "load_time": 21, + "data_size": 14737666736, + "concurrent_qps": null, + "concurrent_error_ratio": null, + "result": [ + [0.602, 0.11, 0.11], + [0.76, 0.29, 0.287], + [1.041, 0.401, 0.396], + [1.143, 0.346, 0.348], + [1.362, 0.832, 0.833], + [1.679, 0.77, 0.784], + [0.798, 0.109, 0.11], + [1.001, 0.291, 0.288], + [1.857, 1.153, 1.157], + [2.221, 1.495, 1.502], + [1.262, 0.61, 0.603], + [1.435, 0.712, 0.723], + [1.958, 1.168, 1.149], + [3.336, 1.461, 1.485], + [1.871, 1.333, 1.297], + [1.841, 1.146, 1.143], + [4.006, 2.477, 2.458], + [3.731, 2.408, 2.414], + [7.174, 4.968, 4.991], + [0.826, 0.217, 0.219], + [10.747, 3.003, 2.995], + [12.519, 3.406, 3.401], + [23.432, 5.067, 5.078], + [12.454, 3.782, 3.662], + [3.042, 0.948, 0.951], + [1.332, 0.711, 0.691], + [3.28, 0.977, 0.967], + [9.99, 2.743, 2.786], + [9.096, 4.596, 4.614], + [0.795, 0.379, 0.378], + [2.615, 1.418, 1.415], + [6.549, 1.886, 1.899], + [7.972, 6.737, 6.69], + [11.679, 5.618, 5.711], + [11.674, 5.732, 5.701], + [1.408, 0.962, 0.958], + [1.192, 0.43, 0.45], + [0.987, 0.257, 0.257], + [1.172, 0.405, 0.405], + [1.627, 0.801, 0.809], + [0.966, 0.216, 0.217], + [0.933, 0.212, 0.212], + [0.945, 0.223, 0.222] +] + } + \ No newline at end of file diff --git a/opteryx/template.json b/opteryx/template.json index 548d083dd8..159898b59b 100644 --- a/opteryx/template.json +++ b/opteryx/template.json @@ -1,9 +1,10 @@ { - "system": "Opteryx", + "system": "Opteryx (Parquet, partitioned)", "proprietary": "no", "hardware": "cpu", "tuned": "no", "tags": [ + "C++", "stateless", "column-oriented", "embedded"