Repository navigation
feat(read): decouple raw warmup and read-ahead caching from prefetch #513
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
dd993eb
4ddc508
a35907d
b755d58
99621f0
7e32d5f
7c83c08
3d675f9
e707b17
5a70ca5
b8fb19f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -83,33 +83,37 @@ the default ``loser-tree`` sort engine additionally warms the first file of | |
| every sorted run before merging. It applies to append-table reads and append | ||
| compaction as well, but does not cross split boundaries. | ||
|
|
||
| How far the next file is prepared is set with | ||
| Whether the next file's raw bytes are fetched early is set with | ||
| ``ReadContextBuilder::SetWarmupLevel()``. Append compaction builds its own read | ||
| context and always uses the default level. | ||
|
|
||
| - ``WarmupLevel::NONE``: do not warm up. | ||
| - ``WarmupLevel::RAW`` (default): fetch the next file's raw bytes through the | ||
| read-ahead cache without decoding them. It has no effect when the read-ahead | ||
| cache is disabled with ``ReadContextBuilder::SetReadAheadCacheEnabled()``. | ||
| - ``WarmupLevel::DECODED``: also start decoding the next file, which hides the | ||
| most latency but uses the most memory. | ||
|
|
||
| Warmup goes through the prefetch reader, so it only applies to Parquet and ORC | ||
| files that are actually read with prefetching, and enabling prefetch alone does | ||
| not guarantee that. With the adaptive strategy, a file whose first read range | ||
| holds more batches than one prefetch queue can buffer is read without | ||
| prefetching, and so without warmup; the ``prefetch.adaptive-disabled-count`` | ||
| metric counts how often this happens. Append compaction uses a prefetch queue | ||
| of three batches, so it only warms files whose first read range spans no more | ||
| than three full batches, such as files with small row groups. | ||
|
|
||
| Warmup and the read-ahead cache work independently of batch prefetch for readers | ||
| implementing the ``PrefetchFileBatchReader`` byte-range interface, including | ||
| Parquet and ORC. Support is determined by the reader's capabilities rather than | ||
| the format name; other readers keep their plain read path. With | ||
| ``EnablePrefetch(false)``, ``RAW`` still warms the cache | ||
| and batches are read synchronously. The same behavior applies when the format | ||
| or the adaptive strategy decides against prefetching. Warmup only fetches raw | ||
| bytes; it never decodes batches. When batch prefetch is active, the first | ||
| actual read starts the parallel decode loop. | ||
|
|
||
| The defaults are batch prefetch off, read-ahead cache on, and ``RAW`` warmup. | ||
| ``WarmupLevel::NONE`` disables early warmup while retaining byte caching during | ||
| normal reads. Disabling the cache makes ``RAW`` a no-op. When the shared cache | ||
| is active, Parquet disables its own pre-buffering to avoid buffering the same | ||
| byte ranges twice. | ||
|
|
||
| ``WarmupLevel::RAW`` issues its fetches from the reading thread through the | ||
| file system's asynchronous read, so it only overlaps I/O when that read is | ||
| really asynchronous. The local file system serves it synchronously, so there | ||
| ``RAW`` warmup reads the next file on the reading thread before the current | ||
| batch is returned, adding latency instead of hiding it. On such file systems, | ||
| prefer ``WarmupLevel::DECODED``, which fetches on the background decode thread, | ||
| or ``WarmupLevel::NONE``. | ||
| use ``WarmupLevel::NONE`` to disable early fetching. | ||
|
|
||
| Warmup preserves the returned rows and their order, but it costs memory: under | ||
| ``WarmupLevel::RAW`` the next file may hold up to the read-ahead cache's | ||
|
|
@@ -118,3 +122,19 @@ on top of the file being read. It may also fetch data from a file that the | |
| reader never consumes: a read that stops early, for example because of a LIMIT, | ||
| may already have warmed the next file, and closing it waits for that file's | ||
| in-flight fetches. | ||
|
|
||
| For synchronous reads, including adaptive fallback, the pre-buffer limit also | ||
| bounds retained range buffers. Completed buffers outside the current window can | ||
| be evicted; later reads can fetch them again. In-flight I/O and buffers retained | ||
| by callbacks count against that budget, so a full budget can temporarily cause a | ||
| read to bypass range caching. The block cache has its separate limit. Parallel | ||
| prefetch retains its existing range-buffer retention policy. | ||
|
|
||
| ORC retains its existing byte-range discovery across all stripes. Preparing | ||
| these ranges can read every stripe footer before the first batch is returned. | ||
|
|
||
| Late-materialization payloads prepare their ranges through the shared internal | ||
| cache context. Format readers expose their initial ranges | ||
| through ``PreBufferRange()``; the public prefetch reader has no range callback | ||
| setter. Plugins overriding the former callback setter must remove that override | ||
| and rebuild against the updated interface. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need to document this in the code? Otherwise the next person will only see a static piece of documentation and won’t necessarily understand the rationale behind this change. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -435,26 +435,27 @@ class PAIMON_EXPORT ReadContextBuilder { | |
| /// | ||
| /// A read-ahead cache is used to prebuffer data ranges before they are needed, | ||
| /// which can improve read performance by reducing redundant I/O operations. | ||
| /// It also works without batch prefetch for readers exposing byte ranges through | ||
| /// PrefetchFileBatchReader, including Parquet and ORC. | ||
| /// @param enabled Whether to enable the read-ahead cache (default: true) | ||
| /// @return Reference to this builder for method chaining. | ||
| ReadContextBuilder& SetReadAheadCacheEnabled(bool enabled); | ||
|
|
||
| /// Set the cache configuration for prefetch read operations. | ||
| /// Set the read-ahead cache configuration for synchronous and prefetch reads. | ||
| /// | ||
| /// @param config The cache configuration to use. | ||
| /// @return Reference to this builder for method chaining. | ||
| ReadContextBuilder& WithCacheConfig(const CacheConfig& config); | ||
|
|
||
| /// Set how far the reader prepares the next file before it is read. | ||
| /// Set whether the reader fetches the next file's raw bytes before it is read. | ||
| /// | ||
| /// Warmup overlaps remote-storage latency with the read of the current file. Higher levels hide | ||
| /// more latency but use more memory, and may warm files that a query never reads (for example | ||
| /// when a LIMIT stops the scan early). | ||
| /// Warmup can overlap remote-storage latency with reading the current file. It may fetch bytes | ||
| /// for files that a query never reads, for example when a LIMIT stops the scan early. | ||
| /// @param level The warmup level to use (default: WarmupLevel::RAW). | ||
| /// @return Reference to this builder for method chaining. | ||
| /// @note WarmupLevel::RAW warms the read-ahead cache, so it has no effect and behaves like | ||
| /// WarmupLevel::NONE when the cache is off (see SetReadAheadCacheEnabled()). | ||
| /// WarmupLevel::DECODED does not depend on the cache. | ||
| /// RAW also works with EnablePrefetch(false) and never starts batch decoding. | ||
| /// @see WarmupLevel | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I feel like the places we’re saying |
||
| ReadContextBuilder& SetWarmupLevel(WarmupLevel level); | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not fully sure I understand—if we’ve already disabled prefetch, why does
PrefetchIoMetricsstill show I/O?