Skip to content

feat(read): decouple raw warmup and read-ahead caching from prefetch - #513

Open
lucasfang wants to merge 11 commits into
apache:mainfrom
lucasfang:codex/decouple-warmup-prefetch
Open

lucasfang wants to merge 11 commits into
apache:mainfrom
lucasfang:codex/decouple-warmup-prefetch

Conversation

@lucasfang

@lucasfang lucasfang commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Purpose

Linked issue: close #512

With batch prefetch disabled, reads previously bypassed the shared read-ahead cache and RAW warmup. This change lets readers that expose byte ranges, including Parquet and ORC, use caching and RAW warmup on the synchronous path, including when the adaptive strategy disables prefetch.

The existing defaults now take effect together: batch prefetch off, read-ahead cache on, and RAW warmup. NONE disables early fetching while retaining caching during normal reads; RAW fetches bytes through the cache and does nothing when caching is disabled. Warmup never decodes batches. When batch prefetch is enabled, the first actual read starts parallel decoding.

  • Share cache creation, range preparation, error handling and metrics through an internal FileReadAhead context. Detect synchronous read-ahead support from reader capabilities, without a format whitelist or rebuilding the reader.
  • Pass the shared context through CacheInputStream so late materialization can prepare payload ranges before its first payload batch, without the public range callback.
  • Use the existing pre-buffer limit to bound synchronous and adaptive-fallback range buffers, including pending I/O and buffers retained by callbacks across resets. Validate ranges against the file size and preserve deferred warmup errors and cleanup on close.
  • Select Parquet's own pre-buffering from the actual shared-cache state to avoid duplicate buffering. ORC retains its existing range discovery across all stripes.

Tests

  • Parameterized Parquet and ORC reader tests cover prefetch and cache switches, NONE/RAW warmup, adaptive fallback, cache hits and unchanged rows across multiple files.
  • Reader-factory tests cover capability detection independently of format names, preserving plain readers, and propagating cache-open failures without rebuilding readers.
  • Warmup tests use controlled streams to check early byte fetching without decoding, repeated calls, schema reset, empty files, deferred failures, close, sparse bitmaps and file-row mapping.
  • Late-materialization tests verify payload ranges are prepared before the first payload batch, skipped for empty matches or disabled caching, and reported through shared error metrics without double counting.
  • Cache tests cover bounded retention, rewind after eviction, pending I/O and callback-held buffers across resets, oversized reads, failed fetches and invalid initial or later ranges.
  • Parquet tests verify buffering follows the actual cache state. Existing core and integration coverage is updated for RAW/NONE semantics and removal of decoded warmup.

API and Format

Breaking C++ API changes:

  • Remove WarmupLevel::DECODED. Callers must use RAW or NONE; warmup no longer starts decoding.
  • Remove PrefetchFileBatchReader::SetPreBufferRangeCallback and PreBufferRangeCallback. Plugins overriding the setter must remove that override and rebuild against the changed virtual interface. Initial byte ranges remain available through PreBufferRange(); late-materialization preparation uses the internal shared context.

The remaining warmup enum values, configuration defaults and ReadHints fields are retained. ReadHints::read_ahead_cache_enabled describes the active shared cache independently of batch prefetch. Cache and optional I/O metrics are available on synchronous reads as well. No configuration options, storage-format changes or protocol changes are introduced.

Documentation

Update the prefetch/warmup guide, I/O metrics guide and API comments to explain independent cache and warmup behavior, synchronous buffer limits and migration from the removed APIs. Document that RAW overlaps I/O only when the filesystem provides genuinely asynchronous reads, and that early termination can leave warmed bytes unused and must wait for in-flight fetches.

Generative AI tooling

Generated-by: OpenAI Codex (codex-cli 0.166.0)

@lucasfang lucasfang left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review by OpenAI Codex of commit dd993eb. This is an author-side review, not an independent maintainer approval.

No blocking correctness findings found in the final implementation.

Reviewed cache ownership and once-per-generation registration; weak late-range callbacks; thread join/discard ordering on schema reset, Close and destruction; deferred and terminal read errors; adaptive/format fallback; no-executor synchronous reads; Parquet buffer handoff; bitmap/filter and file-row-ID forwarding; preservation of other format plugins' plain read path; and cache/I/O metrics lifetime.

Corrected stale callback-lifetime and default-read comments during this review. These final edits only change comments; the implementation was validated by all four complete common/core/Parquet/ORC executables (4,846 passed, 2 conditional skips), and the final source passed read-only formatting/lint and git diff --check.

Remaining validation limits: no local sanitizer, repository-wide unittest or integration-executable runs. Default synchronous reads now incur the explicitly requested cache/RAW memory and I/O behavior; RAW on synchronous filesystems retains its documented latency limitation.

@SteNicholas SteNicholas left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lucasfang, the focused Parquet and ORC cases passed. I found failures under allocation pressure and with caching disabled, along with memory growth on synchronous scans. PTAL.

BREAKING CHANGE: WarmupLevel::DECODED is removed. Use RAW for byte warmup or NONE to disable it.
@lucasfang lucasfang changed the title feat(read): decouple warmup and read-ahead caching from batch prefetch feat(read)!: decouple raw warmup and read-ahead caching from prefetch Nov 5, 2026

@lucasfang lucasfang left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up self-review by OpenAI Codex of commit 4ddc508. This is an author-side review, not an independent maintainer approval.

The final scope is NONE/RAW warmup only. DECODED and the synchronous wrapper's decode thread, pending Arrow batch and join/discard machinery have been removed. Prefetch's background decode loop is still started by actual batch reads, not Warmup().

No blocking correctness findings found. Checked:

  • Raw byte warmup remains independent of prefetch, including format/adaptive fallback.
  • Warmup never calls the batch decoder; synchronous reads stay on the caller thread.
  • Cache generation/reset, weak late-range callbacks, deferred/terminal errors, row IDs, and early close/destruction remain covered.
  • All callers and docs were migrated away from WarmupLevel::DECODED; the public enum removal is explicitly documented as a source API change.
  • The format guard is now named FormatSupportsReadAhead to describe its byte-cache role.

Validation: all four complete common/core/Parquet/ORC executables passed (4,842 passed, 2 conditional cache-disabled skips). Focused checks passed (139 common tests plus 2 skips, 141 context/system/format-table tests). Build, read-only formatting/lint and git diff --check passed.

Repository-wide unittest, integration executables and sanitizer builds were not run locally. RAW retains its documented synchronous-filesystem limitation; NONE disables early fetching.

Detect synchronous byte-range support from reader capabilities and build once.
Share the internal cache context with LM and ORC instead of range callbacks.
Bound synchronous buffer residency, validate ranges before splitting, and
discover ORC stripe ranges on demand while preserving Parquet buffer handoff.
Cover lifecycle, adaptive fallback, payload warmup and high file positions.

BREAKING CHANGE: remove PrefetchFileBatchReader::SetPreBufferRangeCallback
and PreBufferRangeCallback. Plugins must remove overrides and rebuild.
@SteNicholas SteNicholas changed the title feat(read)!: decouple raw warmup and read-ahead caching from prefetch feat(read): decouple raw warmup and read-ahead caching from prefetch Nov 5, 2026

@SteNicholas SteNicholas left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lucasfang, I have reproduced the three issues below against b755d585. PTAL.

Comment on lines +234 to +237
uint64_t next_row = reader_->GetNextRowToRead();
if (read_ahead_ && (!pre_buffered_row_range_ || next_row < pre_buffered_row_range_->first ||
next_row >= pre_buffered_row_range_->second)) {
PAIMON_RETURN_NOT_OK(read_ahead_->PrepareRanges(this));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preparing the next stripe only in NextBatch() is too late when reading involves a seek. The prefetch loop calls EnsureReaderPosition() first, and ORC's seekToRow() can initialize the target stripe, seek within a row group, and read payload before this code runs. In a real three-stripe ORC probe, after warming stripe 0, seeking into stripe 2 synchronously read all 7 bytes of that stripe's payload; the following NextBatch() then fetched the same 7 bytes again through the cache. Please prepare the target stripe from the requested row before the underlying seek performs I/O. A regression should route the format reads through CacheInputStream and assert physical reads/cache hits around the seek. DiscoverStripeRangesOnDemand currently passes a separate raw CountingStream directly to ORC, so its increasing cache-I/O counter does not demonstrate that decoding consumes the fetched bytes.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed reproduction. You're right about both the seek ordering and the test's inability to prove that decoding consumes the cached bytes.

In 5a70ca5, I removed the ORC per-stripe warmup optimization and its dedicated tests from this PR to keep the scope focused on decoupling RAW warmup and read-ahead caching from batch prefetch. ORC now retains its existing all-stripe PreBufferRange() implementation, with no new range-preparation hooks in NextBatch() or SeekToRow(). This removes the late per-stripe preparation path identified here. The existing cost of discovering all stripe ranges is documented; any future per-stripe optimization should cover seek-triggered I/O and verify physical reads/cache hits as you suggested.

The real-ORC decoupling tests remain, covering synchronous reads, parallel prefetch, adaptive fallback, and late materialization. The five affected test targets passed locally (5,228 passed, 2 conditional skips).

Forward synchronous batch reads through the matching underlying API so sparse
bitmap compaction keeps the format reader's file-row mapping and bounds checks.
Avoid opening a cache stream when LM wraps a reader without byte-range support.
Prepare the target ORC stripe before seekToRow can read payload bytes.

Add sparse-bitmap, LM capability and physical seek I/O regression coverage, and
route on-demand ORC stripe tests through CacheInputStream.

Validation: 5,430 tests passed with 2 conditional skips across seven complete
test executables; changed-file pre-commit checks and git diff --check passed.

@SteNicholas SteNicholas left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lucasfang, thanks for updates. I left only comment for updates. PTAL.

@SteNicholas SteNicholas left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lucasfang, thanks for update. I left only one comment for the remaining issue reproduced below: predicate-driven ORC stripe skipping bypasses range preparation and causes duplicate payload I/O. PTAL.

@SteNicholas SteNicholas left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lucasfang, I reproduced the five issues below at 5a70ca5 using the local Parquet fixture, a range-capable test plugin, and a cache with a pinned buffer. PTAL.

input streams. It is not a whole-query or whole-table I/O total. All counters accumulate for the
``PrefetchIoMetrics`` describes only I/O that passes through the instrumented
file input streams, including synchronous reads with batch prefetch disabled.
It is not a whole-query or whole-table I/O total. All counters accumulate for the

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not fully sure I understand—if we’ve already disabled prefetch, why does PrefetchIoMetrics still show I/O?

cache context. Format readers expose their initial ranges
through ``PreBufferRange()``; the public prefetch reader has no range callback
setter. Plugins overriding the former callback setter must remove that override
and rebuild against the updated interface.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to document this in the code? Otherwise the next person will only see a static piece of documentation and won’t necessarily understand the rationale behind this change.

/// WarmupLevel::NONE when the cache is off (see SetReadAheadCacheEnabled()).
/// WarmupLevel::DECODED does not depend on the cache.
/// RAW also works with EnablePrefetch(false) and never starts batch decoding.
/// @see WarmupLevel

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the places we’re saying warmup can be supported even when prefetch isn’t enabled might be a bit too broad—doesn’t it feel a little forced?

}
mutable uint32_t open_count = 0;
bool fail_cache_open = false;
};

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be too much CountingFileSystem usage in the code right now. Please refactor these into the test utilities/tools.

embedded_bytes.size());
EXPECT_OK_AND_ASSIGN(FileStatus file_status,
file_system->GetFileStatus(path + "bucket-0/" + file_name));
return std::make_shared<DataFileMeta>(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change related to warmup in any way?

resident_ranges_.erase(current);
}
return budget_->resident_bytes.load(std::memory_order_acquire) <= limit - size;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Normalize overlapping ranges before merging

MergeRangesLocked() assumes that new_ranges is internally disjoint, but CoalesceByteRanges() can retain partially overlapping ranges when their combined span exceeds the size limit. For example, with range_size_limit=32 and hole_size_limit=0, registering {{10,10},{30,10}} followed by {{0,25},{5,30}} produces [0,10), [10,20), [20,25), [5,30), [30,40). This breaks the ordering required by the binary searches, and MakeRoomLocked() can subsequently fail its offset assertion during eviction. Please normalize the new ranges into a disjoint union before splitting and merging, and add coverage for overlapping ranges whose combined span exceeds the limit.

});
}

ReadAheadCache::ReadAheadCache(const std::shared_ptr<InputStream>& stream,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Make prefetch publication safe under allocation failure

Not blocking. PreBuffer() publishes entries incrementally but dispatches their I/O only after preparing the entire window. If a later buffer allocation throws std::bad_alloc, DispatchFetches() is never reached, and destroying the pending promises leaves previously published entries with broken promises. The allocation exception escapes Warmup(), while subsequent reads can throw std::future_error instead of returning a Status. The new resident_ranges_.insert() introduces another allocation failure point after publication. Please convert allocation failures to Status::OutOfMemory and either roll back undispatched entries or complete their futures with an error, keeping the resident index consistent.


/// A synchronous file reader with independent byte caching and raw warmup.
class WarmupFileBatchReader : public FileBatchReader {
public:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add reader layout comments for the WarmupFileBatchReader in the SplitRead class header. Now it’s hard to tell how each reader is called.

for (uint32_t i = 0; i < reader_count; i++) {
futures.push_back(RunOrSubmit(
executor.get(),
[&builder, &cache, &streams, i, next_stream, io_metrics,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Wait for all submitted tasks before unwinding Open()

CollectAll() exits immediately if a future.get() throws; it does not drain the remaining futures on that path. These tasks capture builder, cache, and streams by reference, so an allocation failure in one task can unwind Open() and destroy its locals while another task is still accessing them, causing a use-after-free. An exception while submitting a later task has the same problem. Destroying these promise-backed futures does not wait for the tasks to finish. Please ensure exception-safe cleanup waits for all accepted tasks before destroying the objects they reference.

Result<bool> ResolvePreBufferEnabled(const std::map<std::string, std::string>& options,
const std::optional<ReadHints>& hints) {
if (hints.has_value() && hints->prefetch_enabled && hints->read_ahead_cache_enabled) {
if (hints.has_value() && hints->read_ahead_cache_enabled) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shared caching sets pre_buffer=false, but the page-filtered path still calls DispatchPreBuffer() → PreBufferRanges() unconditionally. Bitmap point lookups and late-materialization payload reads taking this path can therefore buffer the same raw bytes in both Paimon’s ReadAheadCache and Parquet’s range cache. Please honor the shared-cache setting here, or at least add an TODO documenting the duplicate buffering and the required follow-up.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Decouple raw warmup and read-ahead caching from batch prefetch

3 participants