Skip to content

Cherry pick initial read ranges in AsyncClient support - #16537

Open
leedsalim wants to merge 4 commits into
googleapis:feat/storage-experimental-metricsfrom
leedsalim:feat/storage-experimental-metrics
Open

leedsalim wants to merge 4 commits into
googleapis:feat/storage-experimental-metricsfrom
leedsalim:feat/storage-experimental-metrics

Conversation

@leedsalim

Copy link
Copy Markdown

cherry-picked #16341 and its dependencies #16275, #16309 and #16323

Test Summary
All affected unit test and sample targets were built and executed via Bazel:
--//google/cloud/storage:enable_grpc_metrics=true (default) and
--//google/cloud/storage:enable_grpc_metrics=false.

Compilation fixes applied

@leedsalim
leedsalim requested review from a team as code owners October 5, 2026 16:38
@google-cla

google-cla Bot commented Oct 5, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for pre-warmed initial read ranges when opening an object in the asynchronous Google Cloud Storage client. It adds InitialReadRanges to AsyncClient::Open, propagates these ranges down to the connection implementation, and manages their caching, deduplication, and pacing (with a default 5 MiB buffer limit) in ObjectDescriptorImpl. Tracing and unit tests are also updated to support this feature. The review feedback focuses on adhering to the repository's style guide regarding type deduction (auto), specifically recommending the use of explicit types instead of auto when domain objects, enums, or protobuf messages are obscured, and using auto const& to avoid unnecessary copies of containers or std::shared_ptr instances. Additionally, it suggests moving a std::shared_ptr in ObjectDescriptorImpl to prevent atomic reference count overhead.

// If pre-warmed ranges are configured, populate the initial request
// with these ranges to start downloading them as soon as the stream opens.
if (current->has<ReadRangesOption>()) {
auto const& ranges = current->get<ReadRangesOption>();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When a function returns a reference to a container or a large object, use auto const& instead of auto to avoid unnecessary and inefficient copies.

    auto const& ranges =
        current->get<ReadRangesOption>();
References
  1. When a function returns a reference to a container or a large object (such as std::multimap), use 'auto const&' instead of 'auto' to avoid unnecessary and inefficient copies.

// with these ranges to start downloading them as soon as the stream opens.
if (current->has<ReadRangesOption>()) {
auto const& ranges = current->get<ReadRangesOption>();
for (auto const& r : DeduplicateRanges(ranges)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Avoid using auto when it hides domain objects. Explicitly declare the loop variable type.

    for (DedupedReadRange const& r : DeduplicateRanges(ranges)) {
References
  1. Reject auto when it hides domain objects or function return types. (link)

if (current->has<ReadRangesOption>()) {
auto const& ranges = current->get<ReadRangesOption>();
for (auto const& r : DeduplicateRanges(ranges)) {
auto* proto_range = initial_request.add_read_ranges();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Avoid using auto when it hides protobuf messages or fields. Use the explicit protobuf type instead.

      google::storage::v2::ReadRange* proto_range =
          initial_request.add_read_ranges();
References
  1. Reject auto when it hides protobuf messages/fields. (link)

OpenParams p) override {
auto span = internal::MakeSpan("storage::AsyncConnection::Open");
if (p.options.has<ReadRangesOption>()) {
auto const& ranges = p.options.get<ReadRangesOption>();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When a function returns a reference to a container or a large object, use auto const& instead of auto to avoid unnecessary and inefficient copies.

      auto const& ranges =
          p.options.get<ReadRangesOption>();
References
  1. When a function returns a reference to a container or a large object (such as std::multimap), use 'auto const&' instead of 'auto' to avoid unnecessary and inefficient copies.

// If pre-warmed ranges are specified, initialize their `ReadRange` objects,
// register them as active on the initial stream, and cache them.
if (options_.has<ReadRangesOption>()) {
auto const& ranges = options_.get<ReadRangesOption>();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When a function returns a reference to a container or a large object, use auto const& instead of auto to avoid unnecessary and inefficient copies.

    auto const& ranges =
        options_.get<ReadRangesOption>();
References
  1. When a function returns a reference to a container or a large object (such as std::multimap), use 'auto const&' instead of 'auto' to avoid unnecessary and inefficient copies.

it->active_ranges.emplace(dr.read_id, range);
// Cache them so subsequent `Read()` calls can claim them.
auto [cache_it, inserted] = prewarmed_ranges_.emplace(
range_key, PrewarmedRange{range, dr.read_id});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since range is not used after this point in the loop, we can move it into PrewarmedRange to avoid an unnecessary copy (atomic reference count increment/decrement) of the std::shared_ptr.

            range_key, PrewarmedRange{std::move(range), dr.read_id});
References
  1. Scrutinize copies of non-fundamental C++ types. Is it necessary to copy the data? Can we move the data instead? (link)

Comment on lines +260 to +262
auto cache_status = has_initial_read_ranges_
? InitialReadRangesCacheStatus::kMiss
: InitialReadRangesCacheStatus::kNone;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Avoid using auto when it hides domain objects or enums. Use explicit types instead.

  InitialReadRangesCacheStatus cache_status =
      has_initial_read_ranges_ ? InitialReadRangesCacheStatus::kMiss
                               : InitialReadRangesCacheStatus::kNone;
References
  1. Reject auto when it hides domain objects or function return types. (link)

if (cache_it != prewarmed_ranges_.end()) {
cache_status = InitialReadRangesCacheStatus::kHit;
// Cache hit. Claim the pre-warmed range and return it to the user.
auto prewarmed = std::move(cache_it->second);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Avoid using auto when it hides domain objects. Use explicit types instead.

    PrewarmedRange prewarmed = std::move(cache_it->second);
References
  1. Reject auto when it hides domain objects or function return types. (link)

return false;
};

for (auto& range_data : *response->mutable_object_data_ranges()) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Avoid using auto when it hides protobuf messages or fields. Use the explicit protobuf type instead.

  for (google::storage::v2::ObjectRangeData& range_data :
       *response->mutable_object_data_ranges()) {
References
  1. Reject auto when it hides protobuf messages/fields. (link)

auto const l = copy.find(id);
if (l == copy.end()) continue;

auto range = l->second;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Copying a std::shared_ptr increments the reference count, which is an atomic operation and has overhead. Since range is not modified or moved, use auto const& to avoid the copy.

    auto const& range = l->second;
References
  1. Scrutinize copies of non-fundamental C++ types. Is it necessary to copy the data? Can we move the data instead? (link)
  2. When a function returns a reference to a container or a large object (such as std::multimap), use 'auto const&' instead of 'auto' to avoid unnecessary and inefficient copies.

This branch was successfully deployed

1 active deployment
false — 6b3b4306 Deployed Oct 5, 2026 by leedsalim via Save PR ref #12277
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants