Conversation
…bjectDescriptorImpl (googleapis#16309)
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request introduces support for pre-warmed initial read ranges when opening an object in the asynchronous Google Cloud Storage client. It adds InitialReadRanges to AsyncClient::Open, propagates these ranges down to the connection implementation, and manages their caching, deduplication, and pacing (with a default 5 MiB buffer limit) in ObjectDescriptorImpl. Tracing and unit tests are also updated to support this feature. The review feedback focuses on adhering to the repository's style guide regarding type deduction (auto), specifically recommending the use of explicit types instead of auto when domain objects, enums, or protobuf messages are obscured, and using auto const& to avoid unnecessary copies of containers or std::shared_ptr instances. Additionally, it suggests moving a std::shared_ptr in ObjectDescriptorImpl to prevent atomic reference count overhead.
| // If pre-warmed ranges are configured, populate the initial request | ||
| // with these ranges to start downloading them as soon as the stream opens. | ||
| if (current->has<ReadRangesOption>()) { | ||
| auto const& ranges = current->get<ReadRangesOption>(); |
There was a problem hiding this comment.
When a function returns a reference to a container or a large object, use auto const& instead of auto to avoid unnecessary and inefficient copies.
auto const& ranges =
current->get<ReadRangesOption>();References
- When a function returns a reference to a container or a large object (such as std::multimap), use 'auto const&' instead of 'auto' to avoid unnecessary and inefficient copies.
| // with these ranges to start downloading them as soon as the stream opens. | ||
| if (current->has<ReadRangesOption>()) { | ||
| auto const& ranges = current->get<ReadRangesOption>(); | ||
| for (auto const& r : DeduplicateRanges(ranges)) { |
There was a problem hiding this comment.
Avoid using auto when it hides domain objects. Explicitly declare the loop variable type.
for (DedupedReadRange const& r : DeduplicateRanges(ranges)) {References
- Reject auto when it hides domain objects or function return types. (link)
| if (current->has<ReadRangesOption>()) { | ||
| auto const& ranges = current->get<ReadRangesOption>(); | ||
| for (auto const& r : DeduplicateRanges(ranges)) { | ||
| auto* proto_range = initial_request.add_read_ranges(); |
There was a problem hiding this comment.
Avoid using auto when it hides protobuf messages or fields. Use the explicit protobuf type instead.
google::storage::v2::ReadRange* proto_range =
initial_request.add_read_ranges();References
- Reject auto when it hides protobuf messages/fields. (link)
| OpenParams p) override { | ||
| auto span = internal::MakeSpan("storage::AsyncConnection::Open"); | ||
| if (p.options.has<ReadRangesOption>()) { | ||
| auto const& ranges = p.options.get<ReadRangesOption>(); |
There was a problem hiding this comment.
When a function returns a reference to a container or a large object, use auto const& instead of auto to avoid unnecessary and inefficient copies.
auto const& ranges =
p.options.get<ReadRangesOption>();References
- When a function returns a reference to a container or a large object (such as std::multimap), use 'auto const&' instead of 'auto' to avoid unnecessary and inefficient copies.
| // If pre-warmed ranges are specified, initialize their `ReadRange` objects, | ||
| // register them as active on the initial stream, and cache them. | ||
| if (options_.has<ReadRangesOption>()) { | ||
| auto const& ranges = options_.get<ReadRangesOption>(); |
There was a problem hiding this comment.
When a function returns a reference to a container or a large object, use auto const& instead of auto to avoid unnecessary and inefficient copies.
auto const& ranges =
options_.get<ReadRangesOption>();References
- When a function returns a reference to a container or a large object (such as std::multimap), use 'auto const&' instead of 'auto' to avoid unnecessary and inefficient copies.
| it->active_ranges.emplace(dr.read_id, range); | ||
| // Cache them so subsequent `Read()` calls can claim them. | ||
| auto [cache_it, inserted] = prewarmed_ranges_.emplace( | ||
| range_key, PrewarmedRange{range, dr.read_id}); |
There was a problem hiding this comment.
Since range is not used after this point in the loop, we can move it into PrewarmedRange to avoid an unnecessary copy (atomic reference count increment/decrement) of the std::shared_ptr.
range_key, PrewarmedRange{std::move(range), dr.read_id});References
- Scrutinize copies of non-fundamental C++ types. Is it necessary to copy the data? Can we move the data instead? (link)
| auto cache_status = has_initial_read_ranges_ | ||
| ? InitialReadRangesCacheStatus::kMiss | ||
| : InitialReadRangesCacheStatus::kNone; |
There was a problem hiding this comment.
Avoid using auto when it hides domain objects or enums. Use explicit types instead.
InitialReadRangesCacheStatus cache_status =
has_initial_read_ranges_ ? InitialReadRangesCacheStatus::kMiss
: InitialReadRangesCacheStatus::kNone;References
- Reject auto when it hides domain objects or function return types. (link)
| if (cache_it != prewarmed_ranges_.end()) { | ||
| cache_status = InitialReadRangesCacheStatus::kHit; | ||
| // Cache hit. Claim the pre-warmed range and return it to the user. | ||
| auto prewarmed = std::move(cache_it->second); |
There was a problem hiding this comment.
Avoid using auto when it hides domain objects. Use explicit types instead.
PrewarmedRange prewarmed = std::move(cache_it->second);References
- Reject auto when it hides domain objects or function return types. (link)
| return false; | ||
| }; | ||
|
|
||
| for (auto& range_data : *response->mutable_object_data_ranges()) { |
There was a problem hiding this comment.
Avoid using auto when it hides protobuf messages or fields. Use the explicit protobuf type instead.
for (google::storage::v2::ObjectRangeData& range_data :
*response->mutable_object_data_ranges()) {References
- Reject auto when it hides protobuf messages/fields. (link)
| auto const l = copy.find(id); | ||
| if (l == copy.end()) continue; | ||
|
|
||
| auto range = l->second; |
There was a problem hiding this comment.
Copying a std::shared_ptr increments the reference count, which is an atomic operation and has overhead. Since range is not modified or moved, use auto const& to avoid the copy.
auto const& range = l->second;References
- Scrutinize copies of non-fundamental C++ types. Is it necessary to copy the data? Can we move the data instead? (link)
- When a function returns a reference to a container or a large object (such as std::multimap), use 'auto const&' instead of 'auto' to avoid unnecessary and inefficient copies.
cherry-picked #16341 and its dependencies #16275, #16309 and #16323
Test Summary
All affected unit test and sample targets were built and executed via Bazel:
--//google/cloud/storage:enable_grpc_metrics=true (default) and
--//google/cloud/storage:enable_grpc_metrics=false.
Compilation fixes applied
In
google/cloud/storage/internal/async/connection_tracing_test.cc, the test added by feat(storage): add telemetry for pre-warmed ranges in ObjectDescriptorImpl #16323(ConnectionTracing.OpenSuccessWithInitialReadRanges)referencedOTelAttributeandSpanHasAttributes, whose using::google::cloud::testing_util::... declarations had been added on main by an earlier unrelated commit not present in this feature branch.Added using
::google::cloud::testing_util::OTelAttribute; andusing ::google::cloud::testing_util::SpanHasAttributes; togoogle/cloud/storage/internal/async/connection_tracing_test.cc, amended the feat(storage): add telemetry for pre-warmed ranges in ObjectDescriptorImpl #16323 cherry-pick commit and re-applied feat(storage): expose initial read ranges in AsyncClient #16341.