Skip to content

feat(storage): wire operation metrics in gRPC veneer - #14729

Open
nidhiii-27 wants to merge 99 commits into
mainfrom
feat/storage-grpc-veneer-operation-metrics
Open

nidhiii-27 wants to merge 99 commits into
mainfrom
feat/storage-grpc-veneer-operation-metrics

Conversation

@nidhiii-27

Copy link
Copy Markdown
Contributor

Wires operation metrics into the gRPC veneer implementation.

  • Instruments GrpcStorageImpl across unary calls, multi-step ACL operations, and streaming operations.
  • Plumbs metrics context into GapicCopyWriter, GrpcBlobReadChannel, and GrpcBlobWriteChannel.
  • Verifies operation tracking with unit tests.

Stacked on #14698.

…nfiguration

- Add configuration options to StorageOptions, GrpcStorageOptions, and HttpStorageOptions (enableOtelMetrics, enableOtelDebugMetrics, meterProvider, metricInterval).
- Add development gate checking system properties and environment variables in StorageMetricsConfig.
- Implement package-private instrument registry in StorageClientMetrics.
- Update OpenTelemetryBootstrappingUtils to register client views with custom histogram boundaries and create client meter provider.
- Add comprehensive unit tests in StorageOptionsTest, StorageClientMetricsTest, and OpenTelemetryBootstrappingUtilsTest.
Apply spotify fmt-maven-plugin formatting to StorageMetricsConfig, StorageClientMetrics, StorageOptionsTest, and StorageClientMetricsTest.

[Generated-by: AI]
Use simple class name Duration instead of fully qualified java.time.Duration in GrpcStorageOptions and OpenTelemetryBootstrappingUtils.

[Generated-by: AI]
…trappingUtils

Apply spotify fmt-maven-plugin formatting to OpenTelemetryBootstrappingUtils.

[Generated-by: AI]
… metrics

Refactor addClientHistogramView in OpenTelemetryBootstrappingUtils to remove unit selector and custom view description, preserving instrument default descriptions.

Remove deferred Phase 2 connection metrics (dnsLookupDuration, tcpConnectDuration, tlsHandshakeDuration) from StorageClientMetrics, CLIENT_LATENCY_HISTOGRAMS, and StorageClientMetricsTest.

Ensure clean Duration imports.

[Generated-by: AI]
…rProvider

Default api parameter to "grpc" instead of "storage" in createClientMeterProvider, allow specifying "json" / "http" transport protocols for Cloud Monitoring monitored resource storage.googleapis.com/Client, and add unit test coverage in OpenTelemetryBootstrappingUtilsTest.

[Generated-by: AI]
Update createClientMeterProvider to use prefix custom.googleapis.com with default monitored resource mapping.
Add KEY_URL_TEMPLATE attribute key constant in StorageClientMetrics.

[Generated-by: AI]
… options

- In GrpcStorageOptions and HttpStorageOptions:
  - Add null check in getMetricInterval() defaulting to 60 seconds
  - Use getMetricInterval() in equals() and hashCode()
  - Update Javadoc for setMeterProvider to document nullable/default behavior
- In StorageOptions:
  - Update Javadoc for setMeterProvider on Builder
- In StorageMetricsConfig:
  - Make sysPropResolver and envResolver volatile for thread safety
- In StorageOptionsTest:
  - Add unit test verifying deserialization with null metricInterval defaults to 60s and equals/hashCode match
  - Verify explicit client options take precedence over env vars and sysprops

[Generated-by: AI]
Format StorageOptions, GrpcStorageOptions, HttpStorageOptions, and StorageOptionsTest using com.spotify.fmt:fmt-maven-plugin:2.25:format under JDK 17 to resolve CI lint check failure.

[Generated-by: AI]
…iew comments

Clamp metric export interval to at least 60 seconds when meterProvider is null and no custom OpenTelemetry is configured, preventing invalid sub-60s intervals during deserialization when falling back to default Cloud Monitoring exporter.

[Generated-by: AI]
…ig and tests

Deduplicate flag resolution in StorageMetricsConfig by extracting resolveFlag helper, separate multi-behavior tests per go/tott/726, and add input diagnostic messages in looped assertions.

[Generated-by: AI]
@nidhiii-27
nidhiii-27 requested review from a team as code owners October 22, 2026 05:52
…nd bind list pagination retrier

- Let NotFoundException propagate out of retrier.run and catch 404 outside in delete, internalBlobGet, and internalBucketGet, and set error on 404 in deleteAcl/deleteDefaultAcl so NOT_FOUND is recorded instead of OK.
- Run StorageInternal hooks (internalDirectUpload, internalObjectDelete, internalObjectGet) with null metricsContext so PCU sub-operations do not emit top-level operations, and keep internalObjectGet @nonnull.
- Bind TransformingPageDecorator retrier with multi-step ListBuckets context and use createMultiStepContext for page 2+ in ListBucketsWithPartialSuccessPage and ListObjectsWithSyntheticDirectoriesPage.

[Generated-by: AI]
…RPC updates

- Pass UpdateBucket and UpdateObject operation names into internalBucketGet and internalBlobGet when BucketInfo or BlobInfo has no modified fields in GrpcStorageImpl.update(...), while keeping GetBucket and GetObject for get(...) callers.
- Add unit test in GrpcStorageImplTest verifying no-op update(BucketInfo) and update(BlobInfo) record UpdateBucket and UpdateObject operation metrics.

[Generated-by: AI]
- Move self-causation check to the bottom of the while loop in extractRpcStatusCode and extractHttpStatusCode so top-level self-causing exceptions are inspected first
- Add defensive null checks on getStatus() and getCode() for StatusRuntimeException and StatusException in extractRpcStatusCode
- Add unit tests for top-level self-causing exceptions and null status on gRPC status exceptions

[Generated-by: AI]
…act StorageImplMetricsTest

[Generated-by: AI]

@cpriti-os cpriti-os left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Left two minor comments inline.

nidhiii-27 and others added 21 commits November 5, 2026 09:01
…s, writer(URL), createFrom, and copy

[Generated-by: AI]
…n StorageErrorTypeMapper

- Reword class Javadoc to remove HLD acronym.
- Delegate StatusRuntimeException and StatusException directly to mapGrpcStatus.
- Cap cause-chain traversal depth (MAX_CAUSE_DEPTH = 20) across mapGrpcStatus and mapThrowable to guard against multi-node cause cycles (A -> B -> A).

[Generated-by: AI]
…or-mapping' into feat/storage-metrics-context
…/storage-grpc-veneer-operation-metrics

[Generated-by: AI]
…edundant createContext fallbacks

[Generated-by: AI]
…e-grpc-veneer-operation-metrics

[Generated-by: AI]
Base automatically changed from feat/storage-veneer-operation-metrics to main November 16, 2026 11:43

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants