Skip to content

[metrics] Count per-bucket errors in TabletServer response metrics - #4360

Open
fxbing wants to merge 1 commit into
apache:mainfrom
fxbing:feature/20260914-response-error-metrics
Open

fxbing wants to merge 1 commit into
apache:mainfrom
fxbing:feature/20260914-response-error-metrics

Conversation

@fxbing

@fxbing fxbing commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #4359

Follow up on #4213 to count per-bucket errors returned inside normal RPC responses. These errors currently bypass sendError() and are missing from the per-error metrics.

Brief change log

  • Generate recursive error-code collection for protobuf messages.
  • Count failed response entries in per-error metrics, excluding absent codes and NONE, while preserving the aggregate metric's whole-request failure semantics.
  • Prevent metric failures from triggering another response and avoid counting bucket errors when response encoding fails.

Tests

  • Added regression coverage for mixed-success batches, repeated and unknown errors, nested responses, message reuse, and encoding and metric failures.
  • Common and protogen tests, plus targeted RPC tests, passed.
  • Spotless, Checkstyle, and RAT passed. Full integration tests were not run.

API and Format

No public API, RPC wire-format, or storage-format changes. Per-error metrics now include bucket failures and therefore need not sum to the existing aggregate error metric.

Documentation

Updated the metrics reference to explain the counting semantics.

Generative AI disclosure

  • Yes — OpenAI Codex and Anthropic Claude.

Generate recursive raw error-code collection for protobuf messages.
Count bucket failures without changing aggregate RPC error semantics.
Cover response errors, message reuse, and encoding and metrics failures.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[metrics] Count per-bucket errors in TabletServer response metrics

1 participant