Add error-category telemetry to SshTunnelEvent - #6321
Open
anton-107 wants to merge 1 commit into
Open
Conversation
Collaborator
Integration test reportCommit: 8297e8e
Top 7 slowest tests (at least 2 minutes):
|
Add a coarse, non-PII `error_category` to `SshTunnelEvent`, set at each failure site in the `ssh connect` flow, so we can see why connections fail rather than only that they fail. Also register the telemetry defer before the IDE precondition checks. Those returned before it, so `--ide` failures on a missing `code`/`cursor` command emitted no event at all and were absent from the failure counts. Co-authored-by: Isaac
anton-107
force-pushed
the
deco-28096-error-category-in-sshtunnelevent-7
branch
from
August 20, 2026 11:02
461993e to
8297e8e
Compare
anton-107
marked this pull request as ready for review
August 20, 2026 11:56
Contributor
Approval status: pending
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Add
error_categorytoSshTunnelEvent(libs/telemetry/protos/ssh_tunnel.go) and set it at each failure site in thessh connectflow (experimental/ssh/internal/client/client.go). Categories name the distinct early-return sites —IDE_COMMAND_NOT_ON_PATH,CLUSTER_ACCESS_DENIED,SERVER_START_TIMEOUT,USER_ABORTED, etc. — so no raw error text, cluster name, or path is logged.Two details worth a reviewer's attention:
defernow registers before the IDE precondition checks. It sat after them, so--idefailures on a missingcode/cursorcommand returned early and emitted no event at all. They were not merely uncategorized, they were absent from the failure counts entirely.omitempty, so a success sendsTYPE_UNSPECIFIEDexplicitly rather than collapsing to a null that cannot be told apart from a CLI too old to report the field.The failure outcome is collected in a small
connectOutcomestruct andRunuses a named return, so the deferred logger observes the error the caller sees. A cancelled context maps toUSER_ABORTEDand takes precedence over the category recorded at the failure site, since Ctrl-C surfaces as a cancellation from whichever call happens to observe it first.Why
IDE-mode connections have a 40–55% failure rate, but telemetry only records that a connection failed, so the cause is invisible. The leading hypothesis was that
CheckIDECommandrejects users whose IDE shell command is not on PATH — a permanent per-machine condition, which matches the observed stickiness (a retry after a failed first attempt succeeds only 17–21% of the time).That hypothesis was untestable for a second reason beyond the missing field: those checks ran before the telemetry defer, so they produced no event. Adding the field alone would not have confirmed or refuted it. This also means the sub-5s failure bucket in the original analysis could not have contained the PATH failures.
Deliberately left uncategorized: the malformed
--metadatapaths fall through toUNKNOWN.--metadatais a hidden flag whose value the CLI generates itself inToProxyCommand, so a parse failure is a CLI bug, not a user-environment blocker. Mapping it toSERVER_START_TIMEOUTwould pollute the bucket that tracks unreachable servers.Scope note:
is_successstill carriesomitempty, so failures remain NULL rather thanfalse. That is tracked separately and not touched here. Until it changes, count failures viaerror_category(NOT IN ('TYPE_UNSPECIFIED'), plus anIS NOT NULLguard for rows from CLIs predating this field) rather thanis_success = false.The matching backend schema change has landed, so these values are queryable once this rolls out. Every enum spelling matches the constants added here exactly: the CLI serializes the enum name as a string, so a drift would silently decode to
TYPE_UNSPECIFIEDrather than fail loudly.Tests
Unit tests in
client_internal_test.gocover the category mapping: success reportsTYPE_UNSPECIFIED, an attributed failure keeps its category, an unattributed one falls back toUNKNOWN, a cancellation reportsUSER_ABORTEDand wins over the site category, and a non-zero exit after the tunnel is up is not counted as a connection failure.Verified locally:
./task test-exp-ssh(278 unit + 4 acceptance) and full./task lint(0 issues, all three modules).I also drove
RunwithPATHemptied — the exact condition of the hypothesis above, sinceCheckIDECommandresolves the IDE command withexec.LookPath. The emitted payload is:{"compute_type":"DEDICATED","ide_type":"vscode","client_mode":"IDE", "server_start_time_ms":0,"error_category":"IDE_COMMAND_NOT_ON_PATH"}That confirms an event is emitted at all on this path, and that every category serializes to one of the declared enum names.
Still not verified end to end against real compute — no event has been observed landing in the telemetry table. Worth doing before this is relied on for dashboards.
No changelog fragment: the feature is under
experimental/and this is internal telemetry, matching #4881 and #6058.This PR was written by Claude Code.