Skip to content

Add staged connection diagnosis for opaque dial failures - #1114

Open
rossnelson wants to merge 12 commits into
mainfrom
connection-error-diagnosis
Open

Add staged connection diagnosis for opaque dial failures#1114
rossnelson wants to merge 12 commits into
mainfrom
connection-error-diagnosis

Conversation

@rossnelson

@rossnelson rossnelson commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Related issues

Related to #224 and #851.

What changed?

This PR adds bounded, evidence-based diagnosis after a failed connection, turning opaque dial errors into supported next steps when the evidence allows.

Before this change, failed connections could end as opaque dial errors. After the real failure, the CLI runs bounded DNS, TCP, and TLS checks in sequence. It gives advice only when the evidence supports it. Diagnosis stops after three seconds or cancellation. TEMPORAL_CLI_DISABLE_CONNECT_DIAGNOSIS disables it. A generic TLS handshake failure does not prove mTLS, so the CLI gives no certificate advice.

This PR adds one connection-specific check to the existing CommandOptions.Fail path. It leaves all other commands, generated command flow, Activity errors, extensions, stdout, status, and usage behavior unchanged.

connection-error-diagnosis

Checklist

Stability

  • Breaking changes are marked with 💥 in the PR title and release notes
  • Changes to JSON output (-o json / -o jsonl) are treated as breaking changes

Design

  • This feature does not depend on Cloud-only APIs or behavior (it works against an OSS server)
  • New commands follow temporal <noun> <verb> structure (e.g. temporal workflow start)
  • New flags are named after the API concept, not the implementation mechanism (good: --search-attribute, bad: --index-field)
  • New flags don't duplicate an existing flag that serves the same purpose
  • New flags do not have short aliases without strong justification
  • Experimental features are marked with (Experimental) in commands.yaml

Help text (see style guide at the top of commands.yaml)

  • All flags shown in help text and examples are implemented and functional
  • Summaries use sentence case and have no trailing period
  • Long descriptions end with a period and include at least one example invocation
  • Examples use long flags (--namespace, not -n), one flag per line
  • Placeholder values use YourXxx form (YourWorkflowId, YourNamespace)

Behavior

  • Results go to stdout; errors and warnings go to stderr
  • Error messages are lowercase with no trailing punctuation

Tests

  • Added functional test(s) (SharedServerSuite)
  • Added unit test(s) (func TestXxx) where applicable

Manual tests

Setup

No manual setup was used.

Happy path

No manual happy-path run was used. After merge, the full suite passed in internal/temporalcli in 166.201s:

go test ./... -count=1 -timeout=10m

Error case

Connection-focused go test -race tests passed. The concurrent development-server race still reports the existing upstream global color race in unchanged commands.go lines. This PR does not claim that the full race suite passes.

Composition

No manual composition test was used.

Connection failures now render a classified error with an inline
DNS/TCP/TLS diagnosis and one suggested fix, instead of a bare
'context deadline exceeded' or an empty message.

Fixes #224
Fixes #851
@CLAassistant

CLAassistant commented Jul 8, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Covers classifyGRPCError, connectSummary's grep-compatibility contract,
and an end-to-end case where the failing address comes from a config
profile (exercising the new cliext builder metadata).
- errors.Is(err, syscall.ECONNREFUSED) doesn't match Windows'
  WSAECONNREFUSED; fall back to matching the error message.
- The plaintext test server closed with the client's ClientHello unread,
  sending an RST that on Windows discards the buffered HTTP response
  before the probe reads it; drain before closing.
…nosis

# Conflicts:
#	internal/temporalcli/commands.go
@rossnelson
rossnelson marked this pull request as ready for review July 27, 2026 20:03
@rossnelson
rossnelson requested a review from a team as a code owner July 27, 2026 20:03
@chaptersix

Copy link
Copy Markdown
Contributor

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8f9e46c124

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

case errors.As(err, &hostnameErr):
d.fail("TLS handshake failed: server certificate is not valid for this host: " + shortErr(err))
d.Cause = causeHostnameMismatch
case errors.As(err, &unknownAuthErr), errors.As(err, &certErr):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle non-CA certificate verification failures separately

When the TLS handshake fails because a trusted certificate is expired, not yet valid, or has an incompatible usage, Go wraps the underlying x509.CertificateInvalidError in *tls.CertificateVerificationError, so this broad match classifies it as causeCAVerify. suggestAction then incorrectly tells the user to configure --tls-ca-path, which cannot fix those certificate-validity failures. Reserve causeCAVerify for unknown-authority errors, or classify other verification errors separately so the diagnostic preserves the actual remedy.

Useful? React with 👍 / 👎.

Comment on lines +66 to +68
if runtime.GOOS == "windows" {
shell = displayShellPowerShell
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid assuming every Windows terminal is PowerShell

When the CLI is launched from cmd.exe, runtime.GOOS is still windows, so the suggested command is rendered as PowerShell syntax such as & 'temporal' 'server' 'start-dev'. Command Prompt treats those single quotes as literal characters and cannot execute that command, making the proposed recovery step unusable for cmd users. Detect the active shell or render the current safe invocation in syntax accepted by both Windows shells.

Useful? React with 👍 / 👎.

// (or rejects us at the certificate step, which still means it spoke TLS), the
// mismatch is the likely root cause.
func probeServerSpeaksTLS(ctx context.Context, conn net.Conn, host string) (connectCause, string) {
tlsConn := tls.Client(conn, &tls.Config{InsecureSkipVerify: true, ServerName: host})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Advertise h2 during diagnostic TLS handshakes

When the target TLS endpoint requires ALPN negotiation for h2, this raw crypto/tls probe sends no NextProtos, so the server can reject it with no application protocol. The real gRPC TLS transport advertises h2, meaning the diagnostic follows a different handshake path and can fail to detect that a plaintext-configured client is actually contacting a TLS server; the configured-TLS probe has the same omission and may report a spurious TLS-handshake failure. Add h2 to the cloned/probe TLS configurations so the checks reproduce the gRPC transport.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants