fix(httpx): restore system resolver fallback when no custom resolvers are set

[ezyasin]

Proposed changes

Fixes #2558

httpx resolves through fastdialer with the system nameservers plus the built-in public resolvers (1.1.1.1, 1.0.0.1, 8.8.8.8, 8.8.4.4). When some of them are unreachable, for example a VPN kill switch that only allows DNS to the system nameserver (the write: operation not permitted in the issue), a lookup can spend all of its retries on blocked resolvers and fail with no address found for host. Retries round-robin over a counter shared by all concurrent lookups, so different hosts fail on every run.

fastdialer has a system resolver fallback (EnableFallback) for exactly this. httpx enabled it in 028b046, but #1541, which was meant to disable it only when -r is given, also removed the default EnableFallback = true. It has been off for everyone since v1.3.8.

This PR:

• restores EnableFallback = true when no custom resolvers are set (-r behaviour is unchanged: custom resolvers only, no system resolvers, no fallback)
• moves the fastdialer option building into newFastdialerOptions so it can be unit tested
• adds TestNewFastdialerOptions covering both cases, so this can't silently regress again

Note: the bug only shows with explicit https:// / http:// inputs. With bare hosts the scheme fallback retries the dial, which hides most DNS failures.

Proof

Built the httpx binary with fastdialer's default 1.0.0.1, 8.8.8.8 and 8.8.4.4 replaced by closed local ports, which simulates the VPN blocking them (2 of 5 resolvers reachable, as in the issue). 30 https:// URLs, 5 runs each:

	DNS failures (no address found for host)
dev	3, 4, 3, 3, 1 (different hosts every run)
this PR	0, 0, 0, 0, 0

-r behaviour (same before and after):

• -r 127.0.0.1:1 (dead resolver): 10/10 fail, no fallback to the system resolver
• -r 1.1.1.1: 10/10 probed

Cost: for input made only of non-existent hosts, the fallback adds one system lookup per host. Over 300 random NXDOMAIN subdomains it took ~15.9s vs ~14.1s on dev. This is the same behaviour httpx had before v1.3.8.

TestNewFastdialerOptions fails on dev and passes with this change. go test ./..., integration tests, go build -race and golangci-lint all pass locally.

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests etc.) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Summary by CodeRabbit

• Bug Fixes
  • Improved DNS resolution behavior for HTTP connections. System resolver fallback is used by default, while configured custom resolvers are respected without additional resolver-file or fallback lookups.
  • Existing network policies and connection security settings continue to be applied.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 1bd45814-2454-442c-9e18-451df5e64559

📥 Commits

Reviewing files that changed from the base of the PR and between 482ce7b and 1c41837.

📒 Files selected for processing: 2

• common/httpx/httpx.go
• common/httpx/httpx_test.go

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

The HTTP client now builds fastdialer options through a helper. The helper configures system resolver fallback by default and uses supplied resolvers when custom resolvers are configured. Tests cover both settings.

Changes

DNS resolver configuration

Layer / File(s)	Summary
Build and apply fastdialer options common/httpx/httpx.go, common/httpx/httpx_test.go	A new helper configures resolver behavior, network policy, dialer history, zTLS, and SNI. New uses the helper to create the dialer. Tests check default and custom resolver options.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: dogancanbakir

Merge Risk: ⚪ Minimal · up to 1c418

Default DNS resolution gains the intended system fallback, while custom resolver behavior remains unchanged. The change is mergeable after normal checks.

Security architecture risk: 🔵 Low · up to 1c418

The fallback preserves custom-resolver restrictions and destination-IP policy checks. It does add a system lookup that cannot observe caller cancellation, so slow system resolution can prolong pending work. No new destination-policy bypass was established.

Retained concerns

• Low · reliability · inferred: Default-mode lookup failures now enter an additional system-resolution stage that does not receive the caller context. Slow system resolution can retain lookup work and same-host waiters after cancellation, weakening cancellation-based failure containment. Configured DNS resolution was already context-insensitive; this PR adds another such stage rather than introducing the entire limitation.

Security review details

Security Blast Radius

• inferred — The behavioral expansion affects default-configured HTTPX dialers when configured DNS attempts produce no address and system lookup can proceed. It can make additional destinations resolvable, but does not grant additional destination-policy authority. Exposure depends on the embedding application's inputs and host resolver configuration; no new network-facing service or privilege grant is introduced by the two-file change.

Trust Boundaries and Controls

• observed — System-fallback addresses enter the same IP and port validation loop as configured-DNS results before becoming eligible for dialing. Custom resolver settings continue to suppress the newly enabled syscall path. These controls counter a claim that fallback bypasses destination filtering or overrides explicit DNS resolver selection.

Resilience and Maintainability Implications

• inferred — Failed fallback lookups do not publish successful DNS cache entries, allowing later attempts to recover. However, cancellation cannot interrupt the added net.LookupIP call. Same-host singleflight reduces duplicate lookups but can also leave multiple callers waiting for that lookup; the delay depends on the operating-system resolver.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: restoring system resolver fallback when no custom resolvers are configured.
Linked Issues check		Issue [#2558] requires more reliable DNS resolution during VPN use. newFastdialerOptions enables EnableFallback by default, so the system resolver remains available when built-in resolvers fail. W…
Out of Scope Changes check		The changed production code constructs fastdialer options for the DNS fix. The added unit test verifies the default and custom-resolver configurations. These changes directly support issue [#2558], an…

Full details: Linked Issues check

Explanation

Issue [#2558] requires more reliable DNS resolution during VPN use. newFastdialerOptions enables EnableFallback by default, so the system resolver remains available when built-in resolvers fail. When Options.Resolvers is set, the code disables ResolversFile and EnableFallback and uses BaseResolvers, which preserves the required -r behavior. TestNewFastdialerOptions verifies both configurations.

Full details: Out of Scope Changes check

Explanation

The changed production code constructs fastdialer options for the DNS fix. The added unit test verifies the default and custom-resolver configurations. These changes directly support issue [#2558], and no unrelated change is shown in the supplied change summary.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{A rabbit checks the resolvers with care,
System fallback is ready to spare.
Custom nameservers stay in their place,
Dialer options now share one base.
Test paws confirm each setting is there.}

_{Comment @coderabbitai help to get the list of available commands.}