Skip to content

Add pool tracing/metrics parity and surface pooled-open timeout cause - #4505

Draft
mdaigle wants to merge 1 commit into
dev/automation/channel-pool-v2-followupsfrom
dev/automation/channel-pool-traces-metrics
Draft

Add pool tracing/metrics parity and surface pooled-open timeout cause#4505
mdaigle wants to merge 1 commit into
dev/automation/channel-pool-v2-followupsfrom
dev/automation/channel-pool-traces-metrics

Conversation

@mdaigle

@mdaigle mdaigle commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Stacked on #4504.

Summary

Brings ChannelDbConnectionPool to trace and metric parity with WaitHandleDbConnectionPool, and surfaces the cause of a pooled-open timeout.

Trace parity

Adds TryPoolerTraceEvent calls across the connection lifecycle, following the WaitHandle pool's message conventions:

  • Constructor: Constructed. MinPoolSize=..., MaxPoolSize=...
  • GetInternalConnection: Getting connection., Wait timed out., Pool is shutting down; abandoning wait.
  • GetIdleConnection: Popped from general pool.
  • OpenNewInternalConnection: Errors are set., Creating new connection., Added to pool., rate-limiter saturation, pool-full, and create-threw
  • PutConnectionInIdleChannel: Pushing to general pool.
  • DeactivateAndRouteConnection: the stasis/transacted routing decision
  • RemoveConnection: Removing from pool., Removed from pool., Disposed.
  • PruneConnections: prune start and result
  • IsLiveConnection: the reason a connection was rejected (idle timeout, dead, load balance timeout, stale generation)

Metric parity

Most metric wiring landed in #4504 via IdleConnectionChannel. This closes the two remaining gaps in ReplaceConnection, which disposed the old connection and the failed new connection without counting a HardDisconnectRequest.

Note: the WaitHandle pool leaks an EnterPooledConnection on replace. The channel pool swaps in place via ConnectionPoolSlots.TryReplace, so the pooled gauge is correctly left untouched. That behavior is intentionally not replicated.

Pooled-open timeout cause (#3545)

Today a pooled-open timeout hides why the pool could not produce a connection. This adds IDbConnectionPool.LastConnectionCreateException, recorded on create failure and cleared on success in both pools, and a new ADP.PooledOpenTimeout(Exception inner) overload. SqlConnectionFactory and the WaitHandle pool's pending-opens path now attach it as the inner exception.

Fixes #3545

Out of scope

OpenTelemetry db.client.connections.* semantic-convention metrics (Story 4 of the original request) are deferred to a follow-up.

Suggested release note

  • Connection pool v2 (UseConnectionPoolV2) now emits the same pooler trace events and connection metrics as the default pool.
  • A pooled connection open that times out now carries the last physical connection failure as its inner exception.

Testing

New ChannelDbConnectionPoolInstrumentationTest (17 tests) covering trace emission for each operation category, metric deltas for soft connect/disconnect and hard disconnect, and the last-create-exception plumbing. Adds a matching WaitHandle-pool test. 343 connection-pool unit tests pass on net9.0; net8.0 builds clean.

Checklist

  • Tests added or updated
  • Public API changes documented (no public API changes; IDbConnectionPool is internal)
  • Verified against customer repro (not applicable)
  • Ensure no breaking changes introduced

Instrument ChannelDbConnectionPool with TryPoolerTraceEvent calls across the
connection lifecycle so it matches the categories traced by the WaitHandle
pool: construction, get, create, return, remove/dispose, clear, startup,
shutdown, prune, rate-limit throttle, error state, wait timeout, and the reason
a connection was rejected as not live.

Fill the two remaining metric gaps in ReplaceConnection, which disposed the old
and failed-new connections without counting a hard disconnect.

Also address GH#3545: record the last physical-connection-create exception on
each pool and attach it as the inner exception of the pooled-open timeout, so
callers see why the pool could not produce a connection.

Fixes #3545

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 4, 2026 23:26
@mdaigle
mdaigle requested a review from a team as a code owner August 4, 2026 23:26
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Aug 4, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves connection-pool diagnostics by bringing ChannelDbConnectionPool up to parity with the legacy WaitHandleDbConnectionPool for pooler tracing and metrics, and by surfacing the last physical connection creation failure as the inner exception on pooled-open timeouts (GH#3545).

Changes:

  • Adds pool-lifecycle TryPoolerTraceEvent emissions throughout ChannelDbConnectionPool to match legacy pool trace conventions.
  • Completes metric parity for the channel pool (notably around ReplaceConnection hard-disconnect accounting).
  • Introduces IDbConnectionPool.LastConnectionCreateException plus ADP.PooledOpenTimeout(Exception inner) plumbing so pooled-open timeouts can carry the most recent physical open failure as an inner exception.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/WaitHandleDbConnectionPoolBlockingPeriodTest.cs Adds coverage that the legacy pool records and clears the last-create exception correctly.
src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/TransactedConnectionPoolTest.cs Updates the test mock pool to implement the new LastConnectionCreateException interface member.
src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolInstrumentationTest.cs New test suite validating channel-pool trace emission, metric deltas, and last-create-exception timeout plumbing.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlConnectionFactory.cs Attaches the pool’s last-create exception to pooled-open timeouts thrown from the factory path.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/WaitHandleDbConnectionPool.cs Tracks/clears the last physical create exception and uses it when pending opens time out.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/IDbConnectionPool.cs Adds LastConnectionCreateException to standardize timeout-cause reporting across pool implementations.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs Adds trace parity, last-create-exception storage/clearing, and missing hard-disconnect accounting on replace paths.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/Common/AdapterUtil.cs Adds ADP.PooledOpenTimeout(Exception inner) overload to optionally carry an inner exception.

Comment on lines +582 to +587
private const string SqlClientEventSourceName = "Microsoft.Data.SqlClient.EventSource";

// Mirrors SqlClientEventSource.Keywords.PoolerTrace. Duplicated as a literal because
// that type is not visible to this assembly.
private const EventKeywords PoolerTraceKeyword = (EventKeywords)32;

@mdaigle
mdaigle marked this pull request as draft August 5, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: To triage

Development

Successfully merging this pull request may close these issues.

2 participants