fix(storage): hedged read attempts must not own the connection - #16534
kalragauri wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors HedgedObjectReadSource to pass a std::weak_ptr of the ChildFactory to racing attempts, preventing them from keeping the factory and its associated connection/thread pools alive after the read source is destroyed. Additionally, it re-enables the previously skipped HedgedReadRecordsMetricsOnGlobalMeterProvider test with updated synchronization logic and adds a new unit test QueuedAttemptAfterDestructionDoesNotOpen to verify the lifetime behavior. I have no feedback to provide as there are no review comments to assess.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #16534 +/- ##
==========================================
+ Coverage 92.34% 92.36% +0.01%
==========================================
Files 2262 2262
Lines 217173 217309 +136
==========================================
+ Hits 200549 200708 +159
+ Misses 16624 16601 -23 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This PR re-enables
RetryClientTest.HedgedReadRecordsMetricsOnGlobalMeterProvider(skipped in #16523) and fixes the lifetime bug behind the gmock leak it reported.Two references allowed a pool worker to hold the last reference to
StorageConnectionImplafterRead()had already returned, destroying the connection (and its mock stub and metrics instruments) on a detached workerthread after
main()'s leak check:HedgedObjectReadSource::ReadRaced()capturedchild_factory_byshared_ptrin each attempt task, and the factory captures the connection. Because a worker destroys its task only afterpromise.set_value()unblocks the caller, the winning worker kept the connection alive past the end of the read. Attempts now capture aweak_ptrand lock it only for the open call; an attempt that starts after the source is destroyed retires withkCancelled.RetryObjectReadSource(which holds the connection) until itsClose()returns. This is unchanged: the test now issues one more synchronous read on the single read worker after unblocking the loser, which (FIFO) cannot complete until the loser's task has been destroyed, so the test thread holds the last connection reference. Dropping aClientmid-close in production still tears the connection down on a detached worker, same asAutomaticallyCreatedBackgroundThreads; that is pre-existing and out of scope for this PR.Fixes #16522