Skip to content

fix(bigtable): count pre-OpenSessionResponse stream failures in ChannelPoolDpImpl - #15039

Merged
mutianf merged 6 commits into
mainfrom
fix-channel-pool-pre-open-session-failure
Nov 18, 2026
Merged

mutianf merged 6 commits into
mainfrom
fix-channel-pool-pre-open-session-failure

Conversation

@mutianf

@mutianf mutianf commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Part 2 of 2 for b/584671437 (stacked on #15038).

What changed

  • Only mark a session stream established in ChannelPoolDpImpl when OpenSessionResponse arrives (onMessage), rather than when initial PeerInfo headers arrive (onBeforeSessionStart).
  • Count non-OK stream closes that occur before OpenSessionResponse as consecutive failures and evict the failed routeObservations entry so subsequent OpenSession attempts do not remain pinned to a failing AFE/subzone.

@mutianf
mutianf added this pull request to stack #15040 November 16, 2026 16:59
@mutianf
mutianf requested review from a team as code owners November 16, 2026 16:59

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors channel failure tracking in ChannelPoolDpImpl by introducing a sessionEstablished flag to ensure that consecutive failures are only reset when a session is successfully opened or closed with an OK status, and incremented if a failure occurs before session establishment. A corresponding unit test has been added to verify this behavior. The review feedback correctly identifies a concurrency issue where sessionEstablished is read outside of a synchronized block but written inside one, and suggests declaring it as volatile to guarantee proper memory visibility.

return new SessionStreamImpl(innerCall) {
// null until onBeforeSessionStart fires
@Nullable AfeId afeId = null;
boolean sessionEstablished = false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The sessionEstablished field is read outside of a synchronized block in onMessage (line 332) but written inside a synchronized block (line 334). To ensure proper memory visibility and prevent thread-local caching or instruction reordering under the Java Memory Model (JMM), this field should be declared volatile.

Suggested change
boolean sessionEstablished = false;
volatile boolean sessionEstablished = false;
References
  1. Ensure that fields used for cross-thread state checks, such as exceptions set by background threads and read by application threads, are declared as volatile to guarantee memory visibility and establish a proper happens-before relationship under the Java Memory Model.

@mutianf
mutianf force-pushed the fix-channel-pool-pre-open-session-failure branch from d5c4ee1 to b9617c3 Compare November 16, 2026 20:27
@mutianf
mutianf removed this pull request from stack #15040 November 16, 2026 20:28
@mutianf
mutianf added this pull request to stack #15042 November 16, 2026 20:29
@mutianf
mutianf removed this pull request from stack #15042 November 16, 2026 21:05
@mutianf
mutianf force-pushed the fix-channel-pool-pre-open-session-failure branch from b9617c3 to c626c8b Compare November 16, 2026 21:05
@mutianf
mutianf added this pull request to stack #15043 November 16, 2026 21:05
Base automatically changed from fix-session-pool-pending-rpcs-bound to main November 17, 2026 18:08
@mutianf
mutianf force-pushed the fix-channel-pool-pre-open-session-failure branch from c626c8b to 952a289 Compare November 17, 2026 18:08
@Override
public void onMessage(SessionResponse msg) {
if (!sessionEstablished && msg.hasOpenSession()) {
synchronized (ChannelPoolDpImpl.this) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does this need to synchronized?


@Override
public void onMessage(SessionResponse msg) {
if (!sessionEstablished && msg.hasOpenSession()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a comment why doing this onBeforeSessionStart is not enough.

@mutianf
mutianf enabled stacked auto-merge (squash) November 17, 2026 21:07
@mutianf
mutianf merged commit 3358401 into main Nov 18, 2026
196 checks passed
@mutianf
mutianf deleted the fix-channel-pool-pre-open-session-failure branch November 18, 2026 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants