Skip to content

fix(encryption): detach a coalesced DEK load from its leader - #7

Merged
patrislav merged 1 commit into
masterfrom
fix/detach-coalesced-dek-load
Sep 25, 2026
Merged

patrislav merged 1 commit into
masterfrom
fix/detach-coalesced-dek-load

Conversation

@patrislav

@patrislav patrislav commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #4, from review on waas#132.

DoChan runs the shared load on whichever caller arrived first. Cancel that request mid-load — client disconnect, deadline — and every caller coalesced onto the same key ref gets its context.Canceled, despite their own contexts being live.

Pool DEKs are shared across many rows, so a cold cache or an expired entry reliably produces concurrent misses on one key ref. In waas the failure is silent: ListActiveSessions skips a binding it cannot decrypt, and AdvanceConfig reconciles the on-chain session set to what came back — so an unrelated disconnect can drop live sessions from a wallet's config.

Fix

The load keeps the leader's context values but neither its cancellation nor its deadline, bounded by dekLoadTimeout (30s) so a load nothing waits for still terminates. That bound also covers the hung-load case from #4, which is why there is still no Forget. Waiters are unchanged — they leave early on their own context. cancel is deferred inside the load goroutine: deferring it in fetchDEK would let a returning leader kill the load its waiters are blocked on.

Two things follow from outliving the request:

  • migrateKey opens its own attestation. Re-encrypting shares reaches aescbc.Encrypt(att, …), which draws the IV from the NSM session, and the middleware closes that session when the handler returns. Four mock expectations asserting migration used the caller's attestation had to be relaxed, which confirms it did.
  • tracing.Detach roots the load's span. Span locks its mutations, but the middleware's json.Marshal reads the tree by reflection unlocked — safe only while every span is finished when the request ends.

Test

TestPool_DecryptSurvivesLeaderCancellation holds the leader's share decrypt, cancels the leader, and asserts a follower on the same load still succeeds. Point the load back at ctx and it fails. MockRemoteKey.Decrypt now observes cancellation; without that it ignored ctx and the test passed either way.

🤖 Generated with Claude Code

@patrislav
patrislav requested a review from a team September 25, 2026 09:57
DoChan ran the shared load on whichever caller arrived first. If that request
was cancelled mid-load, every caller coalesced onto the same key ref got its
context.Canceled, even though their own contexts were live. In waas that
surfaces as sessions silently missing from a config reconcile, because the
caller logs and skips a binding it cannot decrypt.

The load now runs on a context carrying the leader's values but neither its
cancellation nor its deadline, bounded by dekLoadTimeout so a load nothing is
waiting for still terminates. Waiters keep leaving early on their own context.

Two consequences of outliving the request, both handled here:

migrateKey re-encrypts shares, which draws the AES-CBC IV from the NSM session;
the leader's session is closed once its handler returns. It now opens its own
attestation and no longer takes one from the caller.

Span mutations are mutex-guarded but the middleware marshals the tree by
reflection without the lock, so a still-running child would be read while it is
written. tracing.Detach starts the load's span as a new root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@patrislav
patrislav force-pushed the fix/detach-coalesced-dek-load branch from ef3d43d to 217881f Compare September 25, 2026 11:21

@marino39 marino39 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@patrislav
patrislav merged commit 5fe26a7 into master Sep 25, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants