test(checkpoint): failing repro for concurrent checkpoint-write lost update (#1917)

[Soph]

Draft — this PR is a reproduction, not a fix. The concurrent subtests are
red on purpose, so CI will fail. It exists to make #1917 concrete and to pin
down which scenario is actually reachable.

What it reproduces

Two agents working in one worktree, whose uncondensed work lands in a single
commit, end up sharing one checkpoint: post-commit condenses both
sessions into it and records it in each session's TurnCheckpointIDs. When
both turns end, both Stop hooks run finalizeAllTurnCheckpoints over that same
checkpoint.

Nothing serializes them:

• each Stop hook holds only its own per-session flock — stateLockPath is
  keyed by session ID, and these are legitimately different sessions;
• both persistent stores advance the checkpoint ref with an unconditional
  Storer.SetReference (checkpoint/refs_store.go:206 for git-refs,
  checkpoint/store.go:91 for git-branch) with no compare-and-swap against the
  tip they read.

Both hooks read the same tip, build sibling commits, and the last ref write
makes the other's finalize unreachable. Both hooks exit 0; nothing is logged.

Results

Real hook subprocesses, ForEachBackend × sequential/concurrent. The two arms
run the identical scenario — only whether the Stop hooks start together differs.

	concurrent	sequential (control)
git-branch	FAIL 5/5	PASS 5/5
git-refs	FAIL 5/5	PASS 5/5

go test -tags integration ./cmd/entire/cli/integration_test \
  -run '^TestConcurrent_TwoStopHooks_SameCheckpoint$' -count=5 -v

Caveats, stated plainly

• The 5/5 is conditional on both hooks starting at the same instant, which
  the test forces with a barrier. Real turn ends are independent, so the field
  rate is roughly finalize duration ÷ spread between the two turn ends. The
  test pads the transcript (400 filler lines) to widen the finalize window;
  real transcripts are larger, so the real window is wider than the tested one,
  not narrower.
• What this does establish: the scenario is built from ordinary operations, not
  a hand-authored interleave, and the loss is silent.
• The losing commit is unreachable and un-reflogged (go-git's
  SetReference writes no reflog), so it is gc-able garbage. The v1 reconcile
  path can't recover it either: ReconcileDisconnectedMetadataRef walks back
  from the local ref tip (strategy/metadata_reconcile.go:170). Per-checkpoint
  directories guarantee cherry-picks apply cleanly; they don't guarantee the
  commit still exists to cherry-pick. Reconcile handles divergence between two
  reachable tips — this destroys reachability inside one clone before any
  reconcile can observe it.

Note on #1917 as filed

The reporter's scenario — a summary backfill racing an attribution backfill on
one checkpoint — is essentially unreachable in practice: those writers are
sequenced by the session lifecycle, and the only unscheduled one is
user-invoked entire checkpoint explain --generate, which runs after the fact.
The defect is real; the reachable path is this one.

Their proposed regression test also asserts the buggy behavior (it passes by
observing the loss). This one asserts the correct behavior, so it goes green
when fixed.

Fix direction (not in this PR)

The ephemeral shadow-branch store already solved this exact shape:
withShadowBranchFlock + casUpdateShadowBranchRef + retry-from-a-fresh-parent
with jitter backoff and dangling-object cleanup (checkpoint/shadow_ref.go,
driven from checkpoint/ephemeral.go:112). The persistent stores never got the
same treatment. Two things worth stating for whoever picks it up:

• retry must rebuild the tree from the new tip, not re-point the commit it
  already built — re-pointing reintroduces the lost update;
• don't use go-git's CheckAndSetReference — see the rationale at
  shadow_ref.go:65: it doesn't interoperate with native git's .lock files,
  and these refs are touched by concurrent hook processes and the user's own git.

Tightening lock scoping alone cannot fix this instance, since the two writers
are legitimately different sessions touching one checkpoint.

Refs #1917

🤖 Generated with Claude Code

---

Note

Low Risk
Test-only change; concurrent subtests are intentionally red and document expected failure until a store-level CAS fix lands.

Overview
Adds integration coverage for #1917: two sessions sharing one post-commit checkpoint, each running a real Claude Stop hook subprocess when their turn ends.

The scenario builds a single commit that condenses both agents, then appends distinct post-commit transcript markers and runs Stop hooks sequentially (control) or concurrently (barrier-started). Helpers stopHookCmd and checkpointBlob support parallel subprocesses and both git-branch / git-refs backends via ForEachBackend.

Assertions require both markers in stored checkpoint transcripts after both hooks exit 0. The concurrent subtests are expected to fail today (silent lost update); sequential should pass. This is a reproduction PR, not a fix.

^{Reviewed by Cursor Bugbot for commit af59539. Configure here.}

[Copilot]

Pull request overview

This PR adds a new integration test that deterministically reproduces #1917’s “lost update” race by running two real Stop-hook subprocesses that concurrently finalize the same checkpoint (across both git-branch and git-refs backends). It’s intended as a concrete reproducer to guide a future CAS-based fix in the persistent checkpoint stores.

Changes:

• Add TestConcurrent_TwoStopHooks_SameCheckpoint, covering sequential (control) vs concurrent Stop-hook finalization.
• Introduce helpers to (a) construct Stop-hook subprocesses without running them and (b) read checkpoint blobs in a backend-aware way.