Fix broken double-checked locking in MemoryOptimizedHistory stream getters#3595
Fix broken double-checked locking in MemoryOptimizedHistory stream getters#3595MattBDev wants to merge 3 commits into
Conversation
…tters The six lazy output-stream getters checked their backing field outside the lock without re-checking inside it, and the four NBT getters were not synchronized at all. None of the six fields were volatile, so two threads could both pass the null check and each construct a separate compressed stream over a fresh buffer, silently discarding one thread's recorded changes. Make the six *Zip fields volatile and give all six getters the standard double-checked-locking shape: check, synchronize, re-check, construct. This mirrors the equivalent fix in DiskStorageHistory so the two classes, which are read side by side, keep the same shape. getBlockOS() additionally now writes the header into a local before publishing the stream to the volatile field, so a fast-path reader cannot observe a stream whose header has not been written yet and interleave block data ahead of it. Adds MemoryOptimizedHistoryConcurrencyTest covering concurrent getter access. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR hardens MemoryOptimizedHistory against a lost-update race caused by broken double-checked locking in its lazy, cached stream getters, aligning the implementation with the concurrency-fix pattern used for the disk-backed history variant.
Changes:
- Make the six cached “*Zip” stream fields
volatileand update all six getters to use check → synchronized → re-check → construct. - Ensure
getBlockOS()fully initializes (writes header) before publishing the stream reference. - Add a concurrency regression test and adjust test runtime dependencies to ensure LZ4 stream classes are present at test runtime.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| worldedit-core/src/main/java/com/fastasyncworldedit/core/history/MemoryOptimizedHistory.java | Fix lazy stream initialization races via volatile + correct double-checked locking; adjust getBlockOS() publication ordering. |
| worldedit-core/src/test/java/com/fastasyncworldedit/core/history/MemoryOptimizedHistoryConcurrencyTest.java | Add regression test validating stream getter returns a single instance under concurrent access. |
| worldedit-core/build.gradle.kts | Add LZ4 library on the test runtime classpath to avoid NoClassDefFoundError during concurrency tests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Both barrier.await() and done.await() were unbounded. If the getter under test ever deadlocked - exactly the failure mode this test exists to catch - the test would hang indefinitely instead of failing, potentially hanging CI. Bring this in line with the equivalent test in DiskStorageHistoryConcurrencyTest (PR #3594), which already does this: bound the barrier wait, and check done.await()'s return value, failing loudly with a diagnostic on timeout instead of silently proceeding. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…etter coverage
Every getter assigned its *Stream buffer field before constructing the
wrapping compressed/NBT stream, which can throw. A failure there (e.g.
getCompressedOS()) left the buffer field non-null while the wrapper field
(*StreamZip) stayed null. flush()/close() gate on the buffer field being
non-null and then dereference the wrapper field unconditionally, so they
would NPE instead of just skipping the failed stream. Build both into locals
and publish them together, so a failed construction leaves neither field set
and flush()/close() correctly treat the stream as never having existed.
Adds getBlockOSFailureLeavesNoPartialStateForFlushOrClose, a deterministic
regression test using a Mockito spy to force getCompressedOS() to throw:
against the pre-fix code it reproduces the exact NPE
("Cannot invoke FaweOutputStream.flush() because idsStreamZip is null");
with the fix, flush()/close() no-op cleanly.
Also, from code review of the DCL fix test:
- worker threads are now daemon, so a getter that deadlocks (the exact
failure mode being guarded against) can't keep the build alive past the
test's own timeout.
- getter results are asserted non-null before being recorded, since null is
a valid IdentityHashMap key and would otherwise let a buggy getter that
returns null pass as "one distinct instance".
- extended coverage to all six getters (previously only getBlockOS and
getTileCreateOS were exercised), so a future edit reintroducing the DCL bug
on getBiomeOS/getEntityCreateOS/getEntityRemoveOS/getTileRemoveOS would be
caught.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Flagging commit 7df0cdb for a second reviewer — this fixes a genuinely critical bug found during a follow-up code review, not just the original DCL race this PR started with: The bug: every one of the six lazy getters (getBlockOS, getBiomeOS, getEntityCreateOS, getEntityRemoveOS, getTileCreateOS, getTileRemoveOS) assigned its buffer field (e.g. idsStream) before constructing the wrapping compressed/NBT stream (getCompressedOS(...), or writeHeader(...) for getBlockOS specifically) - both of which can throw. A failure there left the buffer field non-null while the wrapper field (e.g. idsStreamZip) stayed null. flush()/close() gate on the buffer field being non-null and then unconditionally dereference the wrapper field, so instead of just skipping the failed stream, they NPE - and since MemoryOptimizedHistory.close() calls super.close() first, this NPE would propagate up through AbstractChangeSet.close() and out to whatever called close() on the changeset (e.g. EditSession's session-completion path), rather than the changeset just cleanly closing whatever streams did succeed. The fix: each getter now builds the buffer and its wrapper into locals and only publishes both fields together once construction fully succeeds, so a failure leaves neither field set. What to double-check:
|
Part of Phase 1 (concurrency hardening) of the
com.fastasyncworldedit.core.historyimprovement plan. One of five independent PRs; this one only touchesMemoryOptimizedHistory.The bugs
volatile. Two threads could both construct a compressed stream over a fresh buffer, silently discarding one thread's recorded changes.idsStream) before constructing the wrapping compressed/NBT stream, which can throw. A failure there left the buffer field non-null while the wrapper field (e.g.idsStreamZip) stayed null.flush()/close()gate on the buffer field being non-null and then unconditionally dereference the wrapper field, so instead of cleanly skipping the failed stream they threwNullPointerException— which, sinceclose()callssuper.close()first, would propagate out throughAbstractChangeSet.close()to whatever calledclose()on the changeset.The fix
*Zipfields are nowvolatile.getBlockOS()writes the header before publishing to the volatile field (see Fix broken double-checked locking in DiskStorageHistory stream getters #3594 for the reasoning).Verification
:worldedit-core:compileJava/compileTestJavapass.MemoryOptimizedHistoryConcurrencyTest: 121/121 pass with the fix (extended in 7df0cdb to cover all six getters, daemon worker threads, and a non-null assertion on the getter's return value).Could not initialize class ... ExceptionUtils) rather than failing an assertion cleanly — the race allocates duplicate ~520KB buffers per racing thread until the 1G test heap is exhausted. So there is a real fail→pass transition, but the pre-fix failure mode is a dead executor, not a clean assertion. Flagging that rather than claiming a clean regression test.getBlockOSFailureLeavesNoPartialStateForFlushOrClose(added in 7df0cdb) is a deterministic regression test for the partial-state bug: a Mockito spy forcesgetCompressedOS()to throw, and the test assertsflush()/close()don't NPE afterward. Against the pre-fix code it fails with the exact NPE (Cannot invoke FaweOutputStream.flush() because idsStreamZip is null); with the fix it passes.Notes
git worktreeadditionally need the Grgit fix from Add baseline benchmark for history write path (pre Phase 1 concurrency fixes) #3593; deliberately not included (CI uses a normal checkout).testRuntimeOnly(libs.lz4Java)added: lz4-java iscompileOnlyfor main, so the test JVM otherwise hitsNoClassDefFoundError: LZ4BlockOutputStreamviagetCompressedOS().🤖 Generated with Claude Code