Repository navigation
Make file-backed StoreAsync safe under concurrent writers and readers [patch] - #116
Open
matt-edmondson wants to merge 4 commits into
Open
matt-edmondson wants to merge 4 commits into
matt-edmondson wants to merge 4 commits into
Conversation
… [patch] FileSystemPersistenceProvider and TempPersistenceProvider staged every write in the same "<file>.tmp" and then deleted the target before moving the temp file into place. Overlapping writers to one key moved each other's temp file away and threw. Between the delete and the move, a reader saw the key as missing, so RetrieveOrCreateAsync could store a fresh default over the real record. Both providers now write through Shared/AtomicFileWrite. Each write uses its own "<guid>.tmp" in the target's directory, and the swap is a single overwriting move. netstandard2.1 lacks that overload, so it uses File.Replace, or File.Move when the target does not exist yet. A short, bounded retry covers Windows refusing to replace a file another handle has open, and the staging file is always cleaned up. The staging name is not derived from the key, because filenames are budgeted for only the extension plus ".tmp". A longer suffix pushes long keys past the 255-byte name limit. Fixes #66 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YLmoTmY5N7amW8Z9jazuGV
On windows-latest the concurrency test failed in RetrieveAsync: "The process cannot access the file ... k.json because it is being used by another process." Windows holds the target exclusively for the instant an overwriting move swaps it in. A read that lands at that moment is refused, where it would otherwise be served either version. The shared helper, renamed from AtomicFileWrite to AtomicFile, now also provides ReadAllTextAsync, and both providers' RetrieveAsync use it. It retries only ERROR_SHARING_VIOLATION and ERROR_LOCK_VIOLATION, with the same bounded backoff as the write path. A missing file and every other error still surface at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YLmoTmY5N7amW8Z9jazuGV
SonarCloud S1994 flagged both retry loops in AtomicFile. They were written `for (int attempt = 1; ; attempt++)` with the bound in the catch filter. Each loop now runs `attempt < MaxAttempts` retried attempts, and then makes one final unguarded attempt whose failure surfaces. The swap moves into a Replace helper so the two call sites share it. Behaviour is unchanged. The concurrency test also uses Assert.Contains in place of CollectionAssert.Contains, which MSTEST0068 asked for. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YLmoTmY5N7amW8Z9jazuGV
# Conflicts: # Essentials.Tests/PersistenceProviderTests.cs
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Fixes #66
What was wrong
FileSystemPersistenceProvider.StoreAsyncandTempPersistenceProvider.StoreAsync(and through them ConfigHome and DataHome) had two problems:<file>.tmp, so overlapping writers to one key moved each other's temp file away and the loser threw.RetrieveOrCreateAsynccould store a freshnew T()over the real record.Change
Both providers now go through a new
Shared/AtomicFile.cs, linked into each project the same wayPersistenceDefaults.csalready is:<guid>.tmpin the target's directory. The.tmpextension keeps it out of the*<extension>key listings, and afinallyalways deletes it.File.Move(temp, target, overwrite: true)replaces the old file with no delete in between. netstandard2.1 has no such overload, so there it usesFile.Replacewhen the target exists andFile.Movewhen it doesn't. If another writer creates the target in between, theMovethrows and the retry takes theReplacepath.IOException/UnauthorizedAccessException, at most 50 times, with delays capped at 20 ms.RetrieveAsyncreads throughAtomicFile.ReadAllTextAsync. Windows holds the target exclusively for the instant an overwriting move swaps it, so a read landing then is refused. The first Windows CI run here failed on exactly that. OnlyERROR_SHARING_VIOLATION/ERROR_LOCK_VIOLATIONare retried; a missing file and every other error still surface at once.The staging name is deliberately not
<file>.<guid>.tmp.PersistenceProviderUtilitiesbudgets each filename for the extension plus.tmponly, and a longer suffix pushes long keys past the 255-byte name limit; an earlier attempt brokeLong_Multibyte_Keys_Store_And_Retrieve_On_The_Native_File_Systemthis way. Raising that budget would move where existing long keys get truncated and orphan their stored files.Tests
Concurrent_Stores_To_One_Key_Never_Throw_Or_Hide_The_Keyruns on both FileSystem and Temp. It starts 200 parallelStoreAsynccalls to one key while a reader loops overExistsAsync/RetrieveAsync, then asserts:GetAllKeysAsynclists only that key.tmpfile is left behindResults:
FileNotFoundExceptionin the delete-then-move window.Essentials.Testssuite passes (1036), and both provider projects build for every target framework.CLAUDE.mdnow lists the new shared file. Related but not covered here: #67, theRetrieveOrCreateAsyncpresence check.🤖 Generated with Claude Code
https://claude.ai/code/session_01YLmoTmY5N7amW8Z9jazuGV