Skip to content

Make file-backed StoreAsync safe under concurrent writers and readers [patch] - #116

Open
matt-edmondson wants to merge 4 commits into
mainfrom
fix/66-atomic-store
Open

matt-edmondson wants to merge 4 commits into
mainfrom
fix/66-atomic-store

Conversation

@matt-edmondson

@matt-edmondson matt-edmondson commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #66

What was wrong

FileSystemPersistenceProvider.StoreAsync and TempPersistenceProvider.StoreAsync (and through them ConfigHome and DataHome) had two problems:

  • Every writer staged its data in the same <file>.tmp, so overlapping writers to one key moved each other's temp file away and the loser threw.
  • The swap deleted the target and then moved the temp file in. In between, a reader saw the key as missing, so RetrieveOrCreateAsync could store a fresh new T() over the real record.

Change

Both providers now go through a new Shared/AtomicFile.cs, linked into each project the same way PersistenceDefaults.cs already is:

  • Unique staging file. Each write stages its data in its own <guid>.tmp in the target's directory. The .tmp extension keeps it out of the *<extension> key listings, and a finally always deletes it.
  • One-step swap. File.Move(temp, target, overwrite: true) replaces the old file with no delete in between. netstandard2.1 has no such overload, so there it uses File.Replace when the target exists and File.Move when it doesn't. If another writer creates the target in between, the Move throws and the retry takes the Replace path.
  • Write retry. Windows won't replace a file while another handle has it open, which a concurrent reader or writer does briefly. The move is retried on IOException/UnauthorizedAccessException, at most 50 times, with delays capped at 20 ms.
  • Read retry. RetrieveAsync reads through AtomicFile.ReadAllTextAsync. Windows holds the target exclusively for the instant an overwriting move swaps it, so a read landing then is refused. The first Windows CI run here failed on exactly that. Only ERROR_SHARING_VIOLATION/ERROR_LOCK_VIOLATION are retried; a missing file and every other error still surface at once.

The staging name is deliberately not <file>.<guid>.tmp. PersistenceProviderUtilities budgets each filename for the extension plus .tmp only, and a longer suffix pushes long keys past the 255-byte name limit; an earlier attempt broke Long_Multibyte_Keys_Store_And_Retrieve_On_The_Native_File_System this way. Raising that budget would move where existing long keys get truncated and orphan their stored files.

Tests

Concurrent_Stores_To_One_Key_Never_Throw_Or_Hide_The_Key runs on both FileSystem and Temp. It starts 200 parallel StoreAsync calls to one key while a reader loops over ExistsAsync/RetrieveAsync, then asserts:

  • no store or read throws
  • the key is never missing
  • the final value is one of the payloads that was written
  • GetAllKeysAsync lists only that key
  • no .tmp file is left behind

Results:

  • Without the fix, the test fails on every run on Linux: the reader hits FileNotFoundException in the delete-then-move window.
  • With the fix, it passes 5 out of 5 runs on Linux.
  • The full Essentials.Tests suite passes (1036), and both provider projects build for every target framework.
  • The Windows-only read retry can't be reproduced on Linux; the windows-latest leg is its check.

CLAUDE.md now lists the new shared file. Related but not covered here: #67, the RetrieveOrCreateAsync presence check.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YLmoTmY5N7amW8Z9jazuGV

… [patch]

FileSystemPersistenceProvider and TempPersistenceProvider staged every
write in the same "<file>.tmp" and then deleted the target before moving
the temp file into place. Overlapping writers to one key moved each
other's temp file away and threw. Between the delete and the move, a
reader saw the key as missing, so RetrieveOrCreateAsync could store a
fresh default over the real record.

Both providers now write through Shared/AtomicFileWrite. Each write uses
its own "<guid>.tmp" in the target's directory, and the swap is a single
overwriting move. netstandard2.1 lacks that overload, so it uses
File.Replace, or File.Move when the target does not exist yet. A short,
bounded retry covers Windows refusing to replace a file another handle
has open, and the staging file is always cleaned up.

The staging name is not derived from the key, because filenames are
budgeted for only the extension plus ".tmp". A longer suffix pushes
long keys past the 255-byte name limit.

Fixes #66

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YLmoTmY5N7amW8Z9jazuGV
On windows-latest the concurrency test failed in RetrieveAsync:
"The process cannot access the file ... k.json because it is being used
by another process." Windows holds the target exclusively for the
instant an overwriting move swaps it in. A read that lands at that
moment is refused, where it would otherwise be served either version.

The shared helper, renamed from AtomicFileWrite to AtomicFile, now also
provides ReadAllTextAsync, and both providers' RetrieveAsync use it. It
retries only ERROR_SHARING_VIOLATION and ERROR_LOCK_VIOLATION, with the
same bounded backoff as the write path. A missing file and every other
error still surface at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YLmoTmY5N7amW8Z9jazuGV
SonarCloud S1994 flagged both retry loops in AtomicFile. They were
written `for (int attempt = 1; ; attempt++)` with the bound in the catch
filter. Each loop now runs `attempt < MaxAttempts` retried attempts, and
then makes one final unguarded attempt whose failure surfaces. The swap
moves into a Replace helper so the two call sites share it. Behaviour
is unchanged.

The concurrency test also uses Assert.Contains in place of
CollectionAssert.Contains, which MSTEST0068 asked for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YLmoTmY5N7amW8Z9jazuGV
# Conflicts:
#	Essentials.Tests/PersistenceProviderTests.cs
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FileSystem/Temp StoreAsync: concurrent writes to one key throw, and readers see the key as missing mid-write (shared .tmp path + delete-then-move)

2 participants