Skip to content

fix: views read through their position only under the cache lock - #116

Merged
awolverp merged 2 commits into
awolverp:mainfrom
greenclaw:fix/iterator-lock
Oct 28, 2026
Merged

awolverp merged 2 commits into
awolverp:mainfrom
greenclaw:fix/iterator-lock

Conversation

@greenclaw

Copy link
Copy Markdown
Contributor

Follow-up to #94 / #98.

Problem

#98 made replace() bump the generation version, so iterating while another thread replaces a value now raises RuntimeError instead of reading a freed value. The underlying check-then-read race is still there, though: a view's __next__ compares the generation without holding the cache lock and then reads through its raw position. On free-threaded builds another thread can free or move entries between the check and the read (an insert that grows the table, pop, eviction, …).

Two related spots have the same race:

  • TTL/VTTL views' __len__ and __bool__ read entries (to skip expired ones) after the same unlocked check.
  • TTL view constructors create the iterator under the lock, release it, and only then read the generation, so a mutation in between makes stale positions look current. (VTTL already does both under one lock.)

On current main with CPython 3.14t, eight threads mixing get / set / pop with iterating views crash with SIGSEGV for FIFOCache, LFUCache, LRUCache and TTLCache in 5/5 local runs and VTTLCache in 2/5. The idea of holding the cache lock in the views comes from #94.

Fix

  • implement_view_guard! now generates lock_and_check(): take the cache lock, then compare the generation, and return the guard so it stays held while the view reads. The unlocked check_generation() is removed, so every view method has to go through the lock and the compiler enforces it: __next__ and __len__ of all views, __bool__ of TTL/VTTL views (16 call sites).
  • View structs keep the typed cache (Py<PyXCache> instead of Py<PyAny>) to reach its lock.
  • TTL view constructors create the iterator and read the generation under one lock.
  • Lock order is always cache lock, then the view's own iterator mutex. This is documented on the macro.

Behaviour change

next() / len() / bool() on a view now wait for the cache lock instead of reading unsynchronised. Calling them from Python code that runs inside an operation on the same cache (a key's __eq__, a value's __del__ during eviction) now blocks, like cache[key] from those places already does. I added one paragraph about this to docs/docs/tips.md (Iteration Safety).

Tests

New IterationMixin.test_iterating_while_other_threads_mutate runs for every cache class. A child process runs 8 threads that mix get / set / pop with walking keys() / values() / items() and calling len() / bool() on the views. items() must yield key -> key pairs without duplicates in one pass, which also catches stale reads that don't fault (LRUCache reuses freed list nodes, so a stale read returns another entry instead of crashing). It runs in a child process so a fault fails the test instead of killing pytest, like the existing EQ_TRIGGERING_GC tests, and faulthandler dumps all threads if the child hangs.

  • Red on main with only the test added, GitHub's ubuntu runner: 3.14t (FIFO, LFU, RR, TTL fail) and 3.15t (FIFO, LFU, TTL fail); every GIL build passes.
  • Green with the fix: 10/10 local runs on 3.14t; full suite in the CI configuration passes on 3.13 and 3.14t; fork CI green on the whole matrix, including 3.14t and 3.15t.

What the test catches, reverting one part of the fix at a time (3 runs each, 3.14t):

reverted red
__next__ lock FIFO, LRU, TTL in 3/3
TTL/VTTL __len__/__bool__ lock VTTL in 1/3 (short window)
TTL constructors TTL in 3/3

Cache follows the same pattern but didn't crash in this mix, because its table rarely reallocates under it; RRCache crashed only on the x86 runner. Those are covered by the shared macro rather than by a reliable red run.

Related, not in this PR

While testing I found that methods taking the lock with plain inner.policy() instead of policy_py_attached (clear, shrink_to_fit, popitem, len(cache), bool(cache), capacity, …) can hang free-threaded builds under contention when another thread needs a stop-the-world (GC, starting a thread). It reproduces on main without any iteration. I'll open a separate issue or PR for it, since it's a different fix.

Eight threads mix get/set/pop with walking keys()/values()/items() and
calling len()/bool() on the views, in a child process so a fault fails the
test instead of killing pytest. items() must yield key -> key pairs without
duplicates in one pass, which also catches stale reads that don't fault.

On free-threaded builds this crashes with SIGSEGV on main (FIFO, LFU, LRU
and TTL in every local run, VTTL in some). faulthandler dumps all threads
if the child hangs.
Views (keys/values/items, LFU items_with_frequency, TTL items_with_expire)
hold raw positions into the policy storage and rely on the generation
version to notice changes. They compared the generation without holding
the cache lock and then read, so on free-threaded builds another thread
could free or move entries in between (awolverp#94 reported this crash; awolverp#98 made
replace() bump the generation, but the check-then-read race remained).

- implement_view_guard! now generates lock_and_check(): take the cache
  lock, then compare the generation, and return the guard so it is held
  while reading. The unlocked check_generation() is removed, so every view
  method goes through the lock (__next__, __len__, and TTL/VTTL __bool__).
- View fields hold the typed cache (Py<PyXCache>) to reach its lock.
- TTL view constructors created the iterator and read the generation under
  two separate locks; a mutation in between made stale positions look
  current. Both now happen under one lock, as VTTL already did.
- docs: iterator methods take the cache lock; calling them from code that
  runs inside an operation on the same cache (a key's __eq__, a value's
  __del__) blocks, like cache[key] there.

Refs awolverp#94
@awolverp

Copy link
Copy Markdown
Owner

Thanks ❤️ This PR will be better with a few changes.
I will merge it, make the improvements, and then release a new version.

@awolverp
awolverp merged commit c05acd4 into awolverp:main Oct 28, 2026
12 checks passed
@greenclaw
greenclaw deleted the fix/iterator-lock branch October 29, 2026 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants