dir() can raise RuntimeError: dictionary changed size during iteration on the free-threaded build

Bug report

Bug description:

I believe there is an issue in the free-threaded build when using dir(). Apologies if this isn't considered an issue. I read https://docs.python.org/3/howto/free-threading-python.html#thread-safety and my understanding of:

Built-in types like dict, list, and set use internal locks to protect against concurrent modifications in ways that behave similarly to the GIL.

is that this is something that could be fixed. Possibly this was missed when implementing #114508.

Calling dir() on a class (or on an instance of it) can fail with RuntimeError: dictionary changed size during iteration if another thread concurrently performs a read that lazily stores something in the class __dict__. The most common such read on 3.14+ is the first access to __annotations__ (e.g. through typing.get_type_hints()), which stores __annotations_cache__ on the class.

Same can happen with e.g. copy.copy(), which will set __slotnames__ via copyreg._slotnames().

MRE

import sys
import threading
import typing
from concurrent.futures import ThreadPoolExecutor


def make_class():
    class C:
        x: int

    # Insert many elements in the class dict so that `dir()` spends more time
    # iterating over it (the race also happens without this, just less often):
    for i in range(1000):
        setattr(C, f'attr_{i}', i)
    return C


def main(rounds: int = 100, readers: int = 8) -> None:
    failures = 0
    for _ in range(rounds):
        C = make_class()
        barrier = threading.Barrier(readers + 1)
        errors = []

        def read():
            barrier.wait()
            try:
                dir(C)
            except RuntimeError as e:
                errors.append(e)

        def write():
            barrier.wait()
            # First access to `C.__annotations__` (here, through `get_type_hints()`) stores
            # `__annotations_cache__` in the class `__dict__`.
            typing.get_type_hints(C)

        with ThreadPoolExecutor(max_workers=readers + 1) as executor:
            futures = [executor.submit(read) for _ in range(readers)] + [executor.submit(write)]
            for future in futures:
                future.result()
        failures += len(errors)
        if errors and failures == len(errors):
            print(f'{type(errors[0]).__name__}: {errors[0]}')

    gil = sys._is_gil_enabled()
    print(f'{sys.version.split()[0]} ({"GIL" if gil else "free-threaded"}): {failures}/{rounds * readers} dir() calls failed')


if __name__ == '__main__':
    main()

$ python3.15t mre.py
RuntimeError: dictionary changed size during iteration
3.15.0rc2 (free-threaded): 108/800 dir() calls failed
$ PYTHON_GIL=1 python3.14t mre.py
3.15.0rc2 (GIL): 0/800 dir() calls failed

Analysis

AI analysis pointed me at the following. I'm not knowledgeable to know if this is the actual issue, but can help for initial debugging.

The reader: `dir()`

type.__dir__() and object.__dir__() collect names with merge_class_dict(), which fetches cls.__dict__ and merges it into a fresh dict with PyDict_Update():

cpython/Objects/typeobject.c

Lines 6965 to 6982 in 894af95

    
           merge_class_dict(PyObject *dict, PyObject *aclass) 
        
           { 
        
               PyObject *classdict; 
        
               PyObject *bases; 
        
               assert(PyDict_Check(dict)); 
        
               assert(aclass); 
        
               /* Merge in the type's dict (if any). */ 
        
               if (PyObject_GetOptionalAttr(aclass, &_Py_ID(__dict__), &classdict) < 0) { 
        
                   return -1; 
        
               } 
        
               if (classdict != NULL) { 
        
                   int status = PyDict_Update(dict, classdict); 
        
                   Py_DECREF(classdict); 
        
                   if (status < 0) 
        
                       return -1; 
        
               }

cls.__dict__ is a mappingproxy, not a dict, so dict_merge() doesn't take the fast path (which holds the critical sections of both dicts). It takes the generic path instead, which only holds the critical section of the destination dict:

cpython/Objects/dictobject.c

Lines 4309 to 4324 in 894af95

    
           if (PyAnyDict_Check(b) && (Py_TYPE(b)->tp_iter == dict_iter)) { 
        
               PyDictObject *other = (PyDictObject*)b; 
        
               int res; 
        
               Py_BEGIN_CRITICAL_SECTION2(a, b); 
        
               assert(can_modify_dict(mp)); 
        
               res = dict_dict_merge((PyDictObject *)a, other, override, dupkey); 
        
               ASSERT_CONSISTENT(a); 
        
               Py_END_CRITICAL_SECTION2(); 
        
               return res; 
        
           } 
        
           else { 
        
               /* Do it the generic, slower way */ 
        
               Py_BEGIN_CRITICAL_SECTION(a); 
        
               assert(can_modify_dict(mp)); 
        
               PyObject *keys = PyMapping_Keys(b);

The generic path gets the keys with PyMapping_Keys(), which for a non-dict calls .keys() and turns the returned dict_keys view into a list by iterating over it:

cpython/Objects/abstract.c

Lines 2433 to 2468 in 894af95

    
           method_output_as_list(PyObject *o, PyObject *meth) 
        
           { 
        
               PyObject *it, *result, *meth_output; 
        
               assert(o != NULL); 
        
               meth_output = PyObject_CallMethodNoArgs(o, meth); 
        
               if (meth_output == NULL || PyList_CheckExact(meth_output)) { 
        
                   return meth_output; 
        
               } 
        
               it = PyObject_GetIter(meth_output); 
        
               if (it == NULL) { 
        
                   PyThreadState *tstate = _PyThreadState_GET(); 
        
                   if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError)) { 
        
                       _PyErr_Format(tstate, PyExc_TypeError, 
        
                                     "%T.%U() must return an iterable, not %T", 
        
                                     o, meth, meth_output); 
        
                   } 
        
                   Py_DECREF(meth_output); 
        
                   return NULL; 
        
               } 
        
               Py_DECREF(meth_output); 
        
               result = PySequence_List(it); 
        
               Py_DECREF(it); 
        
               return result; 
        
           } 
        
           PyObject * 
        
           PyMapping_Keys(PyObject *o) 
        
           { 
        
               if (o == NULL) { 
        
                   return null_error(); 
        
               } 
        
               if (PyAnyDict_CheckExact(o)) { 
        
                   return PyDict_Keys(o); 
        
               } 
        
               return method_output_as_list(o, &_Py_ID(keys));

That iteration over the class __dict__ happens without holding its critical section, and the dict iterator checks the size at every step, so any concurrent insertion into the class __dict__ raises RuntimeError.

The writer: a lazy cache stored on the class by a read

The type.__annotations__ getter stores the evaluated annotations as __annotations_cache__ in the class __dict__ on first access:

cpython/Objects/typeobject.c

Lines 2194 to 2201 in 894af95

    
           if (annotations) { 
        
               int result = PyDict_SetItem( 
        
                       dict, &_Py_ID(__annotations_cache__), annotations); 
        
               if (result) { 
        
                   Py_CLEAR(annotations); 
        
               } else { 
        
                   PyType_Modified(type); 
        
               }

If the class has no __annotate__ function, the type.__annotate__ getter (called by the above) also stores __annotate_func__ = None:

cpython/Objects/typeobject.c

Lines 2086 to 2092 in 894af95

    
           else { 
        
               annotate = Py_None; 
        
               int result = PyDict_SetItem(dict, &_Py_ID(__annotate_func__), annotate); 
        
               if (result < 0) { 
        
                   Py_DECREF(dict); 
        
                   return NULL; 
        
               }

Other stdlib operations do the same kind of lazy insertion into a class __dict__, so the issue is not specific to annotations:

copy.copy(), copy.deepcopy() and pickling of an instance store __slotnames__ on the class, on every version, through _PyType_GetSlotNames() / copyreg._slotnames():

cpython/Objects/typeobject.c

Lines 7790 to 7795 in 894af95

    
               /* Use _slotnames function from the copyreg module to find the slots 
        
                  by this class and its bases. This function will cache the result 
        
                  in __slotnames__. */ 
        
               slotnames = PyObject_CallMethodOneArg( 
        
                       copyreg, &_Py_ID(_slotnames), (PyObject *)cls); 
        
               Py_DECREF(copyreg);

cpython/Lib/copyreg.py

Lines 155 to 158 in 894af95

    
           # Cache the outcome in the class if at all possible 
        
           try: 
        
               cls.__slotnames__ = names 
        
           except:

The first instantiation of a concrete subclass of a typing.Protocol replaces its __init__():

cpython/Lib/typing.py

Lines 1925 to 1931 in 894af95

    
           for base in cls.__mro__: 
        
               init = base.__dict__.get('__init__', _no_init_or_replace_init) 
        
               if init is not _no_init_or_replace_init: 
        
                   cls.__init__ = init 
        
                   break 
        
           else: 
        
               # should not happen

Possible fix

In dict_merge() (or in PyMapping_Keys()), unwrap a mappingproxy whose underlying mapping is a dict and use the locked dict-to-dict path. This would fix dir(), dict(vars(cls)), {**vars(cls)} and everything built on them at once. The pure-Python loops over base.__dict__.items() in inspect and enum.Enum.__dir__ would still need to iterate over a copy.

CPython versions tested on:

3.15

Operating systems tested on:

macOS

Linked PRs

gh-157217: Fix dir() race when merging from a mappingproxy #157279

Analysis & Proposed Fix:

The issue diagnosis is spot on. When dir() calls merge_class_dict(), classdict is a mappingproxy (PyDictProxy_Type). Because PyAnyDict_Check(b) evaluates to false for proxies in dict_merge() (Objects/dictobject.c), CPython falls back to the generic PyMapping_Keys() iteration path, bypassing the critical section on the underlying dictionary.

Proposed Solution:
We can handle mappingproxy objects directly in dict_merge() by unwrapping the proxy to its underlying mapping before performing the critical section lock and merge:

Check if PyDictProxy_Check(b) is true.

Extract the underlying dict via ((PyDictProxyObject *)b)->mapping.

Pass the underlying dict to Py_BEGIN_CRITICAL_SECTION2(a, underlying_dict) and run dict_dict_merge().

This will fix thread-safety for dir(), dict(vars(cls)), and {**vars(cls)} in free-threaded builds without adding per-iteration lock overhead in Python land.

I can submit a PR with this fix and a regression test for test_free_threaded if the core devs agree with this approach.

pythongh-157217: Fix dir() race when merging from a mappingproxy

pythongh-157217: Export _PyDictProxy_GetMapping for dict_merge

pythongh-157217: Add free-threading dir() race test

pythongh-157217: Cover mappingproxy in dict.update tests

pythongh-157217: Add mappingproxy accessor for dict_merge

pythongh-157217: Add NEWS blurb for mappingproxy dict_merge fix

pythongh-157217: Add mappingproxy dict() and update tests

pythongh-157217: Add free-threading dir() race test

Add dictobject.c hunk for pythongh-157217

pythongh-157217: Drop unused mappingproxy helper from internal header

pythongh-157217: Follow-up hunk to unwrap mappingproxy locally

pythongh-157217: Unwrap mappingproxy locally in dict_merge

pythongh-157217: Fix dir() race when merging from a mappingproxy

pythongh-157217: Fix ruff F841/E731 in test_type.py for Lint/prek

ci: one-shot squash helper for pythongh-157217-fix

pythongh-157217: Fix dir() race when merging from a mappingproxy

Merge python/cpython main into pythongh-157217-fix

Complete pythongh-157217 mappingproxy race fix on current main

Merge branch 'main' into pythongh-157217-fix

This doesn't look like a bug to me. The interpreter raises RuntimeError: dictionary changed size during iteration because that's precisely what happens in the MRE. The fact that in a with-GIL build this cannot be exercised because of the lack of parallelism doesn't make this a bug.

This doesn't look like a bug to me. The interpreter raises RuntimeError: dictionary changed size during iteration because that's precisely what happens in the MRE. The fact that in a with-GIL build this cannot be exercised because of the lack of parallelism doesn't make this a bug.

So if this isn't a bug/isn't something that should be fixed, what am I misinterpreting in the docs section I mentioned?

From the docs you mention:

See Thread Safety Guarantees for the guarantees provided by built-in types.

Pointing to:

To safely iterate over a dictionary that may be modified by another thread, iterate over a copy:
# Make a copy to iterate safely
for key, value in d.copy().items():
    process(key)
Consider external synchronization when sharing dict instances across threads.

I have no idea why you have an issue with this case specifically, is it a problem that came up as a bug report in a typing library? Or is it just theoretical? The problem of concurrently calling dir() doesn't look like a big issue to me.

Pointing to:
To safely iterate over a dictionary that may be modified by another thread, iterate over a copy:
# Make a copy to iterate safely
for key, value in d.copy().items():
    process(key)

I believe this make sense if you are iterating over the items yourself using items(), but dir() is a builtin, it would feel odd to me to hold an external lock every time one wants to use dir() on a class.

is it a problem that came up as a bug report in a typing library? Or is it just theoretical?

It is theoretical (in the sense that I did not encounter the issue directly, but as seen in repro can still happen in practice), which I found when investigating other free-threaded race conditions in Pydantic. I think this is pretty rare in practice.

PyDict_Update() (used by type.__dir__()) documentation states:

In the free-threaded build, when b is a dict (with the standard iterator), both a and b are locked for the duration of the operation. When b is a non-dict mapping, only a is locked; b may be concurrently modified by another thread.

So in some way, the thread safety concern is documented (in our case, b would be the __dict__ of the dir() argument). But having dir() using PyDict_Update() is an implementation detail, and dir() could very much go through a different variant where it holds a lock for both dicts.

Navigation Menu

Repository navigation

`dir()` can raise `RuntimeError: dictionary changed size during iteration` on the free-threaded build #157217

Bug report

Bug description:

MRE

Analysis

The reader: `dir()`

The writer: a lazy cache stored on the class by a read

Other affected readers

Possible fix

CPython versions tested on:

Operating systems tested on:

Linked PRs

2 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	merge_class_dict(PyObject dict, PyObject aclass)
	{
	PyObject *classdict;
	PyObject *bases;

	assert(PyDict_Check(dict));
	assert(aclass);

	/* Merge in the type's dict (if any). */
	if (PyObject_GetOptionalAttr(aclass, &_Py_ID(__dict__), &classdict) < 0) {
	return -1;
	}
	if (classdict != NULL) {
	int status = PyDict_Update(dict, classdict);
	Py_DECREF(classdict);
	if (status < 0)
	return -1;
	}

	if (PyAnyDict_Check(b) && (Py_TYPE(b)->tp_iter == dict_iter)) {
	PyDictObject other = (PyDictObject)b;
	int res;
	Py_BEGIN_CRITICAL_SECTION2(a, b);
	assert(can_modify_dict(mp));
	res = dict_dict_merge((PyDictObject *)a, other, override, dupkey);
	ASSERT_CONSISTENT(a);
	Py_END_CRITICAL_SECTION2();
	return res;
	}
	else {
	/* Do it the generic, slower way */
	Py_BEGIN_CRITICAL_SECTION(a);
	assert(can_modify_dict(mp));

	PyObject *keys = PyMapping_Keys(b);

	method_output_as_list(PyObject o, PyObject meth)
	{
	PyObject it, result, *meth_output;

	assert(o != NULL);
	meth_output = PyObject_CallMethodNoArgs(o, meth);
	if (meth_output == NULL \|\| PyList_CheckExact(meth_output)) {
	return meth_output;
	}
	it = PyObject_GetIter(meth_output);
	if (it == NULL) {
	PyThreadState *tstate = _PyThreadState_GET();
	if (_PyErr_ExceptionMatches(tstate, PyExc_TypeError)) {
	_PyErr_Format(tstate, PyExc_TypeError,
	"%T.%U() must return an iterable, not %T",
	o, meth, meth_output);
	}
	Py_DECREF(meth_output);
	return NULL;
	}
	Py_DECREF(meth_output);
	result = PySequence_List(it);
	Py_DECREF(it);
	return result;
	}

	PyObject *
	PyMapping_Keys(PyObject *o)
	{
	if (o == NULL) {
	return null_error();
	}
	if (PyAnyDict_CheckExact(o)) {
	return PyDict_Keys(o);
	}
	return method_output_as_list(o, &_Py_ID(keys));

	if (annotations) {
	int result = PyDict_SetItem(
	dict, &_Py_ID(__annotations_cache__), annotations);
	if (result) {
	Py_CLEAR(annotations);
	} else {
	PyType_Modified(type);
	}

	else {
	annotate = Py_None;
	int result = PyDict_SetItem(dict, &_Py_ID(__annotate_func__), annotate);
	if (result < 0) {
	Py_DECREF(dict);
	return NULL;
	}

	/* Use _slotnames function from the copyreg module to find the slots
	by this class and its bases. This function will cache the result
	in __slotnames__. */
	slotnames = PyObject_CallMethodOneArg(
	copyreg, &_Py_ID(_slotnames), (PyObject *)cls);
	Py_DECREF(copyreg);

	# Cache the outcome in the class if at all possible
	try:
	cls.__slotnames__ = names
	except:

	for base in cls.__mro__:
	init = base.__dict__.get('__init__', _no_init_or_replace_init)
	if init is not _no_init_or_replace_init:
	cls.__init__ = init
	break
	else:
	# should not happen

dir() can raise RuntimeError: dictionary changed size during iteration on the free-threaded build #157217

Description

Bug report

Bug description:

MRE

Analysis

The reader: dir()

The writer: a lazy cache stored on the class by a read

Other affected readers

Possible fix

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

search1ofall-maker commented on Sep 10, 2026

2 remaining items

dpdani commented on Sep 25, 2026

Viicos commented on Sep 25, 2026

dpdani commented on Sep 28, 2026

Viicos commented on Sep 28, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`dir()` can raise `RuntimeError: dictionary changed size during iteration` on the free-threaded build #157217

The reader: `dir()`