diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 944a91ed755..c30f6960125 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -730,6 +730,7 @@ peps/pep-0849.rst @JelleZijlstra peps/pep-0850/ @willingc @rgommers peps/pep-0850.rst @willingc @rgommers peps/pep-0851.rst @StanFromIreland +peps/pep-0852.rst @ZeroIntensity # ... peps/pep-2026.rst @hugovk # ... diff --git a/peps/pep-0852.rst b/peps/pep-0852.rst new file mode 100644 index 00000000000..bf430a89798 --- /dev/null +++ b/peps/pep-0852.rst @@ -0,0 +1,815 @@ +PEP: 852 +Title: Asynchronous functions in the C API +Author: Peter Bierma +Discussions-To: Pending +Status: Draft +Type: Standards Track +Created: 02-Nov-2026 +Python-Version: 3.16 +Post-History: `14-Jan-2023 `__, + `09-Jan-2024 `__, + `12-Apr-2024 `__ + + +Abstract +======== + +This PEP describes a new suite of functions in the C API to interact with +:term:`coroutines `. For example: + +.. code-block:: c + + #include + + static int + return_result(PyObject *asyncobj, PyObject *result) + { + PyAsync_SetResult(asyncobj, result); + return 1; + } + + /* Equivalent to: async def await_from_c(coro): return await coro */ + static PyObject * + await_from_c(PyObject *self, PyObject *coro) /* METH_O */ + { + PyObject *asyncobj = PyAsync_New(); + if (asyncobj == NULL) { + return NULL; + } + if (PyAsync_AddAwait(asyncobj, coro, return_result, NULL) < 0) { + PyAsync_Cancel(asyncobj); + Py_DECREF(asyncobj); + return NULL; + } + return asyncobj; + } + + +Motivation +========== + + +The C API has no easy way to invoke asynchronous code +----------------------------------------------------- + +Currently, Python's C API provides no simple interface for defining :term:`coroutine +functions `. This lack of API support has not gone unnoticed by +users. For example, ``mypyc`` still `does not support all asynchronous features +`__, because they are forced to +build coroutine types from scratch, which is difficult to do correctly. + +This problem is not unique to ``mypyc``; it shows up across the extension +ecosystem: + +- PyO3 has had an `open tracking issue + `__ for ``async fn`` support since + 2021. Its experimental support relies on a coroutine type implemented by + PyO3 itself, which still has gaps; for example, it `does not support + `__ the asynchronous context + manager protocol. +- A contributor to ``pydantic-core`` `asked + `__ how to await a + Python coroutine from Rust, and was told by a maintainer that moving the + asynchronous logic out of Python "is *really* hard". +- pybind11 has had an `open feature request + `__ for defining + asynchronous functions since 2020, in which the author resorted to + hand-writing their own awaitable class in C++. + +For extension developers, the current solution to this problem is to do one +of three things: + +1. Don't write the asynchronous code in C, and instead define a pure-Python + function that handles the :keyword:`await`\ s, and then call an + event-loop-specific API from C. Given that a common use case for building + extension modules is to improve performance, this can be a step backwards + for developers. +2. Implement a type that implements the :term:`coroutine` protocol on its own, + complete with all the necessary logic for a state machine. This is possible, + but non-trivial and bad for extension maintenance; C API users should not + have to learn about the inner workings of generators in order to use a + built-in syntax construct. This is the root of ``mypyc``'s problem, for example. +3. Use a binding generator that has done one of the above. To `quote + `__ + Petr Viktorin (on a separate proposal): "CPython should have a usable and + stable C API for all stakeholders, and a list of a few blessed users goes + against that goal." In other words, CPython should aim to make its C API + usable (and ergonomic) where possible -- not encourage users to switch to + binding generators. + +All of these options are less than ideal, and as a result, there is support +from both Python users and core developers for adding such a C API to Python. +To `quote `__ Marc-André Lemburg: + + I very much do think that we need a core maintained async C API in CPython. + + We currently don’t have a way to write Python C extensions which work in + async mode without using a shim written in Python which takes care of + the interactions with the asyncio (and other) event loops. + + +.. _pep-852-no-third-party: + +A third-party library is bad for both maintainers and users +----------------------------------------------------------- + +The API described in this PEP is heavily influenced by the third-party +`PyAwaitable `__ library -- but why does that +need to come into CPython at all? + +First and foremost, adding a dependency to a C extension typically has a very +high bar. In most cases, it's deemed safer to pick one of the solutions described +above, which each come with their own set of problems. + +Second, distributing native dependencies is extremely difficult in Python's extension +ecosystem. As of recently, extension modules are increasingly being written +using C API binding libraries and not the C API itself. Unfortunately, these +binding libraries cannot wrap APIs that don't come directly from CPython, so +native libraries are effectively unusable to projects using other C API bindings. +In fact, PyAwaitable uses a vendoring technique that would make it functionally +impossible to use from something such as PyO3. + +There have been `some `__ `discussions +`__ about making this easier (both of which +arose *because* PyAwaitable was so difficult to distribute), but in general, fixing +this problem seems like a much more massive proposal than what is currently being +proposed. An asynchronous C API in Python's core would be useful, and adding +one to CPython is much simpler and easier for both maintainers and users than +trying to solve a more generic problem in the packaging ecosystem; in other words, +to quote :pep:`20`, "practicality beats purity". + + +Why was a third-party library created in the first place? +********************************************************** + +When PyAwaitable was first created in early 2024, a relatively simple +implementation was possible without changes to the CPython core, and as +such, PyAwaitable was better fit as a library for the time being -- though +always with the intent of eventually becoming part of the public C API. +The idea was that a third-party library would iron out potential design +flaws in the API as a whole before they were immortalized into CPython, +and this was largely successful (see :ref:`the deviations from PyAwaitable +`); as such, the third-party library has served its purpose, +and it is now time to add it to the C API. + +It is also worth noting that the reference implementation of this proposal does +require the use of some internal helper functions (that PyAwaitable, as a +third-party library, had to copy and work around), which is more evidence that PyAwaitable +is a good fit for CPython, as one of the original arguments against adding such +an API was that it was implementable using only the public C API. + + +The C API is allowed to be ergonomic where possible +*************************************************** + +When discussed in early 2024, the main argument against this proposal was that +it was implementable using only Python's public C API. To `quote `__ +Petr Viktorin: + + To set expectations: I don’t think this needs to be in CPython itself. + I know that sounds disappointing, but after all, Python is made to be + extendable; we want to have interesting projects outside the core. + +Since then, there is a much larger precedent for adding convenience functions +to Python's C API (as in, functions that can be implemented in terms of the +existing C API or in standard C). For example: + +- There have been numerous attribute access helpers added in recent years, such + as :c:func:`PyObject_GetOptionalAttr`, :c:func:`PySys_GetAttr`, + :c:func:`PyImport_ImportModuleAttr`, and related functions. +- :c:type:`PyMutex`, while useful for the :term:`free threading` effort, is + a generic locking interface that could have been implemented outside CPython's + core. +- Several private APIs that were promoted to public, such as the + :ref:`c-api-time`. +- The APIs described by :pep:`788` can largely be implemented through + :mod:`atexit` callbacks. + +APIs like these were added because Python's C API should be easy to use where +possible -- merely because something *can* be implemented using the +public C API does not mean that it's the best option for users, especially +given the difficulty of distributing native code that wraps Python's C API. +When something is tightly coupled to a core language feature (in this case, +``async``/``await``), it is typically a better fit for the C API and not for +a third-party library. + + +Specification +============= + +The following APIs are available through ``Python.h`` and require an +:term:`attached thread state`. Unless otherwise specified, an *asyncobj* +argument must be a :c:type:`PyAsync` object; passing any other object is +undefined behavior. Callbacks receive :term:`borrowed references ` to their arguments. + + +Async objects +------------- + +.. c:type:: PyAsync + + An opaque Python object that implements the + :class:`~collections.abc.Coroutine` protocol. A :c:type:`PyAsync` + object stores a queue of asynchronous operations and callbacks to + execute when it is awaited. + + Awaiting a :c:type:`PyAsync` object runs its queued operations in + order. The object suspends whenever an operation suspends, and + completes once the queue is empty, returning the result set with + :c:func:`PyAsync_SetResult` (``None`` by default). A :c:type:`PyAsync` + object is recognized by :func:`asyncio.iscoroutine` and + :func:`inspect.isawaitable`, and can be passed to + :func:`asyncio.create_task`. + + The type cannot be subclassed or instantiated from Python, and is not + exposed in any Python module. + + +.. c:var:: PyTypeObject PyAsync_Type + + The type object for :c:type:`PyAsync` objects. + + +.. c:function:: int PyAsync_CheckExact(PyObject *obj) + + Return nonzero if *obj* is a :c:type:`PyAsync` object, and ``0`` otherwise. + This function cannot fail. + + +.. c:function:: PyObject *PyAsync_New(void) + + Create a new, empty :c:type:`PyAsync` object. Return a new + :term:`strong reference` on success, or ``NULL`` with an exception set + on failure. + + By default, awaiting the object returns ``None``. The result can be changed + with :c:func:`PyAsync_SetResult`. An object can only be awaited once; + attempting to reuse a completed object raises :exc:`RuntimeError`. + + Destroying an object that was never awaited, cancelled with + :c:func:`PyAsync_Cancel`, or closed emits a :exc:`RuntimeWarning`, as + destroying a native coroutine that was never awaited does. + + +.. c:function:: int PyAsync_SetResult(PyObject *asyncobj, PyObject *result) + + Set the object returned by awaiting *asyncobj* to *result*. + *result* must not be ``NULL``. + + This function stores a strong reference to *result*, releasing any + previously stored result. It cannot fail and returns ``0``. + + +Coroutine protocol +------------------ + +A :c:type:`PyAsync` object behaves as a native coroutine object does: + +* ``send(value)`` resumes the object. *value* must be ``None`` if the + object has not started. The object raises :exc:`StopIteration` carrying + its result when it completes. +* ``throw(exc)`` raises *exc* inside the currently awaited operation, as + :keyword:`await` would. If the operation does not handle it, the + operation's error callback is invoked with the exception, and so on as + described under :c:func:`PyAsync_AddAwait`. If the object has not started, + or is between operations, the exception propagates and the object + completes. +* ``close()`` throws :exc:`GeneratorExit` into the object. The currently + awaited operation is closed, pending error callbacks and + :keyword:`async with` exits observe :exc:`GeneratorExit`, and the object + completes. Closing an object that has not started or has completed does + nothing. + +A :c:type:`PyAsync` object may not be resumed while it is already executing; +doing so raises :exc:`ValueError`. Queue and storage mutations are +serialized per object, so on the :term:`free-threaded build` it is safe to +add operations or set values from another thread, but an object can only be +driven by one thread at a time. + + +Awaiting and callbacks +---------------------- + +Callbacks report their outcome through their return value. Every callback +must set an exception if and only if it reports failure; a callback that +returns success with an exception set, or failure without one, causes the +:c:type:`PyAsync` object to raise :exc:`SystemError`. + +.. c:type:: int (*PyAsync_Callback)(PyObject *asyncobj, PyObject *result) + + The type of a result callback. *asyncobj* is the :c:type:`PyAsync` + object executing the callback, and *result* is the value returned by + the awaited operation. + + The callback must return one of the following: + + * ``1`` on success. + * ``0`` with an exception set to invoke the associated error callback, + or propagate the exception if no error callback was provided. + * ``-1`` with an exception set to propagate the exception directly, + skipping the associated error callback. + + +.. c:type:: int (*PyAsync_Error)(PyObject *asyncobj, PyObject *exception) + + The type of an error callback. *exception* is the exception raised by + an awaited operation or its result callback. The error callback is + invoked with no exception set. + + The callback must return one of the following: + + * ``1`` to handle the exception and continue executing queued operations. + * ``0`` to restore and propagate the original exception. + * ``-1`` with a new exception set to propagate that exception instead. + + +.. c:type:: int (*PyAsync_Defer)(PyObject *asyncobj) + + The type of a deferred callback, registered with + :c:func:`PyAsync_DeferAwait`. The callback must return ``1`` on success, + or ``0`` with an exception set to stop execution and propagate the + exception to the caller awaiting *asyncobj*. + + +.. c:function:: int PyAsync_AddAwait(PyObject *asyncobj, PyObject *awaitable, PyAsync_Callback result_callback, PyAsync_Error error_callback) + + Append *awaitable* to the operations to await when *asyncobj* executes. + Operations execute in the order they were added. This function stores a + strong reference to *awaitable*, which must be a non-``NULL`` + :term:`awaitable` object distinct from *asyncobj*; anything else is + rejected with :exc:`TypeError` when this function is called, not when + *asyncobj* is awaited. + + On successful completion, *result_callback* is invoked with *asyncobj* + and the operation's return value. If *result_callback* is ``NULL``, the + return value is discarded. + + If the operation raises an exception, or *result_callback* returns ``0``, + *error_callback* is invoked. If *error_callback* is ``NULL``, the exception + propagates to the caller awaiting *asyncobj*. + + Callbacks may append further operations to *asyncobj*; they are added + to the end of the queue and run after all previously queued operations. + Adding operations to a completed object raises :exc:`RuntimeError`. + + Return ``0`` on success, or ``-1`` with an exception set on failure. + + +.. c:function:: int PyAsync_AddExpr(PyObject *asyncobj, PyObject *expr, PyAsync_Callback result_callback, PyAsync_Error error_callback) + + Like :c:func:`PyAsync_AddAwait`, but steals a reference to *expr* on + both success and failure. If *expr* is ``NULL``, return ``-1`` without + changing the current exception, which is assumed to have been set by + whatever produced *expr*. + + Otherwise, return ``0`` on success, or ``-1`` with an exception set + on failure. + + +.. c:function:: int PyAsync_DeferAwait(PyObject *asyncobj, PyAsync_Defer callback) + + Append a synchronous callback to the operations executed when *asyncobj* + is awaited. *callback* must not be ``NULL``. It is invoked with *asyncobj* + after previously queued operations complete, and may queue further + operations. + + Return ``0`` on success, or ``-1`` with an exception set on failure. + + +.. c:function:: void PyAsync_Cancel(PyObject *asyncobj) + + Clear the queued operations and callbacks, releasing their references + and discarding the current await iterator. This also suppresses the + :exc:`RuntimeWarning` for an object that was never awaited. + + This function may be called from a callback, in which case the object + completes once the callback returns, unless the callback adds new + operations. Further operations may be added to *asyncobj* until it + completes. A pending :keyword:`async with` exit (see + :c:func:`PyAsync_AsyncWith`) is not cleared and still runs. + + The currently awaited operation, if any, is dropped without being + closed. This function is unrelated to :mod:`asyncio` task cancellation, + which reaches the object through ``throw()``. It cannot fail. + + +.. c:function:: int PyAsync_AsyncWith(PyObject *asyncobj, PyObject *context_manager, PyAsync_Callback result_callback, PyAsync_Error error_callback) + + Append an :keyword:`async with` block using *context_manager*. When the + operation runs, the ``__aenter__`` method of *context_manager* is + awaited, *result_callback* is invoked, any operations it queued are + awaited, and finally the ``__aexit__`` method is awaited. + + *result_callback*, which may be ``NULL``, is the body of the block. It + receives a :c:type:`PyAsync` object representing the block and the + value returned by ``__aenter__``. The block object shares its result + and value storage with *asyncobj* (:c:func:`PyAsync_SetResult`, + :c:func:`PyAsync_SetValue` and the other storage functions act on + *asyncobj*), but has its own operation queue: operations added to the + block object run inside the block, before ``__aexit__``. Operations + added to *asyncobj* run after the block, as they would in Python. + + Once ``__aenter__`` has completed, ``__aexit__`` is always awaited, + whether the block completes normally, *result_callback* or an + operation inside the block raises, an exception is thrown into the + object (including :exc:`asyncio.CancelledError`), or the object is + closed. On an exception, ``__aexit__`` receives the exception and may + suppress it by returning a true value. An exception from + ``__aenter__`` propagates without invoking ``__aexit__``. If + ``__aexit__`` itself raises, the block's exception becomes the + ``__context__`` of the new one. + + *error_callback* handles exceptions from the operation as a whole, + including any exception left unsuppressed by ``__aexit__``. It receives + *asyncobj* and the exception. If it is ``NULL``, the exception + propagates to the caller awaiting *asyncobj*. + + Blocks may be nested by calling this function on a block object. + + Return ``0`` on success, or ``-1`` with an exception set on failure. + + +Value storage +------------- + +Each :c:type:`PyAsync` object has one object slot and one arbitrary pointer +slot for storing state between callbacks. Both slots are initially ``NULL``. +Code that needs more than one object stores a tuple or other container. + +.. c:function:: int PyAsync_SetValue(PyObject *asyncobj, PyObject *value) + + Store a strong reference to *value* in the object slot, releasing the + previously stored reference. If *value* is ``NULL``, clear the slot. + + This function cannot fail and returns ``0``. + + +.. c:function:: PyObject *PyAsync_GetValue(PyObject *asyncobj) + + Return a new :term:`strong reference` to the object stored in the object + slot, or ``NULL`` if the slot is empty. This function cannot fail and + does not set an exception when the slot is empty. + + +.. c:function:: int PyAsync_SetArbValue(PyObject *asyncobj, void *value) + + Store *value* in the arbitrary pointer slot, replacing the previous + pointer. *value* may be ``NULL``. The caller is responsible for the + lifetime of any memory referenced by the pointer. + + This function cannot fail and returns ``0``. + + +.. c:function:: void *PyAsync_GetArbValue(PyObject *asyncobj) + + Return the pointer stored in the arbitrary pointer slot, or ``NULL`` + if the slot is empty. This function cannot fail and does not set an + exception when the slot is empty. + + +Limited API +----------- + +All APIs in this PEP are part of the :ref:`Limited API ` +and the :ref:`Stable ABI ` as of Python 3.16. + + +Rationale +========= + + +An API for coroutine functions, not futures +------------------------------------------- + +This proposal aims to add a C API for generic ``async``/``await`` constructs, *not* +an API for describing all capabilities of :mod:`asyncio`. While this PEP may +be reason for adding an ``asyncio`` C API in the future, its only goal is to +make :keyword:`async def` functions possible to implement in the C API. + + +.. _pep-852-deviations: + +Deviations from PyAwaitable +--------------------------- + +The primary purpose of making PyAwaitable a third-party library and not a +formal C API proposal (yet) was to identify the best design choices based on +user experience and feedback. As such, this proposal does not copy PyAwaitable's +API exactly, and instead describes a new API without the original design flaws +of PyAwaitable. + + +``PyAsync`` instead of ``PyAwaitable`` +************************************** + +The most obvious deviation from the original library is the use of the +``PyAsync`` prefix over ``PyAwaitable``. This is because the original +"PyAwaitable" name is a historical artifact; it was chosen as +a parallel to the ``PyCoro`` prefix, with the idea that a ``PyCoro`` was +analogous to a native ``coroutine`` object, while a ``PyAwaitable`` would +be analogous to a native ``awaitable`` object. Given that this API (and new +versions of PyAwaitable) support all of the :class:`~collections.abc.Coroutine` +protocol (in contrast to the :class:`~collections.abc.Awaitable` protocol), +this name is now misleading, and thus the rename to ``PyAsync`` felt necessary. + + +Simpler storage for callback values +*********************************** + +This API uses a single ``PyObject *`` reference and ``void *`` pointer for value +storage between callbacks instead of variadic argument lists that were stored as +arrays. PyAwaitable used the latter because it was optimized for ergonomics when +writing pure C code, but since this API is also designed to be accessible from +binding generators (and FFIs) as well, a single pointer for each felt like the +better call. PyAwaitable's array system also turned out to be a source of +complexity and bugs, and avoiding extra complexity for CPython where possible +is ideal. + +This API also returns a :term:`strong reference` from ``GetValue`` instead of a +:term:`borrowed reference`. PyAwaitable used the latter because it predated the +acceptance of :pep:`703`, and thus any concurrency implications were not +considered. + + +Different callback return values +******************************** + +In PyAwaitable, callbacks may return ``0``, ``-1``, and ``-2``, which correspond +to ``1``, ``0``, and ``-1`` in this proposal, respectively. This is also a historical +artifact; PyAwaitable did not originally have the ``-2`` return value, and changing +the meaning of ``0`` and ``-1`` could not be done for the sake of backwards +compatibility. + + +Backwards Compatibility +======================= + +This PEP has no known backwards compatibility concerns. + + +Security Implications +===================== + +This PEP has no known security concerns. + + +How to Teach This +================= + +The new APIs will be documented in the C API documentation. +A tutorial can also be added based on the tutorial present in PyAwaitable's +documentation. + + +Examples +-------- + + +Example: Forwarding an asynchronous result +****************************************** + +An extension method can call an asynchronous function and forward the +result of awaiting it. :c:type:`PyAsync` handles suspension and resumption +for the extension. + +.. code-block:: c + + static int + return_result(PyObject *asyncobj, PyObject *result) + { + PyAsync_SetResult(asyncobj, result); + return 1; + } + + /* Equivalent to: async def call_async(func): return await func() */ + static PyObject * + call_async(PyObject *self, PyObject *func) /* METH_O */ + { + PyObject *asyncobj = PyAsync_New(); + if (asyncobj == NULL) { + return NULL; + } + if (PyAsync_AddExpr(asyncobj, PyObject_CallNoArgs(func), + return_result, NULL) < 0) { + PyAsync_Cancel(asyncobj); + Py_DECREF(asyncobj); + return NULL; + } + return asyncobj; + } + + +Example: Keeping state across an await +************************************** + +To implement ``return (await get_number()) * factor``, store *factor* with +:c:func:`PyAsync_SetValue` before queuing ``get_number()`` with this callback. +The :c:type:`PyAsync` object keeps the factor alive while execution is +suspended. The callback releases the strong reference returned by +:c:func:`PyAsync_GetValue` after using it. + +.. code-block:: c + + static int + multiply_result(PyObject *asyncobj, PyObject *value) + { + PyObject *factor = PyAsync_GetValue(asyncobj); + PyObject *result = PyNumber_Multiply(value, factor); + Py_DECREF(factor); + if (result == NULL) { + return 0; + } + PyAsync_SetResult(asyncobj, result); + Py_DECREF(result); + return 1; + } + + +Example: Handling a timeout +*************************** + +In ``call_async`` above, replace the final ``NULL`` argument to +:c:func:`PyAsync_AddExpr` with ``timeout_to_none``. Awaiting the result +then returns ``None`` on :exc:`TimeoutError`. Returning ``0`` from the +callback lets other exceptions propagate. + +.. code-block:: c + + static int + timeout_to_none(PyObject *asyncobj, PyObject *exception) + { + if (!PyErr_GivenExceptionMatches(exception, PyExc_TimeoutError)) { + return 0; + } + PyAsync_SetResult(asyncobj, Py_None); + return 1; + } + +For example, :func:`asyncio.timeout` can impose a deadline on an operation: + +.. code-block:: python + + import asyncio + + async def slow_operation(): + async with asyncio.timeout(0.01): + return await asyncio.sleep(1, result=42) + + assert asyncio.run(call_async(slow_operation)) is None + + +Example: Scheduling a dependent read +************************************ + +Suppose a protocol reader exposes asynchronous ``read_size()`` and +``readexactly(size)`` methods. Store the reader with +:c:func:`PyAsync_SetValue`, then queue ``reader.read_size()`` with this +callback. The first read's result determines the next asynchronous call, +which reuses ``return_result`` to return the completed packet. + +.. code-block:: c + + static int + read_body(PyObject *asyncobj, PyObject *size) + { + PyObject *reader = PyAsync_GetValue(asyncobj); + PyObject *coro = PyObject_CallMethod(reader, "readexactly", "O", size); + Py_DECREF(reader); + return PyAsync_AddExpr(asyncobj, coro, return_result, NULL) == 0; + } + + +Example: Acquiring an async lock for native work +************************************************ + +An asynchronous context manager can acquire a lock and return the protected +dictionary to the result callback: + +.. code-block:: python + + import asyncio + from contextlib import asynccontextmanager + + @asynccontextmanager + async def locked_cache(lock, cache): + async with lock: + yield cache + +The C callback updates that dictionary while the lock is held: + +.. code-block:: c + + static int + mark_ready(PyObject *asyncobj, PyObject *cache) + { + return PyDict_SetItemString(cache, "ready", Py_True) == 0; + } + + static PyObject * + update_locked(PyObject *self, PyObject *lock) /* METH_O */ + { + PyObject *asyncobj = PyAsync_New(); + if (asyncobj == NULL) { + return NULL; + } + if (PyAsync_AsyncWith(asyncobj, lock, mark_ready, NULL) < 0) { + PyAsync_Cancel(asyncobj); + Py_DECREF(asyncobj); + return NULL; + } + return asyncobj; + } + +Python code passes the lock and dictionary through the context manager: + +.. code-block:: python + + async def main(): + lock, cache = asyncio.Lock(), {} + await update_locked(locked_cache(lock, cache)) + assert cache["ready"] is True + assert not lock.locked() + + asyncio.run(main()) + +:c:func:`PyAsync_AsyncWith` invokes the result callback after acquiring the +lock, and releases the lock even if the dictionary update raises an exception. + + +Reference Implementation +======================== + +A reference implementation of this PEP can be found +`here `__. + + +Rejected Ideas +============== + + +A single callback with a state number +------------------------------------- + +Instead of one callback per :keyword:`await`, Serhiy Storchaka suggested in +the `2023 discussion `__ a single +callback that is re-entered with a state number and dispatches with a +``switch``. This is closer to a generator API than to :keyword:`await`, and +pushes the state machine onto the user, which is the burden this PEP aims to +remove. Users who prefer that style can implement it on top of the proposed +API. + +In addition, a single function with a ``switch`` would likely encourage users +to use ``static`` variables, which have thread-safety implications that would +likely go against the efforts of the :term:`free threading` project. + + +C++20 coroutines +---------------- + +Steve Dower suggested in the same discussion that native coroutine support +is better left to languages that have it, such as C++20 via pybind11. The C +API is a C API, and most of the extension ecosystem (including binding +generators) cannot consume C++ interfaces. Nothing in this PEP prevents a +C++ wrapper from offering ``co_await`` on top of :c:type:`PyAsync`. + + +Reusing native coroutine objects +-------------------------------- + +Native coroutine objects (instances of :class:`types.CoroutineType`) are bound +to a Python frame with bytecode, and thus there is no way to construct one +around C code without emulating a frame. A new type is simpler and avoids +exposing frame internals. + + +Keeping the API in a third-party library +---------------------------------------- + +See :ref:`pep-852-no-third-party`. + + +Open Issues +=========== + +TBD. + + +Acknowledgements +================ + +Thanks to Petr Viktorin for his help with the original PyAwaitable design +discussion, and to all users of PyAwaitable over the past few years. + + +Change History +============== + +TBD. + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive.