Repository navigation
Zend async Scheduler ABI - #22561
Open
EdmondDantes wants to merge 81 commits into
Open
Zend async Scheduler ABI#22561EdmondDantes wants to merge 81 commits into
EdmondDantes wants to merge 81 commits into
Conversation
EdmondDantes
marked this pull request as draft
July 2, 2026 17:48
EdmondDantes
force-pushed
the
async-core
branch
4 times, most recently
from
July 2, 2026 18:11
615d050 to
ed9fe03
Compare
EdmondDantes
force-pushed
the
async-core
branch
10 times, most recently
from
July 3, 2026 09:58
f3b272d to
6e1a5ce
Compare
bwoebi
reviewed
Jul 3, 2026
bwoebi
reviewed
Jul 3, 2026
bwoebi
reviewed
Jul 3, 2026
bwoebi
reviewed
Jul 3, 2026
EdmondDantes
force-pushed
the
async-core
branch
5 times, most recently
from
July 16, 2026 08:21
3286ac0 to
9e95fa1
Compare
A provider registers once per process but wants async per request: it sets READY with ZEND_ASYNC_INITIALIZE in its RINIT, or when it registers at run time. Registration no longer sets READY. A request left OFF runs without a scheduler, so a run-time provider such as the ext-scheduler-hook bridge is not asked to launch before its PHP scheduler exists. TrueAsync gates its launch the same way.
…h ZEND_ASYNC_CANCEL_EX The comment said delivery is deferred; in TrueAsync a started coroutine cancelled safely is not interrupted at all: it becomes a zombie and runs to its end. ZEND_ASYNC_CANCEL_EX is the fork's macro that passes the flag.
…outine_count slot Nothing kept ZEND_ASYNC_ACTIVE_COROUTINE_COUNT up to date, so it always read 0. The scheduler knows its coroutines; the new slot asks it, and the macro gives 0 when no scheduler provides it. test_scheduler implements it and exposes it as TestScheduler\coroutineCount() for the test.
A coroutine without an object was allowed by the comments but had no use, and ZEND_COROUTINE_ADD_REF(), which new_coroutine tells callers to take, asserted on it. The fiber code now takes and drops its reference with ZEND_COROUTINE_ADD_REF()/ZEND_COROUTINE_RELEASE() instead of skipping a missing object.
The CLI evaluated that code without the launch and the run after main that php_execute_script_ex() gives a file, so the scheduler never started under php -r. The RFC launches it before the script's first line in every SAPI. -r code runs as a file does; -B, every -R line and -E share one main coroutine, and the scheduler runs once, after -E.
Under the scheduler, every coroutine that found the root buffer full awaited the same GC coroutine run and then called gc_adjust_threshold() with its count. N waiters raised the threshold by N steps and grew the buffer to match: 12000 coroutines left a threshold of 90020001 and a buffer with room for 90 million roots after one run. As in the TrueAsync core, a full buffer now records that a step is owed (adjust_threshold, replacing run_deferred) and the GC coroutine takes it once after its run, skipping a run that found the buffer drained. The record is set before the call, since our callers await the run. A run whose waiter was cancelled, or that was started where switching is blocked, now adjusts the threshold too; before, neither ever did. When the GC coroutine cannot be created, the caller takes the step at once, as TrueAsync does. A bailout that cuts the run short drops the record with the coroutine.
zend_gc_collect_cycles() removed the caller's live TMPVARs from the root buffer before creating the GC coroutine. When creation failed it returned 0 without re-rooting them, and they left GC tracking.
PHP prints the path with backslashes on Windows, so str_replace() did not find the one built with a forward slash and the test failed there.
…r the front A coroutine that fills the root buffer parks in the middle of an opcode until the GC coroutine's run ends, and so does every coroutine that finds the buffer full before the run starts. A scheduler that queues the run behind them keeps one parked fiber stack per such coroutine: 100000 coroutines exceed vm.max_map_count. The slot carries the engine's request in TrueAsync's zend_coroutine_priority: HI_PRIORITY for the collector's run, NORMAL for the destructor iterators, so a scheduler can put the run first without reordering the destructor passes. TrueAsync gives the priority the other way round, to its destructor coroutines, because its callers never wait for the run. ext/test_scheduler ignores the request. API version 3.
zend_gc_collect_cycles() in a coroutine returned GC_G(gc_collected), the count of the last run to finish. A scheduler that runs the GC coroutine before the coroutines already queued wakes the waiter behind them, and one of them can start and finish a second run first: the waiter then returned the second run's count. A run cancelled before it started returned the previous run's count too. The run stores its count in its coroutine's result, and the waiter holds a reference to the run to read it, released under gc_active: on a full buffer the root the release adds would start the next run and wait for it, and that run's release would start another. The await slot's caller holds the reference across the call and the slot takes none, so the provider's return releases nothing on the waiter; ext/test_scheduler's ts_await() no longer takes its own.
OPcache validates a cached script by its file time in whole seconds. The test rewrote one .inc file for each case, and when a case wrote it within the same second as an earlier case, its child ran the earlier case's cached script: on WINDOWS_X64_ZTS, where the children share the parent's OPcache memory, the exits case ran the throws case's code. opcache.file_cache keys by path too, so a reused path can go stale on other platforms as well.
zend_hash_update() runs the old value's destructor before it writes the new one: a destructor that grows the table leaves the write in the freed bucket array, one that unsets the key frees the value twice. Both the userland store's string keys and the internal context replaced values this way; the object keys already wrote the new value first and released the old one after. The new value is also copied before the lookup, which may grow the table a C caller's value points into. zend_async_context_destroy() released the store while the coroutine still pointed at it, so a value's destructor reached the store being freed.
…t ran The scheduler finishes such a coroutine without the fiber's entry point, and the entry point is what wakes the coroutine waiting in start(): it waited forever, the fiber stayed in INIT, and a second start() adopted another coroutine while the first kept a pointer to the fiber, written after the Fiber was freed. A finish handler added at start() ends the fiber and wakes the waiter with the coroutine's error, as from a body that threw at once; the entry point removes it once the body runs. TrueAsync calls the entry point with the error thrown instead.
After a fatal out-of-memory error in the script, the call after the destructors made a loop coroutine and a new main coroutine. The object store reuses no handle after the destructors, so a full store had to grow with no memory left, and the error was printed a second time "in Unknown on line 0". That call now finishes main, drains only when there is work, and returns async to READY with no current coroutine, as TrueAsync does after main: what runs before async is turned off is main.c's print of an uncaught exception, on the synchronous paths.
… can run it Output handlers run after the core turned async off. A fiber run as a coroutine whose coroutine a bailout left parked is still suspended there, and resume() or throw() enqueued it and waited as the current coroutine: there is none, which asserts in zend_fiber_await() (an Error from the scheduler without assertions). Both now throw a FiberError before the enqueue once the scheduler has stopped, and the FiberError of a refused switch when the current coroutine is missing or finished, which is what zend_fiber_await() needs. A plain fiber is not affected. TrueAsync has no such check.
The destructor of a fiber run as a coroutine cancels the coroutine and detaches it, and left the fiber SUSPENDED. A destructor that kept the object (a cycle the GC collects, whose __destruct stores the fiber) then reached the legacy path of resume() and throw(), which switched to a context the fiber never had: an assertion in a debug build, a crash without one. Like a legacy fiber the destructor closes, the fiber is now DEAD, and resume() throws "Cannot resume a fiber that is not suspended".
A coroutine fiber stayed SUSPENDED until its body was scheduled, the legacy caller pointer is never set for it, and the yield cleared the waiting caller before that caller had taken the value. A resume() from another coroutine in either window passed the check, overwrote the transfer value and the waiting caller, and the first caller waited for good. resume() and throw() now mark the fiber RUNNING, as a legacy switch does. The yield moves the waiting caller to handoff_coroutine, where it stays until it has taken the value; a body its cancellation runs meanwhile yields to nobody and leaves that value alone. resume(), throw(), isSuspended() and isRunning() count both as the legacy path counts its caller. The transfer slot releases a value nobody takes (a waiter cancelled after the yield, a resume value the cancellation of the body overtook, a failed enqueue), detached first and released after the state change, since its destructor may park or re-enter. A starter cancelled before the body first runs leaves the body no frame to link its backtrace to.
A coroutine fiber stays INIT until its body runs. When the starter was cancelled while the body waited in the queue, a second start() adopted the fiber again: two coroutines ran one body and one of them leaked. start() refuses a fiber that already has a coroutine, and isStarted() says so.
…t ends A body parked outside Fiber::suspend() (an await, a delay) saved EG(active_fiber) with its context and got it back on the next switch in. When the last holder, its starter, was cancelled meanwhile, the destructor freed the fiber and the body's Fiber::getCurrent() or Fiber::suspend() read freed memory. The destructor now keeps the object until the body ends, through a finish handler, as a legacy fiber is held by its resumer while it runs. The body is cancelled, unless it is the running coroutine outside the scheduler's tick, and its Fiber::suspend() refuses as in a force-closed fiber.
…h handler, a defer and the launch The core's refusal paths had no test: the test scheduler could fail only the enqueue of a CREATED coroutine. New INI countdowns, each failing the Nth call once: fail_wake (the enqueue of a coroutine that is not CREATED), fail_cancel, fail_finish_handler (returns 0), fail_defer and fail_launch (returns NULL). A refused enqueue, cancel or defer throws an Error naming its seam; a transferred error is released. Two fixes the tests need. An exception a finish handler leaves at a coroutine's end becomes that coroutine's, as the body's would, instead of staying pending for the next coroutine to meet at its suspend point; one the body ended with comes first. ts_enqueue() stores the error for the suspension point only after its last refusal, so a refused wake leaves no error behind for a later cancel to take as already in flight.
…body's end without losing an error Fiber::suspend() parked the body before waking the waiting caller: a refused wake left its error pending, and the next coroutine to run met it. The wake now comes first; refused, it throws in the body, which goes on. start() added its finish handler after the enqueue and stored the id unchecked: a refused handler left a body cancelled before it ran with nobody to wake the starter. The handler now comes first, and a refused one fails start() with a FiberError. A body's end handed its error to the caller with ownership, so a refused wake released it; the error now stays the coroutine's, under the refusal, and an exit stays an exit. The enqueue contract says what the core relies on: every refusal returns false with an Error thrown and a transferred error released.
…nd, not by a finish handler The destructor of a fiber whose body runs kept the object through a finish handler. A scheduler that refused the handler left the body running on a freed fiber, a heap use after free in Fiber::getCurrent(); a handler that threw or was skipped kept the fiber to the request's end. The destructor now sets ZEND_FIBER_FLAG_HELD_BY_BODY, and the coroutine entry releases the object after it has handed the body's outcome back, on the bailout path too. The hand-back moves into zend_fiber_coroutine_hand_back().
…in charge gc_spawn_destructors_coroutine() set dtor_coroutine and counted the new iterator before the scheduler accepted it, and undid both on refusal. The parked iterator, still running, then saw dtor_coroutine NULL and never disarmed the microtask, which leaked. Both are set only after the handler and the enqueue succeed.
…opped a reference The error of a refused wake of the GC coroutine, a refused defer of the hand-off microtask, or a refused run where switching is blocked (a tick function, the scheduler's own work) stayed pending and surfaced in unrelated code. Each is cleared, as a refused iterator's already was: it belongs to no caller. One that wraps an exception the caller had stays, as clearing it would take that exception with it. A GC coroutine whose wake is refused stays parked inside its run: the collector is off for the rest of the request, and whoever waits for the run stays parked until the scheduler finds nothing else to run and reports a deadlock.
…ow position across the park A reference an unwind drops can fill the root buffer, and the automatic run parks that frame until the GC coroutine ends. The pending exception went into the switch: the next coroutine met it, or, held by the scheduler, it came back with EG(opline_before_exception) overwritten by code that threw and caught meanwhile, so the frame's catch was missed. gc_possible_root_when_full() now holds the exception and its throw position as zend_objects_destroy_object() does around a destructor, without moving the frame back to the throwing opline: the GC's TMPVAR walks would take live ranges the unwind already freed as live. A cancellation delivered meanwhile is thrown over the held exception; an exit wins either way. A refused GC coroutine's error is cleared under the hold, so the exception in flight reaches its catch unchanged. The suspend contract says the caller parks with no exception pending, and the test scheduler asserts it.
…e TMPVARs zend_gc_collect_cycles() shields the caller's live TMPVARs from the run and re-roots them after the await by the frame's opline. A cancellation delivered at the await moves the frame to the exception op, the walk found no live range, and the shielded values fell out of the root buffer: a cycle only they kept was collected at the request's end. The re-root walks at the opline the shield used; nothing between the two frees a TMPVAR.
…for it; one far past the queued run waits
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A lightweight asynchronous core without complex logic.
The idea is:
https://github.com/true-async/php-async-core-rfc/blob/main/scheduler_rfc.md
A detailed explanation of the core integration can be found here:
https://github.com/true-async/php-async-core-rfc/blob/main/core-integration.md
At the moment, both the code and the RFC are still under development. I'd be happy to hear your ideas and feedback.