Repository navigation
[SYCL] Fix host accessor deadlock on scheduler graph lock - #23586
Open
uditagarwal97 wants to merge 1 commit into
Open
uditagarwal97 wants to merge 1 commit into
uditagarwal97 wants to merge 1 commit into
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
4 open issues
This change starts protecting writes toMEnqueueStatuswithMBlockedUsersMutex, but other code… · NewRootCommand->copySubmissionCodeLocation()is executed while holdingCmd->MBlockedUsersMutex,… · New The new behavior thatreleaseHostAccessor()enqueues deferred blocked users (via… · NewenqueueUnblockedCommandsis invoked withBlockedCmd->MBlockedUsersbut the list is not cleared… · New
What changed in this PR
Fixes a scheduler deadlock when creating a host_accessor by avoiding busy-wait under the scheduler graph lock and deferring blocked blocking-enqueue callers until the blocking command is released.
Changes:
- Defers blocking enqueue of blocked commands by registering the root event in
MBlockedUsersand returningSyclEnqueueBlocked. - Updates host-accessor release to mark commands ready under
MBlockedUsersMutexand enqueue deferred waiters. - Removes
BlockingTfromCommand::enqueueand updates unit tests/mocks accordingly.
| File | Description |
|---|---|
| sycl/unittests/scheduler/SchedulerTestUtils.hpp | Updates mock Command::enqueue signature to match API change. |
| sycl/unittests/scheduler/BlockedCommands.cpp | Adjusts unit test expectations for new “blocking enqueue defers” behavior and blocked-user registration. |
| sycl/source/detail/scheduler/scheduler.cpp | Unblocks host-accessor “EmptyCmd” under mutex and enqueues deferred blocked users. |
| sycl/source/detail/scheduler/graph_processor.cpp | Implements deferral path for blocked commands (including blocking callers) and updates enqueue call signature. |
| sycl/source/detail/scheduler/commands.hpp | Updates Command::enqueue API and documentation to remove blocking-wait behavior. |
| sycl/source/detail/scheduler/commands.cpp | Removes busy-wait loop in Command::enqueue and always returns SyclEnqueueBlocked for blocked commands. |
Review effort: Lite
Comment on lines
+395
to
+398
| { | ||
| std::lock_guard<std::mutex> Guard(BlockedCmd->MBlockedUsersMutex); | ||
| BlockedCmd->MEnqueueStatus = EnqueueResultT::SyclEnqueueReady; | ||
| } |
Comment on lines
+86
to
+94
| std::lock_guard<std::mutex> Guard(Cmd->MBlockedUsersMutex); | ||
| // Re-check under the mutex the unblocking thread also takes, so that | ||
| // the root is either seen as a blocked user or enqueued normally. | ||
| if (Cmd->isEnqueueBlocked()) { | ||
| Cmd->addBlockedUserUnique(RootCommand->getEvent()); | ||
| RootCommand->copySubmissionCodeLocation(); | ||
| EnqueueResult = EnqueueResultT(EnqueueResultT::SyclEnqueueBlocked, Cmd); | ||
| return false; | ||
| } |
Comment on lines
+399
to
+400
| // Commands whose blocking waiters deferred on BlockedCmd. | ||
| enqueueUnblockedCommands(BlockedCmd->MBlockedUsers, Lock, ToCleanUp); |
Comment on lines
+395
to
+400
| { | ||
| std::lock_guard<std::mutex> Guard(BlockedCmd->MBlockedUsersMutex); | ||
| BlockedCmd->MEnqueueStatus = EnqueueResultT::SyclEnqueueReady; | ||
| } | ||
| // Commands whose blocking waiters deferred on BlockedCmd. | ||
| enqueueUnblockedCommands(BlockedCmd->MBlockedUsers, Lock, ToCleanUp); |
Copilot stopped reviewing on behalf of
uditagarwal97 due to an error
October 22, 2026 00:46
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Fixes CMPLRLLVM-78891
Problem
When a thread creates a
host_accessor,Scheduler::addHostAccessoradds two commands to the graph:UpdateHostReqCmd: brings the data to the host. The accessor's constructor waits on this command's event.EmptyCmd: created already blocked (MEnqueueStatus=SyclEnqueueBlocked,MIsBlockable= true). It stays blocked until the accessor is destroyed, which is what stops later work on the buffer from running while the host is using it.~AccessorImplHostcallsScheduler::releaseHostAccessor, which marks it Ready.When thread T2 creates an accessor on a buffer while T1 still holds one, T2's
UpdateHostReqCmddepends on T1'sEmptyCmd. T2's wait then runs this path:event_impl::wait
└ Scheduler::waitForEvent ← takes the graph READ lock
└ GraphProcessor::waitForEvent
└ enqueueCommand(root, BLOCKING) → recurses into deps → reaches T1's EmptyCmd
└ Command::enqueue ← while (Blocked); // spins, still holding the read lock
Deadlock: T1 needs the write lock before it can destroy its accessor. The write lock can't be granted while T2 holds a read lock, and T2 won't let go until T1 destroys its accessor. BOOM!
Proposed Solution
Defer instead of spinning, using the same mechanism host tasks already use:
GraphProcessor::enqueueCommand: on a blocked command, blocking callers register the root event inMBlockedUsers(underMBlockedUsersMutex) and returnSyclEnqueueBlocked.waitForEventthen drops the graph lock and sleeps inevent_impl::waitInternal.Scheduler::releaseHostAccessor: marks the commandReadyunder the same mutex, then enqueues the registered waiters withenqueueUnblockedCommands, which wakes them.Command::enqueue: removes the busy-wait loop and the now-unusedBlockingTparameter.Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com