Skip to content

[SYCL] Fix host accessor deadlock on scheduler graph lock - #23586

Open
uditagarwal97 wants to merge 1 commit into
intel:syclfrom
uditagarwal97:private/udit/fix_host_accessor_graph_lock_deadlock
Open

uditagarwal97 wants to merge 1 commit into
intel:syclfrom
uditagarwal97:private/udit/fix_host_accessor_graph_lock_deadlock

Conversation

@uditagarwal97

@uditagarwal97 uditagarwal97 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Fixes CMPLRLLVM-78891

Problem

When a thread creates a host_accessor, Scheduler::addHostAccessor adds two commands to the graph:

  • UpdateHostReqCmd: brings the data to the host. The accessor's constructor waits on this command's event.
  • EmptyCmd: created already blocked (MEnqueueStatus = SyclEnqueueBlocked, MIsBlockable = true). It stays blocked until the accessor is destroyed, which is what stops later work on the buffer from running while the host is using it. ~AccessorImplHost calls Scheduler::releaseHostAccessor, which marks it Ready.

When thread T2 creates an accessor on a buffer while T1 still holds one, T2's UpdateHostReqCmd depends on T1's EmptyCmd. T2's wait then runs this path:

event_impl::wait
└ Scheduler::waitForEvent ← takes the graph READ lock
└ GraphProcessor::waitForEvent
└ enqueueCommand(root, BLOCKING) → recurses into deps → reaches T1's EmptyCmd
└ Command::enqueue ← while (Blocked); // spins, still holding the read lock

Deadlock: T1 needs the write lock before it can destroy its accessor. The write lock can't be granted while T2 holds a read lock, and T2 won't let go until T1 destroys its accessor. BOOM!

Proposed Solution

Defer instead of spinning, using the same mechanism host tasks already use:

  • GraphProcessor::enqueueCommand: on a blocked command, blocking callers register the root event in MBlockedUsers (under MBlockedUsersMutex) and return SyclEnqueueBlocked. waitForEvent then drops the graph lock and sleeps in event_impl::waitInternal.
  • Scheduler::releaseHostAccessor: marks the command Ready under the same mutex, then enqueues the registered waiters with enqueueUnblockedCommands, which wakes them.
  • Command::enqueue: removes the busy-wait loop and the now-unused BlockingT parameter.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uditagarwal97 uditagarwal97 self-assigned this Oct 21, 2026
@uditagarwal97
uditagarwal97 requested a balanced review from Copilot October 21, 2026 23:03
@uditagarwal97
uditagarwal97 marked this pull request as ready for review October 21, 2026 23:08
@uditagarwal97
uditagarwal97 requested a review from a team as a code owner October 21, 2026 23:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

4 open issues
What changed in this PR

Fixes a scheduler deadlock when creating a host_accessor by avoiding busy-wait under the scheduler graph lock and deferring blocked blocking-enqueue callers until the blocking command is released.

Changes:

  • Defers blocking enqueue of blocked commands by registering the root event in MBlockedUsers and returning SyclEnqueueBlocked.
  • Updates host-accessor release to mark commands ready under MBlockedUsersMutex and enqueue deferred waiters.
  • Removes BlockingT from Command::enqueue and updates unit tests/mocks accordingly.
File Description
sycl/​unittests/​scheduler/​SchedulerTestUtils.hpp Updates mock Command::enqueue signature to match API change.
sycl/​unittests/​scheduler/​BlockedCommands.cpp Adjusts unit test expectations for new “blocking enqueue defers” behavior and blocked-user registration.
sycl/​source/​detail/​scheduler/​scheduler.cpp Unblocks host-accessor “EmptyCmd” under mutex and enqueues deferred blocked users.
sycl/​source/​detail/​scheduler/​graph_processor.cpp Implements deferral path for blocked commands (including blocking callers) and updates enqueue call signature.
sycl/​source/​detail/​scheduler/​commands.hpp Updates Command::enqueue API and documentation to remove blocking-wait behavior.
sycl/​source/​detail/​scheduler/​commands.cpp Removes busy-wait loop in Command::enqueue and always returns SyclEnqueueBlocked for blocked commands.

Review effort: Lite

Comment on lines +395 to +398
{
std::lock_guard<std::mutex> Guard(BlockedCmd->MBlockedUsersMutex);
BlockedCmd->MEnqueueStatus = EnqueueResultT::SyclEnqueueReady;
}
Comment on lines +86 to +94
std::lock_guard<std::mutex> Guard(Cmd->MBlockedUsersMutex);
// Re-check under the mutex the unblocking thread also takes, so that
// the root is either seen as a blocked user or enqueued normally.
if (Cmd->isEnqueueBlocked()) {
Cmd->addBlockedUserUnique(RootCommand->getEvent());
RootCommand->copySubmissionCodeLocation();
EnqueueResult = EnqueueResultT(EnqueueResultT::SyclEnqueueBlocked, Cmd);
return false;
}
Comment on lines +399 to +400
// Commands whose blocking waiters deferred on BlockedCmd.
enqueueUnblockedCommands(BlockedCmd->MBlockedUsers, Lock, ToCleanUp);
Comment on lines +395 to +400
{
std::lock_guard<std::mutex> Guard(BlockedCmd->MBlockedUsersMutex);
BlockedCmd->MEnqueueStatus = EnqueueResultT::SyclEnqueueReady;
}
// Commands whose blocking waiters deferred on BlockedCmd.
enqueueUnblockedCommands(BlockedCmd->MBlockedUsers, Lock, ToCleanUp);

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants