Skip to content

Python: share todo mutation locks across providers on one store - #9869

Closed
NingLin Liu (ninglinLiu) wants to merge 1 commit into
microsoft:mainfrom
ninglinLiu:fix/todo-store-level-mutation-lock-9868
Closed

NingLin Liu (ninglinLiu) wants to merge 1 commit into
microsoft:mainfrom
ninglinLiu:fix/todo-store-level-mutation-lock-9868

Conversation

@ninglinLiu

Copy link
Copy Markdown
Contributor

Motivation & Context

TodoProvider kept read-modify-write locks on the provider instance. The store is shareable, so two providers over one store and session each took their own lock, could load the same next_id, and the second save could drop the first provider's items or reuse ids. An asyncio.Lock tied to one event loop also breaks when the same session is reused across asyncio.run boundaries.

Description & Review Guide

  • What are the major changes? Move todo mutation locking to a module-level WeakValueDictionary keyed by (id(store), event loop, id(session), source_id), matching the file-access _store_write_lock pattern. TodoProvider._mutation_lock now returns that shared lock.
  • What is the impact of these changes? Concurrent todos_add / complete / remove from different providers over the same store and session serialize correctly. Locks remain process-local and are not retained after they go unused (weak values).
  • What do you want reviewers to focus on? Lock keying (store + session + source_id + loop) and the new shared-provider file-store concurrency test.

Related Issue

Fixes #9868

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: Assisted with implementation and tests; I reviewed the lock design against _store_write_lock, verified lock sharing with a local script, and accept responsibility for the change.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

TodoProvider kept per-session locks on the provider instance, so two
providers over the same TodoStore and session could each load the same
next_id, append colliding items, and drop the other save. Move the lock
to a store/session/source_id map keyed per event loop, matching the
file-access _store_write_lock pattern.

Fixes microsoft#9868

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

File-backed mutations use object-specific locks, allowing separate AgentSession objects for the same persisted session to allocate duplicate IDs and overwrite concurrent updates.

1 open issue
What changed in this PR

This PR addresses #9868 by moving Python todo mutation locks from individual providers to a shared registry, allowing providers using the same store and session object to coordinate updates.

Changes:

  • Share locks by store, event loop, session object, and source ID.
  • Use weak references so unused locks are released.
  • Add tests for lock sharing and concurrent additions through two providers.
File Description
python/​packages/​core/​tests/​core/​test_harness_todo.py Adds shared-lock assertions and a two-provider file-store concurrency test.
python/​packages/​core/​agent_framework/​_harness/​_todo.py Replaces provider-local locks with a shared, event-loop-aware weak registry.

Review effort: Balanced

lock is in use, because whoever holds or awaits the lock still references the store
and session.
"""
key = (id(store), asyncio.get_running_loop(), id(session), source_id)
@eavanvalkenburg

Copy link
Copy Markdown
Member

Closing since the issue has not been triaged

This branch was previously deployed

1 inactive deployment
github-app-auth — 1d322911 Deployed Oct 30, 2026 by ninglinLiu via team_check #8071
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: todo mutations race when two providers share one todo store

3 participants