New testing utilities package with mock implementations for integration testing

[Rodrigo Brandão (rodrigobr-msft)]

This pull request primarily reorganizes and clarifies the usage of the agent testing utilities by moving them from microsoft_agents.testing to a new namespace, microsoft_agents.hosting.testing, and updates all relevant imports and documentation. It also updates the installation and dependency configuration to reflect this change, and refactors the OAuth continuation test to use a new mock user token client utility.

Testing Utilities Namespace Migration:

• All imports of testing utilities in the integration tests and documentation have been changed from microsoft_agents.testing to microsoft_agents.hosting.testing to reflect the new package structure. This affects files such as test_booking_dialog.py, test_user_profile_dialog.py, test_oauth_continuation.py, test_expect_replies.py, test_quickstart.py, and others, as well as the package's README and utility imports. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17]

Dependency and Installation Updates:

• The pyproject.toml dependencies for integration tests have been updated to include both microsoft-agents-hosting-testing and the relocated microsoft-agents-testing, ensuring compatibility with the new structure.
• The CI/CD configuration files (.github/workflows/python-package.yml and .azdo/ci-pr.yaml) have been updated to install the new microsoft_agents_testing wheel. [1] [2]
• The README has been updated to instruct users to install from the new microsoft-agents-hosting-testing directory. (F8ecb043L8R8)

OAuth Continuation Test Refactor:

• The OAuth continuation integration test (test_oauth_continuation.py) has been refactored to use the new MockUserTokenClient from microsoft_agents.testing, replacing the previous custom fake client with a more robust and reusable mock implementation. The test logic for simulating token exchanges and tracking calls is now encapsulated in a helper function, improving maintainability and clarity. [1] [2] [3] [4]

Documentation and Example Updates:

• The README (now under microsoft-agents-hosting-testing) has been updated to use the new import paths and to clarify usage examples for the new testing utilities. (F8ecb043L8R8, [1] [2] [3] [4] [5]

Minor Cleanups:

• Removed unused code and simplified state management in the OAuth continuation test, such as eliminating the token_available flag and related logic.

These changes collectively improve the organization, clarity, and maintainability of the agent testing infrastructure and its usage in integration tests.

[Copilot]

Pull request overview

This PR introduces a new microsoft_agents.testing package to enable in-memory/integration-style testing of agent turns (including mocked OAuth/token flows), and updates a few type annotations in existing core/protocol libraries to improve typing ergonomics.

Changes:

• Added a new microsoft-agents-testing library with TestAdapter, TestFlow, and MockUserTokenClient.
• Added tests for the new TestFlow / adapter reply-queue behavior.
• Updated adapter/protocol typing (use() returns Self) and simplified TokenStatus type annotations.

Reviewed changes

Copilot reviewed 17 out of 19 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/testing_package/test_test_flow.py	Adds async pytest coverage for TestFlow fluent chaining and adapter reply queuing.
tests/testing_package/init.py	Adds test package marker for the new testing-package test suite.
tests/_common/testing_objects/mocks/mock_user_token_client.py	Minor formatting cleanup for an existing test mock.
libraries/microsoft-agents-testing/setup.py	Adds packaging entrypoint and install requirements for the new testing package.
libraries/microsoft-agents-testing/readme.md	Adds package README for distribution/documentation.
libraries/microsoft-agents-testing/pyproject.toml	Adds PEP 621 project metadata for the new testing package.
libraries/microsoft-agents-testing/microsoft_agents/testing/type_def.py	Introduces shared typing aliases for the testing helpers.
libraries/microsoft-agents-testing/microsoft_agents/testing/test_flow.py	Implements fluent TestFlow send/assert helpers for scripted agent testing.
libraries/microsoft-agents-testing/microsoft_agents/testing/test_adapter.py	Implements the in-memory adapter that runs the normal pipeline and captures outgoing activities.
libraries/microsoft-agents-testing/microsoft_agents/testing/auth/mock_user_token_client.py	Adds an in-memory UserTokenClientBase implementation for OAuth/token-flow testing.
libraries/microsoft-agents-testing/microsoft_agents/testing/auth/_types.py	Adds dataclasses used as keys/records for the token mock’s internal stores.
libraries/microsoft-agents-testing/microsoft_agents/testing/auth/init.py	Exports the auth testing utilities.
libraries/microsoft-agents-testing/microsoft_agents/testing/_defaults.py	Adds default conversation/user/bot constants for deterministic tests.
libraries/microsoft-agents-testing/microsoft_agents/testing/init.py	Exports the primary testing surface area (MockUserTokenClient, TestAdapter, TestFlow).
libraries/microsoft-agents-testing/MANIFEST.in	Adds packaging include rules (VERSION.txt).
libraries/microsoft-agents-testing/LICENSE	Adds license file for distribution compliance.
libraries/microsoft-agents-hosting-core/microsoft_agents/hosting/core/channel_adapter.py	Updates use() to return Self for better chaining/subclass typing.
libraries/microsoft-agents-activity/microsoft_agents/activity/token_status.py	Simplifies TokenStatus type annotations (relies on camel-case alias generator).
libraries/microsoft-agents-activity/microsoft_agents/activity/channel_adapter_protocol.py	Updates protocol use() return type to Self for accurate fluent typing.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 28 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

libraries/microsoft-agents-testing/setup.py:10

• When VERSION.txt is missing and PackageVersion isn't set, package_version falls back to "0.0.0", which makes install_requires pin microsoft-agents-hosting-core==0.0.0 and will typically fail dependency resolution for local/source installs. Also, aiohttp is declared as a hard dependency but this package doesn't import it anywhere (only setup.py references it), which unnecessarily expands the install surface.

    package_version = environ.get("PackageVersion", "0.0.0")

libraries/microsoft-agents-testing/microsoft_agents/testing/auth/mock_user_token_client.py:356

• exchange_token() uses is to compare the _RAISE_EXCEPTION sentinel. Since this is a string value stored in a dict, identity comparison is not guaranteed; use equality so the sentinel check is reliable.

        if key in self._exchangable_tokens:
            token = self._exchangable_tokens[key]
            if token is _RAISE_EXCEPTION:
                raise Exception("Simulated exception during token exchange.")

libraries/microsoft-agents-testing/microsoft_agents/testing/test_adapter.py:232

• process_activity() overwrites activity.id unconditionally. This breaks scenarios where tests intentionally supply an ID (and also causes IDs to be generated twice for send_text_to_bot(), since create_activity() already assigns an ID). Only assign an ID when the inbound activity doesn't already have one.

        activity.recipient = self._conversation.agent
        activity.conversation = self._conversation.conversation
        activity.service_url = self._conversation.service_url
        activity.id = self._gen_id()

tests/hosting_msteams/helpers.py:41

• return [ResourceResponse()] * len(activities) repeats the same ResourceResponse instance for every activity. If any caller mutates a response (or inspects IDs), this can produce incorrect behavior. Return a distinct ResourceResponse per activity instead.

    async def send_activities(self, context, activities):
        self.sent_activities.extend(activities)
        return [ResourceResponse()] * len(activities)

[Copilot]

Pull request overview

Copilot reviewed 28 out of 30 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

libraries/microsoft-agents-testing/microsoft_agents/testing/auth/mock_user_token_client.py:356

• In MockUserTokenClient.exchange_token(), the sentinel check uses identity comparison (is) against a string constant. String interning is not guaranteed, so raise_on_exchange_request() may not reliably trigger the simulated exception.

        if key in self._exchangable_tokens:
            token = self._exchangable_tokens[key]
            if token is _RAISE_EXCEPTION:
                raise Exception("Simulated exception during token exchange.")

libraries/microsoft-agents-testing/microsoft_agents/testing/test_adapter.py:232

• TestAdapter.process_activity() always overwrites activity.id with a new generated ID. This discards IDs already set by callers (including activities created by create_activity()) and also advances the adapter's deterministic counter unnecessarily.

        activity.recipient = self._conversation.agent
        activity.conversation = self._conversation.conversation
        activity.service_url = self._conversation.service_url
        activity.id = self._gen_id()

[Copilot]

Pull request overview

Copilot reviewed 32 out of 34 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

libraries/microsoft-agents-testing/microsoft_agents/testing/auth/mock_user_token_client.py:356

• exchange_token checks token is _RAISE_EXCEPTION, but _RAISE_EXCEPTION is a string sentinel. Identity comparison on strings is not reliable and can fail even when the values are equal, so the mock may silently return a token instead of raising the simulated exception. Use == for the sentinel comparison (or switch to a unique object sentinel).

        if key in self._exchangable_tokens:
            token = self._exchangable_tokens[key]
            if token is _RAISE_EXCEPTION:
                raise Exception("Simulated exception during token exchange.")

libraries/microsoft-agents-testing/setup.py:18

• aiohttp is declared as an install requirement for microsoft-agents-testing, but there are no aiohttp imports/usages in this package (only referenced here in setup.py). This adds an unnecessary runtime dependency footprint for test utilities; consider removing it unless there's a concrete API that requires it.

setup(
    version=package_version,
    install_requires=[
        f"microsoft-agents-hosting-core=={package_version}",
        "aiohttp>=3.11.11",
    ],
)

[Copilot]

Pull request overview

Copilot reviewed 32 out of 34 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (4)

libraries/microsoft-agents-testing/setup.py:17

• aiohttp is listed as an install requirement for microsoft-agents-testing, but nothing in this package imports/uses it. Keeping unused runtime dependencies increases install size and can introduce avoidable dependency conflicts.

    install_requires=[
        f"microsoft-agents-hosting-core=={package_version}",
    ],
)

libraries/microsoft-agents-testing/microsoft_agents/testing/test_adapter.py:384

• get_next_reply_async() appends a Future to _queued_requests but never removes it if the caller times out/cancels (e.g., via asyncio.wait_for). This can leak cancelled futures and can also block immediate dequeuing because _queued_requests stays non-empty even when replies are queued.

        loop = asyncio.get_running_loop()
        future: asyncio.Future[Activity] = loop.create_future()
        self._queued_requests.append(future)
        return await future

libraries/microsoft-agents-testing/microsoft_agents/testing/test_adapter.py:405

• create_activity() builds an Activity shaped like the current conversation, but it does not set channel_id. Setting it makes the returned Activity self-contained for callers that use create_turn_context(create_activity(...)) without going through process_activity().

        return Activity(
            type=ActivityTypes.message,
            text=text,
            locale=self.locale or _DEFAULTS._LOCALE,
            recipient=self._conversation.agent,

tests/hosting_msteams/helpers.py:43

• [ResourceResponse()] * len(activities) returns the same ResourceResponse instance repeated, so mutating one entry would mutate all of them. Returning distinct instances avoids surprising shared-state in tests.

    async def send_activities(self, context, activities):
        self.sent_activities.extend(activities)
        return [ResourceResponse()] * len(activities)

[Copilot]

Pull request overview

Copilot reviewed 32 out of 34 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (7)

libraries/microsoft-agents-testing/microsoft_agents/testing/test_adapter.py:232

• process_activity unconditionally overwrites activity.id, but create_activity() already assigns an ID. This double-increments the adapter's ID counter and discards any caller-supplied inbound activity ID, which can make activity ID assertions flaky. Only generate an ID when one isn't already present.

        activity.recipient = self._conversation.agent
        activity.conversation = self._conversation.conversation
        activity.service_url = self._conversation.service_url
        activity.id = self._gen_id()

libraries/microsoft-agents-activity/microsoft_agents/activity/channel_adapter_protocol.py:8

• Importing Self directly from typing_extensions makes runtime availability depend on typing_extensions being installed even on Python versions (3.11+) where typing.Self exists. Prefer importing Self from typing with a fallback to typing_extensions for Python 3.10, to avoid an unnecessary hard dependency on typing_extensions for newer runtimes.

from typing import Protocol, Callable, Awaitable, Optional

from typing_extensions import Self

libraries/microsoft-agents-hosting-core/microsoft_agents/hosting/core/channel_adapter.py:8

• Importing Self directly from typing_extensions makes runtime availability depend on typing_extensions being installed even on Python versions (3.11+) where typing.Self exists. Prefer importing Self from typing with a fallback to typing_extensions for Python 3.10, to avoid an unnecessary hard dependency on typing_extensions for newer runtimes.

from __future__ import annotations

from typing_extensions import Self

from abc import ABC, abstractmethod

libraries/microsoft-agents-testing/microsoft_agents/testing/test_flow.py:228

• assert_reply_contains accepts a description override but currently drops it when delegating to assert_reply, so failures won't show the caller-provided message. Pass description through so the error text is consistent with other assertions.

        async def validate(reply: Activity) -> None:
            if expected not in (reply.text or ""):
                raise AssertionError(
                    description
                    or f"Expected reply text to contain '{expected}', received '{getattr(reply, 'text', None)}'."
                )

        return self.assert_reply(validate, timeout=timeout)

libraries/microsoft-agents-testing/microsoft_agents/testing/test_adapter.py:282

• TurnContext.send_activities([]) is allowed (it can legitimately result in an empty output list), but TestAdapter.send_activities currently raises for an empty activities list. This makes tests fail if code calls context.send_activities([]) or similar no-op sends. Return an empty response list instead of raising.

        if not activities:
            raise ValueError("Activities list cannot be empty.")

tests/hosting_msteams/helpers.py:43

• Using list multiplication here returns the same ResourceResponse instance repeated N times. If any code later mutates a response object, all entries will appear to change together. Prefer a list comprehension to create distinct instances.

    async def send_activities(self, context, activities):
        self.sent_activities.extend(activities)
        return [ResourceResponse()] * len(activities)

libraries/microsoft-agents-testing/microsoft_agents/testing/auth/mock_user_token_client.py:46

• The attribute name _exchangable_tokens is misspelled (should be _exchangeable_tokens). This is internal but it makes the code harder to read/search and increases the chance of future typos. Consider renaming it (and updating all references in this module).

    _user_tokens: dict[UserTokenKey, str]
    _exchangable_tokens: dict[ExchangeableTokenKey, str]
    _magic_codes: list[TokenMagicCode]

    def __init__(self):
        """Create an empty in-memory token store."""
        self._user_tokens = {}
        self._exchangable_tokens = {}
        self._magic_codes = []

[Copilot]

Pull request overview

Copilot reviewed 131 out of 208 changed files in this pull request and generated no new comments.

Suppressed comments (1)

tests/hosting_msteams/helpers.py:43

• [ResourceResponse()] * len(activities) repeats the same ResourceResponse instance, which can cause surprising aliasing if a test (or production code under test) mutates a returned response. Prefer creating a new instance per activity.

[Copilot]

Pull request overview

Copilot reviewed 131 out of 209 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 133 out of 210 changed files in this pull request and generated no new comments.

Suppressed comments (1)

tests/hosting_msteams/helpers.py:44

• _FakeAdapter.send_activities() returns [ResourceResponse()] * len(activities), which repeats the same ResourceResponse instance. If any test inspects/mutates a response object, all entries alias each other. Prefer creating distinct instances per activity.

[Copilot]

Pull request overview

Copilot reviewed 133 out of 210 changed files in this pull request and generated no new comments.