Skip to content

Python: filter non-assistant messages from workflow agent responses - #8729

Merged
Eduard van Valkenburg (eavanvalkenburg) merged 12 commits into
microsoft:mainfrom
ManoharPaturi:fix/workflow-agent-filter-user-messages
Oct 26, 2026
Merged

Eduard van Valkenburg (eavanvalkenburg) merged 12 commits into
microsoft:mainfrom
ManoharPaturi:fix/workflow-agent-filter-user-messages

Conversation

@ManoharPaturi

Copy link
Copy Markdown
Contributor

Motivation & Context

When converting a workflow to an agent using workflow.as_agent(), WorkflowAgent wraps workflow execution inside the agent contract. In multi-agent orchestrations such as GroupChat or workflows where executors yield conversation history, the emitted output often contains user questions, system instructions, or tool results alongside assistant responses.

Previously, WorkflowAgent._convert_workflow_events_to_agent_response and _convert_workflow_event_to_agent_response_updates forwarded messages from AgentResponse, list[Message], and Message payloads without inspecting message roles. This caused user prompts to be re-emitted as part of the agent response, compounding across conversation turns in streaming and non-streaming modes.

Filtering to assistant-only messages ensures that callers invoking the workflow as an agent receive solely the generated assistant output.

Description & Review Guide

  1. Major changes:
    In python/packages/core/agent_framework/_workflows/_agent.py, updated _convert_workflow_events_to_agent_response so that only assistant-role messages from AgentResponse, list[Message], and Message payloads are added to the returned AgentResponse. Also updated _convert_workflow_event_to_agent_response_updates to drop non-assistant messages and updates during streaming conversion.

  2. Impact of changes:
    WorkflowAgent now returns strictly assistant output. If a workflow output event consists only of non-assistant messages, it is cleanly omitted without crashing or appending dangling raw representations. Internal workflow messaging and inter-executor communication remain completely unchanged.

  3. Review focus:
    Please verify the role filtering logic across the AgentResponse, list[Message], and Message branches in _convert_workflow_events_to_agent_response and _convert_workflow_event_to_agent_response_updates, as well as the unit tests covering multi-turn conversation non-compounding and streaming behavior.

Related Issue

Fixes #4261

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

When running workflows converted to an agent via workflow.as_agent(), WorkflowAgent._convert_workflow_events_to_agent_response and _convert_workflow_event_to_agent_response_updates forwarded messages without checking their role. When underlying executors or orchestrations such as GroupChat yield conversation history containing user inputs, system instructions, or tool outputs, these non-assistant messages were re-emitted as caller-facing agent responses and streaming chunks.

This change filters out non-assistant messages across AgentResponse, list[Message], Message, and AgentResponseUpdate workflow outputs so that WorkflowAgent exclusively returns assistant output. If an event contains only non-assistant messages, it is safely excluded from agent output without altering internal workflow execution.

Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Sep 24, 2026
Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
the list[Message] branch appended the whole yielded list as the
raw_representation even when non-assistant entries were filtered out of the
public messages, so user/system/tool content stayed reachable through the
non-streaming AgentResponse payload. when the list is filtered, append each
assistant message's own raw_representation instead; unfiltered lists keep
the original object.

Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
@ManoharPaturi

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) fixed. the list[Message] branch now appends each assistant message's own raw_representation when the list is filtered, so user/system/tool content no longer leaks through the non-streaming payload; unfiltered lists keep the original object, matching the streaming path. added an assertion to the mixed-list test pinning that no filtered-out role survives in raw_representation. workflow agent suite 64/64 green.

@ManoharPaturi

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) fixed in 4266443 — took the drop direction you offered: an assistant message whose contents are entirely function-call envelopes is now dropped alongside the non-assistant messages, so call → tool(result) → final keeps only the final and never leaves an orphaned call in the persisted response. the alternative (preserving paired call+result history) would reintroduce exactly the tool-role leakage this PR exists to remove, since the results are tool-role by construction — but happy to switch if you'd rather expose the pair.

regression test builds your exact transcript (assistant(get_weather call) → tool(18C result) → assistant(final)) and asserts no function_call content survives while the final text does. workflow agent suite 65/65.

Role-only filtering kept assistant(function_call) while dropping its
tool(function_result), producing an invalid call-without-result transcript
that providers reject on replay when the response is persisted through a
session. Assistant messages whose contents are entirely function-call
envelopes are now dropped alongside the non-assistant messages, so the
public response carries only user-facing assistant output with no dangling
calls. Regression test covers call → tool result → final.

Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
@ManoharPaturi

Copy link
Copy Markdown
Contributor Author

CI note: the failing Pre-commit/Package checks on the previous head were pyupgrade wanting the typing import collapse and a docstring-format fix on the new helper — both applied, plus ruff format. all hooks pass locally now (ruff check clean, format clean, pyupgrade idempotent) and the workflow agent suite is 65/65. fresh matrix running on f320aed.

…oles

One helper now decides what a workflow-as-agent caller sees, for both
response modes: _caller_facing_messages drops non-assistant messages and
assistant messages carrying function-call envelopes whole, reasoning
included, since their tool-role results are excluded by the role policy
and a call without its result is an invalid replay transcript.

Streaming gains a _StreamingRoleGate that carries the last declared role
across updates, so role-less continuation chunks of a user stream can no
longer leak through the assistant-only filter. The foundry-hosting
checkpoint resume path uses the same gate.

Also imports Never/assert_type from typing_extensions; they do not exist
in Python 3.10's typing module and broke the 3.10 CI matrix.
Same reportPrivateUsage suppression the existing call site already
carries.
@eavanvalkenburg

Copy link
Copy Markdown
Member

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (7 commit(s)): f049cb58c3cd, a6eff9e8e746, bcab43884656, cf2a1c0967ea, f320aed5165f, bd8f1e82df0f, c660761cd7b2
Model: gpt-5.6-sol

Overview

The PR centralizes caller-facing message filtering, applies it consistently across response shapes, and adds targeted coverage for role-less continuations, raw payloads, and orphaned function calls. Request-info handling remains separate, and filtered streaming output is what reaches session persistence. Three residual regressions remain: valid informational calls are removed, role inheritance is shared across concurrent streams, and response-level usage metadata is lost when every message is filtered.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
3 verified findings remained after source verification (3 medium) across 1 file. Details are attached to the affected lines below.

Affected areas: python/packages/core/agent_framework/_workflows/_agent.py

…etadata

Four review fixes on the caller-facing filter:

- the streaming role gate remembers the declared role per message id, so
  a filtered user declaration no longer stains an unrelated role-less
  message from another stream, while the user message's own continuations
  stay filtered; id-less updates keep one conservative fallback slot
- informational-only function calls have no tool result by contract and
  are no longer treated as internal call envelopes, so a child agent
  emitting only provider-hosted transcript calls still returns them
- usage details and creation time merge even when every message of a
  child AgentResponse was filtered, keeping billing and telemetry intact

Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
@ManoharPaturi

Copy link
Copy Markdown
Contributor Author

All four findings addressed in 54b276f:

  • role gate: declared roles are now remembered per message id. A user declaration only stains that message's own continuations, so an unrelated role-less message (another stream, or a later message of the run) is forwarded instead of dropped, and the concurrent A-user/B-assistant interleaving resolves per stream. Updates without a message id share one conservative fallback slot, since there is no identity to scope by.
  • informational_only calls are excluded from the internal-envelope check in both the predicate and the streaming filter, so provider-hosted transcript calls without results survive in both modes.
  • usage_details and created_at now merge unconditionally per child AgentResponse, so an all-filtered response still reports its billed tokens and timestamp.

New tests: informational call survives (both modes), per-message gate scoping (user decl on one id, role-less chunk on another forwarded, same-id continuation dropped), and usage retention with zero surviving messages. workflow_agent 71/71; foundry_hosting 879 passed with the 4 conftest state-root failures pre-existing on a clean tree (missing optional deps locally).

Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
…sages' into fix/workflow-agent-filter-user-messages
Merged via the queue into microsoft:main with commit 9337c50 Oct 26, 2026
48 checks passed

This branch was previously deployed

1 inactive deployment
github-app-auth — 80020811 Deployed Oct 15, 2026 by ManoharPaturi via add_label #25433
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: _convert_workflow_event_to_agent_response_updates emits user input again

4 participants