Repository navigation
Python: filter non-assistant messages from workflow agent responses - #8729
Conversation
When running workflows converted to an agent via workflow.as_agent(), WorkflowAgent._convert_workflow_events_to_agent_response and _convert_workflow_event_to_agent_response_updates forwarded messages without checking their role. When underlying executors or orchestrations such as GroupChat yield conversation history containing user inputs, system instructions, or tool outputs, these non-assistant messages were re-emitted as caller-facing agent responses and streaming chunks. This change filters out non-assistant messages across AgentResponse, list[Message], Message, and AgentResponseUpdate workflow outputs so that WorkflowAgent exclusively returns assistant output. If an event contains only non-assistant messages, it is safely excluded from agent output without altering internal workflow execution. Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
the list[Message] branch appended the whole yielded list as the raw_representation even when non-assistant entries were filtered out of the public messages, so user/system/tool content stayed reachable through the non-streaming AgentResponse payload. when the list is filtered, append each assistant message's own raw_representation instead; unfiltered lists keep the original object. Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
|
Eduard van Valkenburg (@eavanvalkenburg) fixed. the list[Message] branch now appends each assistant message's own raw_representation when the list is filtered, so user/system/tool content no longer leaks through the non-streaming payload; unfiltered lists keep the original object, matching the streaming path. added an assertion to the mixed-list test pinning that no filtered-out role survives in raw_representation. workflow agent suite 64/64 green. |
|
Eduard van Valkenburg (@eavanvalkenburg) fixed in 4266443 — took the drop direction you offered: an assistant message whose contents are entirely function-call envelopes is now dropped alongside the non-assistant messages, so call → tool(result) → final keeps only the final and never leaves an orphaned call in the persisted response. the alternative (preserving paired call+result history) would reintroduce exactly the tool-role leakage this PR exists to remove, since the results are tool-role by construction — but happy to switch if you'd rather expose the pair. regression test builds your exact transcript (assistant(get_weather call) → tool(18C result) → assistant(final)) and asserts no function_call content survives while the final text does. workflow agent suite 65/65. |
Role-only filtering kept assistant(function_call) while dropping its tool(function_result), producing an invalid call-without-result transcript that providers reject on replay when the response is persisted through a session. Assistant messages whose contents are entirely function-call envelopes are now dropped alongside the non-assistant messages, so the public response carries only user-facing assistant output with no dangling calls. Regression test covers call → tool result → final. Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
4266443 to
f320aed
Compare
|
CI note: the failing Pre-commit/Package checks on the previous head were pyupgrade wanting the typing import collapse and a docstring-format fix on the new helper — both applied, plus ruff format. all hooks pass locally now (ruff check clean, format clean, pyupgrade idempotent) and the workflow agent suite is 65/65. fresh matrix running on f320aed. |
…oles One helper now decides what a workflow-as-agent caller sees, for both response modes: _caller_facing_messages drops non-assistant messages and assistant messages carrying function-call envelopes whole, reasoning included, since their tool-role results are excluded by the role policy and a call without its result is an invalid replay transcript. Streaming gains a _StreamingRoleGate that carries the last declared role across updates, so role-less continuation chunks of a user stream can no longer leak through the assistant-only filter. The foundry-hosting checkpoint resume path uses the same gate. Also imports Never/assert_type from typing_extensions; they do not exist in Python 3.10's typing module and broke the 3.10 CI matrix.
Same reportPrivateUsage suppression the existing call site already carries.
|
/review |
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (7 commit(s)): f049cb58c3cd, a6eff9e8e746, bcab43884656, cf2a1c0967ea, f320aed5165f, bd8f1e82df0f, c660761cd7b2
Model: gpt-5.6-sol
Overview
The PR centralizes caller-facing message filtering, applies it consistently across response shapes, and adds targeted coverage for role-less continuations, raw payloads, and orphaned function calls. Request-info handling remains separate, and filtered streaming output is what reaches session persistence. Three residual regressions remain: valid informational calls are removed, role inheritance is shared across concurrent streams, and response-level usage metadata is lost when every message is filtered.
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
3 verified findings remained after source verification (3 medium) across 1 file. Details are attached to the affected lines below.
Affected areas: python/packages/core/agent_framework/_workflows/_agent.py
…etadata Four review fixes on the caller-facing filter: - the streaming role gate remembers the declared role per message id, so a filtered user declaration no longer stains an unrelated role-less message from another stream, while the user message's own continuations stay filtered; id-less updates keep one conservative fallback slot - informational-only function calls have no tool result by contract and are no longer treated as internal call envelopes, so a child agent emitting only provider-hosted transcript calls still returns them - usage details and creation time merge even when every message of a child AgentResponse was filtered, keeping billing and telemetry intact Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
|
All four findings addressed in 54b276f:
New tests: informational call survives (both modes), per-message gate scoping (user decl on one id, role-less chunk on another forwarded, same-id continuation dropped), and usage retention with zero surviving messages. workflow_agent 71/71; foundry_hosting 879 passed with the 4 conftest state-root failures pre-existing on a clean tree (missing optional deps locally). |
Signed-off-by: Manohar Paturi <186662190+ManoharPaturi@users.noreply.github.com>
…sages' into fix/workflow-agent-filter-user-messages
Motivation & Context
When converting a workflow to an agent using workflow.as_agent(), WorkflowAgent wraps workflow execution inside the agent contract. In multi-agent orchestrations such as GroupChat or workflows where executors yield conversation history, the emitted output often contains user questions, system instructions, or tool results alongside assistant responses.
Previously, WorkflowAgent._convert_workflow_events_to_agent_response and _convert_workflow_event_to_agent_response_updates forwarded messages from AgentResponse, list[Message], and Message payloads without inspecting message roles. This caused user prompts to be re-emitted as part of the agent response, compounding across conversation turns in streaming and non-streaming modes.
Filtering to assistant-only messages ensures that callers invoking the workflow as an agent receive solely the generated assistant output.
Description & Review Guide
Major changes:
In python/packages/core/agent_framework/_workflows/_agent.py, updated _convert_workflow_events_to_agent_response so that only assistant-role messages from AgentResponse, list[Message], and Message payloads are added to the returned AgentResponse. Also updated _convert_workflow_event_to_agent_response_updates to drop non-assistant messages and updates during streaming conversion.
Impact of changes:
WorkflowAgent now returns strictly assistant output. If a workflow output event consists only of non-assistant messages, it is cleanly omitted without crashing or appending dangling raw representations. Internal workflow messaging and inter-executor communication remain completely unchanged.
Review focus:
Please verify the role filtering logic across the AgentResponse, list[Message], and Message branches in _convert_workflow_events_to_agent_response and _convert_workflow_event_to_agent_response_updates, as well as the unit tests covering multi-turn conversation non-compounding and streaming behavior.
Related Issue
Fixes #4261
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.