Skip to content

Python: Support response_format in GitHubCopilotAgent - #9810

Open
Dineshsuriya D (droideronline) wants to merge 8 commits into
microsoft:mainfrom
droideronline:feature/github-copilot-structured-output
Open

Dineshsuriya D (droideronline) wants to merge 8 commits into
microsoft:mainfrom
droideronline:feature/github-copilot-structured-output

Conversation

@droideronline

Copy link
Copy Markdown
Contributor

Motivation & Context

GitHubCopilotAgent ignored response_format, so structured output was unavailable even though the Copilot SDK supports it (github/copilot-sdk#2590, in SDK v1.0.15 and later, within the existing >=1.0.16 pin).

Description & Review Guide

  • What are the major changes?
    • Added response_format (Pydantic model type or JSON Schema mapping) to GitHubCopilotOptions.
    • The agent forwards it as response_schema to send_and_wait (non-streaming) and send (streaming). Per-run options override default_options.
    • response_format is stripped from the create_session kwargs so it no longer reaches the SDK session factory.
    • AgentResponse.value is parsed from the final text: via response_format= on the non-streaming response and output_format_type= in the streaming finalizer.
  • What is the impact of these changes? Opt-in. Without response_format the SDK receives response_schema=None, which is the existing behavior.
  • What do you want reviewers to focus on? Streaming still yields text deltas, with the value parsed once the stream completes, matching the SDK behavior.

Related Issue

Fixes #9809

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: implementation and tests.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Forward response_format as response_schema to the Copilot SDK send and
send_and_wait calls, keep it out of create_session kwargs, and parse
AgentResponse.value for streaming and non-streaming runs.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Structured-output parsing can incorrectly use trailing subagent text instead of the root answer, while mapping schemas and aliased models can trigger serialization or validation failures.

3 open issues
What changed in this PR

Adds opt-in structured output support to GitHubCopilotAgent, bringing it closer to other Agent Framework providers and addressing #9809.

Changes:

  • Accepts Pydantic models or JSON Schema mappings through response_format.
  • Forwards schemas to streaming and non-streaming SDK calls, excludes them from session creation, and enables AgentResponse.value parsing.
  • Adds tests for schema forwarding, default options, unstructured requests, and streaming results.
File Description
python/​packages/​github_copilot/​tests/​test_github_copilot_agent.py Adds structured-output tests for both response modes.
python/​packages/​github_copilot/​agent_framework_github_copilot/​_agent.py Adds the option, SDK forwarding, and structured response parsing.

Review effort: Balanced

Parse the streaming structured value from root-agent text only, convert
mapping schemas to plain dicts before they reach the SDK, and validate
Pydantic models by alias to match the schema the SDK requests.
@droideronline

Copy link
Copy Markdown
Contributor Author

Hi Eduard van Valkenburg (@eavanvalkenburg) , please take a look when you get a chance

@droideronline

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) - the pipeline failure is fixed. Package Checks (3.11) failed because pyright could not resolve the from_updates overload in the streaming finalizer: options.get("response_format") was partially unknown. It is now typed as Any (a14d0ce), and pyright, ruff and the github_copilot tests are clean locally.

Could you please retrigger the pipeline when you get a chance? Thanks!

@droideronline

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) - I moved the alias parsing into core as you suggested (790ac8e), and fixed the typing failure in the new aliased-model tests (3d1e5c8). Could you please retrigger the pipeline when you get a chance?

If everything is green, would it be possible to get this into the next release? We are waiting on structured output for the GitHub Copilot agent. Thanks!

# Conflicts:
#	python/packages/github_copilot/tests/test_github_copilot_agent.py
@eavanvalkenburg

Copy link
Copy Markdown
Member

Dineshsuriya D (@droideronline) Hard-coding by_name=False in shared response parsing overrides models configured with validate_by_name=True or populate_by_name=True. A model with item_count: int = Field(alias="itemCount") previously accepted {"item_count": 3} through AgentResponse.value and ChatResponse.value; this head now raises ValidationError. Please preserve each model’s configured field-name validation while still supporting alias-keyed output, and add a name-keyed regression test.

@droideronline

Copy link
Copy Markdown
Contributor Author

Eduard van Valkenburg (@eavanvalkenburg) - fixed in 6cdf1e0. Core now validates with the model's own configuration first, so name-keyed output (validate_by_name or populate_by_name) is accepted as before. Only if that fails does it retry with aliases only (pydantic 2.11+) to support alias-keyed provider output, and it re-raises the original error if both fail. Added regression tests for name-keyed output and for the original error on invalid output.

This branch was previously deployed

1 inactive deployment
github-app-auth — 6cdf1e08 Deployed Nov 3, 2026 by droideronline via add_label #26933
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Feature]: Support response_format (structured output) in GitHubCopilotAgent

3 participants