Skip to content

.NET: Preserve non-ASCII text in hosting tool payloads - #10065

Merged
westey (westey-m) merged 3 commits into
microsoft:mainfrom
westey-m:hosting-unicode-serialization
Nov 4, 2026
Merged

westey (westey-m) merged 3 commits into
microsoft:mainfrom
westey-m:hosting-unicode-serialization

Conversation

@westey-m

Copy link
Copy Markdown
Collaborator

Motivation & Context

Tool call arguments containing non-ASCII text are serialized as literal \uXXXX sequences by the OpenAI Responses hosting package. This makes tool calls difficult to read in DevUI and increases payload size for languages such as Cyrillic, Arabic, and CJK.

The same serialization behavior also affects OpenAI Chat Completions hosting and Foundry hosting tool calls, approval requests, and object results.

Description & Review Guide

  • What are the major changes?
    • Configure hosting JSON serialization to preserve characters in the Basic Multilingual Plane while continuing to escape HTML-sensitive characters.
    • Use the shared OpenAI hosting serialization options for Chat Completions tool arguments.
    • Apply the same encoding behavior to Foundry hosting tool calls, approval requests, and object results.
    • Add regression tests for readable non-ASCII text and escaped HTML-sensitive characters.
  • What is the impact of these changes?
    • Common non-ASCII text is readable in hosted tool payloads instead of appearing as \uXXXX sequences.
    • Characters such as <, >, and & remain escaped.
    • There is no public API or breaking behavior change.
  • What do you want reviewers to focus on?
    • Whether JavaScriptEncoder.Create(UnicodeRanges.All) provides the desired balance between readable international text and retaining HTML-sensitive character escaping.

Related Issue

Fixes #10031

The issue has an automated high-confidence reproduction, but does not yet have the ready-for-implementation label or an explicit maintainer approval comment. No competing open PR was found.

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: Copilot assisted with the implementation, tests, investigation, and PR description.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI balanced review requested due to automatic review settings November 3, 2026 18:52
@agent-framework-automation agent-framework-automation Bot added the .NET Usage: [Issues, PRs], Target: .Net label Nov 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Non-string JsonElement and JsonDocument tool results retain Unicode escapes instead of displaying the decoded characters.

1 open issue
What changed in this PR

This PR addresses #10031 by making non-ASCII tool payloads readable across .NET OpenAI and Foundry hosting while retaining HTML-sensitive character escaping.

Changes:

  • Configure hosting serialization to preserve common non-ASCII characters.
  • Reuse shared OpenAI options for Chat Completions and apply equivalent encoding to Foundry tool payloads.
  • Add regression tests for international text and HTML-sensitive characters.
File Description
dotnet/​tests/​Microsoft.Agents.AI.Hosting.OpenAI.UnitTests/​OpenAIResponsesSerializationTests.cs Tests readable Responses arguments and HTML escaping.
dotnet/​tests/​Microsoft.Agents.AI.Hosting.OpenAI.UnitTests/​OpenAIChatCompletionsSerializationTests.cs Tests readable Chat Completions arguments.
dotnet/​tests/​Microsoft.Agents.AI.Foundry.Hosting.UnitTests/​OutputConverterTests.cs Tests tool arguments, approval requests, object results, and HTML escaping.
dotnet/​src/​Microsoft.Agents.AI.Hosting.OpenAI/​OpenAIHostingJsonUtilities.cs Configures the shared HTML-safe Unicode encoder.
dotnet/​src/​Microsoft.Agents.AI.Hosting.OpenAI/​ChatCompletions/​AgentResponseExtensions.cs Uses shared hosting options for tool arguments.
dotnet/​src/​Microsoft.Agents.AI.Foundry.Hosting/​OutputConverter.cs Applies Unicode encoding options to Foundry tool payloads.

Review effort: Balanced

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: No findings
Scope: full PR (1 commit(s)): 674b20a228e6
Model: gpt-5.6-sol

Overview

The PR configures the OpenAI and Foundry hosting serializers to preserve non-ASCII text while retaining HTML-sensitive escaping, and it keeps existing wire shapes intact. Regression tests cover OpenAI Responses, Chat Completions, Foundry function calls, approval requests, and object results; after deduplication against the existing unresolved review thread, no additional publishable concern remains.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
No publishable findings remained after source verification for this scope.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 2

Result: No findings
Scope: 23 net-new commit(s): e2936e2caa8e, 503a9ec155ec, dc6617aa96f2, f34fafb95469, b6ad777cfad2, 52230700d5c9, 84573b186a1e, e2dc4605b7c2, 39dc737178cf, 1c152ab2c43e, 4522e336a18f, cd45c5890ef5, 3f70b72c7f9f, 693b2f5c3140, 91b1d9aad800, 3c301645f447, 989bc9964ed3, c2b8286a60b5, b76bb7639365, f84f1a42efb3, fd1230844b89, 9b3564d0c6c4, b7e7b020d012
Model: gpt-5.6-sol

Overview

The incremental change correctly reserializes non-string JsonElement and JsonDocument results with the shared Foundry options while preserving string unwrapping and the outer JSON-string wire contract. Regression tests cover both result types, and the supplied range's other behavior changes include corresponding guard or compatibility coverage. No Critical, High, or Medium defect anchored to a changed line survived reconciliation.

Reviewed the supplied incremental change set across correctness, security/reliability, architecture, and failure behavior.
No publishable findings remained after source verification for this scope.

@westey-m
westey (westey-m) added this pull request to the merge queue Nov 4, 2026
Merged via the queue into microsoft:main with commit 230a3aa Nov 4, 2026
53 of 59 checks passed
@westey-m
westey (westey-m) deleted the hosting-unicode-serialization branch November 4, 2026 17:54

This branch was previously deployed

1 inactive deployment
github-app-auth — b7e7b020 Deployed Nov 4, 2026 by westey-m via add_label #27076
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Usage: [Issues, PRs], Target: .Net

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.NET: [Bug]: OpenAI Responses hosting escapes non-ASCII characters in function call arguments as \uXXXX

4 participants