Skip to content

Fix orchestration reliability in Development - #1908

Merged
Paul Lizer (paullizer) merged 9 commits into
microsoft:Developmentfrom
paullizer:paullizer-orchestration-reliability-83a
Nov 9, 2026
Merged

Paul Lizer (paullizer) merged 9 commits into
microsoft:Developmentfrom
paullizer:paullizer-orchestration-reliability-83a

Conversation

@paullizer

Copy link
Copy Markdown
Collaborator

Summary

  • Correct the requested target after #1907 merged into paullizer-react-v2-ui. This PR targets Development, preserves its existing UI and release notes, and excludes the separate saved-findings/Sources inspector work. It includes the required earlier plan-contract, authorized action-source binding, and canonical plugin-loader prerequisites missing from Development.
  • Recover recognized planning mistakes with conservative single-source wiring and two shared model corrections, without weakening source permissions, explicit selections, Analyze contracts, capability checks, or approvals. Preserve explicit selections, including empty selections, instead of silently replacing them.
  • Read native Word comments from authorized DOCX/DOCM/DOTX originals, retain supported anchors/replies/resolution information, distinguish verified empty results from unavailable or partial evidence, and reopen exact committed evidence for Classic/V2 citations. Keep existing Excel behavior.
  • Repair one malformed answer-checker report using only allowed evidence labels without unnecessarily regenerating a correct retained answer. Keep substantive grounding failures blocking delivery and add correlated, content-free failure diagnostics.

The published head is a229ed79cc92a232e40b4ae804a8faaa4880ec02. This is a normal integration merge with Development commit 4dd6d80230fe60e3cbfe8544d1d52308a14a7387, not a history rewrite. The PR has 57 target-relative changed files. The original merge remains untouched. The branch is published from the existing paullizer/simplechat fork because direct push permission to the upstream repository is unavailable; the base repository remains microsoft/simplechat.

Linked issue

N/A — no associated issue was requested. The original PR is referenced above as a non-closing target-correction reference.

Release Notes & Latest Features

  • New Feature
  • Bug Fix
  • UI Enhancement
  • Breaking Change
  • Internal only

Is this visible to end users?

  • Yes
  • No

Is this admin-facing (Admin Settings, governance, deployment, config)?

  • Yes
  • No

Should this become a Latest Feature card?

  • Yes
  • No
  • Already added

Screenshot needed for the card?

  • Yes
  • No
  • Attached

Release notes and Latest Feature cards were deliberately left unchanged during this target correction.

Version bump

  • application/single_app/config.py VERSION third segment bumped, or not needed because this is docs-only
  • deployers/version.txt bumped, or not needed because deployers/ was not changed

Application version: 0.261.422 → 0.261.423. No deployer files changed.

Testing / validation

The counts below overlap and are not a live end-to-end success-rate measurement.

Command Result
Combined pipeline command below 515 passed, no deselections.
Local browser command below 49 passed.
Route-policy command below 30 passed.
python .\scripts\check_cosmos_response_boundaries.py --run-contracts 2,015 passed; 431 subtests passed. 1,094 owners and 425 doubles, 0 review gaps.
npm --prefix .\application\v2_ui ci --no-audit --no-fund Passed; restored 217 pinned packages after the missing-tsc build failure. No new frontend dependency/lockfile changes relative to Development.
npm --prefix .\application\v2_ui run build TypeScript/Vite build passed. Existing large-chunk warning remains.
node --check .\application\single_app\static\js\chat\chat-citations.js Passed.
python .\scripts\build_docs_inventory.py Passed; committed inventory is current.
python .\functional_tests\test_docs_app_surface_coverage.py 7/7 passed.
python .\functional_tests\test_docs_site_quality.py 6/6 passed.
Published-commit access-control/XSS commands below Passed for 22 Python files and 25 supported rendering files, respectively.
git --no-pager diff --check and git --no-pager diff --cached --check Passed.

Combined pipeline command — 515 passed:

$env:PYTHONPATH = "$PWD\application\single_app;$PWD\functional_tests"
python -m pytest -q -p no:cacheprovider --disable-warnings --tb=short --show-capture=no `
    .\functional_tests\test_task_completeness_engine.py `
    .\functional_tests\test_task_completeness_runtime.py `
    .\functional_tests\test_orchestration_planner_contract_recovery.py `
    .\functional_tests\test_orchestration_document_planning_recovery.py `
    .\functional_tests\test_orchestration_attachment_native_evidence.py `
    .\functional_tests\test_orchestration_source_kind_planning.py `
    .\functional_tests\test_orchestration_failure_telemetry.py `
    .\functional_tests\test_orchestration_selected_source_planning.py `
    .\functional_tests\test_orchestration_conversation_sources.py `
    .\functional_tests\test_orchestration_planning_persistence.py `
    .\functional_tests\test_orchestration_document_action_binding.py `
    .\functional_tests\test_orchestration_document_action_end_to_end.py `
    .\functional_tests\test_orchestration_document_action_lineage.py `
    .\functional_tests\test_orchestration_document_loader_reliability.py `
    .\functional_tests\test_plugin_discovery_identity.py `
    .\functional_tests\test_orchestration_native_citation_delivery.py `
    .\functional_tests\test_orchestration_citation_persistence.py `
    .\functional_tests\test_orchestration_result_imports.py `
    .\functional_tests\test_orchestration_registry_contract.py

Local browser command — 49 passed:

$env:PYTHONPATH = "$PWD\application\single_app;$PWD\functional_tests;$PWD"
$env:PLAYWRIGHT_SERVICE_URL = ''
python -m pytest -q -p no:cacheprovider --disable-warnings --tb=short --show-capture=no `
    .\ui_tests\test_v2_stream_citation_delivery.py `
    .\ui_tests\test_v2_excel_citation_inspection.py `
    .\ui_tests\test_inline_citation_filename_fallback.py `
    .\ui_tests\test_v2_orchestration_actions.py

The synthetic action-browser fixture now returns the existing pending-question endpoint's empty response only for its exact origin, GET method, path, and conversation. Unexpected requests still abort and fail, and browser-error assertions remain active. No production controller was changed to make these tests pass.

Route-policy command — 30 passed:

$env:PYTHONPATH = "$PWD\application\single_app;$PWD\functional_tests"
python -m pytest -q -p no:cacheprovider --disable-warnings --tb=short --show-capture=no `
    .\functional_tests\route_tests\test_route_blueprint_policy_inventory.py `
    .\functional_tests\route_tests\test_route_unauthenticated_policy_contract.py `
    .\functional_tests\route_tests\test_route_policy_test_coverage.py

Published-commit guardrails — passed:

$base = '4dd6d80230fe60e3cbfe8544d1d52308a14a7387'
$paths = @(git --no-pager diff --name-only $base HEAD -- .\application |
    ForEach-Object { $_.Replace('/', '\') })
python .\scripts\check_broken_access_control.py --base-sha $base --head-sha HEAD -- $paths
python .\scripts\check_xss_sinks.py --base-sha $base --head-sha HEAD -- $paths

The unchanged execution-module Cosmos reviews were restored from Development only after proving exact whole-file and owner/module fingerprint equality. Changed storage owners retain their concrete runtime evidence; the full network-blocked gate subsequently passed.

Not performed: deployment, live beta replay, or a full Jekyll build. Source-level documentation checks passed. The original v0.261.422 validation and its unrelated historical baseline limitations are recorded separately in the fix document; they are not presented as the current Development results.

Documentation

  • Release notes updated, or not needed
  • Feature documentation updated, or not needed
  • Fix documentation updated, or not needed

Updated the orchestration, action-binding, and document accessibility feature documentation; native-comment action and logging references; and docs/explanation/fixes/ORCHESTRATION_RELIABILITY_PIPELINE_FIX.md. The fix document records v0.261.423, required prerequisites, excluded UI work, and current Development validation. Release notes remain unchanged as noted above.

Security checklist

  • New Flask routes include @swagger_route(security=get_auth_security())
  • Settings sent to non-admin frontends use sanitize_settings_for_user()
  • Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS
  • No secrets, keys, connection strings, or local-only artifacts are included

No new Flask routes are introduced. Current source authorization and committed-evidence integrity are rechecked when citations reopen. Word author/comment/anchor text is rendered as inert text. Planner corrections never broaden permissions or replace explicit source selections, and diagnostic properties do not include prompts, raw failed references, source text, filenames, document IDs, or provider messages.

Keep answers focused, disclose retained findings and evidence on demand, and add content-free debug publication correlation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…nce-experience

Unify saved findings and evidence in the existing Sources inspector
Align planner defaults and output contracts, enforce native document action bindings through retention and citations, and add content-free debug checkpoints and authenticated end-to-end coverage. Bump the application version to 0.261.419.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Retain both native document-action diagnostics and saved-findings telemetry when resolving the logging allowlist conflict. Advance the integrated application version to 0.261.420 and synchronize current-version headers.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…e-planning-failures

Fix Orchestrate planning contracts and native document handoffs
Reuse canonical package imports so native document guards receive the same reader class as discovery. Preserve source isolation and recovery policy while adding safe action-stage failures and real-loader format/startup regressions. Pin the required template parser, add the CI gate, and update documentation and release pages for 0.261.421.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ion-stability

Fix canonical plugin discovery for document orchestration
Repair unambiguous selected-source wiring with two shared planner corrections, route native Word comments through the authorized shared reader and retained citations, and repair malformed checker evidence labels without weakening grounding. Add content-free correlated diagnostics, regressions and documentation for v0.261.422.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve Development changes and exclude the separate saved-findings Sources inspector work. Include the required plan-contract, native action source-binding and canonical plugin-loader prerequisites with the reliability fixes. Record v0.261.423 validation and retain published history without force-pushing.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@paullizer
Paul Lizer (paullizer) merged commit 6fd290b into microsoft:Development Nov 9, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant