Skip to content

Fix document orchestration pipeline reliability - #1907

Merged
Paul Lizer (paullizer) merged 1 commit into
paullizer-react-v2-uifrom
paullizer-orchestration-reliability-83a
Nov 8, 2026
Merged

Paul Lizer (paullizer) merged 1 commit into
paullizer-react-v2-uifrom
paullizer-orchestration-reliability-83a

Conversation

@paullizer

Copy link
Copy Markdown
Collaborator

Summary

Document-backed Orchestrate runs could reject straightforward reviews even when ordinary chat succeeded. This fixes the plan, native-reader, and answer-checker handoffs without relaxing permissions or factual checks.

  • Repair missing document bindings locally only for one explicitly authorized, available selection and an existing eligible reader/action. Preserve explicit IDs, dynamic inputs, disabled work, and multi-source scope; allow at most two model corrections shared across recognized planning errors.
  • Route opt-in DOCX/DOCM/DOTX comment inspection through the existing authorized Word reader, retaining supported replies, anchors, resolution information, and exact citation evidence in Classic and V2. Preserve Excel inspection and default ingestion behavior; unavailable or partial evidence never becomes a complete empty inventory.
  • Restrict checker references to supplied evidence labels and provide one actionable malformed-report repair without unnecessarily regenerating the answer. Substantive grounding failures still block delivery.
  • Add content-free, correlated reader/checker diagnostics with debug logging on or off, regression coverage, and current documentation for v0.261.422.

Native Word inventories remain workspace-only. Comment-only inspection can work without stored style metadata, but combined requests still require their stored sections. This PR does not add an original Word preview or replace Analyze/OCR/vision.

Linked issue

N/A. No associated issue was created or authorized for this work.

Release Notes & Latest Features

  • New Feature
  • Bug Fix
  • UI Enhancement
  • Breaking Change
  • Internal only

Is this visible to end users?

  • Yes
  • No

Is this admin-facing (Admin Settings, governance, deployment, config)?

  • Yes
  • No

Should this become a Latest Feature card?

  • Yes
  • No
  • Already added

Screenshot needed for the card?

  • Yes
  • No
  • Attached

Version bump

  • application/single_app/config.py VERSION third segment bumped, or not needed because this is docs-only
  • deployers/version.txt bumped, or not needed because deployers/ was not changed

Application version: 0.261.421 -> 0.261.422. No deployer files changed.

Testing / validation

Python checks used:

$env:PYTHONPATH = "$PWD\application\single_app;$PWD\functional_tests"
Command Result
python .\scripts\check_cosmos_response_boundaries.py --run-contracts Passed: 2,026 tests and 431 subtests; 1,094 owners and 425 doubles with 0 review gaps. Network blocked.
Coupled regression command below Passed: 262 tests and 3 subtests; 1 confirmed baseline case deselected.
npm --prefix .\application\v2_ui run build Passed TypeScript/Vite build.
python .\scripts\build_docs_inventory.py Passed; generated inventory is current.
python .\functional_tests\test_docs_app_surface_coverage.py Passed: 7/7 checks.
python .\functional_tests\test_docs_site_quality.py Passed: 6/6 checks.
git --no-pager diff --check Passed.
python -m pytest -q -p no:cacheprovider --disable-warnings --tb=short --show-capture=no `
    .\functional_tests\test_orchestration_selected_source_planning.py `
    .\functional_tests\test_orchestration_plan_revision_planner.py `
    .\functional_tests\test_orchestration_document_action_binding.py `
    .\functional_tests\test_orchestration_document_action_end_to_end.py `
    .\functional_tests\test_orchestration_composition_metadata_recovery.py `
    .\functional_tests\test_orchestration_compose_output_retry.py `
    .\functional_tests\test_orchestration_retained_analysis_publication.py `
    .\functional_tests\test_orchestration_search_result_integrity.py `
    .\functional_tests\test_orchestration_result_imports.py `
    .\functional_tests\test_plugin_discovery_identity.py `
    .\functional_tests\test_document_accessibility_extraction.py `
    .\functional_tests\test_excel_document_accessibility_extraction.py `
    .\functional_tests\test_orchestration_registry_contract.py `
    -k "not test_an_endpoint_refusing_json_mode_is_asked_again_without_it"

Known baseline limitations: The deselected test has an unchanged exact-log assertion missing fields already emitted by the endpoint client. A separately attempted legacy planner-diagnostics suite has ten setup errors because its AST-only harness omits imported STORAGE_FAILURE_CODE. Both were reproduced against unmodified HEAD and left unchanged; neither exclusion waives the full storage contract gate. Additional focused and browser evidence is recorded in docs/explanation/fixes/ORCHESTRATION_RELIABILITY_PIPELINE_FIX.md.

Not performed: deployment, live beta chat submissions/replays, or a full Jekyll site build. These local results are not a claim of 100% deployed-model reliability.

Documentation

  • Release notes updated, or not needed
  • Feature documentation updated, or not needed
  • Fix documentation updated, or not needed

Updated orchestration, Document Search, accessibility, plan-editing, citation-control, and logging documentation. Added ORCHESTRATION_RELIABILITY_PIPELINE_FIX.md with root causes, contracts, verified results, and reproduction commands. Release notes were left unchanged pending user approval; no Latest Feature card was added.

Security checklist

  • New Flask routes include @swagger_route(security=get_auth_security())
  • Settings sent to non-admin frontends use sanitize_settings_for_user()
  • Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS
  • No secrets, keys, connection strings, or local-only artifacts are included

No new Flask routes or frontend settings-serialization changes. Existing source authorization, current-source/integrity rechecks, factual validation, and inert citation rendering remain in place.

Repair unambiguous selected-source wiring with two shared planner corrections, route native Word comments through the authorized shared reader and retained citations, and repair malformed checker evidence labels without weakening grounding. Add content-free correlated diagnostics, regressions and documentation for v0.261.422.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@paullizer
Paul Lizer (paullizer) merged commit b857909 into paullizer-react-v2-ui Nov 8, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant