Skip to content

Improve native document fidelity and audit answer reliability - #1914

Merged
Paul Lizer (paullizer) merged 4 commits into
Developmentfrom
paullizer-native-audit-improvement-plan
Nov 11, 2026
Merged

Paul Lizer (paullizer) merged 4 commits into
Developmentfrom
paullizer-native-audit-improvement-plan

Conversation

@paullizer

Copy link
Copy Markdown
Collaborator

Summary

Native document reads could succeed while the final audit failed answer checking or misstated missing properties. This keeps source facts authoritative and delivers useful findings with precise limitations instead of invented bindings or absent-as-empty claims.

  • Preserve supported native PDF/PPTX/workbook properties, text/tag and object identities, exact alternatives, decorative states, saved formatting, and real Excel in-cell checkbox bindings. Keep absent, stored-empty, populated, and inaccessible fields distinct.
  • Repair checked answer delivery with immutable source facts, authorized supporting citations, targeted validation/correction feedback, bounded Excel all-section paging, and safe check-only recovery without repeating source reads or outgoing effects.
  • Keep authorization, screened active-source identity, transport bounds, redaction, and honest coverage disclosures intact. Native inventories are not rendered-page reviews or accessibility certification; this adds no public route, cloud service, bulk re-ingestion, or OCR/vision job.

Scope: This targets Development and includes the three previously unmerged native-document fidelity, answer-delivery, and reliability commits (302e2aebb, 5fffd4342, 989d9845a), not just the last follow-up. The Development sync is already included; its existing chat/table UI updates are not duplicated in this PR's delta.

Review boundaries: Native Excel inventories use schema 3.0 and can rebuild from currently authorized retained originals. The private answer/checkpoint contract is v3; older failed preparations upgrade only on an explicit retry. Compatibility fields remain stored, but canonical states drive reasoning and review. A fact digest authenticates the appended facts, not all free-form prose. Full rendered-PDF comparison remains separate work. The exact historical R88 validation subtype remains unproved; the supporting-citation rejection was independently reproduced and repaired.

Linked issue

N/A - no associated issue.

Release Notes & Latest Features

  • New Feature
  • Bug Fix
  • UI Enhancement
  • Breaking Change
  • Internal only

Is this visible to end users?

  • Yes
  • No

Is this admin-facing (Admin Settings, governance, deployment, config)?

  • Yes
  • No

Should this become a Latest Feature card?

  • Yes
  • No
  • Already added

Screenshot needed for the card?

  • Yes
  • No
  • Attached

The original v0.261.424 native-fidelity release-note entries are included. No additional release notes or Latest Feature card were added for the follow-ups, per the user's existing decision.

Version bump

  • application/single_app/config.py VERSION third segment bumped, or not needed because this is docs-only
  • deployers/version.txt bumped, or not needed because deployers/ was not changed

Application version is 0.261.427 after resolving the Development merge's version-only conflict. deployers/ is unchanged.

Testing / validation

Backend selections used PYTHONDONTWRITEBYTECODE=1 and:

$env:PYTHONPATH="$PWD\application\single_app;$PWD\functional_tests;$PWD"
  • Fresh current-head native contract preflight: 56 passed.
python -m pytest .\functional_tests\test_native_audit_reliability.py .\functional_tests\test_native_inspection_request_contract.py -x -q --tb=short -p test_support.cosmos_network_guard -p no:cacheprovider --disable-warnings
  • Merged native reader/delivery/preservation and citation/evidence integration: 270 passed.
python -m pytest .\functional_tests\test_native_audit_reliability.py .\functional_tests\test_native_audit_answer_delivery.py .\functional_tests\test_native_inspection_request_contract.py .\functional_tests\test_native_document_fidelity.py .\functional_tests\test_native_document_fidelity_pipeline.py .\functional_tests\test_native_document_fidelity_http.py .\functional_tests\test_excel_inventory_persistence.py .\functional_tests\test_orchestration_attachment_native_evidence.py .\functional_tests\test_orchestration_search_citations.py .\functional_tests\test_evidence_status.py -x -q --tb=short -p test_support.cosmos_network_guard -p no:cacheprovider --disable-warnings
  • Local saved-analysis/evidence browser integration: 97 passed. Synthetic transport/local Chromium only; no live SimpleChat audit submissions.
$env:PYTHONPATH="$PWD\ui_tests;$PWD\application\single_app;$PWD\functional_tests;$PWD"
$env:PLAYWRIGHT_SERVICE_URL=''
python -m pytest .\ui_tests\test_chat_saved_analysis.py .\ui_tests\test_v2_evidence_status.py -x -q --tb=short -p no:cacheprovider --disable-warnings
  • TypeScript compilation and production build: passed, with Vite's existing large-chunk warning.
npm --prefix .\application\v2_ui run build
  • Documentation, boundary inventory, and whitespace: passed. Coverage 7/7; site quality 6/6; 1,094 storage owners / 425 doubles / zero review gaps. The full registered Cosmos contract sweep was not rerun for this PR preflight.
python .\functional_tests\test_docs_app_surface_coverage.py
python .\functional_tests\test_docs_site_quality.py
python .\scripts\check_cosmos_response_boundaries.py
git --no-pager diff --check

Counts overlap and are not a unique aggregate. Controlled provider replies establish contract/runtime behavior, not general live-model reliability. Deployment, fresh live acceptance, and rendered-PDF comparison were not performed. Private conversation exports and local investigation artifacts are excluded.

Documentation

  • Release notes updated, or not needed
  • Feature documentation updated, or not needed
  • Fix documentation updated, or not needed

Updated the native accessibility/structure and Excel inspection feature references, Document Search reference, and workbook guide. Added native fidelity, answer-delivery, and reliability fix documentation with scope, migration/retention behavior, validation, and limitations. Original release notes are included as described above; no new follow-up entry was requested.

Security checklist

  • New Flask routes include @swagger_route(security=get_auth_security()) (N/A: no new Flask routes)
  • Settings sent to non-admin frontends use sanitize_settings_for_user() (N/A: settings-to-frontend paths are unchanged)
  • Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS
  • No secrets, keys, connection strings, or local-only artifacts are included

Source reads and retained-result use keep current authorization, screening, version/hash, and attempt-ownership guards. Diagnostics contain stable codes and hashed correlations, not candidates, source bodies, or plaintext cursors. The committed fixture ZIP contains verified synthetic documents only.

Preserve native PDF, Excel and PowerPoint source states and associations, add authorized source-backed inspection and validated native audit rendering, and retain Word review-comment behavior.

Add verified synthetic regressions and documentation/release notes for v0.261.424.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep native field states authoritative, accept source-bound citations, and harden answer validation and explicit retry recovery. Clarify native-only PDF coverage and add focused regression coverage and documentation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate Development at f292209 while preserving the native-audit reliability fixes. Resolve the application version conflict as 0.261.427.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@paullizer
Paul Lizer (paullizer) merged commit 5f23901 into Development Nov 11, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants