Skip to content

Classify verify-raw diagnostics on whole phrases, after stripping colour codes - #336

Open
MaxGhenis wants to merge 1 commit into
mainfrom
nzhub/verify-raw-diagnostics
Open

MaxGhenis wants to merge 1 commit into
mainfrom
nzhub/verify-raw-diagnostics

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

Follow-up to #334, fixing the diagnostic nit from its review.

The problem

chronicle verify-raw decides between missing_remote and fetch_error by looking for marker text in Wrangler's stderr. The review of #334 found two misclassifications. Both only affect entries that already fail, so neither could ever report verified:

  • Colour codes inside the missing-object phrase made it a fetch_error.
  • Unrelated text that merely mentioned NoSuchKey made it missing_remote.

The fix

_is_missing_object_diagnostic in chronicle/artifacts.py now:

  • strips ANSI colour sequences and collapses whitespace before matching;
  • matches a marker only as a whole phrase, so NoSuchKeyHandler isn't NoSuchKey;
  • counts a marker only on a line that holds nothing but markers (such as NoSuchKey: specified key does not exist) or is an error line (it contains error, failed or Wrangler's ✘).

Nothing else in verify-raw changes.

Invariants and how they're checked

  • Colour codes anywhere in the message still give missing_remote. One test inserts a colour sequence at every position of [ERROR] The specified key does not exist.
  • A marker inside a longer identifier, or on a line that reports no error, gives fetch_error. Five cases cover this.
  • Existing behaviour holds. The bare markers, the S3-style Code: message line and Wrangler's real captured stderr all still give missing_remote.

Tests

  • tests/test_chronicle_verify_raw.py: 36 passed (25 existing, 11 new).
  • tests/test_chronicle_artifacts.py: 36 passed.
  • Three in-memory mutants each fail the new tests: the old substring logic (8 failures), no colour stripping (2) and never-missing (7).
  • Live, with the real Wrangler and this branch's code: an object absent at its canonical key reports missing_remote, and the published IRD wage and salary workbook reports verified.
  • Ruff 0.14.10 check and format pass.

Repository tooling only: no source package, fact or bundle pin changes.

🤖 Generated with Claude Code

…ripping colour codes

The review of #334 found two misclassifications among failing entries. Colour
codes inside the missing-object phrase produced fetch_error, and unrelated text
that merely mentioned NoSuchKey produced missing_remote. Neither could report
verified.

_is_missing_object_diagnostic now strips ANSI sequences and collapses
whitespace, matches markers only as whole phrases (not inside a longer
identifier), and counts a marker only on a line that holds nothing but markers
or is an error line.

Invariants, each with tests:
- Colour codes at any position of the missing-object message still give
  missing_remote (exhaustive over every insertion point).
- A marker inside a longer identifier, or on a line reporting no error, gives
  fetch_error.
- The bare markers and Wrangler's real captured output keep giving
  missing_remote; no status other than the missing/fetch-error split changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant