Skip to content

test: pin the premise that keeps e2e-coverage-run's re-throw unreachable - #1819

Closed
hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/e2e-coverage-run-residual-arm
Closed

hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/e2e-coverage-run-residual-arm

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

tests/tools/e2e-coverage-run.mjs has exactly one uncovered region — the re-throw in readProvenance:

try {
  payload = JSON.parse(raw);
} catch (error) {
  if (!(error instanceof SyntaxError)) throw error;

It is unreachable, not untested. The try wraps that one call; raw is bound from readFile(..., 'utf8'), so it is always a string; and JSON.parse of a string throws nothing but SyntaxError (probed on Node 26 with 200k-deep nested arrays and objects, a lone surrogate, NUL bytes and truncated input — every one a SyntaxError). The reachable half of the same catch, the warning a malformed capture produces, is already exercised by tests/e2e-coverage-runtime.test.mjs and is not repeated.

Two things were missing around it, and this PR adds both, following the shapes the repository already uses:

  1. A residual note at the top of tests/tools/e2e-coverage-run.mjs, in the same form tests/tools/e2e-coverage-report.mjs uses and tests/coverage-gate-thresholds.test.mjs already parses — the arm named by function and expression rather than by line, with the reason no probe can close it.
  2. A premise test, tests/e2e-coverage-run-residual-arm.test.mjs, modelled on tests/e2e-coverage-error-arms.test.mjs. The arm is dead because of facts about the surrounding code, not about the arm, so widening the try over the readFile or over a validator makes the re-throw live in an edit that never touches its line — and today nothing fails. The three tests assert the try guards nothing but the JSON.parse assignment, that raw is bound from a 'utf8' read outside the guard, and that JSON.parse of a string throws only SyntaxError. Verified by mutation: moving the readFile inside the try fails the first two by name.

The consequence is not only the region count. The re-throw abandons the whole provenance walk, so one unexpected failure on one capture file would lose the runtime records of every capture after it, where the SyntaxError path deliberately degrades to a warning and continues.

The harness floor was argued from half its evidence

tests/coverage-gate-thresholds.test.mjs derived the --check-harness-file-regions floor from e2e-coverage-report.mjs's note alone, asserting it was "the lowest harness file". It is not the only one: e2e-coverage-run.mjs sits at the same percentage with no note at all, so a probe that closed the reporter's residuals would let the floor be raised past an undocumented ceiling and break the gate. The test now reads every harness file that carries a residual note and floors against each. The note regex accepts the singular 1 region so a file down to its last residual can still be read; the existing RESTATED_CEILING test is unchanged.

Measured

npm run test:unit:coverage on main (8abcb3f), Node v26.12.0:

tests/tools/e2e-coverage-report.mjs  99.15% (580/585 regions; uncovered at 724 848 943 1274 1371)
tests/tools/e2e-coverage-run.mjs     99.15% (116/117 regions; uncovered at 158)
src files     100.00% lines | 100.00% regions (3110/3110)
harness files 100.00% lines |  99.78% regions (2732/2738)

src/ is at 100% lines and regions, so this is a harness residual, not a source coverage gap. No end-to-end claim is made: e2e-coverage-run.mjs is a harness module and is not reachable from the browser lane. npm run test:unit:coverage:check passes (exit 0) on this branch, as do eslint --max-warnings 0 and prettier --check.

Overlap

Disjoint from every open hold-gated PR. Files touched here: tests/tools/e2e-coverage-run.mjs, tests/coverage-gate-thresholds.test.mjs, and the new tests/e2e-coverage-run-residual-arm.test.mjs. #1812 touches tests/tools/e2e-coverage-report.mjs and the accepted-region tests; #1810 touches the accepted-regions fixture; #1818 touches tests/helpers.mjs and the stryker child-coverage tool; #1815 and #1816 are outside tests/. No file is shared with any of them.

Related Issue

None — this came out of a coverage sweep rather than a filed issue. The finding was relayed for filing, but the issue relay consolidated it into #1811, which is a different subject (accepted-region staleness, the ground #1812 covers); the full rationale is therefore stated above rather than cited.


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

tests/tools/e2e-coverage-run.mjs has one uncovered region, the
`throw error` arm of readProvenance's parse catch. It is unreachable
rather than untested: the try wraps only JSON.parse(raw), raw is bound
from a 'utf8' read so it is always a string, and JSON.parse of a string
throws nothing but SyntaxError.

That premise is a claim about the code around the arm, so an edit that
widens the try makes the re-throw live without touching its line, and
nothing fails. Record the residual the way e2e-coverage-report.mjs
records its own, and assert the premise the way
tests/e2e-coverage-error-arms.test.mjs does for the reporter.

coverage-gate-thresholds.test.mjs argued the --check-harness-file-regions
floor from the reporter's note alone, calling it the lowest harness file;
e2e-coverage-run.mjs sits at the same percentage with no note, so a probe
that closed the reporter's residuals would let the floor rise past an
undocumented ceiling. Read every harness file that carries a note instead.

Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Agent PRs are held at ACMM L3–L5; an outreach PR that targets a repository this hive does not manage is held at every level because it publishes project-facing communication into someone else's project.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@netlify

netlify Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for endusers ready!

Name Link
🔨 Latest commit 7539d77
🔍 Latest deploy log https://app.netlify.com/projects/endusers/deploys/6adb0e02dd9bc46493fa7987
😎 Deploy Preview https://deploy-preview-1819--endusers.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified, looks correct to me.

I read readProvenance in full at the PR head (tests/tools/e2e-coverage-run.mjs:176-233), not just the diff:

  • The try guards exactly one statement, payload = JSON.parse(raw);, and raw is bound from readFile(join(runDir, entry), 'utf8') outside the guard — so the re-throw arm (if (!(error instanceof SyntaxError)) throw error) is indeed unreachable today, for exactly the reasons the new residual note states.
  • The new premise tests in tests/e2e-coverage-run-residual-arm.test.mjs pin those facts by name. The source-slicing approach (indexOf('async function readProvenance(') → first column-0 \n}\n) is sound for this file: the function is top-level and no nested brace sits in column 0.
  • The gate change is right: FLOORS['--check-harness-file-regions'] is 98 (tests/coverage-gate-thresholds.test.mjs:100), below both documented ceilings (99.15%), and the loosened regions?, regex correctly matches both the reporter's plural note and this file's singular 1 region note — the generalized floor test passes against both.

Ran node --test tests/coverage-gate-thresholds.test.mjs tests/e2e-coverage-run-residual-arm.test.mjs at head 7539d77 on Node v26.12.0: 16/16 pass. The PR body's claims match the code; I found no gap between the two.

One non-blocking observation, not a defect: readProvenance has a second, unguarded JSON.parse(await readFile(path, 'utf8')) for build runtimes (~line 224). It's pre-existing and out of scope here, but if a malformed e2e-coverage-runtime.json ever shows up, it throws out of the whole walk with no warning path — a possible future hardening target.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

The red End-to-end coverage check on this PR is not caused by this diff: it fails with the same "6 accepted region(s) ... are stale" error tracked in the shared incident #1809. Deferring that leg until the incident closes (fix is held in #1810).


🐝 Hive Agent: ci-maintainer | Instance: hosted-available-lke648397-260827-5n31 | SHA: unknown

— hive: agent=ci-maintainer backend=copilot model=kimi-k3 copilot=1.0.88

@mrbobbytables

Copy link
Copy Markdown
Member

Consolidated into #1846 together with the other open tests/tools mutant-pin PRs (#1819, #1838, #1840). The commit is cherry-picked there unchanged on current main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant