Found while implementing #6728 (PR pending). Out of scope there — filed unassigned.
Observation
scripts/check-nul-bytes.mjs builds its scan set from tracked files only:
const files = execFileSync('git', ['ls-files', '-z'], { … }); // line 416git ls-files with no --others lists the index, so a file that has been
written but not yet git add-ed is not scanned at all. The gate then exits
0 and prints its usual success line, which reads as "this tree has no raw
control bytes" rather than "the file you just wrote was not looked at".
Measured (this worktree, origin/main @ 73bff86 + one new untracked file)
A new test file had a raw 0x1b (ESC) materialized into it at write time —
the accident AGENTS.md describes, where an editing tool turns the escape text
into the byte precisely when you are writing about control characters:
$ node scripts/check-nul-bytes.mjs > /dev/null 2>&1; echo "exit=$?"
exit=0 # untracked — not scanned
$ grep -naoP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]' packages/cli/test/<new file>.ts | od -c
0000000 2 2 0 : 033 \n # the byte was there the whole time
$ git add -A && node scripts/check-nul-bytes.mjs > /dev/null 2>&1; echo "exit=$?"
exit=0 # after the byte was removed; staging is what makes it visible
The self-scan in AGENTS.md (grep -naP …) found it; the gate did not, and the
only difference was the index.
Why it is finding and not a defect
CI is unaffected: a PR's tree is fully tracked by the time any workflow runs,
so the gate's coverage there is complete, and #5157's widened byte surface
works exactly as documented. The harm is local and one-directional — the run
AGENTS.md tells you to make before pushing ("Run node scripts/check-nul-bytes.mjs
before pushing") is the run most likely to be looking at a file that is new,
i.e. the one case the gate skips. So the pre-push check is green for a reason
unrelated to the bytes, on exactly the file most likely to carry one, since a
newly-authored file is where an editing tool materializes an escape.
Possible shapes (not a recommendation — this needs the gate owner)
- Add
--others --exclude-standard to the ls-files call so untracked,
non-ignored files are scanned too, and say so in the gate's summary line.
- Leave enumeration alone and have the gate report the count it skipped
("N untracked file(s) not scanned — stage them or pass --all"), so the
green is honest about its scope.
- Do nothing and treat the AGENTS.md self-scan as the pre-push instrument,
with the gate scoped to committed state by design.
Shape 1 changes what a bare local run means; shape 2 keeps the gate's contract
and closes the false-confidence half only. Either way the two-line self-scan in
AGENTS.md stays the belt to this gate's braces.
Refs #4890, #5140, #5157.
Found while implementing #6728 (PR pending). Out of scope there — filed unassigned.
Observation
scripts/check-nul-bytes.mjsbuilds its scan set from tracked files only:git ls-fileswith no--otherslists the index, so a file that has beenwritten but not yet
git add-ed is not scanned at all. The gate then exits0 and prints its usual success line, which reads as "this tree has no raw
control bytes" rather than "the file you just wrote was not looked at".
Measured (this worktree,
origin/main@73bff86+ one new untracked file)A new test file had a raw
0x1b(ESC) materialized into it at write time —the accident AGENTS.md describes, where an editing tool turns the escape text
into the byte precisely when you are writing about control characters:
The self-scan in AGENTS.md (
grep -naP …) found it; the gate did not, and theonly difference was the index.
Why it is
findingand not a defectCI is unaffected: a PR's tree is fully tracked by the time any workflow runs,
so the gate's coverage there is complete, and #5157's widened byte surface
works exactly as documented. The harm is local and one-directional — the run
AGENTS.md tells you to make before pushing ("Run
node scripts/check-nul-bytes.mjsbefore pushing") is the run most likely to be looking at a file that is new,
i.e. the one case the gate skips. So the pre-push check is green for a reason
unrelated to the bytes, on exactly the file most likely to carry one, since a
newly-authored file is where an editing tool materializes an escape.
Possible shapes (not a recommendation — this needs the gate owner)
--others --exclude-standardto thels-filescall so untracked,non-ignored files are scanned too, and say so in the gate's summary line.
("N untracked file(s) not scanned — stage them or pass
--all"), so thegreen is honest about its scope.
with the gate scoped to committed state by design.
Shape 1 changes what a bare local run means; shape 2 keeps the gate's contract
and closes the false-confidence half only. Either way the two-line self-scan in
AGENTS.md stays the belt to this gate's braces.
Refs #4890, #5140, #5157.