Skip to content

fix(glob): bound expansion while it runs; stop ** following symlinks - #242

Open
hyperpolymath wants to merge 23 commits into
mainfrom
claude/gifted-planck-mha9c9
Open

hyperpolymath wants to merge 23 commits into
mainfrom
claude/gifted-planck-mha9c9

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Addresses #191. Fixes the fuzz_glob_expansion out-of-memory failure seen on PR #230 (check run 134118693669, input /**/*, 2629 MB against libFuzzer's 2560 MB limit).

Cause (reproduced before changing anything)

  • The glob crate follows symbolic links to directories under **. On a 7-entry tree with d/up -> .. and self -> ., **/* passed 2.5 GB in 15 s, and **/no-such-name never returned while using only 2 MB. A cap on matches alone therefore cannot bound it.
  • expand_glob collected every path before checking MAX_GLOB_RESULTS. The fuzz target passed /**/* with an empty base dir, so it walked the runner's whole file system, including the loops under /proc and /sys. The real ASan target replays the CI input to the same out-of-memory report.
  • When the cap did fire, external-command expansion treated the error as "no match" and ran the command with the literal pattern.

Change

  • glob.rs walks the tree itself; glob::Pattern still matches each component. Each (directory, component) pair is processed once, in name order. ** lists a symlinked directory but does not descend into it, and skips hidden directories; other components still go through links.
  • New src/grade.rs: Cost (adds up), Occupancy (high-water mark), and a Meter that refuses the charge over a limit. The walk meters file-system reads (10,000,000), matches (100,000) and match bytes (6 MiB, Linux's hard execve ceiling, measured).
  • A limit returns GlobLimitExceeded with the limit, the usage so far and the directory being read. Nothing partial is returned, and the shell no longer substitutes the literal pattern for it.
  • .* and .h* now find hidden files; a name that is not UTF-8 no longer panics.
  • The fuzz target uses a small fixed tree, re-roots absolute inputs into it, and checks the limits on every input. fuzz/Cargo.lock regenerated (it lacked hmac and subtle).
  • confirmation::count_files_in_tree stopped following symlinks. Its test walked the real /tmp and hung for over 10 minutes once /tmp held a symlink loop, which the fuzz fixture leaves behind.

Against bash 5.2 (globstar on)

Case bash vsh before vsh now
**/*, 7-entry looped tree 6 paths, <1 ms out of memory same 6 paths
**/*, 328,420 entries 328,420 paths, 1.7 s, 63 MB error after collecting all, 82 MB limit error, 0.46 s, 11 MB
**/*/**/*, same tree 976,400 paths (328,400 distinct), 182 MB error, 251 MB limit error, 0.33 s, 11 MB
/**/* 643,683 paths, 205 MB out of memory byte-limit error, 20 MB
/usr/**/* 100,053 paths, 0.57 s n/a 97,097 paths, 0.48 s

bash has no limits, so those are deliberate departures. The /usr difference is a bash quirk: when ** is not the first component, bash enters a symlinked directory for one more component. All 2,956 extra paths are below links such as /usr/bin/X11 -> .. For **/... patterns bash and vsh agree. This is documented in glob.rs.

Tests

  • cargo fmt --check, cargo clippy --all-targets --all-features -- -D warnings: clean.
  • cargo test: 865 passed, 0 failed, 15 ignored (was 827/0/15).
  • tests/glob_bounds_tests.rs (14 tests). The two symlink-loop tests fail on main (20 s timeout) and pass here. The read-count tests compare the reported usage with counts made directly from the fixtures.
  • A differential property test checks the walker against the glob crate where the rules did not change; each of three deliberately planted bugs was caught.
  • Fuzzing: 382,036 runs in 10 minutes under -rss_limit_mb=2560, no crash; the CI input replays cleanly.

Status vocabulary: the A-6 work bound in docs/PROOF-OPEN-FRONTIER.adoc is tested-only; no Lean bound and no Rust contract yet.


Generated by Claude Code

…191)

ClusterFuzzLite's pr_fuzzing (address) job on PR #230 (check run
134118693669) hit libFuzzer's out-of-memory limit in fuzz_glob_expansion
with the input `/**/*`: 2629 MB used against 2560 MB, live allocations in
the glob crate's fill_todo, reached from expand_glob. The fault is on main.

Cause, reproduced locally before any change (Linux 6.18, release build):

- The glob crate follows symbolic links to directories under `**`, so a
  link back to a parent never ends the walk. `**/*` over a 7-entry tree
  with `d/up -> ..` and `self -> .` passed 2.5 GB in 15 s;
  `**/no-such-name` over the same tree never returned, at 2 MB, so a cap
  on matches alone cannot bound it.
- expand_glob collected every path and only then compared the count with
  MAX_GLOB_RESULTS. `/**/*` from the fuzz target's empty base dir walked
  the runner's whole file system, including the loops under /proc and
  /sys, and grew to the limit in 16 s. The real ASan fuzz target replays
  the CI input to the same out-of-memory report.
- When the cap did fire, external-command expansion treated the error as
  "no match" and ran the command with the literal pattern.

Fix:

- glob.rs walks the tree itself. Each component is still matched by
  glob::Pattern; the walk processes each (directory, pattern component)
  pair once, in name order. `**` lists a symbolic link to a directory but
  does not descend into it and skips hidden directories; other components
  still go through links, so `link/*` works.
- New src/grade.rs: a Cost grade (sequence adds) and an Occupancy grade
  (high-water mark), and a Meter that refuses the charge over a limit.
  The walk meters file-system reads (10,000,000), matches held (100,000)
  and bytes of matches held (6 MiB, the most Linux execve accepts for
  argument strings at any stack limit, measured here: 6,279,840 bytes).
  The grade laws are property-tested, not proved.
- Reaching a limit returns GlobLimitExceeded: the limit, the usage so far
  and the directory being read. Nothing partial is returned, and the shell
  no longer substitutes the literal word for a limit error.

Comparison with bash 5.2.21 (globstar on, dotglob off) on the same trees:

- 7-entry looped tree: bash gives 6 paths for `**/*` in under 1 ms; vsh
  now gives the same 6 (was: out of memory).
- 328,420-entry tree: bash expands `**/*` to 328,420 paths in 1.7 s at
  63 MB; vsh stops at the 100,000-match limit after 0.46 s at 11 MB.
  `**/*/**/*`: bash lists 976,400 paths (328,400 distinct) in 5.3 s at
  182 MB; vsh lists each path once and stops at the limit in 0.33 s.
- `/**/*`: bash expands it to 643,683 paths in 4.6 s at 205 MB; vsh stops
  at the 6 MiB byte limit at 20 MB. bash has no limits; these are
  deliberate departures.
- `/usr/**/*`: vsh 0.48 s, bash 0.57 s. bash finds 2,956 more paths, all
  below symlinked directories such as /usr/bin/X11 -> . : when `**` is not
  the first component, bash enters such a link for one more component.
  vsh does not; for `**/...` patterns the two agree. Documented in glob.rs.

Also:

- `.*` and `.h*` now find hidden files as in bash (before: only `.`/`..`,
  or nothing), and a file name that is not UTF-8 no longer panics.
- The fuzz target expands over a small fixed tree (two symlink loops, a
  hidden directory, a non-UTF-8 name), re-roots absolute inputs into it,
  skips `..`, runs under small limits and checks every result against the
  limits and the file system. 382,036 runs in 10 minutes with
  -rss_limit_mb=2560: no crash.
- fuzz/Cargo.lock regenerated: it lacked hmac and subtle.

Tests: tests/glob_bounds_tests.rs (14). The two symlink-loop tests fail on
main (timeout after 20 s) and pass here. Limit, residue and determinism
tests use injected limits; the read counts are checked against counts made
directly from the fixtures. A differential property test checks the walker
against the glob crate where the rules did not change; three planted bugs
were each caught by it. security_recursive_glob_bounded now asserts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDhGsL6zrtXcfEnvKbC
count_files_in_tree followed every symbolic link to a directory, and its
only test walked the real /tmp. With a link back to a parent directory
anywhere in /tmp the walk branches at every level and does not finish:
the test ran for over 10 minutes on this machine, where it had passed
before, because /tmp now held the glob reproduction trees.

The glob fuzz target leaves its fixture (with two such links) in the temp
directory on every run, because libFuzzer exits without running
destructors, so any machine that ran the fuzzer would hang this test.

Count links as entries and do not follow them, as `rm -r` does; the test
uses its own fixture, and a second test covers a looped tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDhGsL6zrtXcfEnvKbC
@github-actions

Copy link
Copy Markdown

🔍 Hypatia Security Scan

Findings: 20 issues detected

Severity Count
🔴 Critical 0
🟠 High 3
🟡 Medium 17
View findings
[
  {
    "line": null,
    "reason": "workflow .github/workflows/hypatia-scan.yml runs harden-runner in `egress-policy: audit` — telemetry-only, never blocks; on a protected-branch trigger this is not containment",
    "type": "RE002",
    "file": ".github/workflows/hypatia-scan.yml",
    "level": "epi",
    "rule_module": "research_extensions",
    "action": "report",
    "severity": "high",
    "evidence": [
      {
        "span": {
          "file": ".github/workflows/hypatia-scan.yml",
          "end_line": 0,
          "start_line": 0
        },
        "rule": "research_extensions/RE002",
        "observations": 1,
        "rule_version": "0.1.0"
      }
    ],
    "support": {
      "kind": "pattern"
    }
  },
  {
    "line": 2293,
    "reason": "expect() in hot path (3 occurrences, CWE-754, line 2293, 2322, 2930)",
    "type": "expect_in_hot_path",
    "file": "/home/runner/work/valence-shell/valence-shell/impl/rust-cli/src/parser.rs",
    "level": "epi",
    "rule_module": "code_safety",
    "action": "flag",
    "severity": "medium",
    "evidence": [
      {
        "span": {
          "file": "/home/runner/work/valence-shell/valence-shell/impl/rust-cli/src/parser.rs",
          "end_line": 2293,
          "start_line": 2293
        },
        "rule": "code_safety/expect_in_hot_path",
        "observations": 1,
        "rule_version": "0.1.0"
      }
    ],
    "support": {
      "kind": "pattern"
    }
  },
  {
    "line": 44,
    "reason": "unwrap() without prior check -- DoS via panic (6 occurrences, CWE-754, line 44, 49, 52, 53, 54, 55)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/valence-shell/valence-shell/impl/rust-cli/fuzz/fuzz_targets/fuzz_glob_expansion.rs",
    "level": "epi",
    "rule_module": "code_safety",
    "action": "flag",
    "severity": "high",
    "evidence": [
      {
        "span": {
          "file": "/home/runner/work/valence-shell/valence-shell/impl/rust-cli/fuzz/fuzz_targets/fuzz_glob_expansion.rs",
          "end_line": 44,
          "start_line": 44
        },
        "rule": "code_safety/unwrap_without_check",
        "observations": 1,
        "rule_version": "0.1.0"
      }
    ],
    "support": {
      "kind": "pattern"
    }
  },
  {
    "line": 41,
    "reason": "expect() in hot path (1 occurrences, CWE-754, line 41)",
    "type": "expect_in_hot_path",
    "file": "/home/runner/work/valence-shell/valence-shell/impl/rust-cli/fuzz/fuzz_targets/fuzz_glob_expansion.rs",
    "level": "epi",
    "rule_module": "code_safety",
    "action": "flag",
    "severity": "medium",
    "evidence": [
      {
        "span": {
          "file": "/home/runner/work/valence-shell/valence-shell/impl/rust-cli/fuzz/fuzz_targets/fuzz_glob_expansion.rs",
          "end_line": 41,
          "start_line": 41
        },
        "rule": "code_safety/expect_in_hot_path",
        "observations": 1,
        "rule_version": "0.1.0"
      }
    ],
    "support": {
      "kind": "pattern"
    }
  },
  {
    "reason": "Code scanning (Hypatia): hypatia/structural_drift/SD022 -- Hypatia structural_drift: SD022 -- 1 day(s) old",
    "type": "CSA001",
    "file": "docs/audits/2026-11-16-hypatia-remediation.adoc",
    "level": "epi",
    "rule_module": "code_scanning_alerts",
    "action": "review",
    "severity": "medium",
    "evidence": [
      {
        "span": {
          "file": "docs/audits/2026-11-16-hypatia-remediation.adoc",
          "end_line": 0,
          "start_line": 0
        },
        "rule": "code_scanning_alerts/CSA001",
        "observations": 1,
        "rule_version": "0.1.0"
      }
    ],
    "support": {
      "kind": "pattern"
    }
  },
  {
    "reason": "Code scanning (Hypatia): hypatia/code_safety/expect_in_hot_path -- Hypatia code_safety: expect_in_hot_path -- 1 day(s) old",
    "type": "CSA001",
    "file": "impl/rust-cli/src/parser.rs",
    "level": "epi",
    "rule_module": "code_scanning_alerts",
    "action": "review",
    "severity": "medium",
    "evidence": [
      {
        "span": {
          "file": "impl/rust-cli/src/parser.rs",
          "end_line": 0,
          "start_line": 0
        },
        "rule": "code_scanning_alerts/CSA001",
        "observations": 1,
        "rule_version": "0.1.0"
      }
    ],
    "support": {
      "kind": "pattern"
    }
  },
  {
    "reason": "Code scanning (Hypatia): hypatia/research_extensions/RE002 -- Hypatia research_extensions: RE002 -- 1 day(s) old",
    "type": "CSA001",
    "file": ".github/workflows/hypatia-scan.yml",
    "level": "epi",
    "rule_module": "code_scanning_alerts",
    "action": "update",
    "severity": "high",
    "evidence": [
      {
        "span": {
          "file": ".github/workflows/hypatia-scan.yml",
          "end_line": 0,
          "start_line": 0
        },
        "rule": "code_scanning_alerts/CSA001",
        "observations": 1,
        "rule_version": "0.1.0"
      }
    ],
    "support": {
      "kind": "pattern"
    }
  },
  {
    "reason": "Descriptile in retired location (layout drift, not a security issue); the current canon keeps descriptiles as clauses of the repo deed, converted by kyX (standards D312, D313)",
    "type": "SD004",
    "file": ".machine_readable/6a2/STATE.a2ml",
    "level": "epi",
    "target": ".machine_readable/descriptiles/STATE.a2ml",
    "advisory": true,
    "rule_module": "structural_drift",
    "action": "move",
    "severity": "medium",
    "evidence": [
      {
        "span": {
          "file": ".machine_readable/6a2/STATE.a2ml",
          "end_line": 0,
          "start_line": 0
        },
        "rule": "structural_drift/SD004",
        "observations": 1,
        "rule_version": "0.1.0"
      }
    ],
    "support": {
      "kind": "pattern"
    }
  },
  {
    "reason": "Descriptile in retired location (layout drift, not a security issue); the current canon keeps descriptiles as clauses of the repo deed, converted by kyX (standards D312, D313)",
    "type": "SD004",
    "file": ".machine_readable/6a2/META.a2ml",
    "level": "epi",
    "target": ".machine_readable/descriptiles/META.a2ml",
    "advisory": true,
    "rule_module": "structural_drift",
    "action": "move",
    "severity": "medium",
    "evidence": [
      {
        "span": {
          "file": ".machine_readable/6a2/META.a2ml",
          "end_line": 0,
          "start_line": 0
        },
        "rule": "structural_drift/SD004",
        "observations": 1,
        "rule_version": "0.1.0"
      }
    ],
    "support": {
      "kind": "pattern"
    }
  },
  {
    "reason": "Descriptile in retired location (layout drift, not a security issue); the current canon keeps descriptiles as clauses of the repo deed, converted by kyX (standards D312, D313)",
    "type": "SD004",
    "file": ".machine_readable/6a2/ECOSYSTEM.a2ml",
    "level": "epi",
    "target": ".machine_readable/descriptiles/ECOSYSTEM.a2ml",
    "advisory": true,
    "rule_module": "structural_drift",
    "action": "move",
    "severity": "medium",
    "evidence": [
      {
        "span": {
          "file": ".machine_readable/6a2/ECOSYSTEM.a2ml",
          "end_line": 0,
          "start_line": 0
        },
        "rule": "structural_drift/SD004",
        "observations": 1,
        "rule_version": "0.1.0"
      }
    ],
    "support": {
      "kind": "pattern"
    }
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

hyperpolymath and others added 3 commits November 18, 2026 00:40
The Hypatia code_safety scan on PR #242 flagged expect() (line 41) and
six unwrap() calls (lines 44-55) in the fixture setup of
fuzz_glob_expansion. Setup errors are now returned with `?`; if the
fixture cannot be built, the target skips its inputs instead of
panicking, so a temp-directory failure is never reported as a crash.

Rebuilt under ASan and fuzzed for 60 s (31,108 runs) with
-rss_limit_mb=2560: no crash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDhGsL6zrtXcfEnvKbC
@hyperpolymath
hyperpolymath marked this pull request as ready for review November 18, 2026 00:42
Repository owner deleted a comment from coderabbitai Bot Nov 18, 2026

hyperpolymath commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner Author

Local evidence, not CI

CI could not test this branch. The workflows that use third-party actions or the metadatastician/standards reusable workflows (Rust CLI Tests, Secret Scanner, Governance, Compilation Tests, Lean Verification Build, ECHIDNA Validation, Correspondence Validation) end in startup_failure before any job starts. The most likely cause is an Actions policy change; it is not in this diff. The results below come from a Linux 6.18 container. They do not replace the required checks.

Tool versions: rustc, cargo and clippy 1.97.0 (2026-07-07); rustfmt 1.9.0-stable.

Head 5038be1 (after #244 was merged into this branch)

Run in impl/rust-cli/:

Command Result
cargo fmt --check clean
cargo clippy --all-targets --all-features -- -D warnings exit 0, no warnings
cargo test 887 passed, 0 failed, 14 ignored

Earlier head 0c643700 (this PR's own commits only)

Command Result
cargo fmt --check clean
cargo clippy --all-targets --all-features -- -D warnings exit 0, no warnings
cargo test 865 passed, 0 failed, 15 ignored (827 / 0 / 15 on main before this change)
cargo test --test glob_bounds_tests 14 passed
cargo build --release && ./target/release/vsh --version vsh 0.9.0

Fuzzing at that head, with cargo-fuzz 0.13.2 on nightly with AddressSanitizer:

  • fuzz_glob_expansion ran 382,036 inputs in 600 s with -rss_limit_mb=2560; nothing found.
  • After the fixture fix (4323498) it ran another 31,108 inputs in 60 s; nothing found.
  • The CI out-of-memory input /**/* (base64 LyoqLyo=) replays cleanly.

Not run locally: cargo test in ffi/rust, the Lean build, gitleaks, the code-coverage gate, and fuzzing at 5038be1.

The PR stays blocked on the required scan / gitleaks check until those workflows can start.


Generated by Claude Code

Repository owner deleted a comment from github-actions Bot Nov 18, 2026
@hyperpolymath
hyperpolymath enabled auto-merge (rebase) November 18, 2026 14:56
@hyperpolymath
hyperpolymath enabled auto-merge (rebase) November 18, 2026 14:57
Stacked on #242
(`claude/gifted-planck-mha9c9`), which adds
`glob::expand_glob_with_limits` and `GlobLimitExceeded`. Once #242
merges, retarget this PR to `main`.

## Problem (measured on the debug binary, directory with `a.txt b.txt
c.log`)

| Command | Before | Now (same as bash) |
|---|---|---|
| `wc -c *.txt` | `wc: '*.txt': No such file or directory` | `0 a.txt` /
`0 b.txt` / `0 total` |
| `ls *.txt \| wc -l` | `ls: cannot access '*.txt'` | `2` |
| `echo *.txt` | `*.txt` | `a.txt b.txt` |
| `echo '*.txt'` | `\*.txt` | `*.txt` |
| `echo a{1,2}` / `echo $((2+3))` | `a{1,2}` / `$((2+3))` | `a1 a2` /
`5` |

Cause: argument globbing lived only in `external::expand_glob_args`,
which only `execute_external_with_redirects` called. There was also a
second bug: quoted metacharacters were marked with a backslash in the
token string, and nothing ever removed it.

## Change: typed word IR

- New `src/word.rs`. A `Word` records the quoting of each piece
(unquoted, `'…'`, `"…"`, `\x`). `expand_words` runs, in order: tilde,
parameter, command and arithmetic expansion, process substitution, then
brace expansion, glob expansion and quote removal.
- The tokenizer builds a `Word` next to the old string form.
`Command::External`, `Command::Pipeline` and `Command::Echo` now carry
`Vec<Word>`.
- The new stage is used for external commands (foreground, background,
with or without redirections), every pipeline stage, `echo`,
function-call arguments, and the `External` branch of command
substitution and process substitution.
- Kept from #242:
- A pattern with no match, or an invalid pattern, stays as the literal
word.
- A `GlobLimitExceeded` fails the command without running it (`vsh:
prog: …`, exit 1).
- `external::expand_glob_args` is removed. Its rules and tests moved to
`word.rs`.
- Two smaller fixes:
- `$(…)` is now read to its matching `)`, so `$((1+(2*3)))` is a single
substitution.
  - `$(echo …)` is supported.

### Behaviour changes
- An expansion error used to come back from `execute()` as `Err`. In
`-c` mode that printed `Error:` plus a backtrace. Now the command exits
with status 1 and does not run.
- An unquoted `$X` whose value contains glob characters is now globbed,
as POSIX requires. A quoted `"$X"` is not.

### Not migrated in this PR
These still read the old string encoding, so their behaviour is
unchanged and still differs from bash:
- builtin operands (`touch '*q'` creates `\*q`; `ls *.txt` as a builtin
does not glob)
- redirection targets
- assignment values (`X='*.txt'` stores `\*.txt`)
- `case` patterns, `test`/`[[ ]]` arguments, `for` word lists

Field splitting is still not done. All of this is documented in
`docs/POSIX_COMPLIANCE.adoc`, Milestones 5 and 6.

## Tests
- `tests/word_expansion_e2e_tests.rs` runs the `vsh` binary. It covers:
- an external command without a redirect, with a redirect, and in a
pipeline
  - builtin `echo`
- quoted patterns `'…'`, `"…"` and `\*`, which stay literal with no
backslash
  - a pattern that matches nothing
  - a partly quoted word (`'*'x.*`)
  - braces
  - function arguments
- Unit tests in `word.rs`: the limit error, an invalid pattern, a
literal backslash inside a pattern, quoted vs unquoted parameters, and
refused process substitution.
- A proptest in `parser.rs`: `Display` then `tokenize` gives back the
same `Word` (256 cases). The domain excludes backticks inside double
quotes, because the tokenizer has no `\`` escape.
- Differential run of 30 commands across the old binary, the new binary
and bash: every changed result now matches bash, and no builtin case
changed.
- `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings`:
clean.
- `cargo test`: 887 passed, 0 failed, 14 ignored (the same ignored set
as before: stress tests, the model oracle and 2 in the lib).

---
_Generated by [Claude
Code](https://claude.ai/code/session_01J4QvDfX8Xoq6vedAKdWBif)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
coderabbitai[bot]

This comment was marked as off-topic.

…242

Verified findings from the CodeRabbit review of PR #242:

- word.rs:430 (major): every brace alternative and every word started
  with a fresh glob budget. With up to 10,000 brace alternatives, one word
  could do 10,000 times the intended work, which defeats the bound #191
  adds. glob::expand_glob_continuing starts the meters from the usage
  already spent; word expansion threads that usage through every word and
  alternative of a command, so the error still reports the configured
  limit. New test: five one-match patterns, as brace alternatives and as
  separate words, now exceed a four-match limit.
- glob.rs:415: a match refused by the byte limit still raised the match
  peak by one. Meter::check tests a charge without recording it; both
  limits are checked before either is charged. The byte-limit test now
  asserts both peaks.
- confirmation.rs:221: a symbolic link given as the root was followed.
  The root is now read with symlink_metadata and counted as one entry.
- fuzz_glob_expansion.rs:69 (major): a fixture that could not be built
  made the target skip every input, a green run that tested nothing. It
  now stops with a clear panic message.
- glob.rs:612: the differential test oracle now escapes the temporary
  directory path before using it as a glob prefix.
- EXPLAINME.adoc: the bash compatibility claim names its one documented
  difference.

cargo fmt --check, cargo clippy --all-targets --all-features -D warnings:
clean. cargo test: 889 passed, 0 failed, 14 ignored. Fuzz target rebuilt
under ASan; 23,410 runs in 60 s, no crash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDhGsL6zrtXcfEnvKbC

Copy link
Copy Markdown
Owner Author

Response to the CodeRabbit review (26 inline findings)

Fixed in b600ca3. Locally: fmt and clippy clean; cargo test 889 passed, 0 failed, 14 ignored.

Finding Outcome
word.rs:430 One glob budget for all brace alternatives (major) Fixed, for the words of a command as well: glob::expand_glob_continuing starts the meters from the usage already spent. New test covers both cases.
glob.rs:415 Commit both match charges only after both limits accept Fixed with Meter::check; the test now asserts both peaks.
fuzz_glob_expansion.rs:69 Report fixture failure (major) Fixed: the target panics with a clear message instead of skipping every input.
confirmation.rs:221 Root symbolic link Fixed: a link given as the root counts as one entry.
glob.rs:612 Escape the temporary directory in the test oracle Fixed.
EXPLAINME.adoc:215 Qualify the bash claim Fixed.
glob.rs:449 Charge reads before resolving a link target Not changed. The extra stat happens only for an entry that was already charged, so the work stays bounded by the read limit.
glob.rs:490 Invalid bytes compared with a literal U+FFFD Not changed. Names are matched after lossy decoding, so only a pattern that contains U+FFFD itself can tell them apart. Documented edge case.
grade.rs:253 Overflow before the limit compare Not changed. Addition saturates, so a wrong comparison would need a configured limit of u64::MAX.
confirmation.rs:225 Race between check and open Not changed. It is a race window, not a loop: an entry is only entered when it is a real directory at the time it is read.
fuzz_glob_expansion.rs:67, :86 Fixture owner, literal prefix Not changed. libFuzzer exits without running destructors, and tempfile names hold no glob metacharacters.
CHANGELOG.adoc:54 Read-limit units Not changed. The module docs define a read as a directory opened, an entry listed or a name looked up.
PROOF-OPEN-FRONTIER.adoc:122 Add A-6 to the proof ledger Left for the maintainer: the ledger row format and status are their decision. A-6 is tested-only.
executable.rs:509 (major), parser.rs:969 (major), parser.rs:1184, parser.rs:2381, word.rs:213, word.rs:341, word.rs:363 (major), word.rs:427 (major), tests/word_expansion_e2e_tests.rs:49 (major) and :67, parser.rs:4294, CHANGELOG.adoc:22/:28, POSIX_COMPLIANCE.adoc:185 Not addressed here. These are in the word-expansion change from #244, which was merged into this branch; I have asked the maintainer whether to fix them in this PR or separately.

Generated by Claude Code

…built

Hypatia's code_safety scan flagged the panic! added in b600ca3
(panic_macro, CWE-754). CodeRabbit had asked that a fixture failure
fail the run rather than skip every input. Printing the error and
calling std::process::exit(1) satisfies both: libFuzzer reports an
exiting target as an error.

Checked with TMPDIR pointing at a missing directory: the run prints
"cannot build the fixture tree" and "libFuzzer: fuzz target exited"
and exits non-zero. A normal 30 s run (10,012 inputs) finds nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J4QvDhGsL6zrtXcfEnvKbC
Repository owner deleted a comment from github-actions Bot Nov 18, 2026
- parser: the statement splitters walk bytes and looped for ever on an
  unquoted non-ASCII letter (`echo é` hung); only an ASCII letter can
  start a keyword.
- glob: brace expansion used character counts as byte offsets and
  panicked on `'é'{a,b}`.
- parser: ${...} parsing sliced at character counts; it now uses byte
  offsets (${é}, ${é:-x}).
- parser: ${X:2:N} with a huge N overflowed; the end is now saturating.
- parser: a quoted or escaped parenthesis no longer ends $(...).
- word: a quoted hyphen inside [...] is written last in the class, where
  it cannot form a range; quoted *, ? and [ stay bare inside a class.
- word: a tilde whose prefix has a quoted piece is not expanded.
- word, parser: an unbraced parameter followed by a literal is written
  back with braces, so an AND-OR list reads $A''b as ${A}b, not $Ab.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136P7ZnLF9EM2PnNxFaCFov
coderabbitai[bot]

This comment was marked as resolved.

Code
- parser: an AND-OR list writes its operands back from the typed word, so
  each piece keeps its quoting (`echo '*'x.* && true` lost the glob).
- word, external: expanded fields are OsString; a glob match reaches an
  external command with the exact bytes of its file name.
- glob: a name that is not UTF-8 is matched through a private-use mapping,
  not a lossy copy, so a literal U+FFFD in a pattern no longer matches it.
- glob: resolving a symbolic link in is_dir is a charged read.
- glob: new limit max_listing_bytes for the names of one directory listing.
- glob: the walker's stop error is boxed; glob::escape is new.
- grade: Meter::check computes the attempted level in u128 before the
  grades are composed; Exceeded::level is u128.
- confirmation: the no-follow count is documented as holding for a tree
  that does not change.

Fuzz
- fuzz_glob_expansion removes its fixture at exit (libc::atexit), escapes
  the fixture root in absolute patterns and checks that /**/* reaches it.

Tests
- charged link lookups, the listing limit, exact name matching, the meter
  overflow, list operands, exact argument bytes, a glob in a later pipeline
  stage; wc output is compared without column padding.

Documents
- CHANGELOG: walker read units defined; the exit-status claim is limited
  to argument expansion.
- POSIX_COMPLIANCE: Milestones 5, 6 and 12 are Partial in the overview and
  the detail; `for` word lists do not expand globs.
- PROOF-OPEN-FRONTIER A-6 and the EXPLAINME proof ledger: fixture evidence
  is tested-only, the bound is open; Lean is model evidence only, the Rust
  walker needs Creusot or Verus.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136P7ZnLF9EM2PnNxFaCFov
…rnal path

test_double_quotes_with_expansion parsed `touch`, which is a builtin, so
its Command::External arm never ran, and that arm used the Display form of
the word as if it were the expanded argument. The test now parses an
external command, expands its typed words with word::expand_words, and
checks parameter expansion and quote removal. The builtin form is checked
for the exact value.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136P7ZnLF9EM2PnNxFaCFov
coderabbitai[bot]

This comment was marked as resolved.

A non-final literal component was charged in run() and again in is_dir(),
so real/file cost three reads for two lookups and a budget of two failed
early. The literal is now charged only in the final-component branch;
is_dir charges the intermediate lookup. Adds a regression test.

Answers the CodeRabbit finding on glob.rs line 582 of #242.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136P7ZpKjBuoCRahUbNPywX
Hypatia flagged std::mem::forget in JobTable::drop (mem_forget, CWE-401,
high). ProcessSubstitution::leave_running now drops the Child handle
(which neither kills nor waits) and marks the substitution as not owned,
so its Drop keeps the FIFO and the child. Behaviour at shell exit is
unchanged: a job that still runs can still open its FIFO.

Test written first: a_substitution_left_running_keeps_its_fifo_and_its_child
(failed with a plain drop). cargo test 928 passed, 0 failed, 14 ignored;
fmt, clippy -D warnings and the fuzz check pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136P7Zk179dJt4BhyPzDhFN
coderabbitai[bot]

This comment was marked as resolved.

@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor
Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Owner ruling AD1: a SIGCHLD handler that only sets a flag, which the
reaper checks first, with polling at safe points as the fallback.

- job.rs: on_sigchld stores to a lock-free AtomicBool (async-signal-safe:
  no allocation, lock, I/O or errno change). install_sigchld_handler uses
  sigaction with SA_RESTART. reap_finished returns at once when the
  handler is installed and the flag is clear; it clears the flag before
  the poll, so a signal during the poll is seen at the next safe point.
  Without the handler it polls every time.
- main.rs: install the handler at start (a failure only warns). Drop the
  job table before std::process::exit, which runs no destructors: ended
  jobs of a -c or script run were never cleaned at exit.

Tests written first: the handler sets the flag (failed with a stub); a
raw read(2) interrupted by SIGCHLD is restarted (failed with EINTR when
SA_RESTART was left out); a script's read builtin gets its line while a
job ends during the read (a guard); a job that ended before exit leaves
no FIFO (failed: FIFO left). cargo test 932 passed, 0 failed, 14 ignored;
fmt, clippy -D warnings and the fuzz check pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136P7Zk179dJt4BhyPzDhFN
… notices in scripts

CodeRabbit review on 905833b:

- A job that kill marked Done was never polled, so its substitutions were
  kept until exit. The reaper now polls every job it has not reported yet,
  whatever its display state, and reports each job once.
- A finished job's substitutions were cleaned with the blocking
  cleanup(), so a connected >(cmd) that worked on made the next command
  and the exit wait for it. They now go to a finishing list that is
  checked at each safe point with ProcessSubstitution::try_finish (one
  release, one try_wait, no wait). fg uses the same path. At exit a
  substitution that still works is left to finish, like one of a job
  that still runs (issue 245).
- Done notices are queued only in an interactive shell; a script or -c
  run kept every notice.
- The background-job binary tests read /proc, so they run on Linux only.
- The SIGCHLD handler tests move to their own test binary: they install a
  process-wide handler and read a global flag.

Tests written first and seen failing: a job marked Done before it was
reaped is still reaped; a script queues no notices; a connected
substitution of an ended job blocks no command (the shell took 2.0 s,
now well under 1.5 s, and the substitution still writes its file).
cargo test 935 passed, 0 failed, 14 ignored; fmt, clippy -D warnings
and the fuzz check pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136P7Zk179dJt4BhyPzDhFN

Copy link
Copy Markdown
Owner Author

@coderabbitai review

Head 3e53e52 answers your 4 findings on 905833b (killed jobs reaped, no blocking on a working substitution, no notices in scripts, Linux-only /proc tests). It also adds the SIGCHLD flag handler of owner ruling AD1 (1e6ba50). Local checks: cargo test 935 passed, 0 failed, 14 ignored; fmt, clippy -D warnings and the fuzz check pass.


Generated by Claude Code

Copy link
Copy Markdown
Owner Author

@coderabbitai review

The previous request was rate limited. Head 3e53e52 has not been reviewed yet; it answers the 4 findings of your review on 905833b, and all 16 CI checks pass on it.


Generated by Claude Code

@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor
Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai[bot]

This comment was marked as resolved.

CodeRabbit review on 3e53e52: the test sent a pthread_t through a
channel. On musl pthread_t is *mut c_void, which is not Send, so the
test would not compile there. JoinHandleExt::as_pthread_t gives the
handle on the parent side; the reader stays blocked in read(2) until
the byte is written after pthread_kill.

Checked: the test still fails with EINTR when SA_RESTART is left out,
and passes with it. cargo test 935 passed, 0 failed, 14 ignored; fmt,
clippy -D warnings and the fuzz check pass. Not compiled for a musl
target here (only x86_64-unknown-linux-gnu is installed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136P7Zk179dJt4BhyPzDhFN

Copy link
Copy Markdown
Owner Author

@coderabbitai review

Head 8367afd fixes the one finding of your review on 3e53e52 (the pthread_t channel transfer); that thread is resolved. All 16 CI checks pass on 8367afd, and no review thread is open.


Generated by Claude Code

@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor
Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai[bot]

This comment was marked as resolved.

CodeRabbit review on 8367afd: std's RawPthread is c_ulong on every Linux
target, but libc::pthread_t is *mut c_void on musl, so pthread_kill would
not type-check there. The cast is a no-op on gnu (clippy -D warnings
accepts it).

cargo test 935 passed, 0 failed, 14 ignored; fmt and clippy pass. Not
compiled for a musl target here (only x86_64-unknown-linux-gnu).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0136P7Zk179dJt4BhyPzDhFN
@sonarqubecloud

Copy link
Copy Markdown

Repository owner deleted a comment from github-actions Bot Nov 20, 2026
Repository owner deleted a comment from github-actions Bot Nov 20, 2026
@hyperpolymath
hyperpolymath enabled auto-merge (squash) November 20, 2026 10:21
Repository owner deleted a comment from github-actions Bot Nov 20, 2026
Repository owner deleted a comment from coderabbitai Bot Nov 20, 2026
Repository owner deleted a comment from github-actions Bot Nov 20, 2026
Repository owner deleted a comment from coderabbitai Bot Nov 20, 2026
@hyperpolymath
hyperpolymath dismissed coderabbitai[bot]’s stale review November 21, 2026 14:18

needs work on proven tests, until the not clear if this is the correct 'cure', next stage

Repository owner deleted a comment from github-actions Bot Nov 21, 2026
@hyperpolymath
hyperpolymath enabled auto-merge (rebase) November 21, 2026 14:19
Repository owner deleted a comment from coderabbitai Bot Nov 21, 2026
@hyperpolymath
hyperpolymath enabled auto-merge (rebase) November 21, 2026 14:20

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants