Skip to content

fix: validate an sg rule's language field in verify - #173

Open
thecodedrift wants to merge 1 commit into
mainfrom
fix/verify-sg-language
Open

fix: validate an sg rule's language field in verify#173
thecodedrift wants to merge 1 commit into
mainfrom
fix/verify-sg-language

Conversation

@thecodedrift

Copy link
Copy Markdown
Member

What

taskless verify returned ok: true for any language: spelling. The vendored
ast-grep-rule-schema.json types $defs.Language as a bare string with no enum
(its only hint is an example reading "typescript", which is not even the
canonical spelling), and verify never read the field — so the ast-grep binary
was the first thing with an opinion, at check time, in both of its bad ways:

  • Unrecognized name. SgLang deserialization fails, which aborts parsing of
    the single config Taskless assembles per run. One typo blinds the whole sg
    engine.
  • Recognized name, wrong parser. Tsx and TypeScript are two parsers, not
    aliases. A TypeScript rule scoped to **/*.tsx matches nothing, exits zero,
    and is indistinguishable from a clean codebase.

verify now resolves language: the way ast-grep does and reports both.

Decision 1 — case variants are accepted, with a notice

Accept, and name the canonical spelling in a notice. Hard rejection was the
other option and it is a breaking change for rules that demonstrably work:
onboarding this repository produced four rules spelled typescript, and that
lowercase spelling is the one ast-grep's own JSON Schema shows as the field's
example. Failing them would turn a green project red over a spelling the
engine itself resolves.

Measured against the pinned 0.41.0 binary, through a real rule config:

Spelling Result
TypeScript, Tsx, Cpp, … accepted, canonical
typescript, TYPESCRIPT, Cs, GOLANG, JsX accepted — matching is case-insensitive throughout
ts, tsx, js, jsx, py, rb, rs, kt, hs, ex, sol, cc, cxx, c++, cs, yml, golang accepted off-list aliases
C#, nonsense, h, hpp, mjs, cjs, sh, tf, csx, htm rejected — did not match any variant of untagged enum SgLang
"ts " rejected — whitespace is not folded

Two of those rows changed the implementation. golang is a real alias and was
missing from the first draft of the table. And because "ts " is rejected by
the binary, resolveAstGrepLanguage deliberately does not trim: a trimming
resolver would pass a rule ast-grep refuses to load.

One more measured surprise worth recording: a rule's language: field and
sg run --lang do not share a vocabulary.
--lang C++ is rejected outright
while language: C++ parses fine. Everything here was probed through a config,
because that is the only thing a rule file is ever fed to.

Decision 2 — where the list comes from

AST_GREP_LANGUAGES was already derived-by-pinning: set-equality against
sg run -h, so a version bump that adds or drops a language fails loudly.

The alias table cannot work that way — sg run -h prints only the canonical
list, and nothing in the binary enumerates aliases. capabilities.ts is also a
Worker-safe pure-data module (the build fails if the prompts chunk reaches a
host capability), so it cannot spawn the binary itself.

So the aliases are transcribed there and pinned by probing in
test/ast-grep-vendor-contract.test.ts:

  • Every alias is fed to the binary in a real config, and its resolution is read
    back out of the scan stream's own language field. That field names the
    parser ast-grep settled on, so the mapping is the binary's answer rather than
    ours inferred from which files got scanned.
  • Every canonical name is fed to it lowercased, pinning the case-insensitivity
    verify relies on.
  • A sweep of near-misses (h, hpp, mjs, cjs, sh, tf, csx) asserts
    rejection, which is the direction the table cannot self-check: a bump that
    adds an alias would otherwise leave verify failing a rule that now works.

Those probes key on SgLang in stderr, not on the exit status. Every rule that
fails to load exits 8 with the same top line, and pattern: zzz is legitimately
unparseable in some grammars (Html wants a kind) — so status alone cannot
tell "not a language" from "not a pattern".

Measured output

pnpm build, then verify against deliberately broken rules:

✗ sg/bad-lang
    language: "nonsense" is not a language ast-grep 0.41.0 accepts. It aborts config parsing, so every other sg rule in the project goes unreported too. Accepted spellings: Bash, C, Cpp, CSharp, Css, Elixir, Go, Haskell, Hcl, Html, Java, JavaScript, Json, Kotlin, Lua, Nix, Php, Python, Ruby, Rust, Scala, Solidity, Swift, Tsx, TypeScript, Yaml.
✗ sg/csharpish
    language: "C#" is not a language ast-grep 0.41.0 accepts. It aborts config parsing, so every other sg rule in the project goes unreported too. Did you mean "CSharp"? Accepted spellings: Bash, C, Cpp, …
✗ sg/wrong-parser
    files: every glob names .tsx, but language is TypeScript. TypeScript does not parse .tsx files — Tsx is a separate parser, not an alias, so those globs match nothing and check reports a clean codebase.
✗ sg/no-eval
    No test file found for rule "no-eval" in .taskless/rules/sg/no-eval/.tests/
    notice: language: "typescript" works — ast-grep resolves it to TypeScript — but TypeScript is how ast-grep spells it.

sg/no-eval is this repository's own rule: the failure there is the missing test
file that main already has, and the new line is the notice — the exact case the
issue reported as invisible.

Notes on the issue text

The issue says ast-grep "accepts case variants (cpp and C++ both reach
Cpp)". C++ is not a case variant of Cpp — it is a separate alias, and the
two live in different places in the fix. The accepted vocabulary is the
canonical list compared case-insensitively, plus sixteen aliases
, which is a
wider set than the issue implies.

The issue's "related smaller item" — warning when a TypeScript rule's files:
globs reach .tsx — is included, since it is the only form the wrong-parser trap
takes that is checkable from the rule file alone. It is an error when every
glob names the sibling extension (the rule is entirely dead) and a notice when
only some do (half the scope still reports real findings).

Checks

pnpm typecheck, pnpm lint, pnpm test (843 tests) all pass.

Fixes #165

The vendored ast-grep rule schema types `language:` as a bare string with no
enum and `verify` never read it, so any spelling returned `ok: true` and the
binary was the first thing with an opinion — at `check` time. An unrecognized
name aborts config parsing and takes every other sg rule down with it; a
recognized name for the wrong parser reports nothing and reads as a clean
codebase.

`verify` now resolves the field the way ast-grep does: case-insensitively,
against the canonical list plus the sixteen aliases the binary accepts. An
unresolvable name is an error naming the accepted spellings; a resolvable but
off-list one is a notice naming the canonical spelling, so the lowercase
`typescript` rules already in the wild keep passing. The Tsx/TypeScript split
is checked against `files:`, which is the form that trap takes in practice.

The alias table is the one list `sg run -h` will not enumerate, so it is pinned
by probing: the vendor contract feeds every alias to the binary through a real
config and reads the resolution back out of the scan stream's own `language`
field, and sweeps a set of near-misses to catch a bump that adds one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017cEN93Acyp4zBwP3oDnyy1
Copilot AI lite review requested due to automatic review settings August 25, 2026 04:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@thecodedrift

Copy link
Copy Markdown
Member Author

@claude /review

@claude

claude Bot commented Aug 25, 2026

Copy link
Copy Markdown

Claude finished @thecodedrift's task in 4m 11s —— View job


Review mode: incremental — read 0 prior review thread(s) before reviewing.

.prior-review.json contained no review threads and no prior review with actionable findings (the one copilot-pull-request-reviewer entry is a quota-limit notice, not feedback; the one comment is the @claude /review trigger itself). This is effectively the first review of this PR — nothing to treat as already addressed.

Summary

This is a careful, well-documented fix: validateLanguage in packages/cli/src/rules/verify.ts resolves a rule's language: field the way ast-grep itself does (case-insensitively against the canonical list, plus a small, binary-probed alias table in capabilities.ts), reports a hard error for names ast-grep can't parse at all, and a non-fatal notice for accepted-but-off-canonical spellings. It also adds the TypeScript/Tsx "wrong parser" check against files: globs. The vendor-contract tests probe every alias and every near-miss directly against the pinned 0.41.0 binary rather than assuming, which matches how the rest of this file already pins AST_GREP_LANGUAGES. verify.test.ts covers the error path, the notice path, the canonical-spelling silent path, the full/partial .tsx-scope cases, and the "leave a missing field to Layer 2" boundary.

I found one real gap, posted inline:

  • [New] packages/cli/src/rules/verify.ts (lines 236–243): the files: glob scan for the TypeScript/Tsx mismatch only handles the plain-string form of RuleFileGlob. The vendored ast-grep schema also allows an object form ({ glob: "...", caseInsensitive?: boolean }), which assemble.ts passes through unmodified. The code filters to typeof glob === "string" before extracting extensions, so a rule using the object form for its files: entries — e.g. files: [{ glob: "src/**/*.tsx" }] with language: TypeScript — silently produces no error and no notice. That's the exact "recognized name, wrong parser" trap this feature exists to catch, missed for one of the two legal shapes a glob entry can take.

Everything else — the alias table, the case-insensitive resolution, the deliberate no-trim behavior, the "leave missing language to the required-fields layer" boundary, the notice plumbing through inspect.ts/rules-verify.ts, and the JSON-output shape (additive notice? field, backward compatible) — checked out against the code and the PR's own measured tables.

Comment on lines +236 to +243
const files = ruleData.files;
if (own !== undefined && Array.isArray(files)) {
const sibling = own === "ts" ? "tsx" : "ts";
const siblingLanguage = own === "ts" ? "Tsx" : "TypeScript";
const named = new Set(
files
.filter((glob): glob is string => typeof glob === "string")
.flatMap((glob) => globExtensions(glob))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[New] The files: scan silently misses the object form of RuleFileGlob, which is exactly the shape it's meant to guard.

The vendored schema (src/generated/ast-grep-rule-schema.json, $defs.RuleFileGlob) allows each entry in files: to be either a plain glob string or an object { glob: string, caseInsensitive?: boolean }. assemble.ts passes files through unmodified, so ast-grep sees the object form as-is.

Here, though:

const named = new Set(
  files
    .filter((glob): glob is string => typeof glob === "string")
    .flatMap((glob) => globExtensions(glob))
);

non-string entries are dropped before extracting extensions. A rule like:

language: TypeScript
files:
  - { glob: "src/**/*.tsx" }

produces an empty named set, so named.has(sibling) is false and neither the error nor the notice fires — the exact "recognized name, wrong parser" trap this whole feature exists to catch (per the PR description's sg/wrong-parser example) goes undetected whenever the rule author uses the object form instead of a bare string. It's a silent false negative, not a crash, but it's the one shape a RuleFileGlob object can legally take that the check doesn't handle at all.

Worth at least unwrapping { glob } entries (glob.glob) before calling globExtensions, mirroring what ast-grep itself does.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

verify does not validate an sg rule's language field

2 participants