Validate Vale rule files against a schema, as ast-grep rules already are

[thecodedrift]

An ast-grep rule is structurally validated before it ever reaches the binary. A Vale rule is not, and the failures that gap allows are silent ones: the rule parses, verify returns ok: true, and it reports nothing forever.

The template

The ast-grep path is the shape to copy:

• packages/cli/scripts/fetch-ast-grep-schema.ts fetches schemas/rule.json from the ast-grep tag matching the pinned @ast-grep/cli version
• output lands in packages/cli/src/generated/ast-grep-rule-schema.json
• packages/cli/src/rules/verify.ts consumes it as the schema layer of VerifyResult, ahead of the requirements and test layers

packages/cli/src/rules/vale/verify.ts has no counterpart. All 318 lines are fixture behavior: bucket coverage, then which fixtures fired. Nothing reads the rule's own structure.

One thing to settle first

The ast-grep schema is generated, which is why it cannot drift from the binary. Vale is unlikely to publish an equivalent for its check types — worth confirming before designing around it. If it does not, the schema has to be hand-authored, and then drift is the whole risk.

The repository already has the answer to that: packages/cli/test/vale-vendor-contract.test.ts pins Vale's givens by invoking the pinned binary directly. A hand-authored schema should be held to the same standard, so a claim in the schema is one the binary was measured making, and a Vale upgrade fails the contract test rather than silently invalidating the schema.

What it would catch

Each of these is silent today. Measured on the pinned 3.18.0 binary unless noted.

Check	Failure it prevents
scope enum	scope: fenced (not a Vale scope) verifies clean. Only test catches it, reported as "fail fixture did not fire" — which points the author at their pattern, not at the invalid scope.
%s count per extends	substitution needs two, existence one. The recipe states outright that a wrong count "passes every check below" and only a human reading the message catches it. A schema can require two %s for substitution.
nonword: true for punctuation tokens	A token with no word characters can never match, because Vale wraps tokens in word boundaries. The em-dash rule in #104 has exactly this shape and would ship as a no-op. Expressible as if tokens contains /^\W+$/ then require nonword: true.
consistency id must be word characters only	The recipe records that a hyphen here fails with E201 invalid group name and takes every Vale rule in the project down, since Vale reads one config per run.
required fields per extends	swap for substitution, token+max/min for occurrence, first/second for conditional, and the singular/plural token vs tokens split. Untested: I did not measure whether Vale errors or silently ignores a mismatched field, so this row needs checking before it is designed.

What it would not catch, and where those belong

• raw disables Vale's suppression comments (create-vale-rule: a rule about a command passes its fixtures and catches nothing #167, and the comment there) — runtime behavior, not structure.
• Bare words vs collocations — judgment. landed on fires on "the plane landed on time".
• Fixture design — when a rule's subject normally appears in code, the fail/ fixture must contain it inline, fenced, and in prose. That is a coverage check and belongs next to the existing bucket-coverage logic in rules/vale/verify.ts, not in a schema.

Note on the .vale.ini

Converting the ini to JSON and schema-validating it was considered and is not worth doing. verify already catches both of its silent failures, with good messages:

zz-probe has no .vale.ini, so nothing scopes it and it will never run.
zz-probe/.vale.ini never enables zz-probe.zz-probe, so the rule is present but off.

That is the missing-config case and the id-must-agree-in-four-places case, which are the two things a schema there would buy. The leverage is entirely on the rule file.

Refs #170
Refs #167

🤖 Generated with Claude Code

https://claude.ai/code/session_017cEN93Acyp4zBwP3oDnyy1

[thecodedrift]

The upstream schema question is settled: there is nothing to fetch

Vale's repository has no schemas/ directory and no JSON Schema files — the top level is .github/, .well-known/, cmd/vale/, internal/, testdata/, and config files. So the ast-grep pipeline (fetch the tag's schemas/rule.json, pin it, generate) has no counterpart, and the schema here must be hand-authored.

The schema does effectively exist, but behind Vale's paid cloud MCP server at api.vale.sh/mcp, part of Vale CMS. Its explain_check tool "describes what each check type detects, its fields, and examples," and scaffold_rule emits valid starter rules. That is the machine-readable field knowledge — remote, paid, and not something verify can depend on. Worth knowing it exists so nobody re-searches for a public schema; it is not a viable source.

The public sources are the docs: https://docs.vale.sh/topics/styles for the check types and their fields, https://docs.vale.sh/topics/scopes for scope values.

Measured against the pinned 3.18.0 binary

Docs are the starting point, not the authority. Findings that change the design:

Vale never validates scope at all. scope: fenced (not a scope), comment.line, and meta each parse clean, verify returns ok: true, and the rule simply matches nothing. An unknown scope is indistinguishable from a correct scope whose pattern did not fire. This makes the scope enum the single highest-value part of the schema — it is the one field where nothing downstream will ever catch a typo.

~ negation and & chaining are real and accepted, not just documented: ~code, [~code], and text & ~code all parse and behave. The create-vale-rule recipe presents scope as a single flat value and never mentions either, so a schema modeling scope as a plain enum would reject valid rules. It needs to accept a string, an array, and the &/~ operators.

raw & ~code fires on nothing, so negation does not provide the "see fenced blocks but skip this one" middle ground it looks like it might. That constraint (#167) is unchanged.

The scope vocabulary is much larger than the recipe's list: heading with levels, table.header/cell/caption, figure.caption, list, paragraph, sentence, blockquote, alt, summary, raw, the v3.17.0+ inline scopes (link, code, strong, emphasis), text.class.<name>, the v3.18.0+ meta/meta.class.<kind>, and comment.line/comment.block.

Correction to the "required fields per extends" row

I marked that row untested. It is now measured, and it is not silent — it fails loudly but late:

verify → ok: true
test   → Vale exited 2: E201: has invalid keys: 'tokens' in .../zz-probe.yml

That is the same E201 class that, per the recipe, takes every Vale rule in the project down, because Vale reads one config per run. So this row is not "catch a silent mistake" — it is "stop one malformed rule from blinding the engine," which is a stronger reason to have it, and it should be described that way.

Two more common fields the recipe omits

limit (max triggers per file) and vocab (default true). Neither appears in the recipe's field table. Also: Vale requires the .yml extension, not .yaml.

One discrepancy to resolve before writing the enum

The docs enumerate 11 check types. The MCP guide says scaffold_rule creates rules for twelve. Whoever implements this should reconcile that against the pinned binary rather than picking a number — an enum that omits a real check type would reject valid rules, which is worse than the gap it closes.

🤖 Generated with Claude Code

https://claude.ai/code/session_017cEN93Acyp4zBwP3oDnyy1