diff --git a/.changeset/vale-authoring-polish.md b/.changeset/vale-authoring-polish.md new file mode 100644 index 00000000..d09e76b6 --- /dev/null +++ b/.changeset/vale-authoring-polish.md @@ -0,0 +1,38 @@ +--- +"@taskless/cli": patch +--- + +Close the gaps in `create-vale-rule` that produce a rule which is green +everywhere and reports nothing. + +A malformed Vale rule fails loudly. A Vale rule that is merely _wrong_ passes +`verify`, passes `test`, and never fires — and nobody re-checks a green rule. +The recipe now documents each of those failures as an observed behavior of the +pinned Vale binary rather than as a caution in principle: + +- **The measured `scope` vocabulary**, with what each value actually reaches. + `raw` subsumes `code` and `text`; `~` negation and `&` chaining are accepted; + and a negation over a scope Vale does not know (`~fenced`) is a silent no-op + that removes the exclusion you wrote the rule for. +- **`scope` is per-rule.** Taskless assembles one config per run, which invites + the assumption that scopes interact. They do not. +- **A `raw`-scoped rule cannot be suppressed** by ``, + because it reads the unparsed document — so a rule about a shell command needs + `raw` and trades away per-case exemption. +- **A punctuation-only token needs `nonword: true`**, because Vale wraps every + token in word boundaries and an em dash has no word character on either side. +- **How to scope a rule _out_**, with a second matcher assigning `NO`. +- **Collocation guidance** for a banned word, checked by writing the `pass/` + fixture from the literal sense first. +- **Fixture design for a subject that appears in code**: `fail/` must carry it + inline, fenced, and in prose. +- **`limit` and `vocab`** in the common-fields table, and that Vale loads only + `.yml` — a style file renamed to `.yaml` is silently not loaded at all. +- **Fixtures run under an isolating config**, so a green `test` is not evidence + the rule's matcher glob reaches any real file. + +Vale's check types are now enumerated from the binary rather than the docs. +There are **twelve**, not eleven: the docs fold `readability` into `metric`. +The per-check field tables are measured the same way, which corrects two +published claims — `capitalization` rejects `prefixes`/`suffixes`, and +`occurrence` rejects `exceptions`. diff --git a/openspec/changes/vale-authoring-polish/.openspec.yaml b/openspec/changes/vale-authoring-polish/.openspec.yaml new file mode 100644 index 00000000..e685d45e --- /dev/null +++ b/openspec/changes/vale-authoring-polish/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-08-25 diff --git a/openspec/changes/vale-authoring-polish/design.md b/openspec/changes/vale-authoring-polish/design.md new file mode 100644 index 00000000..a1031baf --- /dev/null +++ b/openspec/changes/vale-authoring-polish/design.md @@ -0,0 +1,197 @@ +## Context + +`verify` runs three layers against an `sg` rule: the ast-grep JSON schema, the Taskless required fields, and then the tests. For a `vale` rule it runs the last two only. `packages/cli/src/rules/vale/verify.ts` is 318 lines of fixture behavior — bucket coverage, then which fixtures fired — plus a `level` check and the `.vale.ini` presence checks. Nothing reads the rule's structure. + +That gap is not theoretical, and the engine does not close it. Measured against the pinned Vale 3.18.0: + +| Rule defect | `verify` | `test` | `check` | +| --------------------------------- | -------------- | --------------------------- | ------------------------------ | +| `level: bananas` | **reports it** | — | — | +| `extends: nonsense` | `ok: true` | "fail fixture did not fire" | silent | +| `scope: fenced` | `ok: true` | "fail fixture did not fire" | silent | +| `tokens` on an `occurrence` check | `ok: true` | `E201: has invalid keys` | **every Vale rule suppressed** | + +The two middle rows are the shape that matters. The author is told their pattern did not match, so they go and edit the pattern. The fourth row is worse in a different direction: Vale reads one assembled config per run, so one stray field takes the whole engine's reporting down. + +The ast-grep side avoids this because ast-grep publishes `schemas/rule.json` per tag, and `packages/cli/scripts/fetch-ast-grep-schema.ts` pins it to the vendored version. Vale publishes nothing equivalent. Its repository has no `schemas/` directory; the machine-readable field knowledge exists behind the hosted MCP server at `api.vale.sh/mcp`, which is part of the paid Vale CMS product. + +## Goals / Non-Goals + +**Goals:** + +- `verify` rejects a Vale rule whose `extends`, `scope`, or per-check fields the binary will not honor, before Vale is invoked. +- The schema's claims are held to the vendored binary, so a Vale upgrade fails loudly rather than silently invalidating them. +- The `create-vale-rule` recipe documents the silent failures, which is the only place they can be prevented for the cases a schema cannot see. + +**Non-Goals:** + +- Depending on Vale's hosted MCP server at author or verify time. It is remote and paid; `verify` must work offline. +- Validating regex _semantics_. Whether a token is too broad, or whether `landed on` also matches "the plane landed on time", is judgment. That belongs in the recipe. +- Changing what `check` reports for any rule that is valid today. +- Reworking the `.vale.ini`. Converting it to JSON and schema-validating it was considered and rejected — `verify` already catches both of its silent failures ("no `.vale.ini`, so nothing scopes it" and "never enables `.`"), which is what a schema there would buy. + +## Decisions + +### Hand-author the schema, pinned to `VALE_VERSION` + +There is no upstream artifact to generate from, so the choice is between transcribing the docs and not having a schema. Transcribe — but treat the transcription as untrusted. + +_Alternative considered: derive the schema by probing the binary at build time._ Attractive, because it cannot drift. Rejected because Vale does not report its accepted values; the only probe available is "author a rule and see whether it fires", which distinguishes "invalid" from "valid but did not match" only through a fixture that is itself hand-written. That is the vendor-contract test, and it belongs in the test suite rather than in a generator. + +_Alternative considered: query the hosted MCP `explain_check` at build time and vendor the result._ Rejected. It makes a build step depend on a paid third-party service, and the output would still need pinning and checking. + +### The vendor-contract test is what makes the transcription safe + +`packages/cli/test/vale-vendor-contract.test.ts` already pins Vale's givens by invoking the vendored binary directly rather than through `runVale`. Extend it, so that every value the schema asserts is one the binary was measured accepting. A Vale bump that adds or removes a check type then fails a test that names the field, instead of leaving the schema quietly wrong. + +This is the same relationship `capabilities.ts` already has with the format tiers, and the precedent is deliberate: an engine claim in this repository is quoted from a pinned binary rather than from release notes. + +### `scope` is a grammar, not an enum + +The recipe presents `scope` as a single value. Measured, the binary accepts a bare string, a list, `~` negation, and `&` chaining — `~code`, `[~code]`, and `text & ~code` all parse and behave. A schema modeling `scope` as a flat enum would reject valid rules, which is worse than the gap it closes, so the enum applies to the _operands_ and the schema accepts the operators around them. + +The vocabulary is also larger than the recipe's list: `heading` with levels, the `table.*` and `figure.caption` forms, `list`, `paragraph`, `sentence`, `blockquote`, `alt`, `summary`, `raw`, the v3.17.0+ inline scopes (`link`, `code`, `strong`, `emphasis`), `text.class.`, the v3.18.0+ `meta` and `meta.class.`, and `comment.line` / `comment.block`. + +`scope` is the highest-value field in the schema, because it is the one field nothing downstream ever validates. An invalid scope is not an error anywhere in the stack — it is a rule that matches nothing. + +### The corpus is the artifact; the schema is derived from it + +The schema is a transcription, so the thing that makes it true is not how carefully it is written — it is a corpus of rule files, each run through **both** the vendored binary and the schema, asserting the two agree. Build the corpus first and let the schema fall out of it, rather than writing the schema and adding tests afterwards. + +Each corpus entry is a minimal rule YAML plus the verdict the binary was measured giving it. The test asserts `schemaAccepts === binaryAccepts` for every entry, and both directions of disagreement are failures: + +- **Schema accepts, binary rejects** — the gap this change exists to close. +- **Schema rejects, binary accepts** — worse, because it blocks work that would have functioned. This is the direction to be paranoid about, and it is why an unclear value is accepted rather than rejected. + +**Deciding the binary's verdict is the tedious part, and it cannot be read off an exit code.** Vale does not report "this is not a scope." An unrecognized `extends` or `scope` parses clean and produces a rule that matches nothing — indistinguishable, from the outside, from a valid rule whose pattern did not fire. So every corpus entry needs a **positive control**: a fixture document the rule _must_ flag when the construct is valid. "Fired" then means accepted, "did not fire" means the binary ignored it, and `E201` means rejected outright. Three outcomes, not two. + +That control has to be written per entry, because it depends on what the rule matches — a `scope: heading` entry needs a heading, a `scope: table.cell` entry needs a table. There is no way to generate it from the check type alone, and this is the bulk of the work. + +Keep the corpus declarative — a table of `{ check, field, value, fixture, expected }` rather than a test per case — so that a Vale upgrade is a re-run rather than a re-authoring, and so a gap in coverage is visible as a missing row rather than as an absent test nobody notices. + +_Alternative considered: assert the schema against the docs and skip the corpus._ Rejected. It would encode the same transcription twice and prove only that we copied consistently. The corpus is the only thing that can catch a docs page that is wrong or stale about the pinned version — and the docs are already inconsistent with the MCP guide about how many check types exist. + +### Follow the `sg` shape rather than inventing one + +`verify` already has a `schema` layer with a `LayerResult` and an error list. The Vale path reports through the same `VerifyResult`. Reuse both, so the two engines fail the same way and the JSON output shape does not gain a special case. + +### Resolve the check-type count against the binary, not the docs + +The docs enumerate eleven check types. The MCP guide says `scaffold_rule` covers twelve. Do not pick a number: enumerate against the vendored binary and record what was measured. An enum that omits a real check type rejects valid rules. + +## Risks / Trade-offs + +- **The schema drifts from a Vale upgrade.** → The corpus is re-run against the new binary and the disagreement names the field. This is the whole reason the corpus is a requirement rather than a nicety. +- **The corpus is the expensive part and will be under-built.** Every entry needs a hand-written positive control, and an entry that silently lacks one asserts nothing — it "passes" because nothing fired on either side. → Assert that each entry's control fires for at least one known-valid variant, so a control that can never fire is itself a test failure rather than a quiet pass. +- **The schema is stricter than the binary and rejects a valid rule.** → Worse than the gap it closes, because it blocks work that would have functioned. Mitigated by deriving every enumerated value from a measurement, and by the `scope` grammar decision above. When a value's status is unclear, accept it. +- **A rule in the wild fails `verify` after this lands.** → Only if it was already inert or already suppressing the engine. Both are defects the author wants to know about. No rule that reports findings today can start failing. +- **Recipe growth.** `create-vale-rule` is already the longest recipe, and this adds to it. → The additions are failure-mode statements with measurements attached, which is the material the recipe is densest in already. If length becomes the problem, the split is by engine surface, not by trimming the measured parts. + +## Migration Plan + +No migration. The schema layer is additive, and it only rejects rules that the engine was already going to ignore or fail on. + +The stack lands forward in two units, per the proposal: the recipe text first, then the schema and its `verify` layer. If the second unit needs to be reverted, the first stands on its own. + +## Measured against Vale 3.18.0 + +Everything below was obtained by authoring a rule and running the vendored +binary directly, in a temporary project with `BasedOnStyles =` so no bundled +style could contribute a finding. Where a verdict is "did not fire", a +known-valid control was run against the same document first. + +### Twelve check types, and the binary says so itself + +An `extends` outside the set is **not** silently ignored. Vale rejects the file, +exits 2, and names the whole set: + +``` +'extends' key must be one of [capitalization conditional consistency existence +occurrence repetition substitution readability spelling sequence metric script]. +``` + +That resolves eleven-versus-twelve without a judgement call: **twelve**. The +docs' eleven omits `readability`, which they fold into `metric`; the binary +treats them as separate checks with disjoint fields (`metrics`/`grade` versus +`formula`/`condition`), and each rejects the other's. + +### `scope` is a grammar, and the operand vocabulary is not the docs' + +Measured firing: `text`, `code`, `raw`, `heading`, `heading.h1`–`h6`, +`paragraph`, `sentence`, `list`, `blockquote`, `link`, `alt`, `summary`, +`strong`, `emphasis`, `table`, `table.header`, `table.cell`, `table.caption`, +`comment`, `comment.line`, `comment.block`, `frontmatter`, `frontmatter.`, +`text.class.`. + +Reach, on one document holding the token in prose, in an inline span, and in a +fenced block: `text` → 1, `code` → 1, `[code, text]` → 2, `raw` → 3. `raw` +subsumes the other two. + +`~` negation, `&` chaining, and the list form all parse and behave. **A negation +over an unrecognized operand is a silent no-op** — `~fenced` and `text & ~fenced` +both fire on everything, so a typo inside a `~` removes the exclusion rather than +narrowing anything. + +Two corrections to this document's own earlier draft: + +- **There is no `meta` scope.** The v3.18.0 addition is `frontmatter` and + `frontmatter.`, which is what the binary carries (`text.frontmatter.`) + and what fires. `meta` and `meta.class.` never fired in Markdown, + MDX, or HTML. +- **`figure.caption` fires, but only once you find a control it can reach.** + The obvious control — a `
` nested inside `
` — never fires, + and neither does a `scope: text` control over the same document: Vale drops + everything inside a `
` element before any check runs. A bare + `
` is linted normally, and `scope: figure.caption` fires on it in + both HTML and Markdown. This is the case task 5.4 exists for: without the + known-valid control alongside it, the first result would have been recorded as + "the binary ignores this scope" and the operand wrongly dropped from the enum. + (AsciiDoc and reStructuredText, the other formats with figure captions, need + `asciidoctor`/`rst2html`, which Taskless excludes.) + +### Per-check fields, and two checks that validate nothing + +Every check accepts `extends`, `message`, `level`, `scope`, `link`, `limit`, +`action`, `description`, `name`. `vocab` is accepted by `existence`, +`substitution`, `capitalization`, `conditional`, `repetition` and **rejected** +by `occurrence`, `metric`, `readability`, `script`, `sequence`. + +The per-check additions are in the recipe's table. Three published claims are +wrong against the binary: `capitalization` takes `prefix` (singular) and rejects +`prefixes` and `suffixes`; `capitalization` rejects `ignorecase`; `occurrence` +rejects `exceptions` and `vocab`. + +**Field keys are matched case-insensitively — but `extends`, `message` and +`level` are not.** Vale decodes a check's own fields through a case-insensitive +map, so `Tokens:` and `ignoreCase:` are read exactly as their lowercase +spellings. The three header keys are read separately and literally: `EXTENDS:` +fails with "Missing the required 'extends' key". Their _values_ are +case-sensitive too — `level: WARNING` and `extends: Existence` are both +rejected. The schema therefore compares field names case-insensitively and +`extends`/`level` values exactly, which is what the binary does. + +**`consistency` and `spelling` accept any key at all.** `bananafield: true` on +either loads without complaint and is ignored — they do not use the strict +decode the other ten do. The schema cannot be strict about their fields without +rejecting rules the binary accepts, so it is not. + +### The failure modes, re-measured + +| Rule defect | Binary | +| ------------------------ | ------------------------------------------------ | +| `extends: nonsense` | rejected, exit 2, **every** Vale rule unreported | +| foreign field | `E201`, exit 2, **every** Vale rule unreported | +| `scope: fenced` | loads, runs, matches nothing — no error anywhere | +| style file named `.yaml` | not loaded at all — no error, no warning | + +**One row of the table in Context is wrong and is corrected here.** It claimed +`extends: nonsense` "verifies clean and simply matches nothing". Against 3.18.0 +it does not: it is the same engine-wide suppression as `E201`. The genuinely +silent case is `scope`, alone — which makes `scope` the highest-value field in +the schema for exactly the reason the decision above gives, and makes the other +two a blast-radius argument rather than a silence argument. + +## Open Questions + +- **How much of the per-check field table to encode.** The full table makes `E201` unreachable; a partial one still leaves the engine-wide suppression possible for the fields it omits. Recommend the full table, since `E201` is the failure with the widest blast radius, but the cost is that every check type must be transcribed and measured rather than just the common header. +- **Whether `verify` should also report each matcher's selected-file count.** Out of scope here, but adjacent: a matcher glob that reaches nothing produces a rule that is well formed, enabled, green, and inert — the one silent failure this change does not close. diff --git a/openspec/changes/vale-authoring-polish/proposal.md b/openspec/changes/vale-authoring-polish/proposal.md new file mode 100644 index 00000000..8ddca3b6 --- /dev/null +++ b/openspec/changes/vale-authoring-polish/proposal.md @@ -0,0 +1,45 @@ +## Why + +A Vale rule that is _malformed_ fails loudly. A Vale rule that is _wrong_ passes `verify`, passes `test`, and reports nothing forever — and nobody re-checks a green rule. Authoring the first house-style rules against this repository produced three of those in a row, each green on every local gate: + +- An em-dash rule whose token can never match, because Vale wraps tokens in word boundaries and an em dash is non-word on both sides. It needed `nonword: true`, which nothing asked for. +- A rule about a shell command that caught zero of two real violations, because commands in a README live in fenced blocks and Vale skips those by default. +- A rule scoped to `packages/cli/src/**/*.ts` whose clean report was indistinguishable from a matcher glob that reached nothing. + +Measured against the pinned Vale 3.18.0, the engine does not close these. `extends: nonsense` and `scope: fenced` both pass `verify` and simply match nothing; a field belonging to a different check type throws `E201`, which takes **every** Vale rule in the project down because Vale reads one config per run. The `create-vale-rule` recipe carries the knowledge that exists, but it is incomplete in the specific places that produce silent rules. + +## What Changes + +- **`verify` validates a Vale rule file structurally**, before Vale sees it, against a schema that models the check types and their fields. Today it validates `level` and the presence of the rule's `.vale.ini`, and nothing else — `extends: nonsense` verifies clean. +- **The schema is hand-authored and pinned to `VALE_VERSION`**, because there is nothing to generate it from. Vale publishes no JSON Schema; the machine-readable field knowledge exists only behind its paid cloud MCP at `api.vale.sh/mcp`, which `verify` cannot depend on. The vendor-contract test is what stops the schema drifting from the binary. +- **`scope` gains a real enum**, including the hierarchical forms, the v3.17.0+ inline scopes, and the v3.18.0+ `meta` scopes — and accepts the `~` negation and `&` chaining syntax that the recipe never documents and that a naive enum would reject. +- **The `create-vale-rule` recipe closes the gaps that produce silent rules**: the measured scope table, `nonword` for punctuation tokens, how to scope a rule _out_ (the recipe only explains scoping in), fixture design for a subject that normally appears in code, and the `limit`/`vocab` fields it omits. +- **No breaking change.** Every rule that verifies today continues to verify; the schema only rejects rules the engine was already going to ignore or fail on. + +## Capabilities + +### New Capabilities + +None. This tightens two existing capabilities rather than introducing a surface. + +### Modified Capabilities + +- `cli-rule-validation`: the `verify` component table currently says a Vale rule is checked "against Vale's own validation." Measured, that covers `level` and not `extends` or `scope`. The requirement changes to name schema validation as its own layer for `vale`, matching how `sg` is already checked against the ast-grep schema. +- `cli-agent-authoring`: the Vale authoring recipe requirement gains the scope, token, and fixture guidance whose absence is what produces a green-but-inert rule. + +## Impact + +- `packages/cli/src/rules/vale/verify.ts` — today entirely fixture behavior (bucket coverage, which fixtures fired); gains a schema layer. +- A new generated-or-authored schema artifact alongside `packages/cli/src/generated/ast-grep-rule-schema.json`, plus whatever pins it to `VALE_VERSION`. +- `packages/cli/test/vale-vendor-contract.test.ts` — extended to hold the schema's claims against the pinned binary. +- `packages/cli/src/agent/create-vale-rule.txt` — the recipe text. +- Closes #171 and #170; #167 is the worked case that motivated both. + +## Delivery shape + +**Stacked, merging forward.** Two units, each independently safe in production: + +1. **The recipe gaps.** Text only, no behavior change, and it is the half that helps an author today. It can land alone. +2. **The schema and its `verify` layer.** Code, tests, and the vendor-contract additions. + +Landing 1 alone leaves `check`, `verify`, and the test suite exactly as they are, so the stack merges forward rather than down. The changeset goes on the bottom branch and grows as the second unit lands. diff --git a/openspec/changes/vale-authoring-polish/specs/cli-agent-authoring/spec.md b/openspec/changes/vale-authoring-polish/specs/cli-agent-authoring/spec.md new file mode 100644 index 00000000..9818a953 --- /dev/null +++ b/openspec/changes/vale-authoring-polish/specs/cli-agent-authoring/spec.md @@ -0,0 +1,79 @@ +## MODIFIED Requirements + +### Requirement: The Vale authoring recipe covers rule, scope, and fixtures + +The `create-vale-rule` recipe SHALL instruct the agent to produce three artifacts, and SHALL state that a rule is incomplete without all three: + +1. A Vale style file at `.taskless/rules/vale//.yml`. +2. That rule's own `.taskless/rules/vale//.vale.ini`, declaring the matchers that scope it and enabling it as `. = YES`. +3. `pass/` and `fail/` fixture documents under `.taskless/rules/vale//.tests/`. + +The recipe SHALL state that scope is declared in the rule's own config, that no shared file is edited, and that the project-wide config is assembled rather than authored. + +It SHALL direct the agent to check its work by running `verify` and then `test` against the rule's directory path. + +The previous version of this requirement taught the agent to add a matcher to a single project-wide `.vale.ini`. Executing that recipe against sandboxed agents found every one of its silent failures in that step and nowhere else — an assignment above the first matcher, a glob that missed the fixture extension, three names that had to agree with nothing reporting when they didn't. The layout change removes the step rather than documenting it further. + +The recipe SHALL additionally carry the guidance below. Each item is a failure observed while authoring rules against this repository, and each produced a rule that passed `verify` and `test` while reporting nothing: + +- **What each `scope` reaches**, as measured: `text` sees prose, `code` sees inline code spans, `[code, text]` sees both, and `raw` sees prose, inline code, and fenced blocks. `raw` subsumes the other two. +- **That `scope` is per-rule.** Vale assembles one config per run, which invites the assumption that scopes interact. They do not. +- **That a rule scoped to `raw` cannot be suppressed** by Vale's `` directive, because it reads the unparsed document. A rule about a command needs `raw`, so it trades away per-case exemption. +- **That a token made only of punctuation needs `nonword: true`**, because Vale wraps every token in word boundaries. +- **How to scope a rule out**, not only in: a second matcher assigning `. = NO`. +- **That a bare word finds senses you did not mean**, and that narrowing to a collocation is checked by writing the `pass/` fixture from the literal sense first. +- **That fixture design follows the rule's subject**: when the subject normally appears in code, the `fail/` fixture SHALL contain it inline, fenced, and in prose. +- **The `limit` and `vocab` common fields**, which the recipe's field table omits. + +#### Scenario: Authoring produces all three artifacts + +- **WHEN** the agent follows `create-vale-rule` +- **THEN** it writes the style file, the rule's own config with a scoping matcher, and both fixture buckets + +#### Scenario: No shared file is edited + +- **WHEN** the agent scopes a rule +- **THEN** it writes matchers into that rule's own config +- **AND** it SHALL NOT be directed to edit a project-wide Vale config + +#### Scenario: The recipe names the commands that check the work + +- **WHEN** the agent has written the three artifacts +- **THEN** the recipe SHALL direct it to run `verify` and `test` against the rule's path + +#### Scenario: An unscoped rule is not silently accepted + +- **WHEN** the agent writes a style file without a matcher enabling it in the rule's own config +- **THEN** the recipe SHALL identify this as incomplete +- **AND** `verify` SHALL report it + +#### Scenario: A rule about a command reaches fenced blocks + +- **WHEN** the agent authors a rule whose subject is a command, flag, or package name +- **THEN** the recipe SHALL direct it to a scope that reaches fenced blocks +- **AND** the `fail/` fixture SHALL carry the subject inline, fenced, and in prose + +#### Scenario: A punctuation token is not left unable to match + +- **WHEN** the agent authors a rule whose token contains no word characters +- **THEN** the recipe SHALL direct it to set `nonword: true` + +## ADDED Requirements + +### Requirement: An authoring recipe states the failure a rule cannot report itself + +An authoring recipe SHALL document the ways a rule of its engine can be well formed, enabled, green on its fixtures, and still report nothing. + +A malformed rule is caught by `verify` and needs no recipe. A rule that is merely _wrong_ is caught by nothing, ships green, and is discovered only when someone notices it has never fired — so the recipe is the only place that failure can be prevented. + +#### Scenario: The recipe names the silent failures for its engine + +- **WHEN** an authoring recipe is written or revised +- **THEN** it SHALL name the failures that pass every local gate for that engine +- **AND** each SHALL be stated as an observed behavior of the pinned engine rather than as a caution in principle + +#### Scenario: Passing fixtures are not presented as proof of reach + +- **WHEN** the recipe directs the agent to run `test` +- **THEN** it SHALL state that fixtures run under an isolating config +- **AND** that a clean `check` SHALL be confirmed against a real file before the rule is believed to be working diff --git a/openspec/changes/vale-authoring-polish/specs/cli-rule-validation/spec.md b/openspec/changes/vale-authoring-polish/specs/cli-rule-validation/spec.md new file mode 100644 index 00000000..406264fe --- /dev/null +++ b/openspec/changes/vale-authoring-polish/specs/cli-rule-validation/spec.md @@ -0,0 +1,66 @@ +## MODIFIED Requirements + +### Requirement: Verify checks a rule's required components + +`verify` SHALL check that a rule has the components its engine requires and that they are well formed, and SHALL NOT require fixtures or test cases to exist. + +The two commands split because they have different preconditions. An agent part-way through authoring has a rule and no fixtures yet, and needs to know the rule itself is valid before it can write a meaningful test for it. + +Per engine, `verify` SHALL check: + +| Engine | Components | +| --------- | ---------------------------------------------------------------------------------------------------- | +| `sg` | `.yml` against the ast-grep schema and the Taskless required fields | +| `vale` | `.yml` against the Vale rule schema and the Taskless required fields, and the rule's `.vale.ini` | +| `runtime` | `check.ts` present, and at least one capture rule under `captures/` | + +The `vale` row previously read "against Vale's own validation." Measured against the pinned 3.18.0 binary, that covers less than it claims: `level: bananas` is reported, while `extends: nonsense` and `scope: fenced` both verify clean and produce a rule that matches nothing. Vale validates a rule when it _runs_ one, and it runs one field at a time — so a name it does not recognize is not an error, it is a check that never fires. Schema validation is therefore its own layer for `vale`, as it already is for `sg`. + +#### Scenario: A rule with no fixtures still verifies + +- **WHEN** `verify` runs against a rule whose fixture buckets are empty or absent +- **THEN** it SHALL report on the rule's components only +- **AND** the absence of fixtures SHALL NOT be a verify failure + +#### Scenario: A malformed rule reports its own error + +- **WHEN** a Vale style declares a `level` outside `suggestion`/`warning`/`error` +- **THEN** `verify` SHALL report that error, naming the field + +#### Scenario: An unrecognized extension point is rejected + +- **WHEN** a Vale style declares an `extends` that is not one of Vale's check types +- **THEN** `verify` SHALL report it, naming the field and the accepted values +- **AND** it SHALL NOT report the rule as valid + +#### Scenario: An unrecognized scope is rejected + +- **WHEN** a Vale style declares a `scope` that is not one of Vale's scope values +- **THEN** `verify` SHALL report it, naming the field +- **AND** a scope using the `~` negation or `&` chaining syntax over recognized values SHALL be accepted + +#### Scenario: A field belonging to another check type is rejected + +- **WHEN** a Vale style declares a field its `extends` does not accept, such as `tokens` on an `occurrence` check +- **THEN** `verify` SHALL report it before Vale is invoked + +The failure it prevents is not a local one: Vale reports this as `E201: has invalid keys` and reads one assembled config per run, so a single rule with a stray field suppresses every other Vale rule's findings. + +## ADDED Requirements + +### Requirement: The Vale rule schema is pinned to the vendored binary + +The Vale rule schema SHALL be authored in this repository and pinned to `VALE_VERSION`, and a vendor-contract test SHALL hold its claims against the vendored binary. + +Vale publishes no JSON Schema for its check types. The machine-readable field knowledge exists only behind its hosted MCP server, which is a paid product and unavailable to `verify`. The schema is therefore a transcription, and a transcription drifts — so the binary, not the documentation, SHALL be the authority for what the schema asserts. + +#### Scenario: A Vale upgrade that invalidates the schema fails loudly + +- **WHEN** `VALE_VERSION` is raised to a version whose accepted check types or scopes differ from the schema +- **THEN** the vendor-contract test SHALL fail +- **AND** the failure SHALL name the field whose accepted values changed + +#### Scenario: The schema is not derived from documentation alone + +- **WHEN** a value is added to the schema's `extends` or `scope` enumerations +- **THEN** it SHALL be one the vendored binary was measured accepting diff --git a/openspec/changes/vale-authoring-polish/tasks.md b/openspec/changes/vale-authoring-polish/tasks.md new file mode 100644 index 00000000..659695a9 --- /dev/null +++ b/openspec/changes/vale-authoring-polish/tasks.md @@ -0,0 +1,65 @@ +## 1. Measure the binary before writing anything down + +- [x] 1.1 Enumerate the check types the vendored Vale accepts, and resolve the docs' eleven against the MCP guide's twelve. Record the measured list and how it was obtained. +- [x] 1.2 Enumerate the accepted `scope` operands, including the hierarchical forms, the v3.17.0+ inline scopes, and the v3.18.0+ `meta` scopes. Confirm `~` negation and `&` chaining parse. **There is no `meta` scope**: the v3.18.0 addition is `frontmatter`/`frontmatter.`, which is what fires. Negation over an unrecognized operand is a silent no-op. See design.md → Measured against Vale 3.18.0. +- [x] 1.3 For each check type, measure which fields it accepts and which it rejects with `E201`. This is the per-check table the schema encodes. +- [x] 1.4 Confirm the failure modes the design asserts still hold on the pinned binary: `extends: nonsense` and `scope: fenced` verify clean and match nothing; a foreign field throws `E201` and suppresses every other Vale rule's findings. **One half did not hold.** `extends: nonsense` is rejected outright on 3.18.0 — exit 2, whole run suppressed, same blast radius as `E201`. `scope: fenced` is the only genuinely silent case. Corrected in design.md. + +## 2. Recipe gaps (unit 1, lands alone) + +- [x] 2.1 Add the measured scope table to `create-vale-rule`, and state that `raw` subsumes `code` and `text`. +- [x] 2.2 State that `scope` is per-rule and that rules do not interact, despite the single assembled config. +- [x] 2.3 State that a `raw`-scoped rule cannot be suppressed by ``, and that a rule about a command needs `raw`. +- [x] 2.4 Add `nonword: true` for punctuation-only tokens, with the em-dash rule as the worked case. +- [x] 2.5 Document scoping a rule _out_ with a second matcher assigning `NO`. +- [x] 2.6 Add the collocation guidance: narrow a banned word to a collocation, and write the `pass/` fixture from the literal sense first. +- [x] 2.7 Add the fixture rule for a subject that appears in code: `fail/` must carry it inline, fenced, and in prose. +- [x] 2.8 Add `limit` and `vocab` to the common-fields table, and note that Vale requires `.yml` rather than `.yaml`. +- [x] 2.9 State that fixtures run under an isolating config, so passing tests do not prove the matcher reaches any real file. +- [x] 2.10 Update the recipe's cross-reference tests if the added sections change what they assert. No change needed: `recipe-cross-references.test.ts` asserts the rendered format lists, which the new sections do not touch, and the full suite is green. +- [x] 2.11 Write the changeset on this branch, so the stack inherits it. + +## 3. The schema (unit 2) + +- [ ] 3.1 Author the schema from the measurements in group 1, alongside `packages/cli/src/generated/ast-grep-rule-schema.json`. Pin it to `VALE_VERSION`. +- [ ] 3.2 Model the common header fields: `extends`, `message`, `level`, `scope`, `link`, `limit`, `vocab`. +- [ ] 3.3 Model `scope` as a grammar over an enum of operands, accepting a string, a list, `~`, and `&`. +- [ ] 3.4 Model the per-check field tables, so a field belonging to another check type is rejected before `E201` can suppress the engine. +- [ ] 3.5 Decide and record how strictly to treat a value whose status the measurements left unclear. The design's rule is to accept it. + +## 4. Wire it into verify + +- [ ] 4.1 Add the schema layer to the Vale verify path, reporting through the existing `LayerResult` and `VerifyResult` shapes so both engines fail the same way. +- [ ] 4.2 Ensure the error names the field and the accepted values, rather than reporting a raw schema path. +- [ ] 4.3 Confirm the layer runs before Vale is invoked, so a rule that would throw `E201` never reaches the binary. +- [ ] 4.4 Confirm `test` still runs `verify` first and stops on its failure. + +## 5. The corpus, and the differential test that makes the schema true + +Build this before the schema is finalized — the schema is derived from it. This is the largest group and the one that will be under-built if rushed. + +- [ ] 5.1 Design the corpus entry shape: a minimal rule YAML, the fixture that acts as its positive control, and the measured verdict. Keep it a declarative table so a Vale upgrade is a re-run rather than a re-authoring. +- [ ] 5.2 Establish the three-outcome verdict, since an exit code cannot express it: **fired** (accepted), **did not fire** (the binary ignored the construct), **`E201`** (rejected outright). An unrecognized `extends` or `scope` parses clean, so "did not fire" is the signal that the construct is invalid. +- [ ] 5.3 Write a positive control per entry. It depends on what the rule matches — a `scope: heading` entry needs a heading, `scope: table.cell` needs a table — so it cannot be generated from the check type. This is the bulk of the work. +- [ ] 5.4 Guard against a vacuous entry: assert each control fires for at least one known-valid variant, so a control that can never fire is a test failure rather than a quiet pass. +- [ ] 5.5 Cover every check type from 1.1, every scope operand from 1.2, and the per-check field tables from 1.3, including the `~` and `&` forms. +- [ ] 5.6 Write the differential test: for every entry, assert `schemaAccepts === binaryAccepts`. Report both directions distinctly — schema-too-lax is the gap being closed, schema-too-strict blocks valid work and is the worse failure. +- [ ] 5.7 Wire the corpus into `packages/cli/test/vale-vendor-contract.test.ts`, or a sibling beside it, following the existing convention of invoking the vendored binary directly rather than through `runVale`. +- [ ] 5.8 Add the version-bump case: raising `VALE_VERSION` past a change in accepted check types or scopes fails a test that names the field. +- [ ] 5.9 Verify the suite is non-vacuous by reverting the schema and watching exactly the expected entries fail. + +## 6. Regression coverage + +- [ ] 6.1 Test that `extends: nonsense` fails `verify`, naming the field and the accepted values. +- [ ] 6.2 Test that an unrecognized `scope` fails `verify`, and that `~` and `&` over recognized operands pass. +- [ ] 6.3 Test that a foreign field for the declared check type fails `verify`. +- [ ] 6.4 Test that every rule under `.taskless/rules/vale/` still verifies, so no valid rule regressed. +- [ ] 6.5 Test that a rule with no fixtures still verifies, preserving the existing requirement. + +## 7. Land it + +- [ ] 7.1 `pnpm typecheck`, `pnpm lint`, `pnpm test` pass. +- [ ] 7.2 `pnpm build`, then author a deliberately-broken rule and confirm the real CLI reports what the specs require. Record the actual output. +- [ ] 7.3 Extend the changeset on the bottom branch with unit 2's scope. +- [ ] 7.4 Open the stack as two PRs merging forward, the recipe unit first. +- [ ] 7.5 Archive the change on the tip PR. diff --git a/packages/cli/src/agent/create-vale-rule.txt b/packages/cli/src/agent/create-vale-rule.txt index 11440a21..5b1ced95 100644 --- a/packages/cli/src/agent/create-vale-rule.txt +++ b/packages/cli/src/agent/create-vale-rule.txt @@ -59,8 +59,19 @@ it. 1. **Say what the rule reads, then pick an extension point.** Write one sentence: "this fires when a document contains ___." Vale rules are - built by extending one of its eleven checks, and the sentence tells - you which: + built by extending one of its twelve checks, and the sentence tells + you which. Twelve is measured, not counted off the docs: give Vale + v%(VALE_VERSION)s an `extends` it does not know and it names the whole + set back at you — + + ``` + 'extends' key must be one of [capitalization conditional consistency + existence occurrence repetition substitution readability spelling + sequence metric script]. + ``` + + The docs enumerate eleven, folding `readability` into `metric`. They + are separate checks with separate fields. | If the rule is about… | extends | |-----------------------------------------------------------------------------------------|------------------| @@ -71,7 +82,8 @@ it. | a word repeated back to back | `repetition` | | picking one of two acceptable spellings, consistently | `consistency` | | "if X appears, Y must also appear" | `conditional` | -| readability or length thresholds | `metric` | +| a document-level length or ratio threshold | `metric` | +| a readability grade, against a named formula | `readability` | | a misspelling, against a dictionary | `spelling` | | phrases that must appear in a fixed order | `sequence` | | anything the above cannot express (Tengo script) | `script` | @@ -85,9 +97,12 @@ it. `match: GitHub` reports `We host on Github and it is fine. should be GitHub`. - For the two this recipe does not describe, read - https://docs.vale.sh/styles before inventing something — Vale has no - facility for a rule that does not extend one of these eleven. + For the five this recipe has no worked example of — `metric`, + `readability`, `spelling`, `sequence`, `script` — read + https://docs.vale.sh/styles before inventing something. Vale has no + facility for a rule that does not extend one of these twelve, and an + `extends` outside the set is not a rule that misbehaves: Vale exits 2 + and **every** Vale rule in the project goes unreported for that run. **Nine of them have a worked rule at the end of this recipe**, each with the near-miss that fails and why. Read the one closest to your @@ -102,38 +117,153 @@ it. only** (`izeise`, not `ize-ise`). That check compiles the id into its pattern as a regex group name, and a hyphen there fails the entire Vale run. `verify` catches it. Kebab-case is correct for the other - ten. + eleven. Every rule carries: | Field | Required | Notes | |-----------|----------|-----------------------------------------------| -| `extends` | yes | one of the eleven above | +| `extends` | yes | one of the twelve above | | `message` | yes | shown to the user; see the `%%s` table below | | `level` | no | `suggestion` (default), `warning`, or `error` | | `scope` | no | narrow to part of a document — see below | | `link` | no | a URL the reader can follow for the reasoning | +| `limit` | no | cap findings from this rule per scope | +| `vocab` | no | `false` opts the rule out of `Vocab` filtering | + + **The file extension must be `.yml`.** Measured: rename a working + style file to `.yaml` and Vale loads nothing — no error, no warning, + zero findings, and `. = YES` still parses. It is + indistinguishable from a rule whose pattern never matched. **`scope` decides where the rule looks**, so getting it wrong is a - silent over- or under-fire rather than an error. The useful values for - markdown: `heading` (and `heading.h1`…`heading.h6`), `paragraph`, - `sentence`, `list`, `blockquote`, `link`, `code`, `alt`, `summary`, - `table.header`, `table.cell`, `table.caption`, `figure.caption`, - `strong`, `emphasis`, `raw`. Omit it and the rule sees all prose. + silent under-fire rather than an error. An unrecognized scope is not + rejected by Vale: `scope: fenced` loads, runs, and matches nothing. + It is the worst of the three failures on this page, because unlike a + bad `extends` or a foreign field it does not even take the run down to + tell you — the rule is simply inert, forever. Copy the value from the + table below rather than typing it. + + Every value below was measured against Vale v%(VALE_VERSION)s by + authoring a rule with that scope and a document the rule had to flag. + "Fires" means the finding appeared; a scope that never fired is not on + this list. + +| `scope` | reaches | +|------------------------|-----------------------------------------------------------| +| *(omitted)* | everything the format exposes as prose | +| `text` | prose only — not inline code, not fenced blocks | +| `code` | inline code spans only | +| `raw` | the unparsed document: prose, inline code, fenced blocks | +| `heading` | every heading | +| `heading.h1`…`h6` | headings of that level | +| `paragraph` | one paragraph at a time | +| `sentence` | one sentence at a time | +| `list` | list items | +| `blockquote` | quoted blocks | +| `link` | link text, not the URL | +| `alt` | image alt text | +| `summary` | `` of a disclosure | +| `strong`, `emphasis` | bold and italic runs | +| `table` | any part of a table | +| `table.header` | header cells | +| `table.cell` | body cells | +| `table.caption` | a table's caption | +| `figure.caption` | a figure's caption — but see below | +| `frontmatter` | every YAML front-matter value | +| `frontmatter.` | one front-matter key's value | +| `text.class.` | HTML elements carrying that class | +| `comment` | every comment, in a comment-tier format | +| `comment.line` | `//`-style comments | +| `comment.block` | `/* … */`-style comments | + + **`raw` subsumes `code` and `text`.** Measured on one document holding + the token in prose, in an inline span, and in a fenced block: `text` + found one, `code` found one, `[code, text]` found two, `raw` found all + three. If you want prose and inline code but not fenced blocks, write + the list — `raw` is not "a bit wider", it is everything. + + **Vale drops everything inside a `
` element.** Measured: a + `
` nested in `
` is invisible to *every* scope, + `text` and `raw` included, so a `scope: figure.caption` rule over a + normally-marked-up figure reports nothing and looks like a bad scope. + A bare `
` is linted, and `figure.caption` fires on it. If a + fixture is not firing, check whether its subject is inside a `
` + before you touch the pattern. + + **`scope` also takes `~` and `&`.** `~code` is everything but inline + code; `text & ~code` chains two operands; a list (`[code, text]`) is a + union. All three parse and behave. + + **A negation over a scope Vale does not know is a silent no-op.** + Measured: `~banana` and `text & ~banana` both fire on everything, + because there is no such scope to subtract. A typo inside a `~` does + not narrow the rule and does not widen it visibly — it removes the + exclusion you wrote the rule for. Check the spelling inside a `~` + against the table as carefully as a bare one. + + **`scope` is per-rule, and rules do not interact.** Taskless assembles + every rule's matchers into one config for the run, which invites the + assumption that one rule's `scope` narrows another's, or that two + rules over the same file compete. They do not: each check carries its + own scope and is evaluated independently. If a rule is over-firing, + the cause is in that rule's own `scope` and glob, never in a + neighbour's. + + **A `raw`-scoped rule cannot be turned off in a document.** Vale's + `` directive is applied to the *parsed* + document, and `raw` reads the unparsed one. Measured: a `text`-scoped + rule is silenced by the directive; the same rule at `raw` fires + through it. So a rule about a shell command, a flag, or a package name + needs `raw` — commands live in fenced blocks, which nothing else + reaches — and takes that trade: it can no longer be exempted case by + case, only removed. **Then the fields the extension point adds** — this is where the rule actually lives, and each check reads only its own: -| extends | its fields | -|------------------|-------------------------------------------------------------------------------------| -| `existence` | `tokens` (a list) or `raw`; `ignorecase`, `nonword`, `exceptions`, `append` | -| `substitution` | `swap` (a map of observed → expected); `ignorecase`, `nonword`, `exceptions` | -| `capitalization` | `match`; `exceptions`, `style` (with `$title`), `threshold`, `indicators`, `prefix` | -| `occurrence` | `token`, `max` and/or `min` | -| `repetition` | `tokens`; `alpha`, `ignorecase` | -| `consistency` | `either` (a map of the two acceptable forms) | -| `conditional` | `first`, `second`; `exceptions` | -| `metric` | `formula`, `condition` | +| extends | its fields | +|------------------|------------------------------------------------------------------------------| +| `existence` | `tokens` (a list) or `raw`; `ignorecase`, `nonword`, `exceptions`, `append` | +| `substitution` | `swap` (a map of observed → expected); `ignorecase`, `nonword`, `exceptions`, `capitalize`, `pos` | +| `capitalization` | `match`; `style` (with `$title`), `exceptions`, `threshold`, `indicators`, `prefix` | +| `occurrence` | `token`, `max` and/or `min`; `ignorecase` | +| `repetition` | `tokens`; `alpha`, `ignorecase`, `exceptions`, `max` | +| `consistency` | `either` (a map of the two acceptable forms); `ignorecase`, `nonword` | +| `conditional` | `first`, `second`; `exceptions`, `ignorecase` | +| `metric` | `formula`, `condition` | +| `readability` | `metrics` (a list of formula names), `grade` | +| `spelling` | `aff`, `dic`, `custom`, `filters`, `ignore`, `threshold` | +| `sequence` | `tokens` (each a `pattern`/`tag` map); `ignorecase` | +| `script` | `script` (Tengo source) | + + The list above is measured, not transcribed: every entry was added to + a minimal rule of that check and the run watched for `E201`. Three + corrections fall out of it, all against the published docs — + `capitalization` takes `prefix` (singular) and rejects both `prefixes` + and `suffixes`, it rejects `ignorecase`, and `occurrence` rejects + `exceptions` and `vocab`. + + **Field names are matched case-insensitively, but `extends`, `message` + and `level` are not.** Measured: `Tokens:` and `ignoreCase:` are read + exactly as their lowercase spellings, while `EXTENDS:` fails with + "Missing the required 'extends' key". Their *values* are case-sensitive + too — `level: WARNING` and `extends: Existence` are both rejected. + Write everything lowercase and none of this can bite you. + + **A field from the wrong check is the loudest failure Vale has.** + `tokens` on an `occurrence` check gives + `E201 … has invalid keys: 'tokens'`, exit 2, and — because Vale reads + one assembled config per run — **no** Vale rule in the project reports + anything. Take the field names from the table above rather than from + memory: this is the mistake with the widest blast radius. + + **Two checks are exempt, and that is not a licence.** Measured, + `consistency` and `spelling` accept any key at all: `bananafield: + true` on either loads without complaint and is ignored. A misspelled + field on those two is a silent no-op instead of a loud one, so the + schema cannot catch a typo there and neither can Vale. Read the field + list twice when writing those two. **What `%%s` fills with depends on the extension point.** Getting this wrong is the one mistake in this recipe that passes every check below @@ -239,6 +369,42 @@ it. `ignorecase: false` to protect the correct spelling. - `raw` takes a full regex when `tokens` is too restrictive; `nonword` removes the implicit boundaries. + - **A token made only of punctuation can never match without + `nonword: true`.** The boundaries above are `\b`, which needs a word + character on the inside. An em dash has none, on either side. + Measured against `This is a sentence — with an em dash.`: + + ```yaml + # fires on nothing, ever — and reports no error + extends: existence + message: "Use a comma, not an em dash" + tokens: + - '—' + ``` + + ```yaml + # fires + extends: existence + message: "Use a comma, not an em dash" + nonword: true + tokens: + - '—' + ``` + + The first rule verifies, tests green if its `fail/` fixture is + missing the dash, and reports nothing forever. Any token whose + pattern contains no `\w` — punctuation, an emoji, a bare symbol — + needs `nonword: true`. + + - **A bare word finds senses you did not mean.** `landed on` in a rule + about jargon also matches "the plane landed on time"; `simply` in a + rule about hedging also matches "simply connected" in a maths doc. + Narrow the token to the **collocation** you actually object to + (`landed on a decision`, not `landed on`), and check that you got it + right by writing the `pass/` fixture from the literal sense *first*: + put the innocent sentence in `pass/` before you write the guilty one + in `fail/`. A rule whose `pass/` bucket was written afterwards tends + to contain only sentences the author already knew were safe. Vale also understands the markup, which decides what counts as text before your pattern ever runs. Measured in markdown: @@ -288,6 +454,29 @@ it. the style, which is this rule's directory; the second is the check inside it, which is the file. Both are the id. + **Scope a rule *out* with a second matcher, not a cleverer glob.** A + glob says which files a rule sees; it has no way to say "these but not + those". The exclusion is a second matcher that assigns `NO`, and + because precedence here is positional — a later matcher wins — the + exclusion goes **after** the inclusion: + + ```ini + # Every markdown file… + [*.md] + tskl) rule = no-simply + BasedOnStyles = + no-simply.no-simply = YES + + # …except the changelog, which quotes release notes verbatim. + [CHANGELOG.md] + tskl) rule = no-simply + no-simply.no-simply = NO + ``` + + Write the breadcrumb in the second matcher too, or assembly attributes + it to nobody. Reversing the two blocks silently re-enables the rule on + the file you meant to exempt, and nothing reports that. + **Do NOT write `StylesPath` or `MinAlertLevel` here.** Those describe the run rather than a rule, the assembler supplies them, and a copy in a rule's config is dropped on the way in. @@ -383,6 +572,34 @@ it. That is the half of the fixture set that has to work for you. + **When the rule's subject normally appears in code, the `fail/` + fixture must carry it three ways** — inline in a code span, inside a + fenced block, and in ordinary prose — in that one document. A rule + about a command, a flag, a package name or an env var has a subject + that lives in fenced blocks in every real README, and the default + scope cannot see fenced blocks at all. A `fail/` fixture written only + in prose therefore fires, goes green, and the rule then catches none + of the real violations. Measured on one document holding the token in + all three places: the default scope found one of three, `raw` found + three. If the fixture fires on the prose line and not on the other + two, the answer is `scope: raw` — see step 2 for what that costs. + + **Fixtures run under a config that isolates this rule, so a green + `test` is not evidence the rule reaches any real file.** `test` + generates its own `.vale.ini` pointing at the fixture directory and + enabling only `.`; your rule's own matcher globs are not + consulted. So a glob of `packages/cli/src/**/*.ts` that matches + nothing in the repository still produces a rule that verifies, tests + green, and reports forever. The only check for that is a real + `check` over a real file: + + ``` + %(TASKLESS_CLI)s check --json + ``` + + Do that once, on a file you have deliberately made violate the rule, + before you believe the rule works. + 6. **Verify, then test.** Two commands, both taking the rule's directory as their argument, both run from the project root: