docs(agent): show a scope exclusion with folder fixtures, and say so in the never-read error

.changeset/show-scope-exclusion-fixture.md

@@ -0,0 +1,5 @@
+---
+"@taskless/cli": patch
+---
+
+Say how to show that a rule's scope excludes a path. The fixture layout in every authoring recipe, and `create-sg-rule`, now explain the working pattern: put the excluded file in a `pass/` case beside a clean file the scope admits. When `verify` rejects a `pass/` case because the scope excluded every file in it, the error now suggests the same thing alongside moving the fixture.

packages/cli/src/agent/create-runtime-rule.md

@@ -1,4 +1,4 @@
-# Topic: create-runtime-rule     (CLI v%(CLI_VERSION)s / topic v8)
+# Topic: create-runtime-rule     (CLI v%(CLI_VERSION)s / topic v9)
 
 %(ONE_COMMAND_PER_CALL)s
 

packages/cli/src/agent/create-sg-rule.md

@@ -1,4 +1,4 @@
-# Topic: create-sg-rule     (CLI v%(CLI_VERSION)s / topic v16)
+# Topic: create-sg-rule     (CLI v%(CLI_VERSION)s / topic v17)
 
 %(ONE_COMMAND_PER_CALL)s
 
@@ -136,7 +136,10 @@ whole rule.
    Each case file takes the extension ast-grep parses as the rule's
    `language:` (`.ts` for `TypeScript`, `.tsx` for `Tsx`, `.yml` for
    `Yaml`), and sits at a path the rule's `files:` admits and its
-   `ignores:` does not.
+   `ignores:` does not. To show that `files:` or `ignores:` excludes a
+   path, put the offending file at that path in a `pass/` case beside a
+   clean file the scope admits, as the layout above describes. A `pass/`
+   case holding only the excluded file is never read and fails.
 
    %(FIXTURE_SOURCES)s
 
@@ -175,12 +178,13 @@ whole rule.
    - `ok: true` → go to step 8.
    - `ok: false` → read `errors` and fix. Repeat up to 3 times.
 
-| what `errors` says                   | fix                                             |
-|--------------------------------------|-------------------------------------------------|
-| the YAML doesn't match the schema    | check field types against the upstream schema   |
-| a Taskless-required field is missing | add `id`/`language`/`severity`/`message`/`rule` |
-| a case didn't behave as expected     | fix the rule pattern OR the test case           |
-| the rule's scope excluded a case     | move the fixture to a path `files:` admits      |
+| what `errors` says                              | fix                                             |
+|-------------------------------------------------|-------------------------------------------------|
+| the YAML doesn't match the schema               | check field types against the upstream schema   |
+| a Taskless-required field is missing            | add `id`/`language`/`severity`/`message`/`rule` |
+| a case didn't behave as expected                | fix the rule pattern OR the test case           |
+| the rule's scope excluded a case                | move the fixture to a path `files:` admits      |
+| a pass case showing an exclusion was never read | add a clean file the scope admits to that case  |
 
    Every `regex`, at any depth, needs a `kind` beside it, or
    validation fails. That includes `constraints`, `utils`, and every

packages/cli/src/agent/create-vale-rule.md

@@ -1,4 +1,4 @@
-# Topic: create-vale-rule     (CLI v%(CLI_VERSION)s / topic v21)
+# Topic: create-vale-rule     (CLI v%(CLI_VERSION)s / topic v22)
 
 %(ONE_COMMAND_PER_CALL)s
 

packages/cli/src/agent/improve-rule.anonymous.md

@@ -1,4 +1,4 @@
-# Topic: improve-rule (anonymous)     (CLI v%(CLI_VERSION)s / topic v6)
+# Topic: improve-rule (anonymous)     (CLI v%(CLI_VERSION)s / topic v7)
 
 %(ONE_COMMAND_PER_CALL)s
 

packages/cli/src/agent/validate-rule.md

@@ -1,4 +1,4 @@
-# Topic: validate-rule     (CLI v%(CLI_VERSION)s / topic v10)
+# Topic: validate-rule     (CLI v%(CLI_VERSION)s / topic v11)
 
 %(ONE_COMMAND_PER_CALL)s
 

packages/cli/src/prompts/recipes.ts

@@ -508,6 +508,15 @@ engine, keeps its cases beside it in \`tests/\`:
   never read". Either the scope excluded every file in it (move the file to
   a path the scope admits), or the engine walked none of them (an ignored
   path, an empty file, or an extension the engine does not read).
+- **To show the scope excludes a path, put the excluded file beside a
+  clean file the scope admits, in one \`pass/\` case.** A case holding
+  only excluded files is never read, so it fails however correct the
+  scope is. Being read is decided per case, so the clean file makes the
+  case read, and the case passes when the rule stays quiet on both. For an
+  ast-grep rule with \`files: ["src/**/*.ts"]\` and
+  \`ignores: ["src/generated/**"]\`, the case
+  \`tests/pass/excluded-paths/\` holds a clean \`src/clean.ts\` beside an
+  offending \`src/generated/a.ts\` and an offending \`scripts/a.ts\`.
 - **"The rule's scope excluded it"** means a \`fail/\` case fires once its
   scope is removed, so the pattern is fine and the fixture's path is not.
   Move the fixture to a path the scope admits, or widen the scope. Do not

packages/cli/src/rules/fixtures.ts

@@ -309,11 +309,20 @@ export function describeCaseFailure(
   if (casePassed(outcome)) return undefined;
   const path = projectRelative(cwd, outcome.fixture.root);
   if (outcome.unread === "scope-excluded") {
-    return (
+    const message =
       `${outcome.fixture.bucket} case was never read, because the rule's ` +
       `scope excluded every file in it: ${path}/. No file sits at a path ` +
-      `admitted by ${scope ?? "the scope"}. Move the fixture to such a path.`
-    );
+      `admitted by ${scope ?? "the scope"}. Move the fixture to such a path.`;
+    // A pass case made only of excluded files is usually an attempt to show
+    // that the exclusion works. The never-read check is per case, so one clean
+    // file the scope admits makes the case read, and the excluded file beside
+    // it then shows the rule stays quiet on it.
+    return outcome.fixture.bucket === "pass"
+      ? `${message} To show the scope excludes a file, keep that file in ` +
+          `this case and add beside it a clean file at a path the scope ` +
+          `admits: the case is then read, and passes when the rule stays ` +
+          `quiet on both.`
+      : message;
   }
   if (outcome.unread === "not-walked") {
     return (

packages/cli/test/sg-fixtures.test.ts

@@ -199,6 +199,80 @@ describe("an sg fixture case is judged under the rule's scope", () => {
     expect(errors).toContain("pass case was never read");
     expect(errors).toContain("scope excluded every file in it");
     expect(errors).toContain("tests/pass/outside-scope/");
+    // An author writing this case usually means to show the exclusion, so
+    // the error says how to do that as well as how to move the file.
+    expect(errors).toContain("Move the fixture to such a path");
+    expect(errors).toContain(
+      "To show the scope excludes a file, keep that file in this case and " +
+        "add beside it a clean file at a path the scope admits"
+    );
+  });
+
+  it("does not suggest the exclusion pattern for a fail case the scope never read", async () => {
+    await sgRule(WORKFLOW_RULE, WORKFLOW_RULE_YAML, {
+      pass: { pinned: { ".github/workflows/ci.yml": PINNED_STEP } },
+      fail: { "outside-scope": { "ci.yml": PINNED_STEP } },
+    });
+
+    const result = await verifyRule(cwd, WORKFLOW_RULE);
+
+    expect(result.tests.valid).toBe(false);
+    const errors = result.tests.errors.join("\n");
+    expect(errors).toContain("fail case was never read");
+    expect(errors).toContain("Move the fixture to such a path");
+    expect(errors).not.toContain("To show the scope excludes a file");
+  });
+
+  it("passes a pass case that shows an exclusion beside a clean file the scope admits", async () => {
+    // The never-read check is per case, so one read file is enough. The
+    // offending files outside `files:` and inside `ignores:` then show the
+    // rule stays quiet on what its scope excludes.
+    await sgRule(
+      EVAL_ID,
+      `${EVAL_RULE}files:\n  - "src/**/*.ts"\nignores:\n  - "src/generated/**"\n`,
+      {
+        pass: {
+          "excluded-paths": {
+            "src/clean.ts": "foo(x);\n",
+            "src/generated/a.ts": "eval(x);\n",
+            "scripts/a.ts": "eval(x);\n",
+          },
+        },
+        fail: { "eval-call": { "src/a.ts": "eval(x);\n" } },
+      }
+    );
+
+    const result = await verifyRule(cwd, EVAL_ID);
+
+    expect(result.tests.errors).toEqual([]);
+    expect(result.tests.valid).toBe(true);
+    expect(result.tests.passed).toBe(2);
+    expect(
+      result.tests.findings.filter((finding) => finding.bucket === "pass")
+    ).toEqual([]);
+  });
+
+  it("fails each excluded file alone in a pass case as never read", async () => {
+    await sgRule(
+      EVAL_ID,
+      `${EVAL_RULE}files:\n  - "src/**/*.ts"\nignores:\n  - "src/generated/**"\n`,
+      {
+        pass: {
+          ignored: { "src/generated/a.ts": "eval(x);\n" },
+          unlisted: { "scripts/a.ts": "eval(x);\n" },
+        },
+        fail: { "eval-call": { "src/a.ts": "eval(x);\n" } },
+      }
+    );
+
+    const result = await verifyRule(cwd, EVAL_ID);
+
+    expect(result.tests.valid).toBe(false);
+    expect(result.tests.failed).toBe(2);
+    const errors = result.tests.errors.join("\n");
+    expect(errors).toContain("tests/pass/ignored/");
+    expect(errors).toContain("tests/pass/unlisted/");
+    expect(errors).toContain("scope excluded every file in it");
   });
 
   it("fails a case under a path the project's .gitignore covers", async () => {