feat(check): per-rule summary in check --json, --summary, and one finding schema shared with verify

.changeset/check-json-summary.md

@@ -0,0 +1,7 @@
+---
+"@taskless/cli": patch
+---
+
+`check --json` now carries a `summary`: total findings, distinct files, counts by severity, and one entry per rule that fired, keyed by `engine` and `ruleId`. The new `--summary` flag returns only that summary (under `--json` it omits `results`), so an agent can measure a whole repository in one call whose output stays small; without `--json` it prints per-rule counts. Findings from `check --json` and `verify --json` now share one schema, published as `findingSchema` from `@taskless/cli/schemas` alongside `checkOutputSchema`.
+
+Patch: the package is 0.y.z and the change is additive. `results` and every existing key are unchanged unless you pass the new `--summary` flag, so no existing consumer has to react.

openspec/changes/archive/2026-11-11-check-json-summary/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-11-11

openspec/changes/archive/2026-11-11-check-json-summary/proposal.md

@@ -0,0 +1,29 @@
+## Why
+
+`check --json` has no per-rule summary, so measuring a repository means reading every finding: one agent run reported 300KB+ of output, and a harness that forbids piping to `jq` had to count by reading its saved output file. Its findings were also declared separately from the ones `verify --json` prints under `rules[].findings`, so a script written for one broke on the other (#678).
+
+## What Changes
+
+- `check --json` carries a `summary` on every run: totals, distinct files, counts by severity, and one entry per rule that fired, keyed by `engine` and `ruleId` (the pair `verify` groups `rules[]` by), most findings first.
+- New `--summary` flag. With `--json` it omits `results` and keeps `summary`, so the output stays small on any repository. Without `--json` it prints per-rule counts instead of findings. The exit code is unchanged either way.
+- One finding schema, `findingSchema`, declared once and used by both `check --json`'s `results[]` and `verify --json`'s `rules[].findings[]` (which extends it with `bucket`). The two had drifted: `check` allowed a `null` `note` and `fix`, `verify` did not. The shared schema takes `check`'s spelling.
+- `@taskless/cli/schemas` publishes `findingSchema` and `checkOutputSchema`.
+- The envelopes stay different, on purpose: `check` scans a codebase and its natural unit is a finding, while `verify` tests rules and its natural unit is a rule. `summary.rules[]` gives `check` the same per-rule grouping without moving `results`, which every existing consumer reads.
+- The `check` recipe documents `summary` and `--summary` (topic v10).
+- The stale JSONL description of `check --json` in the `cli-check` spec is corrected to the envelope the command has printed for a long time.
+
+## Capabilities
+
+### New Capabilities
+
+None.
+
+### Modified Capabilities
+
+- `cli-check`: `--json` output carries a per-rule summary, `--summary` returns only that, and findings share one schema with `verify`.
+
+## Impact
+
+- `packages/cli/src/schemas/finding.ts` (new), `src/schemas/check.ts`, `src/schemas/rule-check.ts`, `src/schemas/index.ts`, `src/rules/summarize-findings.ts` (new), `src/commands/check.ts`, `src/util/format.ts`, `src/agent/check.md`, `packages/cli/README.md`.
+- Additive for existing `--json` consumers: `results` is unchanged unless `--summary` is passed, and `summary` is a new key.
+- Delivery shape: **single PR**. The schema, the flag, the recipe and the spec only make sense together, and the diff fits one review.

openspec/changes/archive/2026-11-11-check-json-summary/specs/cli-check/spec.md

@@ -0,0 +1,66 @@
+## REMOVED Requirements
+
+### Requirement: Check subcommand formats JSON output when --json is set
+
+**Reason**: It described a JSONL stream, one finding per line, that `check --json` has not printed for a long time. It prints one JSON object, and this change adds a key to it.
+
+**Migration**: Replaced by "Check subcommand prints one JSON envelope when --json is set", which states the envelope as printed.
+
+## ADDED Requirements
+
+### Requirement: Check subcommand prints one JSON envelope when --json is set
+
+When the `--json` flag is set, the CLI SHALL write one JSON object to stdout: `{ success, results, summary }`, plus the optional `skipped`, `failures`, `notices`, `entitlement`, `integrity` and `runDirectory` keys. `results` SHALL hold every finding, each parsing against the published `findingSchema`. `summary` SHALL be present on every run. `results` SHALL be present on every run except one given `--summary`.
+
+#### Scenario: JSON output carries every finding
+
+- **WHEN** the `--json` flag is set and the scanner produces results
+- **THEN** stdout SHALL be a single JSON object whose `results` array holds one entry per finding
+- **AND** its `summary` SHALL count the same findings
+
+#### Scenario: JSON output with no issues
+
+- **WHEN** the `--json` flag is set and the scanner produces zero results
+- **THEN** `results` SHALL be an empty array
+- **AND** `summary.total` SHALL be 0 and `summary.rules` SHALL be empty
+
+### Requirement: Check reports finding counts per rule
+
+`check --json` SHALL carry a `summary` with the total number of findings, the number of distinct files they are in, the counts by severity, and a `rules` array. Every severity SHALL be present in a count, zero included. `rules` SHALL hold one entry per rule with at least one finding, keyed by `engine` and `ruleId`, carrying that rule's total, distinct files and counts by severity, ordered by total descending and then by `ruleId`. Two engines holding the same rule id SHALL be two entries. A rule with no findings SHALL NOT be listed, because a finding list cannot tell a rule that found nothing from one that did not run.
+
+#### Scenario: Summary counts per rule
+
+- **WHEN** `check --json` runs and one error rule and one warning rule each match once in the same file
+- **THEN** `summary` SHALL report a total of 2 across 1 file, with 1 error and 1 warning
+- **AND** `summary.rules` SHALL hold one entry for each rule, each with a total of 1
+
+#### Scenario: Summary on a project with no rules
+
+- **WHEN** `check --json` runs in a project with no rules configured
+- **THEN** `summary` SHALL be present with a total of 0 and an empty `rules`
+
+### Requirement: Check --summary reports counts without findings
+
+`check --summary` SHALL report the run's counts in place of its findings. Under `--json` it SHALL omit `results` and keep every other key, `summary` included. Without `--json` it SHALL print one line per rule that fired, with its counts by severity and its distinct files, then a total line. The exit code SHALL be the same as without `--summary`.
+
+#### Scenario: JSON summary only
+
+- **WHEN** `check --json --summary` runs and a rule of error severity matches
+- **THEN** stdout SHALL be a JSON object with `success` and `summary` and no `results`
+- **AND** the process SHALL exit with code 1
+
+#### Scenario: Text summary
+
+- **WHEN** `check --summary` runs without `--json` and one error rule and one warning rule each match once
+- **THEN** stdout SHALL include a line per rule naming its engine, its id, its counts and its files
+- **AND** stdout SHALL include `2 issues (1 error, 1 warning) from 2 rules across 1 file`
+- **AND** stdout SHALL NOT include the matched code
+
+### Requirement: Check and verify share one finding schema
+
+A finding in `check --json`'s `results` and a finding in `verify --json`'s `rules[].findings` SHALL both parse against the one `findingSchema` published from `@taskless/cli/schemas`. `verify`'s findings SHALL extend it with `bucket` and SHALL NOT redeclare its fields.
+
+#### Scenario: A finding from each command parses against one schema
+
+- **WHEN** `check --json` and `verify --json` run against a project whose rule fires in both the code and the rule's `fail/` fixture
+- **THEN** a finding from each SHALL parse against the published `findingSchema`

openspec/changes/archive/2026-11-11-check-json-summary/tasks.md

@@ -0,0 +1,21 @@
+## 1. One finding schema
+
+- [x] 1.1 Declare `findingSchema` once in `schemas/finding.ts`
+- [x] 1.2 Use it for `check --json`'s `results[]`, and extend it with `bucket` for `verify`'s fixture findings
+- [x] 1.3 Publish `findingSchema` and `checkOutputSchema` from `@taskless/cli/schemas`
+
+## 2. Per-rule summary
+
+- [x] 2.1 Add `summarizeFindings`, grouping by engine and rule id, and the `summary` schema
+- [x] 2.2 Emit `summary` on every `check --json` run, early returns included
+- [x] 2.3 Add `--summary`: omit `results` under `--json`, print per-rule counts in text mode
+
+## 3. Docs
+
+- [x] 3.1 Document `summary` and `--summary` in the `check` recipe (topic v10) and the README
+
+## 4. Tests
+
+- [x] 4.1 Unit-test `summarizeFindings`: empty, ordering, file spellings, one id under two engines, an unmapped source
+- [x] 4.2 Assert `summary` and `--summary` end to end, in JSON and text, and on a project with no rules
+- [x] 4.3 Parse a finding from the built `check --json` and one from `verify --json` against the one published `findingSchema`

openspec/specs/cli-check/spec.md

@@ -124,20 +124,6 @@ When `CI` is `true` or `1` and `--include-warnings` is not set, the human output
 - **WHEN** `CI=true` and `--json` is set
 - **THEN** `results` SHALL include findings of every severity
 
-### Requirement: Check subcommand formats JSON output when --json is set
-
-When the `--json` flag is set, the CLI SHALL output each `CheckResult` as a JSON object per line (JSONL format) to stdout.
-
-#### Scenario: JSON output streams results
-
-- **WHEN** the `--json` flag is set and the scanner produces results
-- **THEN** each `CheckResult` SHALL be written as a single-line JSON object to stdout, one per line
-
-#### Scenario: JSON output with no issues
-
-- **WHEN** the `--json` flag is set and the scanner produces zero results
-- **THEN** stdout SHALL contain no output lines
-
 ### Requirement: Check subcommand exit codes reflect error severity
 
 The CLI SHALL exit with code 0 when no error-severity matches are found (including when only warnings, info, or hints exist) and no reconcile outcome below requires failure. The CLI SHALL exit with code 1 when at least one error-severity match is found. The CLI SHALL also exit with code 1, whatever the findings and in both human and `--json` modes, when a completed reconcile:
@@ -895,3 +881,60 @@ The measurement SHALL NOT change the exit code, the results, or which rules run.
 
 - **WHEN** `taskless check` runs outside CI and the latest-version lookup fails or times out
 - **THEN** the run SHALL complete with the results and exit code the scan alone would produce
+
+### Requirement: Check subcommand prints one JSON envelope when --json is set
+
+When the `--json` flag is set, the CLI SHALL write one JSON object to stdout: `{ success, results, summary }`, plus the optional `skipped`, `failures`, `notices`, `entitlement`, `integrity` and `runDirectory` keys. `results` SHALL hold every finding, each parsing against the published `findingSchema`. `summary` SHALL be present on every run. `results` SHALL be present on every run except one given `--summary`.
+
+#### Scenario: JSON output carries every finding
+
+- **WHEN** the `--json` flag is set and the scanner produces results
+- **THEN** stdout SHALL be a single JSON object whose `results` array holds one entry per finding
+- **AND** its `summary` SHALL count the same findings
+
+#### Scenario: JSON output with no issues
+
+- **WHEN** the `--json` flag is set and the scanner produces zero results
+- **THEN** `results` SHALL be an empty array
+- **AND** `summary.total` SHALL be 0 and `summary.rules` SHALL be empty
+
+### Requirement: Check reports finding counts per rule
+
+`check --json` SHALL carry a `summary` with the total number of findings, the number of distinct files they are in, the counts by severity, and a `rules` array. Every severity SHALL be present in a count, zero included. `rules` SHALL hold one entry per rule with at least one finding, keyed by `engine` and `ruleId`, carrying that rule's total, distinct files and counts by severity, ordered by total descending and then by `ruleId`. Two engines holding the same rule id SHALL be two entries. A rule with no findings SHALL NOT be listed, because a finding list cannot tell a rule that found nothing from one that did not run.
+
+#### Scenario: Summary counts per rule
+
+- **WHEN** `check --json` runs and one error rule and one warning rule each match once in the same file
+- **THEN** `summary` SHALL report a total of 2 across 1 file, with 1 error and 1 warning
+- **AND** `summary.rules` SHALL hold one entry for each rule, each with a total of 1
+
+#### Scenario: Summary on a project with no rules
+
+- **WHEN** `check --json` runs in a project with no rules configured
+- **THEN** `summary` SHALL be present with a total of 0 and an empty `rules`
+
+### Requirement: Check --summary reports counts without findings
+
+`check --summary` SHALL report the run's counts in place of its findings. Under `--json` it SHALL omit `results` and keep every other key, `summary` included. Without `--json` it SHALL print one line per rule that fired, with its counts by severity and its distinct files, then a total line. The exit code SHALL be the same as without `--summary`.
+
+#### Scenario: JSON summary only
+
+- **WHEN** `check --json --summary` runs and a rule of error severity matches
+- **THEN** stdout SHALL be a JSON object with `success` and `summary` and no `results`
+- **AND** the process SHALL exit with code 1
+
+#### Scenario: Text summary
+
+- **WHEN** `check --summary` runs without `--json` and one error rule and one warning rule each match once
+- **THEN** stdout SHALL include a line per rule naming its engine, its id, its counts and its files
+- **AND** stdout SHALL include `2 issues (1 error, 1 warning) from 2 rules across 1 file`
+- **AND** stdout SHALL NOT include the matched code
+
+### Requirement: Check and verify share one finding schema
+
+A finding in `check --json`'s `results` and a finding in `verify --json`'s `rules[].findings` SHALL both parse against the one `findingSchema` published from `@taskless/cli/schemas`. `verify`'s findings SHALL extend it with `bucket` and SHALL NOT redeclare its fields.
+
+#### Scenario: A finding from each command parses against one schema
+
+- **WHEN** `check --json` and `verify --json` run against a project whose rule fires in both the code and the rule's `fail/` fixture
+- **THEN** a finding from each SHALL parse against the published `findingSchema`

packages/cli/README.md

@@ -53,6 +53,7 @@ pipeline:
 ./.taskless/bin/cli.mjs check                                   # scan everything
 ./.taskless/bin/cli.mjs check $(git diff --name-only main...HEAD)  # PR-only scan
 ./.taskless/bin/cli.mjs check --json                            # machine-readable
+./.taskless/bin/cli.mjs check --json --summary                  # per-rule counts only
 ```
 
 Paths that no longer exist are dropped silently, so raw `git diff` output can be

packages/cli/src/agent/check.md

@@ -1,4 +1,4 @@
-# Topic: check     (CLI v%(CLI_VERSION)s / topic v9)
+# Topic: check     (CLI v%(CLI_VERSION)s / topic v10)
 
 %(ONE_COMMAND_PER_CALL)s
 
@@ -62,7 +62,7 @@ plan does not include restoring rules, the git steps that do instead
 Notices about skipped runtime rules are human-readable stderr only.
 Under `--json` they do NOT appear as warnings; instead an additive
 optional `skipped: [{ rule, reason }]` array is included alongside the
-unchanged `{ success, results }`, the not-verified notice is an entry
+unchanged `{ success, results, summary }`, the not-verified notice is an entry
 in `notices`, and the CI backstop
 (`%(TASKLESS_CLI)s agent ci`) is the enforcement point.
 
@@ -179,7 +179,10 @@ delete the rules to make `check` pass, and do not suggest
 `entitlement` with an empty `withheld` does not fail the run.
 
 ## Flags
-- `--json`: machine output (`{ success, results, skipped?, failures?, notices?, integrity?, entitlement? }`).
+- `--json`: machine output (`{ success, results, summary, skipped?, failures?, notices?, integrity?, entitlement? }`).
+- `--summary`: per-rule counts instead of every finding. With `--json`
+  it omits `results` and keeps `summary`, so the output stays small on
+  any repository. Use it when the question is "how many", not "where".
 - `--anonymous`: run only static rules; skip runtime rules.
 - `--dangerously-run-scripts`: run runtime `check.ts` unverified.
 - `--timeout <seconds>`: per-runtime-check wall-clock bound (default 10).
@@ -249,10 +252,31 @@ delete the rules to make `check` pass, and do not suggest
          "matchedText": "eval(input)",
          "fix": null
        }
-     ]
+     ],
+     "summary": {
+       "total": 1,
+       "files": 1,
+       "severities": { "error": 1, "warning": 0, "info": 0, "hint": 0 },
+       "rules": [
+         { "engine": "sg", "ruleId": "no-eval", "total": 1, "files": 1,
+           "severities": { "error": 1, "warning": 0, "info": 0, "hint": 0 } }
+       ]
+     }
    }
    ```
 
+   Each `results` entry has the same shape as a finding in
+   `verify --json`'s `rules[].findings` (which adds `bucket`), so one
+   parser reads both. `summary` is always present; its `rules` lists
+   only rules that fired, most findings first, keyed by `engine` and
+   `ruleId` as `verify` keys `rules[]`.
+
+   **To measure a whole repository, add `--summary`.** `results` can run
+   to hundreds of kilobytes; `check --json --summary` returns only
+   `summary`, one entry per rule that fired. Do not count `results`
+   yourself, and do not read an absent `results` as clean: read
+   `summary.total`.
+
 4. **Report findings to the user.** Group by `file`. Show `severity`,
    `message`, and `ruleId` for each finding; the `range.start` is the
    useful line/column to surface. The `success` field reflects