Skip to content

coverage_matrix.md

Latest commit

 

History

History
36 lines (27 loc) · 2.25 KB

coverage_matrix.md

File metadata and controls

36 lines (27 loc) · 2.25 KB
Raw

Coverage Matrix

The coverage matrix ties every requirement to a verifying artifact so that the spec, fixtures, and implementation stay aligned.

Command

Use the matrix subcommand documented in the command cheatsheet. When invoked with --out spec/extras/trace_matrix.json, it writes the canonical report alongside other toolkit outputs.

  • spec/ — root of the spec artifacts.
  • --out — optional path for the generated JSON report (- to print to stdout).

The output lists each FR ↔ API ↔ fixture ↔ NFR chain. CI can diff this file to detect drift.

Expected Links

Spec Step Artifact Downstream Target
04_fr_list.json Functional requirements Trace to API contracts (05_interface_contracts.json) via traceRef.
05_interface_contracts.json API contracts Targets fixtures (08_fixtures.json) and scaffold routes (15_scaffold.json).
06_invariants.json Invariants Referenced by fixtures and runtime validators.
07_nfrs.json NFRs Linked to dashboards/alerts in 16_impl_context.json (Plan/Review).
11_redteam.json Threat scenarios Feed additional fixtures and updates in 16_impl_context.json (Plan).
16c_review Drift plan.drift checks defined in 16_impl_context.json.

CI Integration

Add the matrix command to CI alongside the core validation commands so drift is detected automatically. Always run:

mkdir -p spec/extras && ./tools/run_specdev.sh matrix spec --repo-root ./devspec_toolkit --out spec/extras/trace_matrix.json

CI should fail if spec/extras/trace_matrix.json differs from the committed expectation without matching spec changes or if required trace links are missing.

Troubleshooting

  • Missing FR trace: ensure each FR in 04_fr_list.json has at least one traceRef pointing to an API ID.
  • Dangling API reference: confirm the API exists in 05_interface_contracts.json and is targeted by a fixture.
  • Unlinked fixture: verify targets reference valid API IDs and that expectations match schema revisions.
  • Matrix diff noise: regenerate after legitimate changes and commit the new output to keep CI deterministic.