Skip to content

fix(validate): report requirements outside delta sections - #1804

Open
dwin-gharibi wants to merge 2 commits into
Fission-AI:mainfrom
dwin-gharibi:fix-orphaned-requirement-report
Open

fix(validate): report requirements outside delta sections#1804
dwin-gharibi wants to merge 2 commits into
Fission-AI:mainfrom
dwin-gharibi:fix-orphaned-requirement-report

Conversation

@dwin-gharibi

@dwin-gharibi dwin-gharibi commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Closes #1803.

Why

The delta reader only looks inside the four delta sections. A canonical
### Requirement: block written anywhere else — under ## Notes, under a
misspelled ## Add Requirements, or above the first ## header — is dropped
with no error, no warning and no note.

That is inconsistent with the treatment of the adjacent mistake. A
non-canonical ### header inside a delta section has been recorded in
skippedHeaders and surfaced as INFO by validate <change> since #498. The
well-formed requirement in the wrong place — arguably the costlier error,
because the block reads exactly like one that would apply — said nothing at all.
validate reports the change valid and archive exits 0.

What Changes

  • parseDeltaSpec returns DeltaPlan.orphanedRequirements: every canonical
    ### Requirement: header outside the delta sections, with the section it sits
    under (null above the first ## ) and its 1-based line number.
  • validate <change> reports each one as a WARNING naming the section and
    the line, and listing the four headers it could move under.
  • buildUpdatedSpec emits the same thing through its existing warn() channel,
    so archive says it too — the last point at which the author can still notice.

Behaviour is otherwise unchanged: the orphan is still not applied. This PR
only ends the silence.

WARNING rather than ERROR is deliberate. Scanning every delta spec.md in this
repository with its own parser found 8 blocks of this shape, all in archived
pre-format changes from 2025-08-19 (add-skip-specs-archive-option,
structured-spec-format). An ERROR would retroactively fail those; the fix for a
real occurrence is to move the block, not to reject the change.

Testing

test/core/parsers/orphaned-requirements.test.ts — 11 tests, written first and
watched fail (10 failed / 1 passed before, 11 passed after).

Edge cases covered:

  • an orphan under ## Notes, under a misspelled ## Add Requirements, and above
    the first ## section
  • several orphans reported in document order with correct line numbers
  • nothing reported when every requirement sits in a delta section
  • every casing of the four delta headers accepted without reporting
  • a requirement shown inside a code fence is not reported
  • a delta that legitimately opens with ## Purpose (new capability) is not
    reported
  • buildUpdatedSpec warns, and still does not apply the orphan
  • a well-formed delta produces no such warning
  • validate reports it as a WARNING and the verdict stays valid (no ERROR)

Full suite green on this branch.

Changeset

Not added. Per .changeset/README.md the default path is the normal release
cadence; happy to run pnpm changeset if a maintainer wants dedicated release
notes.

Summary by CodeRabbit

  • New Features

    • Detects requirement blocks placed outside recognized delta sections.
    • Identifies each misplaced requirement by name, section, and line number.
  • Bug Fixes

    • Change application warns about misplaced requirements and leaves them unapplied.
    • Validation reports misplaced requirements as warnings, with guidance to move them into a valid section.
    • Requirement examples inside code fences are ignored.
    • Delta headers are recognized regardless of capitalization.

Copilot AI lite review requested due to automatic review settings September 6, 2026 08:21
@dwin-gharibi
dwin-gharibi requested a review from a team as a code owner September 6, 2026 08:21
@dwin-gharibi
dwin-gharibi requested review from clay-good and removed request for a team September 6, 2026 08:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 37a3e6fb-cf77-4b93-91f0-382c66bec98d

📥 Commits

Reviewing files that changed from the base of the PR and between 008b876 and 5e975d6.

📒 Files selected for processing (1)
  • test/core/parsers/orphaned-requirements.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/core/parsers/orphaned-requirements.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review.


📝 Walkthrough

Walkthrough

The parser detects canonical requirement blocks outside the four delta sections. Apply and validation flows report these blocks as warnings with their location, while leaving them unapplied.

Changes

Orphaned requirement handling

Layer / File(s) Summary
Detect orphaned requirements
src/core/parsers/requirement-blocks.ts, test/core/parsers/orphaned-requirements.test.ts
parseDeltaSpec records requirement names, sections, and 1-based line numbers outside recognized delta sections. Fenced content and valid delta sections remain excluded.
Report and preserve orphaned requirements
src/core/specs-apply.ts, src/core/validation/validator.ts, test/core/parsers/orphaned-requirements.test.ts
Apply and validation flows emit warnings for orphaned requirements. The apply flow does not include them in the rebuilt specification. Tests verify warning levels and messages.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 5e975

The change adds diagnostics for misplaced requirement blocks while preserving their unapplied behavior. No concrete merge-blocking risk is currently identified.

Sequence Diagram(s)

sequenceDiagram
  participant DeltaSpec
  participant Parser
  participant ApplyOrValidator
  DeltaSpec->>Parser: parse requirement blocks
  Parser->>Parser: detect blocks outside delta sections
  Parser-->>ApplyOrValidator: return orphanedRequirements
  ApplyOrValidator->>ApplyOrValidator: emit warning
  ApplyOrValidator->>ApplyOrValidator: leave orphaned requirement unapplied
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: reporting requirements placed outside delta sections. It does not mention archive diagnostics, but it remains concise and directly related.
Linked Issues check ✅ Passed The changes address issue #1803. They detect orphaned requirements, report each requirement with its section and line number during validation and archive operations, preserve valid status without an …
Out of Scope Changes check ✅ Passed The parser changes, validation and archive warnings, and focused tests all support the linked issue and stated objectives. No unrelated code changes are identified.
Docstring Coverage ✅ Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 4 files.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@alfred-openspec alfred-openspec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed parser, validation, and archive behavior and ran the focused 11-test suite locally. Warning-only reporting closes the silent-drop gap without breaking legacy archived changes. Approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A requirement written outside a delta section is dropped with no diagnostic

3 participants