Activity

1. mnriem

   added

   bug-assess

   on Oct 20, 2026
2. github-actions commented on Oct 20, 2026

   

   github-actionsBot

   on Oct 20, 2026 – with GitHub Actions

   Contributor

   More actions

   Bug assessment — bug-extension-reassessment: Valid · severity medium

   ---

   Bug Assessment: Bug extension command names and stale reports after reassessment

   • Slug: bug-extension-reassessment
   • Created: 2026-10-20T19:06:55Z
   • Source: issue [Bug]: bug extension: README uses dotted command names, and re-assessing a bug leaves the old fix and test reports in place #5036
   • Verdict: valid
   • Severity: medium

   Report (summarized)

   The reporter identifies two related defects in the bundled bug extension v1.0.1 (Specify CLI 1.2.2): the README shows dotted invocations such as /speckit.bug.assess, while skills-based integrations install and invoke hyphenated skill names such as speckit-bug-assess; and reassessing an existing bug can replace assessment.md without invalidating already-generated fix.md and test.md. The latter allows the test stage to read a new assessment together with an old fix. The report cites Claude Code and Codex CLI on macOS and notes that the stale-report scenario was inspected in source rather than run end to end.

   Symptom

   Skills-based users following the README may invoke a command name that is not present in their installed integration. Separately, after an interactive reassessment overwrites assessment.md, the existing fix.md and test.md remain and speckit.bug.test reads the new assessment with the old fix, so its validation can no longer be trusted as validation of the current assessment.

   Reproduction

   1. Install the bug extension with a skills-based integration such as Claude Code or Codex.
   2. Compare the README examples in extensions/bug/README.md with the generated skill names; the README uses /speckit.bug.*, while skills use hyphenated names such as /speckit-bug-assess or $speckit-bug-assess.
   3. Run assessment, fix, and test for one bug slug.
   4. Run assessment again for the same slug and confirm overwriting the existing assessment.
   5. Run the test command for that slug and observe that it reads the newly written assessment.md and the pre-existing fix.md together.

   The first mismatch is directly grounded in the repository. The stale-report path is also directly specified by the command templates, but the issue's end-to-end execution of steps 3–5 is [NEEDS CLARIFICATION: not independently reproduced in this assessment].

   Suspected Code Paths

   • extensions/bug/README.md:26-28 — documents the canonical command IDs as the only command forms, without the integration-specific hyphenated skill invocations.
   • extensions/bug/README.md:59-68 — all three usage examples use dotted slash commands (/speckit.bug.assess, /speckit.bug.fix, /speckit.bug.test).
   • extensions/bug/extension.yml:17-24 — registers the canonical dotted command IDs. These IDs are valid extension identifiers, but skills integrations derive names by replacing dots with hyphens rather than preserving the dotted spelling.
   • src/specify_cli/integrations/base.py:1753-1754,1863 — the shared skills integration renders invocation/skill names as speckit- plus the command stem with dots replaced by hyphens.
   • src/specify_cli/extensions/__init__.py:1438,5661 — extension command skill naming likewise maps dotted command IDs to speckit-<dotted-name-with-hyphens>.
   • extensions/bug/commands/speckit.bug.assess.md:36 — the overwrite guard addresses only BUG_DIR/assessment.md; it does not discard, mark, or otherwise invalidate fix.md or test.md when reassessment is confirmed.
   • extensions/bug/commands/speckit.bug.test.md:36-41 — test prerequisites require both reports and explicitly read assessment.md and fix.md, with no freshness or provenance check between them.
   • extensions/bug/commands/speckit.bug.fix.md:37-38 — fix generation similarly treats the assessment as the contract but does not provide a mechanism for an assessment overwrite to invalidate an existing fix.

   Root Cause Hypothesis

   Confidence: high. The README treats the canonical extension command ID (speckit.bug.*) as a universal invocation syntax, although skills integrations intentionally transform that ID to speckit-bug-* and use integration-specific prefixes. Independently, the assessment command's overwrite contract is scoped to assessment.md, while the test command accepts any existing fix.md in the directory and has no generation, timestamp, or content linkage check. Consequently, reassessment can change the contract without changing the downstream artifacts that consume it.

   Proposed Remediation

   Preferred: Update extensions/bug/README.md to distinguish canonical command IDs from invocation syntax, and show the supported forms for dot-command integrations, hyphen/slash skills integrations, and Codex-style $speckit-bug-* skills where applicable. Keep the dotted IDs in extension.yml; the issue is documentation, not a need to rename the registered commands.

   Make reassessment invalidate dependent artifacts as one explicit workflow decision. When an existing assessment is overwritten after confirmation, require the user to discard (or explicitly retain as archived evidence) fix.md and test.md; the safest default is to remove or move them before writing the new assessment and state that a new fix/test cycle is required. Also add a defensive check in the test command so it refuses to validate when the fix is not associated with the current assessment (for example, by requiring a recorded assessment fingerprint/generation in the reports). This prevents stale validation even if artifacts were copied or retained outside the normal reassessment path.

   Alternatives (optional):

   • Preserve all reports but write generation metadata into each report and have speckit.bug.test require matching assessment and fix generations. This retains history but adds report-format and migration complexity.
   • On reassessment, automatically choose a new slug rather than overwrite the existing bug directory. This is safest for immutability but makes deliberate iterative reassessment less convenient and diverges from the interactive overwrite behavior described by the command.

   Files likely to change:

   • extensions/bug/README.md
   • extensions/bug/commands/speckit.bug.assess.md
   • extensions/bug/commands/speckit.bug.test.md
   • extensions/bug/commands/speckit.bug.fix.md (if report linkage or stale-artifact guidance is added there)
   • Relevant extension/template tests covering README content and reassessment behavior

   Tests to add or update:

   • Verify generated README examples/documentation cover the invocation forms emitted by Claude/Codex and other skills-based integrations.
   • Verify reassessment with an existing fix.md and test.md requires discard/archive handling and does not leave them usable as current reports.
   • Verify speckit.bug.test rejects a fix tied to an older assessment and accepts matching assessment/fix metadata.
   • Verify the existing non-overwrite behavior remains intact in automated mode and for unconfirmed interactive overwrites.

   Risks & Considerations

   • Deleting dependent reports can lose useful history; archiving or clearly retaining them as stale artifacts is safer if history matters.
   • Adding report linkage changes the Markdown contract and must account for existing bug directories that lack metadata; those should fail clearly and instruct the user to rerun the fix rather than silently pass.
   • README examples should reflect each integration's actual prefix and syntax without implying that every integration supports every form.
   • This is a workflow-integrity issue rather than a source-code execution or data-security issue, but false-positive verification can cause maintainers to believe an unvalidated fix satisfies a newer assessment.

   Open Questions

   • [NEEDS CLARIFICATION: Should reassessment archive stale fix.md/test.md, delete them, or preserve them with an explicit stale marker?]
   • [NEEDS CLARIFICATION: Which complete set of integration invocation forms should the README document, beyond Claude Code and Codex?]
   • [NEEDS CLARIFICATION: Should report freshness use a generated assessment ID/fingerprint, or should reassessment always force a new slug?]

   Generated by 🐛 Assess Bug from Labeled Issue for #5036 · copilot · gpt52codex · 1.55 AIC · ⌖ 0.609 AIC · ⊞ 22.3K · ◷
3. github-actions

   added

   severity-medium

   on Oct 20, 2026
4. mnriem

   added

   triage-must-haveVerdict: high-value, important work for Spec Kit — do firstVerdict: high-value, important work for Spec Kit — do first

   on Oct 21, 2026