Skip to content

chore(changelog): one fragment per PR instead of a bullet in the shared file (proposal) - #11215

Merged
MarkusNeusinger merged 3 commits into
mainfrom
infra/changelog-fragments
Sep 3, 2026
Merged

chore(changelog): one fragment per PR instead of a bullet in the shared file (proposal)#11215
MarkusNeusinger merged 3 commits into
mainfrom
infra/changelog-fragments

Conversation

@MarkusNeusinger

Copy link
Copy Markdown
Owner

This is a process change, and it is a proposal. It changes what every future PR has to do. The on/off paths are at the bottom — turning it off is deleting one workflow file; turning it back on is restoring it.

Why

CHANGELOG.md's [Unreleased] section is the one place sibling PRs reliably conflict each other. On the night of 2026-09-02/03 it did so three times in a row — each a hand-resolved rebase for text neither branch disagreed about, and each one a chance to resolve it wrong.

The usual remedies do not apply. .gitattributes already normalizes line endings and does nothing for this. A union merge driver would heal the local rebase for pure additions, but GitHub's own mergeability check ignores merge drivers, so the PR still shows as conflicted — and a branch that moves changelog lines comes out of a union rebase with the block duplicated.

Fragments remove the shared spot instead of healing it: a PR writes changelog.d/<slug>.md and touches CHANGELOG.md not at all. Two PRs never meet at the same line, because there is no longer a line they share.

Ported from the sibling repo kurrentschrift, where the same tool has run since 2026-08-30 across ~40 fragments and one release cut.

What lands

  • tools/changelog — standard library only, three verbs:
    • check [--base REF] — every fragment parses; with --base the branch must carry one, and must not have written a bullet into [Unreleased].
    • preview — the pending section as the next cut would write it.
    • release X.Y.Z --title "…" — the cut: folds every fragment under the new heading (newest first within a category, by the commit that added the fragment), bumps pyproject.toml, uv.lock and app/package.json, repoints the compare links, deletes the fragments. --dry-run prints and writes nothing.
    • It lives outside the distribution: [tool.setuptools.packages.find] names api*, core*, automation*, and the API image's allowlist .dockerignore does not name tools/ either.
  • .github/workflows/ci-changelog.yml — the job "Changelog (fragment)", pull_request only. uv run --no-project keeps it at seconds, since the tool syncs nothing.
  • changelog.d/README.md — the format, the rules, the exemptions.
  • The docs that carry the rule, all in step: CLAUDE.md, .github/copilot-instructions.md, agentic/commands/pull_request.md, agentic/commands/release.md, .github/pull_request_template.md, .claude/skills/open-pr/SKILL.md, and CHANGELOG.md's own header prose, which now opens with "Do not edit this file in a feature PR."

Two adaptations this repository needed

anyplot's release does two things the sibling's does not, and the tool respects both:

  1. The compare links at the bottom of CHANGELOG.md. release.md calls keeping them in step "easy to forget" — which is exactly the kind of step a tool should hold instead of a checklist, so release repoints [Unreleased] and inserts the new version's link. A changelog without a link block keeps none; the tool does not invent one.
  2. The two aggregate lines — the italic *Catalog: …* and the single **Dependencies:** bullet. The tool deliberately does not write these: they summarize a release window rather than any PR, so they stay a hand step in release.md, written into the section the tool has just laid out. release.md step 3 and the CLI's own closing message both say so.

The exemptions are anyplot's existing ones, now enforced rather than remembered: catalogue-only PRs (everything under plots/) pass on the path; the plot pipeline (github-actions[bot]) and Dependabot pass on the author, so the gate never sits red on a bot batch; skip-changelog is the human escape hatch. A pipeline PR that strays outside plots/ still needs a fragment — there is a test for exactly that.

Verification

  • pytest tests/unit tests/integration1946 passed, 1 skipped. 25 of those are the ported tool tests, including the PR gate exercised against a throwaway git repository — the same call the CI job makes.
  • ruff check . and ruff format --check . clean; mypy api core unchanged.
  • tests/unit/test_agent_instructions.py pins the rule as mirrored in both guides; its keywords were updated to carry the prohibition ("never a bullet in CHANGELOG.md") and not just the topic, per the note that file already makes about that failure mode.
  • Run against the real repository: check parses today's [Unreleased] without complaint, and release 3.3.0 --dry-run produces the correct section, the four file writes and no changes on disk.
  • This PR carries its own fragment and passes its own gate — check --base origin/main is green on this branch.

Turning it off, and back on

Off: delete .github/workflows/ci-changelog.yml. Nothing else enforces anything — the tool is inert unless called, and CHANGELOG.md still works exactly as before, since the cut reads [Unreleased] and the fragments alike. Any fragments already written can be folded in by hand or left for the next release run.

Back on: restore that one file.

Partial: keep the workflow but drop the "no bullets in [Unreleased]" half by removing the check --base call and running plain check — that keeps fragments well-formed without requiring them.

The docs edits are the part that would need reverting too, but they are prose in six files and no behaviour depends on them.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UEScQMZFvxxNNyNJYryfa3

…ed file

The [Unreleased] section is the one place sibling PRs reliably conflict each
other — three times in a row on the night of 2026-09-02/03, each a hand-resolved
rebase for text neither branch disagreed about. .gitattributes does nothing for
it, and a union merge driver would not fix it: GitHub's mergeability check
ignores merge drivers, and a branch that MOVES changelog lines comes out of a
union rebase with the block duplicated.

Fragments remove the shared spot rather than healing it. A PR writes
changelog.d/<slug>.md in the changelog's own format and touches CHANGELOG.md
not at all.

tools/changelog is the standard-library tool behind it — `check` (also the CI
job "Changelog (fragment)", which refuses a missing fragment AND a bullet
written into [Unreleased]), `preview`, and `release`, which folds the fragments
newest-first under the new heading, bumps pyproject.toml, uv.lock and
app/package.json, repoints the compare links — a step release.md called easy to
forget, so it is no longer a step — and deletes the fragments.

The exemptions are the ones this repository already had, now enforced rather
than remembered: catalogue-only PRs under plots/, the plot pipeline and
Dependabot by author, and the skip-changelog label. The release's two aggregate
lines stay by hand: they summarize a window, not a PR.

Ported from kurrentschrift, where the same tool has run since 2026-08-30.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UEScQMZFvxxNNyNJYryfa3
Copilot AI balanced review requested due to automatic review settings September 3, 2026 21:33
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UEScQMZFvxxNNyNJYryfa3

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Moderate CI, validation, and release-reporting issues remain unresolved.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Introduces per-PR changelog fragments to avoid conflicts in the shared [Unreleased] section.

Changes:

  • Adds changelog validation, preview, and release tooling.
  • Adds CI enforcement and unit tests.
  • Updates contributor and release documentation.

Review comments:

  • Moderate (1 vote).github/workflows/ci-changelog.yml:20: Include labeled and unlabeled pull-request activity types so applying or removing skip-changelog reruns the gate. Also applies at line 52.
  • Nit (2 votes)agentic/commands/release.md:36: Update agentic/docs/project-guide.md:1094-1106, which still documents the old release process.
  • Nit (1 vote)changelog.d/README.md:3: Clarify that fragments apply to non-code PRs too.
  • Nit (2 votes)changelog.d/README.md:57: Point the aggregate-line cross-reference to step 3.
  • Nit (1 vote)changelog.d/README.md:26: Distinguish machine-enforced syntax from human-reviewed writing guidance.
  • Nit (1 vote)tools/__init__.py:3: Replace the nonexistent root .dockerignore explanation with the actual Dockerfile copy allowlist.
  • Moderate (2 votes)tools/changelog/__init__.py:126: Require a closing ** in bold fragment titles. Also applies at lines 290, 373, 374, and 378.
  • Moderate (2 votes)tools/changelog/__main__.py:73: For non-dry runs, apply the release plan before reporting successful writes or deletions. Also applies at line 87.
File summaries
File Description
tools/changelog/__main__.py Implements the changelog CLI.
tools/changelog/__init__.py Implements parsing, release cuts, and PR validation.
tools/__init__.py Defines the developer-tools package.
tests/unit/tools/test_changelog_tool.py Tests the changelog tool and gate.
tests/unit/tools/__init__.py Defines the test package.
tests/unit/test_agent_instructions.py Updates instruction synchronization checks.
CLAUDE.md Documents the fragment workflow.
CHANGELOG.md Redirects feature PRs to fragments.
changelog.d/README.md Documents fragment formatting and exemptions.
changelog.d/changelog-fragments.md Adds this PR’s changelog fragment.
agentic/commands/release.md Updates the release-cut procedure.
agentic/commands/pull_request.md Adds fragment checks to the PR workflow.
.github/workflows/ci-changelog.yml Adds the fragment CI gate.
.github/pull_request_template.md Updates the contributor checklist.
.github/copilot-instructions.md Synchronizes Copilot guidance.
.claude/skills/open-pr/SKILL.md Updates PR-opening guidance.
Review details

Suppressed comments (9)

.github/workflows/ci-changelog.yml:55

  • Skipping every github-actions[bot] PR means the path-sensitive rule is never evaluated for that author. Consequently, a pipeline PR that changes core/ or a workflow passes without a fragment, contradicting the PR description and test_a_pipeline_pr_that_strays_outside_plots_still_needs_one. Either rely on the existing plots/ exemption for this bot or revise the stated requirement and test.
    if: >-
      !contains(github.event.pull_request.labels.*.name, 'skip-changelog')
      && github.event.pull_request.user.login != 'dependabot[bot]'
      && github.event.pull_request.user.login != 'github-actions[bot]'

changelog.d/README.md:5

  • The new procedure makes fragments mandatory for documentation, workflow, and other non-code PRs too, but this opening limits the duty to PRs that change code. That conflicts with the CI gate and the mirrored “every PR” rule and can lead docs-only contributors to omit the required fragment.
Every PR that changes code adds ONE file here instead of editing
`CHANGELOG.md`: `changelog.d/<slug>.md`, the slug naming the change (the branch
name minus its prefix does fine — `origin-gate-rest.md`, `csp-hashes.md`).

changelog.d/README.md:26

  • The check enforces structural syntax, but it cannot enforce the English and what/where/why prose conventions listed below. Calling every rule enforced gives contributors a false guarantee; distinguish machine-checked format rules from human-reviewed writing guidance.
Rules, all enforced by `uv run python -m tools.changelog check`:

tools/init.py:5

  • There is no root .dockerignore in this checkout, so this explanation cites a nonexistent allowlist. The API image excludes tools/ because api/Dockerfile:60-65 copies only the virtualenv plus api/, core/, and plots/; document that actual mechanism instead.
`[tool.setuptools.packages.find]` in pyproject.toml lists `api*`, `core*` and
`automation*`, so nothing here reaches the distribution, and the API image's
root `.dockerignore` is an allowlist that does not name this directory either.

tools/changelog/init.py:378

  • Any path under changelog.d/ currently satisfies the gate, even README.md, a .txt file that load_fragments() ignores, or a deleted fragment. A non-exempt code PR can therefore bypass the one-fragment rule without contributing a releasable entry. Count only added or modified non-README .md fragments.
    touches_fragments = any(p.startswith(f"{FRAGMENT_DIR_NAME}/") for p in changed)

tools/changelog/init.py:373

  • Any edit that changes or removes the newest version heading sets release_cut=True, bypassing the fragment requirement even when no release was cut. Require a higher new version, an emptied [Unreleased] section, and retention of the previous version heading before granting the release exemption.
        release_cut = after.newest_version != before.newest_version

tools/changelog/init.py:377

  • Set subtraction catches only newly added bullet text. A feature PR can delete, rewrite, reorder, or move existing [Unreleased] bullets and still pass when it also has a fragment, despite the new rule requiring that shared section to remain untouched. Compare the section body for non-release PRs instead.
        gained = _bullets(after.unreleased) - _bullets(before.unreleased)
        for bullet in sorted(gained):
            title = bullet.split("\n", 1)[0][:72]
            problems.append(f"{CHANGELOG_NAME} [Unreleased] gained a bullet — it belongs in a fragment: {title}…")

tools/changelog/init.py:291

  • The date check validates only the string shape, so values such as 2026-02-31 are accepted and written permanently into the release heading. Parse it as an actual calendar date and add an impossible-date regression case.
    if not _DATE.fullmatch(date):
        raise ChangelogError(f"date '{date}' is not YYYY-MM-DD")

tools/changelog/main.py:87

  • This reference is stale: the updated release procedure adds aggregate lines in step 3, not step 2. The completion message currently sends release operators to the cut step.
        f"single **Dependencies:** bullet — agentic/commands/release.md step 2), review the diff,\n"
  • Files reviewed: 15/16 changed files
  • Comments generated: 5
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/ci-changelog.yml
Comment thread tools/changelog/__init__.py
Comment thread tools/changelog/__main__.py
Comment thread agentic/commands/release.md
Comment thread changelog.d/README.md Outdated
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

1. The `skip-changelog` escape hatch stayed red after labelling — the default
   pull_request activity types exclude labeled/unlabeled, so the documented way
   out did not re-run the check. Types listed explicitly now.
2. `- **unterminated title` parsed as well-formed. The closing `**` cannot be
   checked on the opening line, because a real title regularly runs onto the
   continuation line before it, so it is checked when the bullet ends and the
   error names the line the bullet opened on. Three tests: the wrapped title
   that must pass, and two unterminated ones that must not.
3. `release` printed "wrote …"/"deleted …" before apply_release() did any I/O,
   so a failed write left a record of success above the traceback. Apply first,
   report after; the dry run keeps its "would" wording.
4. agentic/docs/project-guide.md still described the old process end to end —
   bullets under [Unreleased], the section moved by hand, no app/package.json,
   the release published verbatim. It now describes the fragment flow and the
   condensation rule.
5. changelog.d/README.md pointed at release.md step 2 for the aggregate lines,
   which is the cut; they are step 3.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UEScQMZFvxxNNyNJYryfa3
@MarkusNeusinger
MarkusNeusinger merged commit d278124 into main Sep 3, 2026
21 checks passed
@MarkusNeusinger
MarkusNeusinger deleted the infra/changelog-fragments branch September 3, 2026 21:57
MarkusNeusinger added a commit that referenced this pull request Sep 3, 2026
The fragment convention landed on main in #11215 while this PR was open: a PR
writes changelog.d/<slug>.md and leaves CHANGELOG.md alone.

The one CHANGELOG.md edit that stays is not this PR's own entry but the
correction of a PREVIOUS one — #11207's frontend bullet still says the API
pushes :latest alongside the deploy, which is exactly what this PR changes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UEScQMZFvxxNNyNJYryfa3
MarkusNeusinger added a commit that referenced this pull request Sep 3, 2026
One line of `CHANGELOG.md`, so the `[Unreleased]` section describes one
rollout state instead of two.

#11207's frontend entry said:

> …then `update-traffic` to exactly that revision (the chains are not
identical — this one pushes `:latest` only after the promotion, **where
the API still pushes it alongside the deploy**).

#11212 is what changed that, so the sentence became false the moment it
merged. Now:

> …to exactly that revision (the chains **were** not identical **at the
time** — this one **pushed** `:latest` only after the promotion, where
the API still **pushed** it alongside the deploy; **the API caught up in
#11212**).

Past tense, and the closing PR named rather than a position: the release
cut folds fragments in **above** the existing bullets, so "the entry
above" would be wrong today and right after the cut, while `#11212` is
true in both.

## Why `skip-changelog` rather than a fragment

This corrects an existing entry; it is not a change worth a
release-notes line of its own. And it *needs* the label, because the
fragment gate compares bullet **sets** — an edited bullet is
indistinguishable from an added one. Measured rather than assumed, when
this same edit first rode along in #11212:

```
error: CHANGELOG.md [Unreleased] gained a bullet — it belongs in a fragment:
- **The frontend deploys through a candidate revision instead of straigh…
```

That is the reason it was taken back out of #11212 and moved here.

The `skip-changelog` label did not exist in this repository — #11215
documented it as the escape hatch without creating it — so it was
created for this PR (grey, "No changelog fragment needed: this PR
changes nothing worth a release-notes line"). The gate's own `if:`
condition now has something to match.

**Noted for later, not built here:** the gate could tell an *added*
bullet from a *changed* one by treating the bold title as the bullet's
identity, which would let a correction like this pass without a label.
That is a sibling-pair change for both repos, deliberately out of scope
tonight.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01UEScQMZFvxxNNyNJYryfa3

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
MarkusNeusinger added a commit that referenced this pull request Sep 4, 2026
## Summary

- **A corrected bullet is no longer an added one.** `check` compared
bullet SETS in `CHANGELOG.md [Unreleased]` against the base, so
re-wording an entry the base already carried was indistinguishable from
writing a new one and drew the same refusal — "gained a bullet — it
belongs in a fragment" — for a change that added nothing. A bullet is
now identified by its **bold title** (`bullet_title`),
whitespace-collapsed so the identity survives a correction that reflows
the very line the title runs over, and **counted** rather than
set-differenced (`_added_bullets`) so a second copy cannot slip in
behind the first: a title the base lacks — or one more copy of a title
it has — is an ADDED bullet and is refused exactly as before, a title it
has is a CHANGED bullet and passes. The closing-`**` check #11215 moved
to the bullet end is what makes that identity total.
- **`check` refuses a fragment that still says `(#NNNNN)`.** The
reference arrives from `/pull_request` step 7 once the PR exists, so a
fragment is written without one — but the placeholder shipped as written
reads as a reference in the released section and points nowhere, and
nothing caught it. The complaint names the file and the line. A
placeholder quoted in backticks is prose *about* the rule, not a
reference, and passes, which is what lets `changelog.d/README.md` and
this PR's own fragment name it.
- **The docs that told an agent otherwise.** The README example no
longer hands the writer a `(#NNNNN)` to leave behind;
`agentic/commands/pull_request.md` steps 4 and 7,
`agentic/docs/project-guide.md`, the PR template, the `CHANGELOG.md`
header, and the § Changelog + releases entry AND the Important Rules
summary in both `CLAUDE.md` and `.github/copilot-instructions.md` now
say added-vs-changed and "leave the reference out, step 7 appends the
real number" — every copy of the rule kept in sync as they require.

Sibling PR: kurrentschrift #520 carries the identical change to the
implementation this repo ported in #11215. Nothing else diverged between
the two files while porting — the repo-specific parts (exempt prefix,
version files, compare links, the aggregate lines) stayed untouched, and
the new code is byte-identical apart from the two places that name each
repo's own reference workflow.

## Plan

N/A — a two-edge fix decided directly.

## Test plan

- [x] `uv run --extra test pytest tests/unit` — 1901 passed
- [x] New cases in `tests/unit/tools/test_changelog_tool.py`: added vs.
changed bullet (a correction, a correction that reflows a wrapped title,
and a second bullet reusing an existing title — all against a throwaway
git repo), placeholder present in either width / absent / a real number
/ quoted in backticks, and the complaint naming the fragment file and
line
- [x] `uv run --extra dev ruff check tools tests` and `ruff format
--check tools tests` clean
- [x] `uv run python -m tools.changelog check --base origin/main` — 5
fragments well-formed, the diff passes (this PR edits the `CHANGELOG.md`
header, which the gate correctly reads as no bullet added)

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants