Skip to content

Add ADR 0008: scope agent findings to user-observable change - #1848

Merged
mrbobbytables merged 1 commit into
cncf:mainfrom
mrbobbytables:adr/0008-agent-remit
Oct 23, 2026
Merged

mrbobbytables merged 1 commit into
cncf:mainfrom
mrbobbytables:adr/0008-agent-remit

Conversation

@mrbobbytables

Copy link
Copy Markdown
Member

Why

The site is launched and its content is stable, but the repository is not. In the ten days to 2026-10-23 the Hive fleet opened 138 of the last 150 issues (about 15 a day), and 65 of the 80 pull requests merged since 2026-10-17 changed only tests/, scripts/, .github/ or root markdown; one changed docs/. ADR 0006 and ADR 0007 each closed one series of findings, and the fleet moved to the adjacent category each left open (mutants in the test harness, clones under the ceiling, re-filed CSS decisions, tracking issues for planning-doc bullets).

What

adr/0008-scope-agent-findings-to-user-observable-change.md states one remit in bright lines, with no qualifiers on purpose (ADR 0007's qualifiers each became their own series and an amendment within two days):

  1. An agent-filed issue or PR must name a user-observable defect on the production site, a security or supply-chain exposure, a required check failing on main or fleet-wide, a data or dependency refresh through its existing lane, or maintainer-filed work.
  2. Everything else, in particular any change whose only effect is to move a metric, is out of remit: at most one comment on 🐝 Hive Advisory Report #23.
  3. The coverage, mutation and clone thresholds are frozen regression gates, listed at their current values; changing one is a maintainer-authored amendment.
  4. A decided question stays decided across rewordings (the [architect] Duplicated freshnessNote CSS rule across PeopleFreshness, GroupLinkStatus, MemberDirectory, with one copy already drifted #1669 / [architect] duplicated freshness-note CSS across GroupLinkStatus, DirectoryFreshness, PeopleFreshness #1681 / refactor: extract shared FreshnessNote component for data-freshness notes #1683 / [architect] Duplicated .freshnessNote CSS rule across PeopleFreshness and GroupLinkStatus #1814 loop).
  5. A PR for a folded finding uses Refs, never Fixes, on its host issue ([architect] ci.yml duplicates Playwright setup across e2e and e2e-coverage jobs #1829 / test: fail on a symlink under static/ instead of skipping it #1839).
  6. No planning checklists return to the tree.

AGENTS.md and GOVERNANCE.md carry rules 1, 2, 4 and 5 in the same words, since those are the files the agents quote back. ADR 0007 gets a status note for the narrowing of its decision 3. No code or threshold values change; the gates already block regressions and the ratchets only move when a change that was in remit anyway removes clones.

The ADR sets a 30-day checkpoint (2026-11-23). If agent-filed volume has not fallen materially, the remaining lever is the Hive configuration (schedule, autonomy level, folding), which lives outside this repository.

Verification

  • npm run test:unit:coverage:check green (ADR contract, markdown reference and governance parity tests included).
  • prettier, cspell and markdownlint-cli2 clean on the five changed files.

Governance change per GOVERNANCE.md, so this takes the independent-review path rather than the low-risk one.

Refs #1788, #1718.

The site is launched and stable, but the fleet opened 138 of the last 150
issues and 65 of the last 80 merged PRs touched only tests, scripts,
workflows or root markdown. ADR 0006 and ADR 0007 each closed one series
of findings and the agents moved to the next category.

ADR 0008 states one remit in bright lines: an agent-filed issue or PR must
name a user-observable defect, a security or supply-chain exposure, a
required check failing, a data or dependency refresh, or maintainer-filed
work. Metric-only changes are out of remit, the coverage, mutation and
clone thresholds are frozen regression gates, decided questions are not
re-filed, folded findings use Refs rather than Fixes, and planning
checklists do not return to the tree.

AGENTS.md and GOVERNANCE.md carry the same rules so the agents read them
where they already look, and ADR 0007 gets a status note for the
narrowing of its decision 3.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
@netlify

netlify Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for endusers ready!

Name Link
🔨 Latest commit c574dc6
🔍 Latest deploy log https://app.netlify.com/projects/endusers/deploys/6adba41e8bfd99d607d5788c
😎 Deploy Preview https://deploy-preview-1848--endusers.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@mrbobbytables

Copy link
Copy Markdown
Member Author

Why this has been happening

A longer explanation of the mechanics behind the numbers in the ADR, for anyone (human or agent) reading this later and wondering why a launched, content-stable site produces fifteen issues a day.

1. Hive has no terminal state

Hive is a scheduled scan-and-file loop. Each agent (quality, architect, sec-check, guide, scanner, strategist, ci-maintainer, dedupe) runs on a cadence, reads the repository and its own instructions, and files whatever its heuristics surface. Nothing in that loop asks "is the product done?" The governor's own issue #23 says so explicitly: "Do not close this issue. It is a living document." The schedule, the agent roster, the autonomy level (ACMM) and the folding behaviour are all set in Hive's project configuration outside this repository (AGENTS.md, "Hive Configuration"), so the repository cannot turn any of them off; it can only change what the agents read.

2. The dominant agent measures things with no floor

quality filed 62 of the last 138 agent issues. Its inputs are metrics: unit and browser coverage, the weekly full-tree Stryker report, the jscpd clone count, and the shape of test prose. Every one of those has an asymptote rather than a finish line. Coverage can always be a region higher; a mutation report always lists survivors (equivalent mutants are, by definition, unkillable); a clone count can always be lower. An agent that treats "metric is not at its limit" as a defect will therefore never run out of defects, and the work it files is self-similar: the open PRs this ADR consolidated (#1845, #1846) are annotations of equivalent mutants in scripts/lib/ and in the tests/tools/ harness that tests the tests, plus tests that pin the premises behind those annotations. #1843 goes one layer further and files mutants in the mutation runner's own error accounting.

3. Every fix became the next feed

The repository has responded to each series by adding a gate, and each gate became a new metric for the same agent to drive:

This is a ratchet by construction. The repository's own comments say so: CLONE_CEILING "only ever ratchets down", the Stryker break "must never sit above the score main actually has", and AGENTS.md tells agents to "lower a ceiling (never raise it) when the gate reports headroom". Each of those is sensible as a regression guard and perverse as an objective function for an autonomous filer.

4. Scoping by category does not converge

ADR 0006 narrowed test-only work (citation pins, listing reach, harness-of-harness tests). ADR 0007 narrowed browser-coverage findings. Both were read and honoured: the guide agent filed #1646 asking that GOVERNANCE.md be tightened per ADR 0006, and citation-pin findings stopped. But each ADR named the category it was closing, and the agents moved to the nearest category it did not name. ADR 0007 is the clearest example: it carried qualifiers ("unless there is surviving-mutant evidence", "integration-boundary evidence", "accepted regions"), and each qualifier was treated as a new area to explore (#1736, #1752, #1811). A remit written as policy-with-exceptions reads, to an agent, as a list of places where work is still permitted. That is why ADR 0008 is written as bright lines and names what is in remit rather than what is out.

5. Decisions do not persist across rewordings

Hive's duplicate detection matches on text and file citations, not on the rule that was decided. The maintainer decision on #1669 (keep per-component CSS local, do not compose .freshnessNote) was re-filed as #1681 and #1683 four hours later, and again as #1814 with PR #1815 four days later, each time with slightly different wording. Nothing in the loop reads a closed issue's resolution and generalises it to the rule. ADR 0008 rule 4 makes the maintainer's decision bind the rule, file or component rather than the sentence.

6. Folding corrupts the issue graph

When a new finding cites a file that an open agent issue already cites, Hive folds the new finding into that issue instead of opening another. The intent is fewer issues; the effect is issues whose title no longer describes their contents and PRs that close the wrong thing. #1829 is "ci.yml duplicates Playwright setup"; the static-asset symlink finding was folded into it because its evidence mentioned ci.yml, and PR #1839 was filed with Fixes #1829 even though its own body said it deliberately did not fix #1829. #1837 is a launch-announcement decision request with a flaky-reporter mutant finding folded into it. #1823 (an ip-address advisory) had the braces advisory folded in. Each of these costs a maintainer the work of untangling before anything can be merged. Rule 5 addresses the half of this the repository controls (what the PR says); the folding itself is Hive configuration.

7. Planning prose was a finding generator

strategist reads markdown. While ROADMAP.md, MILESTONES.md and LAUNCH.md existed, every unchecked bullet in them was "work with no tracking issue" and got one (#1775, #1797, #1830, #1837). Those files were retired in #1841; rule 6 records why they should not come back as a different file name.

8. What the numbers look like from the site's side

Of the 80 pull requests merged between 2026-10-17 and 2026-10-23, 65 changed only tests/, scripts/, .github/, adr/ or root markdown. Twelve touched src/ (mostly CSS composition and mutant annotations). One touched docs/. The end-user-facing site received effectively no change in a week in which the repository merged eleven PRs a day. Meanwhile the agents that did find real problems in that period (sec-check: MDX gate bypasses in #1732 and #1740, a token on argv in #1768; guide: broken community-group links in #1755; ci-maintainer: a fleet-wide failing check in #1809) account for about a quarter of the volume and would all pass rule 1 unchanged.

What this ADR can and cannot do

It can change what the agents read, which the ADR 0006 experience shows they do honour, and it can take the metrics off the table as objectives by naming them frozen. It cannot change the cadence, the autonomy level, or the folding behaviour. If agent-filed volume has not fallen materially by 2026-11-23, the next step is a request to the Hive maintainers with this ADR attached, asking for quality and architect to move to advisory mode, for folding to be disabled, and for the schedule to be reduced.

@hivecommons-hive

Copy link
Copy Markdown
Contributor
c574dc6

ℹ️ Sentinel notice — informational; author is trusted (owner), merge is not blocked

Hive flagged this PR (author @mrbobbytables, head c574dc693362) because it matches behaviors that can override security controls, escalate privileges or damage the codebase. This is a heuristic, not an accusation — a maintainer should confirm the change is intended before it merges.

  • sensitive_path — changes 1 sensitive path(s) (Touches a sensitive path (OWNERS, workflows, policies, hive config, security docs, …))
    • GOVERNANCE.md

Hive did not add the sentinel-alert label because this author is trusted. The finding remains recorded for audit and the dashboard Security tab; set sentinel.trusted_authors_block: true to require the blocking label for trusted authors too.

@mrbobbytables
mrbobbytables added this pull request to the merge queue Oct 23, 2026
Merged via the queue into cncf:main with commit 7897289 Oct 23, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant