measured-elsewhere: a goal can pass FOR-THIS-REPO while every fact in it was measured elsewhere - #644
Merged
Merged
Conversation
… it was measured elsewhere Answers #502 D — the one finding its reporter explicitly asked for a read on, and the most valuable thing in that report. ⛔ THE GAP IN ONE LINE. `scripts/check-goal-conformance.py` separates DECLARED scope from MENTIONED scope, and cannot separate DECLARED scope from MEASURED scope. `onboard.md` step 3 tells an installer to re-scope each goal's `**Repository:**` line. Doing exactly that makes the checker report FOR-THIS-REPO for all five role goals while their BODIES still describe the estate they were written in. In the reporter's real foreign install, certified-in-scope goals asserted "no test infrastructure at all … no test files of any kind" against a repo with 233 test files, and "no Kubernetes … no runtime to observe" against 19 k8s manifests and three production clusters. ★ AND THE FILES WERE ALREADY HONEST ABOUT IT. Those paragraphs carry `[measured: nForma-NEXT 2026-08-19]`. The provenance is written down. Nothing read it. ⛔⛔ USE VS MENTION DECIDED WHETHER THIS WAS EVEN NEEDED, and a grep says the opposite of the truth: `grep -rl 'measured:'` finds 12 files under tools/ and scripts/, which reads as "plenty of things consume this tag". `grep -rl '\[measured'` finds ZERO. Twelve mentions of an English word, zero parsers. ⛔ AND A LINE-BASED READER UNDERCOUNTS THE CORPUS BY SIX. My first pass used `grep -ohE '\[measured:[^]]*\]'` and got 101; the tool reads 107. The gap is exactly the six tags that SPAN A LINE BREAK — `[measured: Blazing-Back ⏎ 2026-08-19]` and five like it — which grep cannot see because it matches within a line. 107 − 6 = 101, exactly: the corpus did not change, the READER did. ★ And the six are not a random sample: a tag long enough to wrap is one carrying provenance DETAIL, the kind most worth reading. FIRST RUN: 107 tags — OWN 80 · ELSEWHERE 19 (all one sibling estate) · UNDATED 8. ⚠ ELSEWHERE IS A LOCATION, NOT A VERDICT, and the output says so on every run. A goal may legitimately cite a sibling estate's measurement; what this makes knowable is HOW MANY and WHERE. After a vendoring re-scope it becomes ~107 of 107 and the same checker still says FOR-THIS-REPO. ⚠ EVERY COUNT IS A LOWER BOUND — it reads the TAG, never the claim, and nothing enforces tagging, so an untagged paragraph measured elsewhere is invisible by construction. IMPORTED, NOT COPIED (#345): `declared_scope` and `this_repo` come from check-goal-conformance via importlib, because a hyphenated filename cannot be imported and that friction is exactly what produces a second, drifting reading of one noun. CALLER — fleet-preflight, NON-GATING, and placed where the question arises: preflight is the acceptance test for the install that just re-scoped those files. warn measured-elsewhere: some tagged claims were measured in ANOTHER repository ELSEWHERE 19 · OWN 80 · UNDATED 8 ⛔ TWO THINGS THE INDEX CHECKER CAUGHT IN THIS PR, both fair: · the file was untracked, so the row named something git could not see · my FIXTURES named a real sibling estate in EXECUTABLE POSITION — a string literal is how the quarantine leg detects contamination, and it was right to. The real instance belongs in the docstring, where it is a citation; in `cases` it claims belonging. Fixtures now use a synthetic name. SUITE — 11 checks, count derived. Known-positive a tag naming the declared repo; known-negative the #502 D shape; controls for owner/repo form, placeholder dates, UNDATED checked BEFORE the repo comparison, an unparseable Repository line, case-insensitivity, two tags on one line, an empty input, and — the load-bearing one — that the bare word "measured:" is NOT a tag. Breaking the repo comparison takes it to exit 3; `-O` to exit 2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Warning Review limit reachedNext included review available in 28 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Answers #502 D — the one finding its reporter explicitly asked for a read on, and the most valuable thing in that report. TEAMLEAD (session
15b69750), 2026-09-07.The gap, in one line
scripts/check-goal-conformance.pyseparates declared scope from mentioned scope, and cannot separate declared scope from measured scope.onboard.mdstep 3 tells an installer to re-scope each goal's**Repository:**line. Doing exactly that makes the checker reportFOR-THIS-REPOfor all five role goals while their bodies still describe the estate they were written in. In the reporter's real foreign install, certified-in-scope goals asserted:pytest, a coverage gate★ And the files were already honest about it. Those paragraphs carry
[measured: nForma-NEXT 2026-08-19]. The provenance is written down. Nothing read it.⛔ Use vs mention decided whether this was even needed — and grep says the opposite of the truth
Twelve mentions of an English word. Zero parsers.
⛔ And a line-based reader undercounts the corpus by six
My first pass used
grep -ohE '\[measured:[^]]*\]'and got 101. The tool reads 107. The gap is exactly the six tags that span a line break — grep matches within a line and cannot see them:107 − 6 = 101, exactly. The corpus did not change; the reader did. ★ And the six are not a random sample — a tag long enough to wrap is one carrying provenance detail, the kind most worth reading. A reimplementation withgrepwould silently reproduce the undercount.First run
⚠
ELSEWHEREis a location, not a verdict, and the output says so on every run. A goal may legitimately cite a sibling estate's measurement; what this makes knowable is how many, and where. ⇒ After a vendoring re-scope it becomes ~107 of 107 — and the same checker still saysFOR-THIS-REPO.⚠ Every count is a lower bound. It reads the tag, never the claim, and nothing enforces tagging — so an untagged paragraph measured elsewhere is invisible by construction.
⚠
UNDATEDis checked before the repo comparison and reported separately:[measured: nForma-NEXT <date>]names the right repo and establishes nothing about when, and one verdict would merge two different failures.Imported, not copied
declared_scopeandthis_repocome fromcheck-goal-conformanceviaimportlib— a hyphenated filename cannot beimported, and that friction is exactly what produces a second, drifting reading of one noun (#345). The two tools must not be able to disagree about what "this repo" is.Caller — non-gating, placed where the question arises
Preflight is the acceptance test for the install that just re-scoped those files:
⛔ Not gating: in the origin estate 19 of 107 tags legitimately cite a sibling, and scoring that as failure would be red on every clean run here.
⛔ Two things the index checker caught in this PR, both fair
casesit would be a claim of belonging. Fixtures now use a synthetic name.Suite — 11 checks, count derived
Known-positive a tag naming the declared repo; known-negative the #502 D shape. Controls for the
owner/repoform, placeholder dates,UNDATEDchecked before the repo comparison, an unparseableRepository:line, case-insensitivity, two tags on one line, empty input, and — the load-bearing one — that the bare wordmeasured:is not a tag. Breaking the repo comparison → exit 3;python3 -O→ exit 2.🤖 Generated with Claude Code