Skip to content

fix: run the real ownership checks when validate/gv are given paths - #125

Draft
perryqh wants to merge 3 commits into
mainfrom
fix/validate-paths-parity-gap
Draft

fix: run the real ownership checks when validate/gv are given paths#125
perryqh wants to merge 3 commits into
mainfrom
fix/validate-paths-parity-gap

Conversation

@perryqh

@perryqh perryqh commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Summary

validate <paths> and gv <paths> were not equivalent to the same commands with no paths, and one of the gaps was a silent false pass: gv <dual-owned file> exited 0 with empty output.

Three commits:

  1. test: — five failing tests documenting the gap.
  2. fix: — ownership for supplied paths is now resolved through the mappers, the same way the whole-project run does. Four of the five tests now pass and are un-ignored.
  3. fix: — review follow-ups: deduplicate a path supplied twice (it was reported twice), correct the complexity claim below, make the unused FileGenerator structurally impossible in the scoped path rather than merely unused, give the scoped span a distinct name, and rewrite the --help text for files.

The fix is cheap because two of the three checks were already per-file: validate_file_ownership iterates file_to_owners(), a par_iter over project.files, and validate_invalid_team's file half does the same. Scoping them to the supplied paths is a filter, not a rewrite. The old code was doing more I/O (re-reading and re-parsing CODEOWNERS per path) to learn less.

To be precise about the cost, since an earlier revision of this description overstated it: this scopes the per-file work, not all of it. Building the owner matchers stays O(repo) — TeamFileMapper::owner_matchers enumerates every annotated file. So the scoped path costs a fixed O(repo) term plus O(supplied paths × matchers), against O(repo × matchers) for the whole-project run. The saving is in the variable term, which dominates on a large repo with a small changeset, but it is not free and it has not been measuredgv with no paths still isn't a perf-harness case.

Verified scoping is real, not accidental whole-repo validation — gv ruby/app/services/multi_owned.rb on the invalid_project fixture reports only that file's dual ownership, and stays silent about calculator.rb, blockchain.rb, and unowned.rb. A cleanly-owned file still exits 0.

One behavior change

Unowned files supplied by path now report as "Some files are missing ownership" — the wording the whole-project run uses — rather than "Unowned files detected:". Same defect, same words, whether or not paths are passed, which is the point. Three test assertions were updated. The wrapping code_ownership gem already parses this format, since validate with no paths has always produced it.

Absolute paths now render project-relative rather than as the caller wrote them.

Still open

Staleness is not checked for a supplied path list and cannot be — it compares the whole generated file against the whole on-disk one. gv <paths> makes it moot by regenerating first, but a team file or .codeowner change can still alter ownership of files outside the changeset without being caught. That gap wants an escalation path (run the full check when team files are in the changeset), which this PR does not add.

The fifth test stays #[ignore]d: non-canonical absolute paths are dropped by the owned_globs filter before any check runs. Separate pre-existing bug, detailed below. Run it with:

cargo test --test validate_files_parity_test -- --ignored

Why

Passing file paths swaps validate_all() for validate_files() (runner.rs:124):

pub fn validate(&self, file_paths: Vec<String>) -> RunResult {
    if file_paths.is_empty() { self.validate_all() } else { self.validate_files(file_paths) }
}

validate_allValidator::validate runs three checks (validator.rs:40-57):

  1. validate_invalid_team — annotation or package naming a nonexistent team
  2. validate_file_ownership — file owned two ways
  3. validate_codeowners_file — CODEOWNERS is stale

validate_files runs none of them. It only asks whether each path resolves to a team when reading the CODEOWNERS file.

gv <paths> regenerates before validating, so check 3 is moot by construction. Checks 1 and 2 are still skipped — generate() only calls generate_file() and writes the string (ownership.rs:167); it validates nothing.

The false pass

gv ruby/app/services/multi_owned.rb, a file owned by both a @team Payments annotation and ruby/app/services/.codeowner naming Payroll:

code=0
stdout=""

gv with no paths reports "Code ownership should only be defined for each file in one way."

The mechanism is the concerning part: regenerating writes the file into CODEOWNERS under one of its two owners, so the per-path check finds an owner and passes. Regenerating conceals the defect rather than exposing it. Reproduces independently through owned_gems on gems/payroll_calculator/calculator.rb, so it is not one mapper misbehaving.

The differential

gv with every owned path vs. gv with none, same repo:

no paths every path
dual ownership (2 files) reported missed
invalid team Web3 reported missed
unowned files reported reported

This is the general form of the targeted tests — it needs no knowledge of which defects the fixture holds, so it keeps working as fixtures change. It is also the check requested back in December when #89 landed, which was never added.

Wrong diagnosis, not just a miss

gv ruby/app/models/blockchain.rb (annotated @team Web3, not a team) exits 1 — but says Unowned files detected: ruby/app/models/blockchain.rb. An invalid team yields no owner, so the file is simply absent from the generated CODEOWNERS and reads as unowned. The developer goes looking for missing ownership instead of fixing a typo.

Separately: absolute paths are silently skipped

validate <non-canonical absolute path> exits 0 having never checked the file:

codeowners --project-root /var/folders/.../tmp --no-cache validate \
  /var/folders/.../tmp/ruby/app/unowned.rb
code=0
stdout=""

cli.rs canonicalizes --project-root, so a caller-supplied /var/... path fails strip_prefix against the canonical /private/var/... root, stays absolute, and is then rejected by the owned_globs filter — dropped before any ownership query runs. Easy to hit on macOS (TMPDIR) and with symlinked checkouts.

Unrelated to the parity gap and older: it dates to the owned_globs filter added by #89 for #88. The symptom is config-dependent — with a **-leading owned_globs the same path survives the filter and is reported spuriously unowned instead. Both wrong, same cause.

What this deliberately does not decide

This takes the option that needed no product decision: make the scoped check correct. It is a strict improvement under any subsequent choice about whether the fast path should exist at all. Still open, and better argued with numbers than in the abstract:

  • Drop the files param. gv <paths> already pays for the full project build, so running the real validate_all() costs only the validator passes — not a second build. If that is within noise, the param buys nothing.
  • Add escalation (--fail-fast-and-retry): run the full check when team files, .codeowner, or CODEOWNERS are in the changeset; otherwise the scoped check. Closes the staleness blind spot above.

Unmeasured, because gv with no paths is not currently a case in the perf harness — worth adding before deciding.

The --help text for files is updated here: it used to promise "fast mode for git hooks" with no hint it skips the staleness check, and now says so outright and points at generate-and-validate. That is deliberately the only place the caveat surfaces — a note on stdout was rejected because successful runs are silent by contract and pre-commit hooks depend on that.

One follow-up left alone: #116 overlaps the invalid-team reporting this now runs on supplied paths. If #116 lands first there may be a conflict in validate_invalid_team; if this lands first, #116's fix applies to the scoped path for free.

Two things reviewed and knowingly not addressed:

  • invalid_package_ownership runs unscoped. Validating one file can therefore fail over a package that file has nothing to do with. Documented in the doc comment; defensible, since packages are orders of magnitude fewer than files and skipping them would be a second blind spot — but it is a behavior change and it has no test.
  • No unit test on Validator::validate_files. Coverage is integration-only through the binary, so the scoping predicate is not exercised in isolation.

Consequence for #124

#124 optimized reading the CODEOWNERS file back per path. This deletes that code path, so #124 is obsolete rather than parked — it can be closed.

🤖 Generated with Claude Code

Passing file paths swaps validate_all() for validate_files() (runner.rs:124).
validate_all runs three checks — validate_invalid_team,
validate_file_ownership, validate_codeowners_file (validator.rs:40-57).
validate_files runs none of them; it only asks whether each path resolves to
a team when reading the CODEOWNERS file.

gv <paths> regenerates before validating, which cures staleness by
construction, but not the other two. Worse, regenerating writes a dual-owned
file into CODEOWNERS under one of its owners, so the per-path check then sees
an owner and passes. Regenerating conceals that defect rather than exposing
it.

Five tests, all failing, all #[ignore]d so the suite stays green:

- gv <dual-owned file> exits 0 with no output, twice over — once for
  annotation vs .codeowner, once for annotation vs owned_gems. They travel
  through different mappers, so a fix could catch one and miss the other.
- gv <invalid-team file> fails, but reports "unowned" instead of naming the
  nonexistent team, sending the developer after the wrong problem.
- gv with every owned path disagrees with gv with no paths about which
  defects exist. This is the general form, and needs no knowledge of what
  the fixture contains.
- validate <non-canonical absolute path> exits 0 having never checked the
  file. cli.rs canonicalizes --project-root, so a /var/... path fails
  strip_prefix against a /private/var/... root, stays absolute, and is then
  dropped by the owned_globs filter. Silent, and it fails in the unsafe
  direction.

The last one is unrelated to the parity gap and predates it — it dates to
the owned_globs filter added by #89 for #88.

No production code changes. Run with:
cargo test --test validate_files_parity_test -- --ignored

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
validate_files answered only "does this path have an owner in the CODEOWNERS
file". That could not see the two defects validate_all catches per file:

- A file owned two ways. Generation picks one winner and writes it, so reading
  CODEOWNERS back finds an owner and passes. `gv <dual-owned file>` exited 0
  with empty output — regenerating first concealed the defect instead of
  exposing it.
- An annotation naming a nonexistent team. That yields no owner, so the file
  was absent from the generated CODEOWNERS and reported as merely "unowned",
  sending the developer after missing ownership rather than a typo'd team.

Now ownership for supplied paths is resolved through the mappers, the same way
the whole-project run does. Validator gains a scoped entry point that runs
validate_invalid_team and validate_file_ownership over just the named files, so
a caller pays O(changed files) rather than O(repo). Both checks were already
per-file — validate_file_ownership iterates file_to_owners(), which is a
par_iter over project.files — so scoping them is a filter, not a rewrite. The
mappers are built either way, by the project build both paths already pay.

Package ownership is checked in full regardless of the path list. Packages are
orders of magnitude fewer than files, and skipping them would leave a second
blind spot.

Staleness is still not checked for a supplied path list, and cannot be: it
compares the whole generated file against the whole on-disk one. `gv <paths>`
makes it moot by regenerating first. A team file or .codeowner change can
therefore still alter ownership of files outside the changeset without being
caught — that gap wants an escalation path, which this commit does not add.

One behavior change worth noting: unowned files supplied by path now report as
"Some files are missing ownership", the same wording the whole-project run uses,
rather than "Unowned files detected:". Same defect, same words, whether or not
paths are passed — which is the point. Three test assertions updated for the
new wording, and test_validate_only_checks_codeowners_file is renamed, since it
documented the very behavior this removes.

Absolute paths now render project-relative rather than as the caller wrote them,
because the validator reports relative paths.

Four of the five parity tests from the previous commit now pass and are
un-ignored. The fifth stays ignored: non-canonical absolute paths are still
dropped by the owned_globs filter before any check runs, which is a separate
pre-existing bug.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@perryqh perryqh changed the title test: expose the validate/gv <paths> parity gap fix: run the real ownership checks when validate/gv are given paths Aug 21, 2026
@perryqh
perryqh marked this pull request as draft August 21, 2026 15:44
Four things, all found by reviewing the previous commit rather than by a test
failing.

Deduplicate absent paths. The missing-path branch iterated the raw argument
list, so `validate ghost.rb ghost.rb` reported the file twice. It now iterates
the deduped set. Paths the project does know about were already deduped for
free, by going through project.files, so only this branch was inconsistent.

Correct the complexity claim. The previous commit said the scoped path costs
O(changed files) rather than O(repo). That is wrong: file_to_owners builds the
owner matchers from every mapper, and TeamFileMapper::owner_matchers enumerates
every annotated file in the project. The real cost is a fixed O(repo) term plus
O(supplied paths x matchers), against O(repo x matchers) for the whole-project
run. The saving is in the variable term, which dominates on a large repo with a
small changeset -- but it is not free, and it is still unmeasured.

Make the unused FileGenerator impossible rather than merely unused. The scoped
constructor built one and never used it, since nothing there generates. The
generator is now a parameter of Validator::validate instead of a field, so the
scoped path cannot be handed one. Also documents why the whole-project path
builds mappers twice: FileGenerator owns them and Box<dyn Mapper> is not Clone.

Rename the scoped span to validator_validate_scoped, paralleling
validator_validate, and drop the "(scoped)" suffixes from the debug! lines. The
per-check spans are shared between both paths, so a profile distinguishes them
by parent span; #121 added those span names precisely to stop distinct work
collapsing into one bucket, and that reasoning applies here too.

Help text for `validate <files>` now states that staleness is not checked and
points at running without files or using generate-and-validate. It previously
advertised "fast mode for git hooks" with no hint it checks less. This is where
the mental model forms; a note on stdout was rejected because successful runs
are silent by contract and pre-commit hooks depend on that.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

1 participant