Refuse to run the suite against a different checkout's engine

[willhea]

Related issue

Closes #435

What does this change?

The test suite could report a full green about source nobody was editing. It happened in
the situation where a green matters most: reviewing a pull request in a git worktree.

pyproject.toml sets pythonpath = [".", "tools"] and excludes src on purpose (#398),
so the engine is importable only through the installed package, which records one
absolute path — the checkout where uv sync last ran. Point another checkout's
interpreter at a worktree and pythonpath = ["."] collects that worktree's tests/ while
deltatrack still resolves somewhere else entirely. Red-green verification then proves
nothing, because reverting the file under review changes nothing the run can see.

Measured before the guard existed. A top-level raise RuntimeError appended to a
worktree's src/deltatrack/bill_tree.py, then run with another checkout's interpreter:

$ /path/to/other-checkout/.venv/bin/python -m pytest tests/test_bill_tree.py -q
133 passed, 1 skipped

The injected fault was unreachable code as far as that run was concerned.

AGENTS.md already asserted that this "fails loudly on the first import rather than
silently reading the main checkout's source." That held only for a worktree with no
environment. Supplying an interpreter was the loophole, and nothing enforced the claim.
This makes the existing claim true and records the loophole beside it.

Why not just add src to pythonpath

That one-line change would make the symptom disappear, and it is the obvious fix. It is
not the right one, but not for the reason that first suggests itself: under an editable
install the suite already reports on the working tree rather than the distribution, which
is exactly why tests/test_engine_installs.py exists and says so in its own docstring.
Wheel fidelity is not what the src exclusion buys.

What it buys is one import-resolution story. Putting src on pythonpath would make
pytest the only consumer that resolves the engine differently from everything else, so an
env-less worktree would quietly pass against its own code while ./diff_bill.py sitting
beside it imported another checkout or failed outright. A split brain is worse than a hard
stop, and the stop names the actual problem — a tree running against a foreign environment
— instead of papering over the one symptom that happened to surface.

Changes

• tests/conftest.py — refuses to run when deltatrack resolves outside this
  checkout's root, naming both paths. Anchored on the checkout root rather than pytest's
  rootdir, which is what keeps test_corpus_manifest.py's child sessions (rootdir =
  tmp_path) from tripping it.

  Second branch, same root cause: when the editable install points at a deleted path, runs
  previously died at ModuleNotFoundError: No module named 'deltatrack', which reads as
  "this branch broke the package." It now names the stale pointer and the repair — but
  only for the engine's own top-level name. A failure from inside the engine (a typo'd
  deltatrack.something) is a fault in the branch and is re-raised untouched.

• tests/test_fixture_layout.py — two tests, because the guard is silent when it
  passes and there are two different ways for it to go quiet.

• AGENTS.md — the worktree convention, the loophole, and the rule that an editable
  install must never be run from a worktree against a shared .venv.

The check lives at conftest import rather than in a test on purpose: a test asserting this
would itself be collected by the wrong-tree run, and could only demonstrate that the wrong
tree agrees with itself.

How to test

The suite passed while the defect was active, so "tests pass" cannot demonstrate this fix.
Each claim was verified against the known-bad setup.

The guard fires where the suite previously went green:

RuntimeError: the tests are running in <worktree> but `deltatrack` imported from
<other-checkout>/src/deltatrack/__init__.py, so this run would report on a DIFFERENT
checkout's source. ...

Both correct setups still work, and now actually see the worktree's code. With the same
injected fault in place, a worktree using its own uv sync environment and a run using
PYTHONPATH=$PWD/src both fail on the fault itself, at
src/deltatrack/bill_tree.py: RuntimeError: FAULT INJECTED IN WORKTREE. The escape hatch
satisfies the guard by making the import correct, not by bypassing the check.

The branch's own broken imports are not misreported. Injecting
from deltatrack.missing_helper import Oops into the engine now surfaces
src/deltatrack/bill_tree.py:1: ModuleNotFoundError: No module named 'deltatrack.missing_helper'
rather than an environment message. This matters because pytest prints conftest import
errors without the raise ... from chain, so rewriting the exception would have hidden the
real module name completely.

Both tests were shown to fail. engine_is_foreign was neutered to return False,
which reddens the unit test. Separately the guard's three calling lines were deleted: the
unit test stayed green while the child-session test failed, which is the fail-open that
test exists to close.

All five CI gates on this branch:

ruff check .                                  All checks passed!
ruff format --check .                         178 files already formatted
pytest -m "not slow and not browser"          1415 passed, 9 skipped, 15 xfailed
pytest -m browser                             6 passed
pytest -m slow --deselect tests/test_govinfo_corpus_parity.py
                                              554 passed, 7 skipped

1415 against a 1413 baseline: the two new tests. Skips are pre-existing.

Breaking changes

None for any current workflow. The guard is stricter by design: a run whose engine does not
come from this tree's own src/ now stops instead of reporting. Two setups it would newly
reject, neither of which anything in this repo does today:

• A non-editable install into a virtualenv outside the checkout (a Docker or tox-style
  layout). Called out rather than silently accepted — if such a workflow is adopted, the
  guard needs an explicit exemption at that point.
• A non-editable install into the checkout's own .venv/. That is a copied snapshot
  which cannot see an edit to src/, so it is the same wrong-tree green from closer range.
  uv sync, uv run and source ./init all produce an editable install, so no documented
  path reaches this state.

An editable install into an external environment is unaffected, since its pointer still
resolves into the checkout's src/.

Known limitations

deltatrack.__file__ being None, which exotic namespace-package resolution can produce,
would raise a bare TypeError rather than a clear message. Left alone rather than adding
hardening for a case that could not be reproduced.

Checklist

• Linked the issue above (Closes #...)
• Ran the CI gates locally and they pass (see What CI checks)
• New or changed behavior has tests
• For a bug fix: the test fails without the fix, and I ran it both ways to check
• Disclosed AI assistance below, if any

AI assistance

Written with Claude Code, and reviewed by a second model before this pull request was
opened. That review caught two defects in the first version, both fixed here and both
reproduced independently before being acted on: the error handler misreported a branch's
own broken import as an environment fault, and the guard's wiring was unpinned so deleting
three lines would have restored the silent green with every test still passing. It also
correctly refuted the original justification for rejecting the pythonpath fix, which is
why the reasoning above rests on import uniformity rather than on wheel fidelity.

---

Follow-up commit: review findings resolved

A review of the first commit raised four items, all resolved in bec302b. Each was
reproduced before being acted on, and each fix was shown to redden exactly one test.

The guard was blind to the worktree layout this repo actually uses. engine_is_foreign
compared against the checkout root, but a worktree lives inside the checkout that owns
it (.claude/worktrees/<name>), so a root comparison reads a sibling working tree as "this
checkout". Measured: with an engine copy at <root>/.claude/worktrees/other/src, the run
imported it and the guard said nothing. It is reachable through the footgun the AGENTS.md
bullet above it names — uv pip resolves an activated VIRTUAL_ENV ahead of the checkout
you are standing in, and otherwise walks up parents, so an editable install run from a
nested worktree re-points the shared environment at it. Now anchored on root/src, which
also rejects a non-editable install under <root>/.venv/.

The repair advice led to that same state. The ModuleNotFoundError message recommended
uv pip install -e . --no-deps — the one command that is dangerous from the other realistic
seat, and the repo's only recommendation of it (init, AGENTS.md and the RuntimeError
twenty lines below all say uv sync). It now names uv sync, which targets the cwd
project's own .venv from either seat. The "why not the other one" reasoning moved into the
code comment: someone stuck at that error needs one action, not the argument.

That handler had no test in either direction. Two child-session tests now pin them:
an absent engine is reported as an environment fault with a safe repair, and a broken import
inside the engine reaches the developer with its module name intact. Neither can fail open
— the handler runs only when the import has already failed, so the session is red either way
— so what they protect is the diagnostic, not the gate. That is stated in both docstrings so
the next reader does not mistake them for fail-open gates.

The comment justifying import-time placement did not hold. It argued a guard test "would
itself be collected by the wrong-tree run"; it would not, since the tests come from the tree
under test, so such a test would be collected correctly and fail correctly. The load-bearing
reason is selection: pytest tests/test_bill_tree.py never collects a guard test at all, and
that single-module run is the scenario this issue measured. The placement was right, only the
stated reason was wrong — which matters, because someone who correctly rebuts a wrong
rationale might move the guard into a test file and reopen the hole.

Verification

Both directions proven end to end, in a worktree nested at .claude/worktrees/:

this tree running a NESTED worktree's engine     rc=4, guard fires (was silent)
main checkout's interpreter on this tree         rc=4, guard fires (unchanged)
PYTHONPATH=$PWD/src escape hatch                 rc=0, 133 passed

Each new assertion was shown to fail against the fault it claims to catch, scoped so that
exactly one test reddens and the rest stay green:

injected fault	reddens
drop the /src anchor	test_the_foreign_engine_rule_can_fire
restore uv pip install -e . as the repair	test_conftest_names_a_broken_environment
rewrite every ModuleNotFoundError as an environment fault	..._does_not_blame_the_environment_for_a_branch_fault
delete the guard's call site	the wiring test only; the other three stay green

Gates on bec302b, exit codes captured directly rather than read off the summary line:
ruff check 0, ruff format --check 0, fast suite 0 (1411 passed, +2 for the new tests),
external validation 0 (33), corpus gates 0 (360), remaining slow suites 0 (155), packaging
gate 0 (6). Run from a worktree, which does not carry the fetched bills/ corpus, so the
fast-suite count sits 6 below a main-checkout run; every one of those is a corpus-gated skip
naming a missing fetched file.