Skip to content

test: keep tests away from real GitHub, HOME, and network remotes - #78

Merged
micahjsmith merged 1 commit into
mainfrom
micahjsmith/stack/11
Oct 13, 2026
Merged

micahjsmith merged 1 commit into
mainfrom
micahjsmith/stack/11

Conversation

@micahjsmith

@micahjsmith micahjsmith commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Stacked PRs:


Summary

A test reached real GitHub with the developer's credentials and landed PRs: test_run_autoland_requires_merge_queue called the real commands.run_autoland in the developer's checkout, relying on the merge-queue gate to stop it. When a later change removed the gate, the test ran a real autoland. Nothing in the suite stopped a test from using the developer's HOME, gh auth, or network remotes.

This adds an autouse fixture, isolated_env in tests/conftest.py, that for every test:

  • points HOME and XDG_CONFIG_HOME at a fresh temp directory, and sets GIT_CONFIG_NOSYSTEM=1, so ~/.stack-pr state and git config are never the developer's. Local runs now match CI, which has no global git identity.
  • empties GH_TOKEN / GITHUB_TOKEN / GH_ENTERPRISE_TOKEN / GITHUB_ENTERPRISE_TOKEN and points GH_CONFIG_DIR at an empty directory, so gh has no credentials.
  • sets GIT_ALLOW_PROTOCOL=file, so git can reach only local remotes (the tests' bare repos), never https or ssh.
  • puts a gh shim first on PATH. The shim logs its arguments and exits 1 with "tests must not run the real gh". A test that ran it errors at teardown, listing the calls, even if the test caught the failure.
  • starts the test in an empty temp directory instead of the checkout, so code that acts on "the current repo" finds no repo unless the test made one and chdir-ed into it.
  • unsets GIT_DIR, GIT_WORK_TREE and similar variables that would point git somewhere else.

Opt-out: a test that means to run gh and see it fail marks itself @pytest.mark.allow_gh_shim. It still gets the shim, never the real gh, and can read the shim's log through isolated_env.gh_calls(). No test needs this today. Tests that remove gh from PATH (e.g. check_gh_installed) keep working.

Other test changes:

  • test_run_autoland_requires_merge_queue now replaces the fresh/resume/replan flows with ones that fail the test, so it can't run a real land if the gate goes away.
  • The branch-name tests in tests/test_misc.py read the current branch, and passed only because pytest ran inside the checkout. They now run in a throwaway repo.

Test plan

  • New tests/test_isolation.py checks the guard itself:
    • HOME is the temp home, not the real one (looked up with pwd), and ~/.stack-pr resolves under it.
    • gh tokens are empty and GH_CONFIG_DIR is empty.
    • The cwd is the temp directory, not the checkout.
    • git ls-remote to https and ssh GitHub URLs fails with "transport ... not allowed".
    • An inner pytest session (pytester): a test that runs gh and catches the failure errors at teardown with the shim's message and its argv; a test marked allow_gh_shim passes and sees its call in the log; a test that doesn't touch gh passes.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com

## Summary

A test reached real GitHub with the developer's credentials and landed PRs: `test_run_autoland_requires_merge_queue` called the real `commands.run_autoland` in the developer's checkout, relying on the merge-queue gate to stop it. When a later change removed the gate, the test ran a real autoland. Nothing in the suite stopped a test from using the developer's HOME, gh auth, or network remotes.

This adds an autouse fixture, `isolated_env` in `tests/conftest.py`, that for every test:
- points `HOME` and `XDG_CONFIG_HOME` at a fresh temp directory, and sets `GIT_CONFIG_NOSYSTEM=1`, so `~/.stack-pr` state and git config are never the developer's. Local runs now match CI, which has no global git identity.
- empties `GH_TOKEN` / `GITHUB_TOKEN` / `GH_ENTERPRISE_TOKEN` / `GITHUB_ENTERPRISE_TOKEN` and points `GH_CONFIG_DIR` at an empty directory, so gh has no credentials.
- sets `GIT_ALLOW_PROTOCOL=file`, so git can reach only local remotes (the tests' bare repos), never https or ssh.
- puts a `gh` shim first on `PATH`. The shim logs its arguments and exits 1 with "tests must not run the real gh". A test that ran it errors at teardown, listing the calls, even if the test caught the failure.
- starts the test in an empty temp directory instead of the checkout, so code that acts on "the current repo" finds no repo unless the test made one and `chdir`-ed into it.
- unsets `GIT_DIR`, `GIT_WORK_TREE` and similar variables that would point git somewhere else.

**Opt-out:** a test that means to run `gh` and see it fail marks itself `@pytest.mark.allow_gh_shim`. It still gets the shim, never the real gh, and can read the shim's log through `isolated_env.gh_calls()`. No test needs this today. Tests that remove gh from `PATH` (e.g. `check_gh_installed`) keep working.

Other test changes:
- `test_run_autoland_requires_merge_queue` now replaces the fresh/resume/replan flows with ones that fail the test, so it can't run a real land if the gate goes away.
- The branch-name tests in `tests/test_misc.py` read the current branch, and passed only because pytest ran inside the checkout. They now run in a throwaway repo.

## Test plan

- [x] New `tests/test_isolation.py` checks the guard itself:
  - `HOME` is the temp home, not the real one (looked up with `pwd`), and `~/.stack-pr` resolves under it.
  - gh tokens are empty and `GH_CONFIG_DIR` is empty.
  - The cwd is the temp directory, not the checkout.
  - `git ls-remote` to https and ssh GitHub URLs fails with "transport ... not allowed".
  - An inner pytest session (`pytester`): a test that runs `gh` and catches the failure errors at teardown with the shim's message and its argv; a test marked `allow_gh_shim` passes and sees its call in the log; a test that doesn't touch gh passes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
stack-info: PR: #78, branch: micahjsmith/stack/11
@micahjsmith
micahjsmith added this pull request to stack #80 October 13, 2026 00:52
@micahjsmith
micahjsmith merged commit 9b395cc into main Oct 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant