Skip to content

Commit b9eb46e

Browse files
committed
perf(tests): reduce Windows submodule process overhead
Windows process startup makes repeated Git version probes and missing-hook checks costly in the submodule tests. A relative `git` executable also prevents the existing minimum-version cache from being shared on Windows. Resolve `git.exe` from `PATH` to an absolute `GIT_PYTHON_GIT_EXECUTABLE` for Windows CI and the local runner, preserving explicit overrides. Use the selected executable for the runner's direct Git commands. Pass `skip_hooks=True` at 70 submodule fixture commit sites, preserving normal `IndexFile.commit()` defaults and dedicated hook coverage. Match CI's private configuration by appending the alias fixture directly and retaining `core.autocrlf=true` on Windows. The previous `include.path` forced Gix repository reloads during local measurements. Document the Git Bash invocation and distinguish these local setup corrections from CI gains. Validation also exposed a daemon-test deadline shorter than Windows' closed loopback-port refusal delay: fresh closed ports returned `TimeoutError` at two seconds and `ConnectionRefusedError` just after 2.02 seconds. Allow five seconds for the post-shutdown check while retaining its strict refusal assertion and restart coverage. A controlled, serial 37-case Windows Gix sample with identical flat config, profiling enabled and coverage disabled improved from 87.10s to 77.20s. Git launches fell from 1,437 to 1,128: 132 fewer version probes and 177 fewer hook checks. This is about 11% faster in the sample, not a full-suite estimate. Validation used CPython 3.12.13, Git 2.55.0.windows.5 and `GixPython` 0.1.0: - Both complete submodule modules and related checks ran with CI's coverage options. CLI finished with 542 passed, 1 skipped, 1 xfailed and 3 xpassed. - The initial Gix selection had 546 passed, 1 failed, 1 xfailed and 3 xpassed. Its sole failure was the daemon deadline corrected here; that regression passed on a targeted rerun and in the subsequent CLI run. - Replaying the original submodule source from `c3a2832b` reproduced the CLI `test_base_rw` file-lock skip, confirming it predates these changes. - Pre-commit checks passed for all changed files. Git Bash checks covered executable resolution, explicit overrides and the non-Windows branch; changed Python files also parse with Python 3.8 syntax.
1 parent 863bb76 commit b9eb46e

6 files changed

Lines changed: 143 additions & 74 deletions

File tree

‎.github/workflows/pythonpackage.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -223,6 +223,11 @@ jobs:
223223
224224
- name: Test with pytest
225225
run: |
226+
if [[ "$RUNNER_OS" == "Windows" && ! -v GIT_PYTHON_GIT_EXECUTABLE ]]; then
227+
# Absolute paths let Windows share minimum-version probes.
228+
git_executable="$(python -c 'import os, shutil; print(os.path.abspath(shutil.which("git")))')"
229+
export GIT_PYTHON_GIT_EXECUTABLE="$git_executable"
230+
fi
226231
pytest --color=yes -p no:sugar --instafail -vv --durations=30 \
227232
--junitxml=test-results/pytest.xml --backend-report=test-results/backend.json
228233
continue-on-error: false

‎doc/gix-backend.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,24 @@ The prepared environments in this checkout are `.venv` (CLI) and `.tox/gix`
5353
--backend-report=.cache/gix-coverage.json
5454
```
5555

56+
On Windows, use Git Bash, as CI does, and the environments' `Scripts`
57+
directories. To retain CI's coverage and pytest options:
58+
59+
```sh
60+
.venv/Scripts/python.exe test/run-local.py --color=yes -p no:sugar --instafail -vv --durations=30 \
61+
--junitxml=.cache/windows-cli.xml --backend-report=.cache/windows-cli.json
62+
.tox/gix/Scripts/python.exe test/run-local.py --color=yes -p no:sugar --instafail -vv --durations=30 \
63+
--junitxml=.cache/windows-gix.xml --backend-report=.cache/windows-gix.json
64+
```
65+
66+
The Windows runner resolves `git.exe` from `PATH` to an absolute path so
67+
GitPython can share minimum-version checks. An explicit
68+
`GIT_PYTHON_GIT_EXECUTABLE` override is preserved and also used for the runner's
69+
direct Git commands. Git Bash normally selects `mingw64/bin/git.exe`; the
70+
`cmd/git.exe` and `bin/git.exe` launchers add overhead to local measurements.
71+
The private config retains CI's `core.autocrlf=true` on Windows and appends
72+
the test aliases directly, matching CI without introducing config includes.
73+
5674
The runner uses local version tags, creates an isolated Git configuration,
5775
prepares the historical test fixture inside a temporary shared clone, gives
5876
pytest a separate temporary root for each run, and disables package-index
@@ -518,6 +536,35 @@ repository-open benchmark above measures the affected operation separately.
518536
Logs, profiles, JUnit results and backend reports are retained locally under
519537
`.cache/ci-performance/`.
520538

539+
### Submodule fixture process overhead on Windows
540+
541+
Windows CI and the local runner now resolve `git.exe` from `PATH` to an
542+
absolute `GIT_PYTHON_GIT_EXECUTABLE` when no override was supplied. This enables
543+
the existing minimum-version cache without changing the library's executable
544+
lookup or cache invalidation. Submodule fixtures also pass `skip_hooks=True`
545+
when committing their test history; the library default and dedicated hook
546+
tests continue to run hooks.
547+
548+
A serial Windows comparison on CPython 3.12.13, Git 2.55.0.windows.5 and
549+
GixPython 0.1.0 selected the Windows destination-name rejection tests,
550+
`test_update_no_fetch_restores_deinitialized_submodule`, and
551+
`test_update_no_fetch_is_recursive`. Both runs used the direct Git executable
552+
on `PATH`, CI's flat alias config and `core.autocrlf=true`, with profiling
553+
enabled and coverage disabled. All 37 cases passed in both runs.
554+
555+
| Test invocation and fixtures | Pytest time | Git launches | Version probes | Hook launches |
556+
| --- | ---: | ---: | ---: | ---: |
557+
| `GIT_PYTHON_GIT_EXECUTABLE=git`, fixture hooks enabled | 87.10s | 1,437 | 403 | 177 |
558+
| Absolute executable, fixture hooks skipped | 77.20s | 1,128 | 271 | 0 |
559+
560+
The 309 eliminated launches comprise 132 redundant version probes and three
561+
hook checks for each of 59 fixture commits. This sample was about 11% faster;
562+
it is not a full-suite speedup estimate. Git Bash CI already selects
563+
`mingw64/bin/git.exe` and appends the alias config directly, so correcting the
564+
local runner's launcher/config differences is not an additional CI gain.
565+
Logs, profiles, JUnit results and operation reports are retained locally under
566+
`.cache/test-cleanup/windows-submodule-{baseline,optimized}*`.
567+
521568
### Test-suite setup measurements
522569

523570
The fixture optimizations below are implemented as separate commits, each

‎test/run-local.py‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66

77
import os
88
from pathlib import Path
9+
import shutil
910
import socket
1011
import subprocess
1112
import sys
@@ -18,7 +19,14 @@ def main():
1819
with TemporaryDirectory(prefix="gitpython-local-tests-") as directory:
1920
temporary = Path(directory)
2021
config = temporary / "gitconfig"
21-
config.write_text("[user]\nname = GitPython Tests\nemail = tests@example.invalid\n", encoding="utf-8")
22+
text = "[user]\nname = GitPython Tests\nemail = tests@example.invalid\n"
23+
if sys.platform == "win32":
24+
# Match CI's Git for Windows setting despite disabling system config.
25+
text += "[core]\nautocrlf = true\n"
26+
# CI appends these aliases directly. An include forces Gix to reload its
27+
# configuration before every operation because included files can change.
28+
text += (root / "test/fixtures/.gitconfig").read_text(encoding="utf-8")
29+
config.write_text(text, encoding="utf-8")
2230
env = {
2331
**os.environ,
2432
"GIT_CONFIG_NOSYSTEM": "1",
@@ -28,11 +36,17 @@ def main():
2836
"PIP_FIND_LINKS": os.environ.get("PIP_FIND_LINKS", str(root / ".cache/gix-wheels")),
2937
"UV_OFFLINE": "1",
3038
}
39+
if sys.platform == "win32" and "GIT_PYTHON_GIT_EXECUTABLE" not in env:
40+
executable = shutil.which("git")
41+
if executable is None:
42+
raise RuntimeError("Git executable not found in PATH")
43+
# Windows can share minimum-version checks for an absolute path.
44+
env["GIT_PYTHON_GIT_EXECUTABLE"] = os.path.abspath(executable)
45+
git_executable = env.get("GIT_PYTHON_GIT_EXECUTABLE", "git")
3146

3247
def git(*args, cwd=root):
33-
return subprocess.check_output(["git", *map(str, args)], cwd=cwd, env=env, text=True).strip()
48+
return subprocess.check_output([git_executable, *map(str, args)], cwd=cwd, env=env, text=True).strip()
3449

35-
git("config", "--file", config, "include.path", root / "test/fixtures/.gitconfig")
3650
fixture = temporary / "repo"
3751
git("clone", "--shared", "--no-checkout", root, fixture)
3852
git("checkout", "--detach", git("rev-parse", "HEAD"), cwd=fixture)

‎test/test_cleanup.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -198,8 +198,9 @@ def test_git_daemon_releases_port_and_can_restart(tmp_path):
198198
with suppress(ConnectionResetError): # Git for Windows resets rejected requests.
199199
assert connection.recv(1) == b""
200200

201+
# Windows can take just over two seconds to refuse a closed loopback port.
201202
with pytest.raises(ConnectionRefusedError):
202-
with socket.create_connection(address, timeout=2):
203+
with socket.create_connection(address, timeout=5):
203204
pass
204205

205206

0 commit comments

Comments
 (0)