Skip to content

copilot: deflake the Windows shell step and GPT file reads in integration journeys - #1229

Closed
xi-db wants to merge 1 commit into
mainfrom
xi-db/stack/copilot-deflake-journeys
Closed

xi-db wants to merge 1 commit into
mainfrom
xi-db/stack/copilot-deflake-journeys

Conversation

@xi-db

@xi-db xi-db commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

🥞 Stacked PR

Use this link to review incremental changes.


What did you change, and why?

Two Copilot integration journeys fail intermittently because of their environment, not ug:

  • Windows: Copilot has no shell. The revoke and sign-out journeys have Copilot run a shell step mid-session. Copilot's shell tool tries pwsh.exe first, but the runner's restricted test PATH only has Windows PowerShell, whose startup probe sometimes exceeds Copilot's 40s limit (PowerShell is not available ... Command timed out after 40000ms). The step then never runs and the journey fails with "Copilot never ran the revoke step". scripts/run_integration.py now adds the directory of pwsh to the PATH for Copilot lanes on Windows, when the runner has it.
  • GPT drops a path segment. The GPT journeys run in a project directory named project with spaces. gpt-5-4-nano sometimes rebuilds the file's absolute path without the part after the space (.../project/input-....txt) and gives up. FileTask.prompt_with_path gives the full path, and the Copilot GPT journeys (headless GPT, configure/repeat/revert, upgrade) use it.

No product code changes.

How is this tested?

  • tests/test_integration_runner.py, tests/test_integration_evidence.py and tests/test_integration_contract.py pass, and lint is clean.
  • The Copilot integration lanes on this PR exercise both fixes on Linux and Windows.

This pull request and its description were written by Isaac.

…tion journeys

Copilot's shell tool tries pwsh first; with only Windows PowerShell on the test
PATH its 40s startup probe can time out, leaving Copilot without a shell, so the
revoke/sign-out step never runs. Expose pwsh to Copilot lanes on Windows.

gpt-5-4-nano sometimes drops the path segment after the space in
'project with spaces' when it rebuilds the file's absolute path. Give the GPT
journeys the full path.

Co-authored-by: Isaac <no-reply@databricks.com>
@xi-db xi-db added the ug-review Run the automated UG review label Oct 21, 2026
# startup probe on a cold runner and leave Copilot without a shell.
pwsh = shutil.which("pwsh", path=base_env.get("PATH"))
if "copilot" in agents and pwsh:
runtime_paths.append(Path(pwsh).parent)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test · low
what happens if pwsh lives in a shared dir like %LOCALAPPDATA%\Microsoft\WindowsApps (Store install)? that puts every alias there on PATH, which breaks the isolation the tool_bin comment above relies on. can we link pwsh into tool_bin instead?

@xi-db xi-db closed this Oct 21, 2026
@xi-db
xi-db deleted the xi-db/stack/copilot-deflake-journeys branch October 21, 2026 21:39
@xi-db

xi-db commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Closing: folding these fixes into #1218 instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ug-review Run the automated UG review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant