Skip to content

Add Copilot review orchestration E2E coverage - #2414

Merged
Alexander Holstrup (aholstrup1) merged 2 commits into
mainfrom
aholstrup1-copilot-review-e2e
Oct 21, 2026
Merged

Alexander Holstrup (aholstrup1) merged 2 commits into
mainfrom
aholstrup1-copilot-review-e2e

Conversation

@aholstrup1

Copy link
Copy Markdown
Collaborator

❔What, Why & How

Add E2E coverage for Copilot review orchestration and verify that ordinary upgrades do not enable automatic review.

The new scenario creates one PTE repository, sets only CodeReviewAgent.enabled = true, updates system files, and opens one PR with a small AL change. It checks that both review workflows succeed and the final review artifact exists. It uses default models and skills, does not inspect AI findings or comments, and skips the Linux invocation to avoid running the review twice. Waiting is bounded and does not rerun reviews.

The existing upgrade test accepts absent review workflows or manual-only workflows with event guards, accounting for older updaters that copy template files directly. Run-count expectations are unchanged. This follows the behavior introduced by #2413 and should run against templates/actions containing that fix.

Local validation: 16 checks in a session-local mocked harness passed, PowerShell parsing and PSScriptAnalyzer passed, and git diff --check passed. No live E2E or paid inference was run.

Related to issue: N/A

✅ Checklist

  • Add tests (E2E, unit tests)
  • Update RELEASENOTES.md - N/A, test-only change
  • Update documentation (e.g. for new settings or scenarios) - N/A, scenario steps are documented in the script header
  • Add telemetry - N/A

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 20, 2026 14:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Compatibility depends on the still-open prerequisite #2413, and live end-to-end orchestration remains unverified.

No open issues 🎉

What changed in this PR

Adds end-to-end coverage for AL-Go’s opt-in Copilot review orchestration and verifies that ordinary upgrades leave automatic reviews disabled.

Changes:

  • Adds a Windows-only scenario checking first-attempt workflow success and the final review artifact, with bounded polling and no retries.
  • Extends upgrade checks to accept absent review workflows or guarded, manual-only workflows.
File Description
e2eTests/​Test-AL-Go-Upgrade.ps1 Verifies review workflows remain inactive without opt-in.
e2eTests/​scenarios/​CopilotCodeReview/​runtest.ps1 Tests enabled review orchestration using one repository and PR.

Review effort: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@aholstrup1

Copy link
Copy Markdown
Collaborator Author

@aholstrup1
Alexander Holstrup (aholstrup1) merged commit 03ee88d into main Oct 21, 2026
120 of 138 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants