Also fail on Pester block/container failures (not just failed tests)#128
Merged
tablackburn merged 2 commits intoJul 7, 2026
Merged
Conversation
In Pester 5 a run has three independent failure counts. Gating only on FailedCount misses FailedBlocksCount (BeforeAll/AfterAll) and FailedContainersCount (files that fail to discover/load), so a failing BeforeAll or an unloadable test file passes the build green. This adds those counts to the gate. Generated with AI (GitHub Copilot CLI). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
@nohwnd should we also make this change to ? |
Contributor
Author
|
Yes. You can also check the result property. |
Contributor
|
@nohwnd do you want to include that change in this PR or do you want me to take care of it? I'm good either way. |
tablackburn
pushed a commit
that referenced
this pull request
Jul 7, 2026
Test-PSBuildPester gated only on FailedCount, so a BeforeAll/AfterAll that threw or a test file that errored during discovery left the count at zero and the build passed despite tests never running. Gate on the run's aggregate Result property instead, which Pester derives from all failure categories (failed tests, failed blocks, failed containers). This applies the shipped-function half of the fix discussed in #128; that PR covers the repository's own psakeFile.ps1. Verified against crash fixtures: a discovery-crash file yields FailedCount=0 / Result=Failed and now throws; a BeforeAll crash still throws; a healthy suite still passes. Full suite: 301 passed, 0 failed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa
3 tasks
5 tasks
tablackburn
reviewed
Jul 7, 2026
tablackburn
left a comment
tablackburn
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the fix! This is correct for the repo's own build file, and we've verified the bug class is real here — while migrating this repo's test suite to Pester 6 (#132), a test file died during discovery and the build still reported success (FailedCount=0, Container failed: 1, exit 0).
Per the discussion above, the shipped-function half (Test-PSBuildPester.ps1) is now covered in #133 using the Result property you suggested, so this PR can stay scoped as-is. Left one optional inline suggestion if you'd like the two gates to match.
Generated by Claude Code
tablackburn
pushed a commit
that referenced
this pull request
Jul 7, 2026
Address review feedback on #133: the Unreleased entry linked #128, but this change lands via #133. Point the entry at #133 and note the companion relationship to #128 explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa
tablackburn
approved these changes
Jul 7, 2026
tablackburn
pushed a commit
that referenced
this pull request
Jul 7, 2026
Replace the three-counter sum merged in #128 with the aggregate Result property, matching the gate in Test-PSBuildPester. Same behavior, one source of truth for what "failed" means. Suggested by nohwnd in the #128 discussion. Verified both directions locally: a discovery-crash container fails the build (exit 1), and a healthy run under Pester 5.8.0 passes (301 passed, exit 0). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa
tablackburn
added a commit
that referenced
this pull request
Jul 7, 2026
Test-PSBuildPester gated pass/fail only on FailedCount, so a BeforeAll/AfterAll that threw or a test file that errored during discovery left the count at zero and the build passed despite tests never running. Gate on the run's aggregate Result property instead, which Pester derives from all failure categories. Also switch this repo's own psakeFile.ps1 Pester gate (the three-counter sum from #128) to the same Result check so both gates share one definition of "failed". Companion to #128. Same bug lineage as #31, #52, and #57.
Contributor
Author
|
Thanks for merging, because see you using Claude here, just pointing out there is now skill for v6 migration, as well as for should -* to should-* migration. https://pester.dev/docs/migrations/v5-to-v6#migrate-with-an-ai-assistant |
tablackburn
pushed a commit
that referenced
this pull request
Jul 10, 2026
Add tests/Test-PSBuildPester.tests.ps1 (#102). Every invocation runs in a Start-Job subprocess with a pinned inner Pester version, and the scenarios run against both installed Pester majors (5.x and 6.x) to verify the shipped function keeps supporting Pester 5 consumers. Crash fixtures are generated into $TestDrive at runtime, never checked in. Scenarios: healthy suite, failing test (regression #52), BeforeAll crash and discovery crash (#128/#133 gate), NUnit result output, and code coverage output path/format (regression Add a second, named Pester dependency (PesterLegacy, 5.9.0) to requirements.psd1 so CI installs both majors side by side. Fix two latent Test-PSBuildPester bugs the tests exposed, test-first (all 14 tests failed before the fixes, pass after): - The finally block called Remove-Module with an empty -Name when the optional ModuleName parameter was omitted, raising a parameter- binding error that -ErrorAction SilentlyContinue cannot suppress. - An unconditional Import-Module Pester -MinimumVersion 5.0.0 loaded the newest installed Pester on top of an already-loaded one, which crashes with a Pester.dll version conflict when two versions are installed side by side. The function now respects a loaded Pester (with a clear error below 5.0.0) and imports the newest installed version only when none is loaded. Closes #102 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa
4 tasks
tablackburn
added a commit
that referenced
this pull request
Jul 10, 2026
) Add tests/Test-PSBuildPester.tests.ps1 (#102). Every invocation runs in a Start-Job subprocess with a pinned inner Pester version, and the scenarios run against both installed Pester majors (5.x and 6.x) to verify the shipped function keeps supporting Pester 5 consumers. Crash fixtures are generated into $TestDrive at runtime, never checked in. Scenarios: healthy suite, failing test (regression #52), BeforeAll crash and discovery crash (#128/#133 gate), NUnit result output, and code coverage output path/format (regression #62). Pester 5.9.0 is installed side by side with 6.0.0 through the new install-only requirements.pester-matrix.psd1; the bootstrap must not import it, because importing two Pester majors into one session crashes with a Pester.dll version conflict. Fix two latent Test-PSBuildPester bugs the tests exposed, test-first: - The finally block called Remove-Module with an empty -Name when the optional ModuleName parameter was omitted, raising a parameter- binding error that -ErrorAction SilentlyContinue cannot suppress. - An unconditional Import-Module Pester -MinimumVersion 5.0.0 loaded the newest installed Pester on top of an already-loaded one. The function now respects a loaded Pester (with a clear error below 5.0.0) and imports the newest installed version only when none is loaded. Closes #102
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I've been seeing this commonly across Pester 5 build scripts and I'm fixing it where it looks like it could hide real failures.
In Pester 5 a run reports three independent failure counts:
FailedCount— failed tests (It)FailedBlocksCount— failedBeforeAll/AfterAllFailedContainersCount— files that error during discovery / fail to loadGating on only
FailedCountmeans aBeforeAllthat throws, or a test file that fails to load, leavesFailedCount = 0and the build passes green despite real failures. Pester itself derives the overall pass/fail from the sum of all three.This change includes all three counts in the gate. Display/log messages are left unchanged.