test!: migrate to Pest 4 and Mantle Testkit for real WordPress integration tests - #38
Open
darylldoyle wants to merge 2 commits into
Open
test!: migrate to Pest 4 and Mantle Testkit for real WordPress integration tests#38darylldoyle wants to merge 2 commits into
darylldoyle wants to merge 2 commits into
Conversation
Replace PHPUnit 9 and Brain Monkey with Pest 4 (built on PHPUnit 12) and Mantle Testkit, so the suite runs against an actual WordPress install and MySQL database instead of mocked WordPress functions. The point is confidence rather than tooling fashion. A mocked register_post_type() can only prove the function was called; it cannot show the post type exists, that labels resolved, or that taxonomies attached. AbstractPostType and AbstractTaxonomy each had exactly one test for that reason, and now have sixteen between them, including inserting and reading back a real post and term. Structure: - tests/Unit/ uses Testkit's Unit_Test_Case, for code that calls no WordPress functions (the class cache, the bin script, asset sidecars). - tests/Integration/ uses Integration_Test_Case, for everything touching WordPress. Each test runs in a transaction that rolls back. - tests/Arch/ holds Pest architecture rules: strict_types everywhere, no debugging leftovers, and src/ never referencing test namespaces. Notable replacements of assertions that could not fail: - HeadOverrides and Emoji asserted on the *source code* of register() as a string, because mocked remove_action() proves nothing. They now assert the hooks are detached and that the markup disappears from wp_head. - BlockRegistrar pointed at directories that never existed, so most tests only showed register_blocks() did not throw. It now builds real block.json files and asserts against WP_Block_Type_Registry. - LoaderDebug's page rendering now runs with a real administrator and real nonces rather than stubbed capability checks. Test isolation notes, which are not obvious: - Testkit's Unit_Test_Case carries #[RunTestsInSeparateProcesses], but the attribute does not survive Pest's generated test classes. A constant defined in a test therefore leaks, so those cases shell out to tests/scripts/ instead. - WP_CORE_DIR is set per-package. Mantle defaults to a bare /tmp/wordpress and reuses any wp-tests-config.php it finds there, so a shared temp directory points the destructive installer at whichever database another project configured. The bootstrap also refuses to run if the resolved config does not name the expected database. - The database rollback does not reset process state: the ModuleInitialization singleton, LoaderDebug's static records, BlockRegistrar's registries and the post type/block registries all need explicit teardown. Also: - composer test no longer forces XDEBUG_MODE=coverage, and phpunit.xml.dist declares no coverage reports. Either makes PHPUnit 12 enable coverage collection and hard-fail when no driver is installed. Use composer test-coverage when Xdebug or PCOV is present. - The CI test job gains a MySQL service and per-package WP_CORE_DIR. - Exclude PHPCompatibility's ThisFoundOutsideClass from tests/: Pest binds $this in every closure, and the warning concerns PHP 5.3. BREAKING CHANGE: contributors now need a MySQL database to run the test suite. Nothing about the shipped package changes; this is development tooling only. See the Testing section of CLAUDE.md.
The pull_request trigger filtered on branches: [trunk, develop], and that filter matches the base branch of the PR. Every pull request opened against a release branch such as feature/2.0.0 was therefore ineligible, and the checks tab reported no checks for the commit rather than a failure. Drop the filter so all pull requests run, keep the push filter on the long-lived branches so PR commits do not run twice, and add workflow_dispatch for manual runs.
There was a problem hiding this comment.
Pull request overview
Migrates the repository’s development test tooling from PHPUnit 9 + Brain Monkey mocks to Pest 4 (PHPUnit 12) + Mantle Testkit running against a real WordPress install and MySQL, improving confidence in framework behavior while keeping shipped library code unchanged.
Changes:
- Replaces mock-based unit tests with real integration tests (post types, taxonomies, blocks, loader debug, core hook modules) using Mantle-installed WordPress + DB transactions.
- Adds/updates unit tests for cache/CLI/asset-sidecar behavior that intentionally run without WordPress, plus helper utilities and scripts for isolation.
- Updates CI and dev configuration (composer scripts/deps, PHPUnit config, PHPCS exceptions, workflow MySQL service, docs/changelog, gitignore).
Reviewed changes
Copilot reviewed 40 out of 44 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Unit/SmokeTest.php | Adds a unit-suite smoke test confirming WordPress is not required. |
| tests/Unit/ModuleInitialization/DisableCacheConstantTest.php | Verifies cache-disable constant behavior via isolated subprocess. |
| tests/Unit/ModuleInitialization/ClassCacheTest.php | Adds unit coverage for cache generation/reading/corruption fallback. |
| tests/Unit/Cache/ReadOnlyFileDiscoverCacheDriverTest.php | Adds unit coverage for read-only cache driver behavior. |
| tests/Unit/Bin/GenerateClassCacheTest.php | Adds unit coverage for the class-cache generator CLI (subprocess). |
| tests/Unit/Assets/GetAssetInfoTest.php | Adds unit coverage for asset sidecar resolution and fallback behavior. |
| tests/TestMultiDirectoryBlockRegistrar.php | Removes old fixed-path block registrar test double. |
| tests/TestEmptyDirectoryBlockRegistrar.php | Removes old empty-directory block registrar test double. |
| tests/TestBlockRegistrar.php | Removes old fixed-path block registrar test double. |
| tests/Taxonomies/AbstractTaxonomyTest.php | Removes old mock-based taxonomy test. |
| tests/scripts/disable-class-cache.php | Adds helper script to test constant-defined behavior in isolation. |
| tests/PostTypes/AbstractPostTypeTest.php | Removes old mock-based post type test. |
| tests/Pest.php | Adds Pest configuration binding Unit vs Integration suites to Testkit bases. |
| tests/ModuleInitializationTest.php | Removes large mock-based module initialization test suite. |
| tests/Integration/Taxonomies/AbstractTaxonomyTest.php | Adds real WP taxonomy registration/labels/options/term storage tests. |
| tests/Integration/SmokeTest.php | Adds integration-suite smoke tests for WP boot and DB availability. |
| tests/Integration/PostTypes/AbstractPostTypeTest.php | Adds real WP post type registration/labels/supports/taxonomy/post storage tests. |
| tests/Integration/ModuleInitializationTest.php | Adds real WP module discovery/registration/action firing tests. |
| tests/Integration/ModuleInitialization/LoaderRecordingTest.php | Adds admin-context tests for loader debug recording. |
| tests/Integration/Debug/LoaderDebugTest.php | Adds real admin-page rendering + nonce/capability + formatting helper tests. |
| tests/Integration/Core/HeadOverridesTest.php | Replaces source-string assertions with real wp_head hook/markup assertions. |
| tests/Integration/Core/EmojiTest.php | Replaces source-string assertions with real hook/filter/markup assertions. |
| tests/Integration/BlockRegistrarTest.php | Adds real block.json registration/rendering/conflict/source-tracking tests. |
| tests/Helpers.php | Adds shared helper utilities for temp dirs, subprocess runs, and state resets. |
| tests/FrameworkTestSetup.php | Removes Brain Monkey-based shared test setup trait. |
| tests/Doubles/SecondBlockRegistrar.php | Adds a distinct registrar class to exercise conflict behavior. |
| tests/Doubles/ConfigurableBlockRegistrar.php | Adds configurable registrar double to point at real temp block dirs. |
| tests/Debug/LoaderDebugTest.php | Removes old mock-based LoaderDebug test suite. |
| tests/Core/HeadOverridesTest.php | Removes old mock-based HeadOverrides test suite. |
| tests/Core/EmojiTest.php | Removes old mock-based Emoji test suite. |
| tests/Cache/ReadOnlyFileDiscoverCacheDriverTest.php | Removes old mock-based cache driver test suite. |
| tests/bootstrap.php | Boots Mantle installer, sets safer DB/WP_CORE_DIR defaults, adds drift guard. |
| tests/BlockRegistrarTest.php | Removes old mock-based BlockRegistrar test suite. |
| tests/Bin/GenerateClassCacheTest.php | Removes old PHPUnit-based CLI integration test file. |
| tests/Assets/GetAssetInfoTest.php | Removes old mock-based GetAssetInfo test file. |
| tests/Arch/ConventionsTest.php | Adds Pest architecture checks (strict types, no debug leftovers, no test deps). |
| phpunit.xml.dist | Updates PHPUnit 12 schema usage and reworks suites/source coverage config. |
| phpcs.xml | Excludes a PHPCompatibility closure sniff for Pest tests. |
| composer.json | Swaps dev deps to Pest/Testkit, adds scripts, and autoload-dev helpers file. |
| CLAUDE.md | Adds repo guidance including new Pest/Testkit testing model and gotchas. |
| CHANGELOG.md | Documents the testing migration and contributor impact. |
| .gitignore | Ignores the CI WordPress install directory used by integration tests. |
| .github/workflows/php.yml | Updates CI to run Pest and provision MySQL + WP env for Mantle installer. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+8
to
+9
| * - Unit/ does not. Testkit's Unit_Test_Case runs each class in a separate process with | ||
| * global state discarded, so unit tests can never see integration state. |
Comment on lines
+6
to
+7
| * discover_live() make no unguarded WordPress calls, so they need no WordPress. Testkit's | ||
| * Unit_Test_Case also runs every test in its own process, which keeps the singleton clean. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Moves the test suite from PHPUnit 9 + Brain Monkey to Pest 4 + Mantle Testkit.
Pest 4 is built on PHPUnit 12, so this is also how we get onto a current PHPUnit. Testkit is the piece that boots a real WordPress:
tests/bootstrap.phpcalls\Mantle\Testing\install(), which downloads and installs WordPress into a temporary directory and runs the suite against it with a MySQL database. No WordPress checkout and noinstall-wp-tests.shneeded.Everything here is development tooling. Nothing about the shipped package changes.
Why: these are real integration tests now, not mocks we can't trust
This is the actual point of the PR.
Under Brain Monkey we mocked WordPress functions, which meant a test could only ever assert that we called something, never that WordPress did anything with it. A mocked
register_post_type()proves the function was invoked. It cannot prove the post type exists, that labels resolved correctly, that supports were applied, or that taxonomies attached.That gap showed up plainly in the coverage.
AbstractPostTypeandAbstractTaxonomyhad exactly one test each — the two most-used classes in the library — because there was almost nothing meaningful you could assert against a mock. They now have 16 tests between them, asserting against real WordPress state: the post type is registered, every derived label is correct, editor supports applied, the declared taxonomy genuinely attached, and a real post and term can be inserted and read back.Three more places where the old tests could not fail:
HeadOverridesandEmojiasserted on the source code ofregister()as a string — literally reading the method body and checking it contained"remove_action( 'wp_head', 'wp_generator' )". That was the only option whenremove_action()is a mock. They now assert the hooks are actually detached, and that the generator tag and emoji detection script genuinely disappear from renderedwp_headoutput.BlockRegistrarpointed at directories that never existed (/test/blocks/), so most of its 20 tests only demonstrated thatregister_blocks()returned without throwing. It now writes realblock.jsonfiles to disk and asserts againstWP_Block_Type_Registry, including rendering a block through itsmarkup.php.LoaderDebug's admin page is now rendered as a real administrator with real nonces, rather than with stubbed capability and nonce checks.The intent is durable confidence in the base classes downstream projects extend, rather than a scattering of assertions that pass regardless of whether the framework works.
Structure
tests/Unit/Mantle\Testkit\Unit_Test_Casetests/Integration/Mantle\Testkit\Integration_Test_Casetests/Arch/arch()rulesTestkit shipping both base classes means the genuinely WordPress-free tests stay fast instead of booting WordPress needlessly.
The architecture tests earned their place immediately — the "no debugging leftovers" rule caught a real
error_log()inBlockRegistrar. It's a deliberate diagnostic, so the exception is scoped to that one class and any newerror_log()elsewhere still fails.Isolation gotchas worth knowing about
These are documented in
CLAUDE.md, and cost real debugging time:Unit_Test_Casecarries#[RunTestsInSeparateProcesses], but the attribute does not survive Pest's generated test classes. Adefine()in a test therefore leaks into every later test in the process. Cases that must define a constant shell out totests/scripts/instead.WP_CORE_DIRis now per-package. Mantle defaults to a bare/tmp/wordpressand reuses anywp-tests-config.phpit finds there rather than regenerating one from the environment. On a machine where another project has run its tests, that config points at that project's database — and Mantle's installer drops and recreates tables. The bootstrap also refuses to start if the resolved config doesn't name the expected database.ModuleInitializationsingleton,LoaderDebug's static records,BlockRegistrar's static registries, and the post type / taxonomy / block registries all need explicit teardown.tests/Helpers.phpprovides resets._doing_it_wrong()fails a test unless declared withsetExpectedIncorrectUsage().One finding surfaced by writing a real conflict test:
register_blocks()callsregister_block_type_from_metadata()before checkingself::$block_sources, and bails on a falsy return. When two registrars claim the same block name in one request, WordPress's own duplicate rejection short-circuits the conflict branch, so its log line never runs. The end state is still correct — first registrar wins, name recorded once — but the conflict logging is effectively unreachable. Left as-is and documented in the test; worth a follow-up if we want that log to fire.Commands
composer testno longer forcesXDEBUG_MODE=coverage, andphpunit.xml.distdeclares no<coverage><report>block. Either one makes PHPUnit 12 enable coverage collection and hard-fail when no driver is installed — PHPUnit 9 only warned.Contributor impact
Running the suite now needs a MySQL database. That's the one cost of this change. Defaults are in
tests/bootstrap.php(wp_framework_tests,root, no password,127.0.0.1) and every value can be overridden by environment variable. The CI test job provisions a MySQL service.Verification
Previously 94 test methods (106 cases). Now 126, with the added weight in the base classes rather than in restating what mocks were told to return.