Skip to content

test!: migrate to Pest 4 and Mantle Testkit for real WordPress integration tests - #38

Open
darylldoyle wants to merge 2 commits into
feature/2.0.0from
feature/pest-4-mantle-testkit
Open

test!: migrate to Pest 4 and Mantle Testkit for real WordPress integration tests#38
darylldoyle wants to merge 2 commits into
feature/2.0.0from
feature/pest-4-mantle-testkit

Conversation

@darylldoyle

Copy link
Copy Markdown
Collaborator

What this does

Moves the test suite from PHPUnit 9 + Brain Monkey to Pest 4 + Mantle Testkit.

Pest 4 is built on PHPUnit 12, so this is also how we get onto a current PHPUnit. Testkit is the piece that boots a real WordPress: tests/bootstrap.php calls \Mantle\Testing\install(), which downloads and installs WordPress into a temporary directory and runs the suite against it with a MySQL database. No WordPress checkout and no install-wp-tests.sh needed.

Everything here is development tooling. Nothing about the shipped package changes.

Why: these are real integration tests now, not mocks we can't trust

This is the actual point of the PR.

Under Brain Monkey we mocked WordPress functions, which meant a test could only ever assert that we called something, never that WordPress did anything with it. A mocked register_post_type() proves the function was invoked. It cannot prove the post type exists, that labels resolved correctly, that supports were applied, or that taxonomies attached.

That gap showed up plainly in the coverage. AbstractPostType and AbstractTaxonomy had exactly one test each — the two most-used classes in the library — because there was almost nothing meaningful you could assert against a mock. They now have 16 tests between them, asserting against real WordPress state: the post type is registered, every derived label is correct, editor supports applied, the declared taxonomy genuinely attached, and a real post and term can be inserted and read back.

Three more places where the old tests could not fail:

  • HeadOverrides and Emoji asserted on the source code of register() as a string — literally reading the method body and checking it contained "remove_action( 'wp_head', 'wp_generator' )". That was the only option when remove_action() is a mock. They now assert the hooks are actually detached, and that the generator tag and emoji detection script genuinely disappear from rendered wp_head output.
  • BlockRegistrar pointed at directories that never existed (/test/blocks/), so most of its 20 tests only demonstrated that register_blocks() returned without throwing. It now writes real block.json files to disk and asserts against WP_Block_Type_Registry, including rendering a block through its markup.php.
  • LoaderDebug's admin page is now rendered as a real administrator with real nonces, rather than with stubbed capability and nonce checks.

The intent is durable confidence in the base classes downstream projects extend, rather than a scattering of assertions that pass regardless of whether the framework works.

Structure

Directory Base class For
tests/Unit/ Mantle\Testkit\Unit_Test_Case Code calling no WordPress functions — the class cache, the bin script, asset sidecars
tests/Integration/ Mantle\Testkit\Integration_Test_Case Anything touching WordPress; each test in a transaction that rolls back
tests/Arch/ none Pest arch() rules

Testkit shipping both base classes means the genuinely WordPress-free tests stay fast instead of booting WordPress needlessly.

The architecture tests earned their place immediately — the "no debugging leftovers" rule caught a real error_log() in BlockRegistrar. It's a deliberate diagnostic, so the exception is scoped to that one class and any new error_log() elsewhere still fails.

Isolation gotchas worth knowing about

These are documented in CLAUDE.md, and cost real debugging time:

  • Testkit's Unit_Test_Case carries #[RunTestsInSeparateProcesses], but the attribute does not survive Pest's generated test classes. A define() in a test therefore leaks into every later test in the process. Cases that must define a constant shell out to tests/scripts/ instead.
  • WP_CORE_DIR is now per-package. Mantle defaults to a bare /tmp/wordpress and reuses any wp-tests-config.php it finds there rather than regenerating one from the environment. On a machine where another project has run its tests, that config points at that project's database — and Mantle's installer drops and recreates tables. The bootstrap also refuses to start if the resolved config doesn't name the expected database.
  • The database rollback does not reset process state. The ModuleInitialization singleton, LoaderDebug's static records, BlockRegistrar's static registries, and the post type / taxonomy / block registries all need explicit teardown. tests/Helpers.php provides resets.
  • _doing_it_wrong() fails a test unless declared with setExpectedIncorrectUsage().

One finding surfaced by writing a real conflict test: register_blocks() calls register_block_type_from_metadata() before checking self::$block_sources, and bails on a falsy return. When two registrars claim the same block name in one request, WordPress's own duplicate rejection short-circuits the conflict branch, so its log line never runs. The end state is still correct — first registrar wins, name recorded once — but the conflict logging is effectively unreachable. Left as-is and documented in the test; worth a follow-up if we want that log to fire.

Commands

composer test           # Pest; no coverage driver needed
composer test-coverage  # Pest with coverage; needs Xdebug or PCOV

composer test no longer forces XDEBUG_MODE=coverage, and phpunit.xml.dist declares no <coverage><report> block. Either one makes PHPUnit 12 enable coverage collection and hard-fail when no driver is installed — PHPUnit 9 only warned.

Contributor impact

Running the suite now needs a MySQL database. That's the one cost of this change. Defaults are in tests/bootstrap.php (wp_framework_tests, root, no password, 127.0.0.1) and every value can be overridden by environment variable. The CI test job provisions a MySQL service.

Verification

composer test      126 passed (262 assertions), 2.5s
composer lint      0 errors
composer static    [OK] No errors, type coverage still 100%
composer validate  --strict clean

Previously 94 test methods (106 cases). Now 126, with the added weight in the base classes rather than in restating what mocks were told to return.

Replace PHPUnit 9 and Brain Monkey with Pest 4 (built on PHPUnit 12) and
Mantle Testkit, so the suite runs against an actual WordPress install and
MySQL database instead of mocked WordPress functions.

The point is confidence rather than tooling fashion. A mocked
register_post_type() can only prove the function was called; it cannot show
the post type exists, that labels resolved, or that taxonomies attached.
AbstractPostType and AbstractTaxonomy each had exactly one test for that
reason, and now have sixteen between them, including inserting and reading
back a real post and term.

Structure:

- tests/Unit/ uses Testkit's Unit_Test_Case, for code that calls no
  WordPress functions (the class cache, the bin script, asset sidecars).
- tests/Integration/ uses Integration_Test_Case, for everything touching
  WordPress. Each test runs in a transaction that rolls back.
- tests/Arch/ holds Pest architecture rules: strict_types everywhere, no
  debugging leftovers, and src/ never referencing test namespaces.

Notable replacements of assertions that could not fail:

- HeadOverrides and Emoji asserted on the *source code* of register() as a
  string, because mocked remove_action() proves nothing. They now assert
  the hooks are detached and that the markup disappears from wp_head.
- BlockRegistrar pointed at directories that never existed, so most tests
  only showed register_blocks() did not throw. It now builds real
  block.json files and asserts against WP_Block_Type_Registry.
- LoaderDebug's page rendering now runs with a real administrator and real
  nonces rather than stubbed capability checks.

Test isolation notes, which are not obvious:

- Testkit's Unit_Test_Case carries #[RunTestsInSeparateProcesses], but the
  attribute does not survive Pest's generated test classes. A constant
  defined in a test therefore leaks, so those cases shell out to
  tests/scripts/ instead.
- WP_CORE_DIR is set per-package. Mantle defaults to a bare /tmp/wordpress
  and reuses any wp-tests-config.php it finds there, so a shared temp
  directory points the destructive installer at whichever database another
  project configured. The bootstrap also refuses to run if the resolved
  config does not name the expected database.
- The database rollback does not reset process state: the
  ModuleInitialization singleton, LoaderDebug's static records,
  BlockRegistrar's registries and the post type/block registries all need
  explicit teardown.

Also:

- composer test no longer forces XDEBUG_MODE=coverage, and
  phpunit.xml.dist declares no coverage reports. Either makes PHPUnit 12
  enable coverage collection and hard-fail when no driver is installed.
  Use composer test-coverage when Xdebug or PCOV is present.
- The CI test job gains a MySQL service and per-package WP_CORE_DIR.
- Exclude PHPCompatibility's ThisFoundOutsideClass from tests/: Pest binds
  $this in every closure, and the warning concerns PHP 5.3.

BREAKING CHANGE: contributors now need a MySQL database to run the test
suite. Nothing about the shipped package changes; this is development
tooling only. See the Testing section of CLAUDE.md.
@darylldoyle darylldoyle added this to the 2.0.0 milestone Jul 30, 2026
@darylldoyle darylldoyle self-assigned this Jul 30, 2026
The pull_request trigger filtered on branches: [trunk, develop], and that
filter matches the base branch of the PR. Every pull request opened against
a release branch such as feature/2.0.0 was therefore ineligible, and the
checks tab reported no checks for the commit rather than a failure.

Drop the filter so all pull requests run, keep the push filter on the
long-lived branches so PR commits do not run twice, and add
workflow_dispatch for manual runs.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates the repository’s development test tooling from PHPUnit 9 + Brain Monkey mocks to Pest 4 (PHPUnit 12) + Mantle Testkit running against a real WordPress install and MySQL, improving confidence in framework behavior while keeping shipped library code unchanged.

Changes:

  • Replaces mock-based unit tests with real integration tests (post types, taxonomies, blocks, loader debug, core hook modules) using Mantle-installed WordPress + DB transactions.
  • Adds/updates unit tests for cache/CLI/asset-sidecar behavior that intentionally run without WordPress, plus helper utilities and scripts for isolation.
  • Updates CI and dev configuration (composer scripts/deps, PHPUnit config, PHPCS exceptions, workflow MySQL service, docs/changelog, gitignore).

Reviewed changes

Copilot reviewed 40 out of 44 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/Unit/SmokeTest.php Adds a unit-suite smoke test confirming WordPress is not required.
tests/Unit/ModuleInitialization/DisableCacheConstantTest.php Verifies cache-disable constant behavior via isolated subprocess.
tests/Unit/ModuleInitialization/ClassCacheTest.php Adds unit coverage for cache generation/reading/corruption fallback.
tests/Unit/Cache/ReadOnlyFileDiscoverCacheDriverTest.php Adds unit coverage for read-only cache driver behavior.
tests/Unit/Bin/GenerateClassCacheTest.php Adds unit coverage for the class-cache generator CLI (subprocess).
tests/Unit/Assets/GetAssetInfoTest.php Adds unit coverage for asset sidecar resolution and fallback behavior.
tests/TestMultiDirectoryBlockRegistrar.php Removes old fixed-path block registrar test double.
tests/TestEmptyDirectoryBlockRegistrar.php Removes old empty-directory block registrar test double.
tests/TestBlockRegistrar.php Removes old fixed-path block registrar test double.
tests/Taxonomies/AbstractTaxonomyTest.php Removes old mock-based taxonomy test.
tests/scripts/disable-class-cache.php Adds helper script to test constant-defined behavior in isolation.
tests/PostTypes/AbstractPostTypeTest.php Removes old mock-based post type test.
tests/Pest.php Adds Pest configuration binding Unit vs Integration suites to Testkit bases.
tests/ModuleInitializationTest.php Removes large mock-based module initialization test suite.
tests/Integration/Taxonomies/AbstractTaxonomyTest.php Adds real WP taxonomy registration/labels/options/term storage tests.
tests/Integration/SmokeTest.php Adds integration-suite smoke tests for WP boot and DB availability.
tests/Integration/PostTypes/AbstractPostTypeTest.php Adds real WP post type registration/labels/supports/taxonomy/post storage tests.
tests/Integration/ModuleInitializationTest.php Adds real WP module discovery/registration/action firing tests.
tests/Integration/ModuleInitialization/LoaderRecordingTest.php Adds admin-context tests for loader debug recording.
tests/Integration/Debug/LoaderDebugTest.php Adds real admin-page rendering + nonce/capability + formatting helper tests.
tests/Integration/Core/HeadOverridesTest.php Replaces source-string assertions with real wp_head hook/markup assertions.
tests/Integration/Core/EmojiTest.php Replaces source-string assertions with real hook/filter/markup assertions.
tests/Integration/BlockRegistrarTest.php Adds real block.json registration/rendering/conflict/source-tracking tests.
tests/Helpers.php Adds shared helper utilities for temp dirs, subprocess runs, and state resets.
tests/FrameworkTestSetup.php Removes Brain Monkey-based shared test setup trait.
tests/Doubles/SecondBlockRegistrar.php Adds a distinct registrar class to exercise conflict behavior.
tests/Doubles/ConfigurableBlockRegistrar.php Adds configurable registrar double to point at real temp block dirs.
tests/Debug/LoaderDebugTest.php Removes old mock-based LoaderDebug test suite.
tests/Core/HeadOverridesTest.php Removes old mock-based HeadOverrides test suite.
tests/Core/EmojiTest.php Removes old mock-based Emoji test suite.
tests/Cache/ReadOnlyFileDiscoverCacheDriverTest.php Removes old mock-based cache driver test suite.
tests/bootstrap.php Boots Mantle installer, sets safer DB/WP_CORE_DIR defaults, adds drift guard.
tests/BlockRegistrarTest.php Removes old mock-based BlockRegistrar test suite.
tests/Bin/GenerateClassCacheTest.php Removes old PHPUnit-based CLI integration test file.
tests/Assets/GetAssetInfoTest.php Removes old mock-based GetAssetInfo test file.
tests/Arch/ConventionsTest.php Adds Pest architecture checks (strict types, no debug leftovers, no test deps).
phpunit.xml.dist Updates PHPUnit 12 schema usage and reworks suites/source coverage config.
phpcs.xml Excludes a PHPCompatibility closure sniff for Pest tests.
composer.json Swaps dev deps to Pest/Testkit, adds scripts, and autoload-dev helpers file.
CLAUDE.md Adds repo guidance including new Pest/Testkit testing model and gotchas.
CHANGELOG.md Documents the testing migration and contributor impact.
.gitignore Ignores the CI WordPress install directory used by integration tests.
.github/workflows/php.yml Updates CI to run Pest and provision MySQL + WP env for Mantle installer.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/Pest.php
Comment on lines +8 to +9
* - Unit/ does not. Testkit's Unit_Test_Case runs each class in a separate process with
* global state discarded, so unit tests can never see integration state.
Comment on lines +6 to +7
* discover_live() make no unguarded WordPress calls, so they need no WordPress. Testkit's
* Unit_Test_Case also runs every test in its own process, which keeps the singleton clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants