Skip to content

test(spanner): optimize and fix system tests and enable in CI#9329

Open
cy-yun wants to merge 1 commit into
googleapis:mainfrom
cy-yun:fix-spanner-tests-9230
Open

test(spanner): optimize and fix system tests and enable in CI#9329
cy-yun wants to merge 1 commit into
googleapis:mainfrom
cy-yun:fix-spanner-tests-9230

Conversation

@cy-yun

@cy-yun cy-yun commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

This PR addresses and resolves the Spanner emulator system test flakiness and optimizations outlined in Issue #9230, enabling them to run more reliably in CI environments.

Key Changes:

  • BackupTest.php Optimizations:
    • Added the @group flakey annotation so these tests can be safely skipped in standard CI runs.
    • Removed redundant Long Running Operation polling in testCancelBackupOperation and testDeleteBackup to significantly reduce overall execution time.
    • Fixed testCreateBackupRequestFailed flakiness where a pending backup from previous tests threw a FailedPreconditionException.
    • Fixed multiple PHPCS linter formatting errors (multi-line function arguments & trailing whitespace).
  • Idempotent Mutations for Emulator Resilience:
    • Swapped insert() to insertOrUpdate() in TransactionTest.php and PgTransactionTest.php.
    • Changed insert() to insertOrUpdateBatch() in PartitionedDmlTest::testPdml().
    • Reason: The Spanner emulator occasionally drops the connection and returns an ABORTED error after it has already internally committed a mutation. When our client's runTransaction loop catches the abort and automatically retries the transaction closure, it was failing with a ConflictException (ALREADY_EXISTS). Making the mutations idempotent handles this retry edge-case gracefully.
  • Core LRO Deprecation Fix:
    • Fixed a PHP 8.1 deprecation warning ("using null as array offset" / passing null to string parameter) in LongRunningClientConnection::resumeOperation by properly omitting the $method argument when it's not explicitly provided. This resolves the warning without breaking any internal mock expectations (like those in InstanceConfigurationTest).
  • CI Configuration:
    • Removed Spanner from the phpunit-system.xml.dist exclusion list so that non-emulator system tests can be successfully executed in Kokoro.

🔗 Related Issues

@cy-yun
cy-yun force-pushed the fix-spanner-tests-9230 branch 2 times, most recently from d69b03a to 169514a Compare July 8, 2026 19:41
@cy-yun cy-yun changed the title Fix spanner tests 9230 test(spanner): optimize and fix system tests and enable in CI Jul 8, 2026
@cy-yun
cy-yun force-pushed the fix-spanner-tests-9230 branch 2 times, most recently from bfd11cb to 41272ef Compare July 8, 2026 20:18
@cy-yun
cy-yun marked this pull request as ready for review July 8, 2026 20:34
@cy-yun
cy-yun requested a review from a team as a code owner July 8, 2026 20:34
Comment thread Spanner/tests/System/BackupTest.php Outdated
// Poll for completion with the extended timeout
$op->pollUntilComplete([
$op->pollUntilComplete(
[

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did the linter suggested this change? This change feels wrong to me. I mean it still is valid, but I would expect either the previous iteration or something like:

$op->pollUntilComplete(
    [
        'timeoutMillis' => ...
    ]
)

As it is right now, having the braces on the same level as the key feels off.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I still think the previous one should be valid 🤷

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the formatting of the code such that it is consistent with the original & phpcbf is not upset


$this->assertTrue($backup->exists());
// Cancellation usually drops the backup. We don't assert exists()
// to avoid flakiness with asynchronous deletion.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, isn't this testing that if you execute a backup deletion and cancel it, it shouldn't be still available? Or am i misreading the test?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this test is meant to test canceling a backup creation operation, not a deletion. Previously, the test was calling $op->pollUntilComplete() before $op->cancel(). That meant it was waiting for the backup to fully finish creating, and then issuing a cancel command on a completed operation (which basically does nothing), hence why it used to assert that the backup still existed.

By removing the polling, we are now properly canceling the operation while it is still in-progress. Because we cancelled it mid-flight, Spanner usually drops the incomplete backup, so asserting that it still exists makes the test extremely flaky.

@cy-yun
cy-yun force-pushed the fix-spanner-tests-9230 branch from 41272ef to 2e68fb4 Compare July 9, 2026 07:57
@product-auto-label product-auto-label Bot added the api: spanner Issues related to the Spanner API. label Jul 9, 2026
Fixes googleapis#9230 by:
- Adding @group flakey to BackupTest.php so it can be safely skipped in normal CI runs.
- Removing redundant Long Running Operation polling in BackupTest.php (testCancelBackupOperation and testDeleteBackup) to significantly reduce the execution time.
- Removing Spanner from the phpunit-system.xml.dist exclusion list so non-emulator system tests run in Kokoro.
- Fix PHP 8.1 deprecation warning (using null as array offset) in LongRunningClientConnection.php by explicitly passing an empty string for the method name.
- Fix BackupTest::testCreateBackupRequestFailed failing due to a pending backup from previous tests throwing FailedPreconditionException.
- Fix PartitionedDmlTest::testPdml failing with ConflictException if the table is already populated by using insertOrUpdateBatch.
- Change insert to insertOrUpdate in TransactionTest and PgTransactionTest. This prevents ConflictException (ALREADY_EXISTS) when runTransaction retries the transaction closure after a transient emulator error (e.g. ABORTED) where the initial mutation was still applied.
@cy-yun
cy-yun force-pushed the fix-spanner-tests-9230 branch from 2e68fb4 to 921a05c Compare July 9, 2026 20:33
@cy-yun cy-yun added kokoro:force-run Add this label to force Kokoro to re-run the tests. and removed kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: spanner Issues related to the Spanner API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improvements to Spanner System Tests

2 participants