Skip to content

compile: drop the dead pg / mysql2 async-runtime rule; pg / mysql2 programs link no tokio (tokio lane L3) - #11331

Merged
proggeramlug merged 2 commits into
mainfrom
tokio-laneL3-db-wrappers
Sep 25, 2026
Merged

proggeramlug merged 2 commits into
mainfrom
tokio-laneL3-db-wrappers

Conversation

@proggeramlug

@proggeramlug proggeramlug commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Tokio lane L3 covers the database wrappers. The planned work was to remove the pg and mysql2 "decline" paths, which call Handle::current().block_on inside perry_ffi_spawn_blocking, and to stop the driver selecting async-runtime for them. That code is already gone: the pg and mysql2 wrappers, decline paths included, were deleted in #10677 and #10680. What was left was a module-name rule in the auto-optimize driver that could never run, plus prose that still described the old setup. This PR removes both. It changes nothing in the runtime.

What changed

  • optimized_libs/driver.rs: the rule that inserted async-runtime for "pg" | "mysql2" | "mysql2/promise" | "mongodb" now covers only "mongodb". The rule sits inside the loop over well-known bindings, and that loop continues at lookup_well_known for any module without a row in well_known_bindings.toml. pg and mysql2 have no row, so these arms were unreachable. Their npm packages compile from source over net / tls and select only what those modules select. mongodb keeps its rule because perry-ext-mongodb still blocks on tokio; that wrapper is being deleted separately.
  • New test well_known::tests::pg_and_mysql2_have_no_well_known_row: it fails if either module gets a row again, which would make the removed rule reachable.
  • Stale prose updated: stdlib_features.rs (comment and test comment), freshness.rs (the binding_bundles_tokio doc said the driver selects async-runtime "for their decline paths by module name", and an alias example used mysql2), a comment in perry-stdlib/Cargo.toml, the PG / MYSQL subsystem-slot docs in perry-db-turnloop (now marked reserved, values unchanged), and the perry-stdlib entry of scripts/tokio_inventory.json (reached_when, plus the group-B step in blocker). tokio_inventory.py --update reports no edge or lockfile change.

Evidence

Three arms were built the same way (cargo build --release -p perry -p perry-runtime-static -p perry-stdlib-static, codegen-units 16) on perrymaster (Linux x86_64):

pg 8.23.0 and mysql2 3.24.4 are pinned in the fixture, and the oracle is Node 26.5.1. PostgreSQL 16.15 and MySQL 8.0.46 ran as private instances on their own ports.

Tokio in the linked stdlib archive (ar t | grep -c tokio) and in the binary (strings | grep -c tokio-1.):

program mode A (main) B (this PR) C (this PR + L2)
pg-only (probe_pg, pg_parity) auto-optimize 16 / 52, feature set includes async-runtime 16 / 52, identical feature set 0 / 0, no async-runtime
mysql2-only (probe_mysql, mysql_parity, mysql_pool_parity, t_414, t_8745_8746, t_9310) auto-optimize 16 / 51 16 / 51 0 / 0
all of the above PERRY_NO_AUTO_OPTIMIZE=1 16 / 53 16 / 53 16 / 53 (full implies async-runtime, as documented)

B matching A is the expected result for a rule that could not run: on main, the tokio in pg and mysql2 programs came from the net / tls pump features, which #11277 moved off tokio. With #11277 and this PR together, a pg-only or mysql2-only auto-optimized program links no tokio. The census counted no tokio* OS threads on the primary agent or in the worker.

Behaviour of the pg probe (connect, simple query, parameters, a rolled-back and a committed transaction, the error code, an 8-query Pool with max 3, a transaction on a pool.connect() client, then the same driver on a worker_threads agent, which is the former decline scenario):

  • With trust or md5 auth, probe_pg and pg_parity print exactly what node prints on A, B and C, including the worker query.
  • The worker half of probe_pg is flaky on all three arms. Repeated runs on each arm fail intermittently with TypeError: Cannot read properties of undefined (reading 'setBuffer'), or time out; the pg_parity runs did not fail. This fails on main too.

Failures that also happen on main (not caused by this PR, same on A, B and C)

  1. pg with scram-sha-256 auth: password authentication failed for user "perry" on every arm, while node authenticates with the same credentials. md5 and trust work.
  2. mysql2 segfaults (SIGSEGV) during createConnection on every arm, with both mysql2 3.24.4 and 3.23.2 and both caching_sha2_password and mysql_native_password. A minimal program prints loaded function and then crashes. So none of the mysql2 tests can check behaviour on Linux at the moment; they only show tokio linkage.
  3. The pg probe's worker flake described above.

I found no open issues for these.

Not run

  • drz/t_9330_drizzle_mysql2_transaction: the fixture's compilePackages setup does not compile, and it would hit failure 2 anyway.
  • test_issue_9330_9517_mysql2_bundled.ts: fails under node itself with mysql2 3.24.4, because it calls an unbound getConnection.
  • TLS database connections.
  • A full gap sweep.
  • The lint gate compile tier.
  • The Windows cargo xwin check: cargo-xwin is not installed in the host's login shell. This PR changes no runtime or stdlib Rust code.
  • macOS.

Checks run

  • cargo test --release -p perry --bin perry -- well_known stdlib_features optimized_libs shared_tokio: all pass, including the new test and bundled_database_copies_map_to_no_stdlib_features.
  • cargo test --release -p perry-db-turnloop: 9 passed.
  • RUSTFLAGS=-D warnings cargo check -p perry -p perry-db-turnloop --all-targets: clean.
  • cargo fmt --check, check_file_size.sh, gc_runtime_root_holders.py, tokio_inventory.py (plus --update, no diff, and --self-test): all pass.
  • SKIP_COMPILE_GATES=1 scripts/run_lint_gates.sh: 89 of 91 script gates pass. The two failures are Public benchmark evidence freshness (red on main) and cargo xwin (not installed). The compile tier was not run.

Other suites

No test outside the touched files is expected to change. optimized_libs/tests.rs uses perry-ext-mysql2 only as an opaque cache-key string, so it is not affected.

Part of the tokio removal (plan L, group B). What remains for the database wrappers is perry-ext-mongodb, which goes by deleting the wrapper.

Summary by CodeRabbit

  • Changes
    • PostgreSQL and MySQL imports now compile from their npm packages instead of using bundled native wrappers.
    • Automatic Tokio runtime selection no longer applies to PostgreSQL and MySQL imports; MongoDB remains supported.
    • With auto-optimization enabled, PostgreSQL-only and MySQL-only programs no longer link Tokio through these imports. Builds with PERRY_NO_AUTO_OPTIMIZE still include Tokio through the full configuration.

proggeramlug pushed a commit that referenced this pull request Sep 25, 2026
@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 17cdd760-5b50-4106-b365-c4a8f084f373

📥 Commits

Reviewing files that changed from the base of the PR and between a2a091c and 60d92a7.

📒 Files selected for processing (2)
  • crates/perry-stdlib/Cargo.toml
  • scripts/tokio_inventory.json
🚧 Files skipped from review as they are similar to previous changes (1)
  • crates/perry-stdlib/Cargo.toml

Included review availability: Your plan provides up to 8 included reviews per hour; 2 remain after this review.


📝 Walkthrough

Walkthrough

The module-name async-runtime selection now names MongoDB. A test checks that PostgreSQL and MySQL module names have no well-known binding row. Database comments and inventory records describe the removed wrappers and the remaining MongoDB wrapper.

Changes

Database binding and runtime selection

Layer / File(s) Summary
Database binding status
crates/perry-db-turnloop/src/lib.rs, crates/perry-stdlib/Cargo.toml, crates/perry/src/commands/stdlib_features.rs, scripts/tokio_inventory.json, changelog.d/11331-tokio-lane-l3-db-wrappers.md
Comments and inventory describe the removed PostgreSQL and MySQL wrappers, the reserved slots, npm package feature selection, and MongoDB as the remaining database wrapper. The changelog records reported Tokio-linking results and unresolved database failures.
Compiler runtime selection and validation
crates/perry/src/commands/compile/optimized_libs/driver.rs, crates/perry/src/commands/compile/optimized_libs/freshness.rs, crates/perry/src/commands/compile/well_known.rs
The module-name async-runtime selection names MongoDB. A test checks that pg, mysql2, mysql2/promise, and node:pg have no well-known binding row. Updated comments use HTTP aliases and describe the removed database wrapper paths.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Refactor

Merge Risk: ⚪ Minimal · up to 60d92

The selector cleanup does not change pg or mysql compilation behavior, so no actionable merge-blocking risk is established.

Architecture Summary

Architecture risk: 🔵 Low · up to 60d92

The change affects 3 systems.

Changed systems: crates, changelog.d, scripts

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — crates (service) was modified; 6 changed files map to changed impact.
  • observed — changelog.d (service) was modified; 1 changed file maps to changed impact.
  • observed — scripts (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in changelog.d/11331-tokio-lane-l3-db-wrappers.md: The changelog describes the removed pg and mysql2 Tokio rule, the test guarding against restoring their well-known rows, reported Tokio-linking results for optimized and PERRY_NO_AUTO_OPTIMIZE builds, and database failures that remain unresolved.
  • observed — Modified behavior in crates/perry-db-turnloop/src/lib.rs: Updated the PG and MySQL slot comments to describe the removed bindings and their replacements; PG remains reserved, and the MySQL reservation is stated to preserve numbering and avoid stale-archive collisions.
  • observed — Modified behavior in crates/perry/src/commands/compile/optimized_libs/driver.rs: Updated the archive-alias example from mysql2 and mysql2/promise to http, https, and http2; the uniqueness behavior is unchanged.
  • observed — Modified behavior in crates/perry/src/commands/compile/optimized_libs/driver.rs: The module-name async-runtime selection now applies only to mongodb. The removed pg, mysql2, and mysql2/promise cases had no well-known binding rows, so they could not reach this branch and instead rely on feature selection for their npm packages’ net/tls imports.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: removal of the dead pg/mysql2 async-runtime rule. It is specific and related to the reported Tokio-linkage outcome.
Description check ✅ Passed The description is detailed and covers the summary, concrete changes, test evidence, known failures, and untested areas. It does not use the template headings or include an explicit Related issue or C…
Docstring Coverage ✅ Passed Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 5 files. (2 skipped: 2 u…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@proggeramlug

Copy link
Copy Markdown
Contributor Author

Ready to merge once CI is clean (tokio lane L3). The pg/mysql2 wrappers were already deleted (#10677/#10680); this removes the dead driver rule that still listed them for async-runtime (they have no well-known row, so the arm never ran) and fixes stale docs and comments. With #11277, pg-only and mysql2-only programs link 0 tokio (from 16 members / ~52 strings). A new test guards against pg or mysql2 getting a well-known row again.

Ralph Küpper added 2 commits September 25, 2026 18:30
The auto-optimize flip loop selected perry-stdlib's tokio async-runtime
for pg / mysql2 by module name. Their wrappers were removed (#10677,
continues past them at lookup_well_known and the rule was unreachable;
their tokio decline paths went with the wrappers. Keep the rule for
mongodb only, pin the no-row fact in a test, and fix the stale prose.
@proggeramlug

Copy link
Copy Markdown
Contributor Author

Merge queue: rebased onto main after #11308 (lane L2b) landed. The only conflict was scripts/tokio_inventory.json's async-runtime prose. It now states both lanes' facts: L2b's http pumps are off tokio, leaving mongodb the only bundled wrapper; and L3 deleted the dead pg/mysql2 module-name rule, so mongodb's decline path is the last B item. Counts were re-measured with --update. The code is unchanged from the CI-green head. Stacked on main with #11325: lint, a strict workspace compile and the perry suite (1203/0) are clean. Merging per the owner's instruction for rebased, previously green PRs.

@proggeramlug
proggeramlug merged commit 441d2ab into main Sep 25, 2026
50 of 51 checks passed
@proggeramlug
proggeramlug deleted the tokio-laneL3-db-wrappers branch September 25, 2026 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant