testdrive: bound the prepare by the query timeout

[bosconi]

Motivation

QAR-167. On release-qualification#1373, Checks 0dt upgrade to a bumped version ran SELECT ... FROM alter_index_source in AlterIndex.validate(), hit the 300 s platform-checks budget (query timed out) at 23:41 UTC, printed one rows didn't match; sleeping to see if dataflow catches up line, and then produced nothing at all until Buildkite cancelled the step at 02:51.

The retry loop in run_sql is bounded by max_duration(state.timeout), so it was not the loop. In try_run_sql, only query_prepared(...) is inside tokio::time::timeout(state.timeout, ...); the pgclient.prepare(query) before it is not. Dropping a timed-out query future does not stop the statement on the server, and tokio-postgres serializes statements on one connection, so the retry's prepare waits behind the stuck statement for as long as the server keeps it. On a dataflow that never produces, that is until the step is killed.

Description

Wrap the prepare in the same timeout. A connection held by a stuck statement now fails the attempt within the budget, the retry loop runs out of max_duration and reports the failure against the query, and the step ends in minutes instead of hours. The timeout message names the phase (preparing query timed out) so it is distinguishable from query timed out in a log.

This does not cancel the stuck statement server-side; testdrive only does that for background queries (cancel_background_query). With platform-checks' statement_timeout='300s' the server ends it on its own, and for other compositions the bounded failure is still the right outcome: the underlying problem, a query that never returns, is what the test should report.

Why the underlying alter_index_source query never returned after ZeroDowntimeBumpedVersion is the product half of QAR-167 and is not addressed here.

Verification

cargo check and cargo clippy -p mz-testdrive -D warnings clean. No unit-test seam exists for try_run_sql (it drives a live pgwire client); the behavior change is a deadline on an existing await, and every testdrive-based CI job exercises the happy path.

🤖 Generated with Claude Code