Repository navigation
fix(connection): don't block in set_query_timeout() while a query runs - #67
Conversation
|
Review (correctness, CI green, no local build) — logic is sound, no race/deadlock/GIL blocker. Two majors before ship:
|
|
Thanks — both in faf7159, on top.
|
The engine applies the query timeout and the per-query thread limit under the same per-connection lock that a running query holds for its whole duration. Connection.set_query_timeout() and set_max_threads_for_exec() called the engine right away, so with a query running on the connection they blocked until it finished. On the pybind backend they also held the GIL while waiting, which froze every Python thread, including an asyncio event loop that called AsyncConnection.set_query_timeout(). Both setters now record the value and return immediately. The value is applied before the next execute(), query_as_arrow() or Arrow-table call on that connection, by one thread at a time so that the latest value wins. The pybind bindings of the two native setters release the GIL, so a thread that does wait for the lock does not stall other threads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… value set_query_timeout() and set_max_threads_for_exec() now check the value before queueing it: an integer (TypeError otherwise) between 0 and 2**64 - 1 (ValueError otherwise). Only the native call is deferred to the next query. Before, a bad value surfaced on the next execute() on the pybind backend (TypeError from the uint64 conversion), and the C-API backend wrapped negative values into huge ones. Negative values now raise ValueError on both backends. The connection no longer keeps its own copy of the timeout: the C-API short-circuit reads the backend connection's value, which is set when the native setter runs, so both backends see the timeout from the same point. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3c14af7 to
6a35191
Compare
|
Non-blocking follow-up nits (post-rebase verified, content-identical to the merge — no action required before merge):
|
|
Thanks. 1 and 2 go in a follow-up PR after this merges, so the head you verified stays as is.
|
This work was produced with the help of language models.
The engine applies the query timeout and per-query thread limit under the mutex a running query holds for its whole duration, so
Connection.set_query_timeout()andset_max_threads_for_exec()blocked until the running query finished, and the pybind bindings held the GIL meanwhile, freezing every Python thread:AsyncConnection.set_query_timeout()with 4 long queries in flight froze the event loop for 12-13 s (CPython 3.13/3.14).Fix. Both setters record the value and return immediately; it is applied before the connection's next
execute(),query_as_arrow()or Arrow-table call, latest value wins. The two pybind bindings release the GIL, so a thread that does wait no longer stalls the others.Tests. Two new tests on a fresh connection: the setters return while a query runs (fails on
main: 6.05 s), and applying a pending setting while a query runs does not stall other threads (fails with the old bindings: 6.4 s stall). Both pass here on pybind and C-API. Full suites: no new failures.Fixes #63