concurrent.interpreters.Queue.get() and Queue.put() mishandle their
timeout argument in three related ways.
-
The value is converted with int(timeout), which truncates a floating-point
timeout to whole seconds. A timeout in the interval [0, 1) becomes a
non-blocking call, and e.g. timeout=1.9 waits for only about one second.
queue.Queue, which this queue is meant to be compatible with, accepts a
floating-point number of seconds.
-
Because of the same int() conversion, a small negative float such as
timeout=-0.5 is truncated to 0 and passes the timeout < 0 check
instead of raising ValueError.
-
The deadline is computed with time.time() (the wall clock), while
queue.Queue uses time.monotonic(). The timeout can therefore over- or
under-wait if the system clock is adjusted (NTP step, manual change) during
the call.
Reproducer (3.14.5)
import time
from concurrent.interpreters import create_queue
q = create_queue(maxsize=1)
q.put(b"x") # fill it
start = time.perf_counter()
try:
q.put(b"y", timeout=0.5) # expected: block ~0.5s, then QueueFull
except Exception as exc:
print(type(exc).__name__, f"{(time.perf_counter() - start) * 1000:.1f} ms")
# -> QueueFull 0.0 ms (expected ~500 ms)
q2 = create_queue()
start = time.perf_counter()
try:
q2.get(timeout=0.9) # expected: block ~0.9s, then QueueEmpty
except Exception as exc:
print(type(exc).__name__, f"{(time.perf_counter() - start) * 1000:.1f} ms")
# -> QueueEmpty 0.0 ms (expected ~900 ms)
For comparison, queue.Queue().get(timeout=0.5) blocks for about 500 ms.
The fix is to use the timeout value as given, reject a negative or NaN timeout
with ValueError, and base the deadline on time.monotonic().
Linked PRs
concurrent.interpreters.Queue.get()andQueue.put()mishandle theirtimeoutargument in three related ways.The value is converted with
int(timeout), which truncates a floating-pointtimeout to whole seconds. A timeout in the interval
[0, 1)becomes anon-blocking call, and e.g.
timeout=1.9waits for only about one second.queue.Queue, which this queue is meant to be compatible with, accepts afloating-point number of seconds.
Because of the same
int()conversion, a small negative float such astimeout=-0.5is truncated to0and passes thetimeout < 0checkinstead of raising
ValueError.The deadline is computed with
time.time()(the wall clock), whilequeue.Queueusestime.monotonic(). The timeout can therefore over- orunder-wait if the system clock is adjusted (NTP step, manual change) during
the call.
Reproducer (3.14.5)
For comparison,
queue.Queue().get(timeout=0.5)blocks for about 500 ms.The fix is to use the timeout value as given, reject a negative or NaN timeout
with
ValueError, and base the deadline ontime.monotonic().Linked PRs