Skip to content

Commit 5cf73cd

Browse files
committed
fix(run-engine): stop requeued runs with a lapsed ttl being orphaned in the queue
A run triggered with a ttl registers a TTL entry that the first dequeue removes ("executing, not expired"). Nacking the run back onto the queue kept the original ttlExpiresAt in the rewritten message without re-registering that entry, so the next dequeue pass took the expired-TTL branch: it removed the run from every queue structure and deferred finalization to a TTL consumer that could never find it. The run then sat QUEUED in the database forever, invisible to dequeue, the TTL consumer, and queue repair. Two changes: - nackMessage drops ttlExpiresAt from the rewritten message. TTL only applies to runs that have never been dequeued, matching the existing includeTtl re-enqueue contract, so a requeued run is never expired or dropped by its original deadline. - The dequeue expired-TTL branches re-register the TTL entry instead of assuming it exists, so any message still carrying a lapsed ttlExpiresAt with no TTL entry (e.g. written before this fix) finalizes as EXPIRED instead of orphaning.
1 parent 2496a8a commit 5cf73cd

3 files changed

Lines changed: 559 additions & 3 deletions

File tree

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: fix
4+
---
5+
6+
Runs triggered with a `ttl` could get permanently stuck in the queued state if they started executing and were then requeued after a failure (for example a worker dying mid-run) once the TTL had already elapsed. Requeued runs now dequeue normally: a run's TTL only applies while it is waiting to start for the first time.

0 commit comments

Comments
 (0)