The freeze occurs after the running execution queue processing and before the queued execution queue processing.
https://shopify.slack.com/archives/C0BR60F2FK2/p1786731801539789
08:33:54.431 Before processing running container
08:33:54.431 Container execution remains in ContainerStatus.RUNNING state.
08:33:54.431 After processing running container execution (duration: 41.29431ms)
08:51:40.895 Before processing queued execution
There only things that the Orchestrator code does in that interval: Sleep 1 second and query a single next queued execution.
Logically, the query is the only culprit. Usually, it takes 0 time. But network is always unpredictable. It could be a connection just hanging for X minutes.
The freeze occurs after the running execution queue processing and before the queued execution queue processing.
https://shopify.slack.com/archives/C0BR60F2FK2/p1786731801539789
There only things that the Orchestrator code does in that interval: Sleep 1 second and query a single next queued execution.
Logically, the query is the only culprit. Usually, it takes 0 time. But network is always unpredictable. It could be a connection just hanging for X minutes.