Skip to content

limits_concurrency on_conflict: :discard looking only into running jobs and not blocked jobs #804

Description

@MRigal

In the uncommon case that (because of whatever error, possibly ProcessPrunedError), a job is not running anymore but the Semaphore hasn't expired yet, SolidQueue is piling up new jobs in "Blocked", thus not fulfilling the on_conflict: discard promise.

I see two possible solutions:

  1. Consider checking if a semaphore is blocking for the key
  2. Check if a job is "running" not only by looking if there is a job in running or ready but also possibly in blocked

It's slightly an edge case, but if the cron is set up to a quite high frequency, it can pile up quite a bit in the few hours the semaphore hasn't expired.

NB: this happened on SolidQueue 1.3.1. We want to update soon and I have not seen in the changelog what could have fixed it "en passant"

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions