Skip to content

[14.0][FW] queue_job: use advisory lock instead of queue_job_lock table - #993

Open
florentx wants to merge 46 commits into
OCA:14.0from
camptocamp:14.0-queue_job-lock
Open

florentx wants to merge 46 commits into
OCA:14.0from
camptocamp:14.0-queue_job-lock

Conversation

@florentx

Copy link
Copy Markdown
Contributor

florentx and others added 30 commits September 22, 2026 11:13
Without this fix, if db_replica_host is set, Odoo might pass a readonly
database cursor and the FOR UPDATE in the method would fail.
Without this, we leak connections to Databases that don't have queue_job
installed.
Without this we risk connection leaks in case of exceptions in the
constructor.
…eue jobs in timeout

[IMP] queue_job: increment 'retry' when re-queuing job that have been killed
'name' attributes for odoo's exception has been deprecated and produces a warning message. This commit fixes the behavior
datetime.datetime.utcnow() is now deprecated and should be replaced by datetime.datetime.now() (optional TZ parameter).
As the original _odoo_now() doesn't contain the Timezone, the parameter datetime.UTC is not added into this improvement
When a SerializationFailure occurs when updating the state of dependent
jobs, the cursor is not usable anymore so the retry failed with `current transaction is aborted`. A savepoint fixes that.
In this commit we cleanly separate the job acquisition (i.e. verifying the job is in the exepected state, marking it started and locking it) from job execution.

We also avoid trying to start the job if it is already locked by using SKIP LOCKED
and exiting early. Indeed in such situations the job is likely already being handled by another worker so there is no point trying to start it, so we exit early
and let it be handled either by the other worker or the dead job requeuer.
Extract the logic to run one job out of the /queue_job/runjob route.
Towards making this logic reusable in other job executors.
Since we are not going to delete records nor modify foreign keys,
we can take a weaker lock.
This would release the job lock, causing spurious restarts by the dead jobs requeuer.
There is no parameter auto_spec to mock.patch. This is likely a typo.

Running the tests with Python 3.12 give a RuntimeError
```
  File "/usr/local/lib/python3.12/unittest/mock.py", line 1311, in __init__
    _check_spec_arg_typos(kwargs)
  File "/usr/local/lib/python3.12/unittest/mock.py", line 1287, in _check_spec_arg_typos
    raise RuntimeError(
RuntimeError: 'auto_spec' might be a typo.
```
Every time a job is done, even if it is not part of a graph, it runs a
query to look for dependents to enqueue. Storing the dependent uuids in
the "dependencies" field was on purpose to know that we have no further
jobs in the graph and that we can skip the check entirely and have no
overhead in this case.

It looks like an oversight, we can add the missing condition.
In this case, when a module adds a value in context, the tests fail
It is forbidden to commit inside a job, because it releases the job lock
and can cause it to start again, while still being run, by the dead jobs
requeuer. For some use cases, it may actually be legitimate, or at least
be needed in the short term before actual updates in the code.

A new option on the job function, false by default, allow to run the job
in a new transaction, at the cost of an additional connection +
transaction overhead.

Related to OCA#889
As the controller changes env on Job instances.
GuillemCForgeFlow and others added 14 commits September 22, 2026 17:45
…ethod

We add the `_subscribe_job_creator` method in `queue.job` which will return True for the cases where we want to subscribe the job creator, False otherwise.
Jobs are always created through sudo() in the Job class, and always
deleted by the cron, never by users.

See discussion on OCA#802 (comment)
…rts patches

Otherwise monkey-patches could be added twice when instantiating new test classes.
(cherry picked from commit 47b8c48)
[ported to 18.0: isinstance union syntax for ruff UP038]
Add a new "paused" option on channel configuration.
A paused channel is equivalent to a channel with no capacity:
it yields no job and also blocks all its subchannels.

It still needs a restart of the jobrunner, so it does not allow dynamic
pause/resume, but hot reload may be added later in OCA#765.

It still is an improvement over setting the capacity at 0, because it
keeps the capacity in the configuration (removing the option in the
config restores the initial capacity).

Also, it does not require changing the channel of a job, which is an
issue when resuming paused jobs (they should go back to their former
channel to be resumed with their expected properties related to
capacity/sequential/throttle).
Allows to execute a model function when the job fails and will
not be retried.

[REF] queue_job: Move on fail definition to job function

Rename on_fail_hook to on_fail
Job.on_fail_method_name was only set as an instance attribute when
the job function configured on_fail_method, so on_fail() raised
AttributeError for every job that didn't opt into it.
This adds `capacity_default` and `sequential_default` options
that set `capacity` and `sequential` for autocreated subchannels.
It also allows non-root channel configurations to omit capacity.
Logging the usage of queue_job__no_delay makes sense only in 'production mode'.
We have tons of tests explicitly bypassing jobs and that generates a lot of useless entries in the logs.
@oca-cla-bot

oca-cla-bot Bot commented Sep 23, 2026

Copy link
Copy Markdown

The following commit authors need to sign the Contributor License Agreement:

CLA not signed

@OCA-git-bot

Copy link
Copy Markdown
Contributor

Hi @guewen,
some modules you are maintaining are being modified, check this out!

@OCA-git-bot OCA-git-bot added mod:queue_job Module queue_job mod:test_queue_job Module test_queue_job series:14.0 mod:test_base_import_async Module test_base_import_async labels Sep 23, 2026
@florentx florentx changed the title [14.0] [FW] queue_job: use advisory lock instead of queue_job_lock table [14.0][FW] queue_job: use advisory lock instead of queue_job_lock table Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mod:queue_job Module queue_job mod:test_base_import_async Module test_base_import_async mod:test_queue_job Module test_queue_job series:14.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.