Fix job list performance and SQLite pagination

[bgentry]

Listing completed jobs can scan the entire job table even though an index exists for this query. Earlier work explicitly aimed to use that index (#304). The finalized_at IS NULL / IS NOT NULL conditions may have been lost along the way when tweaking the job list API or moving this logic out of its original home in riverui.

When filtering by one state, use state = ... instead of state = ANY(...). For completed, cancelled, or discarded jobs sorted by finalized time, also add finalized_at IS NOT NULL. Both changes are needed to make the measured query fast. River UI benefits without any changes to its calls. This optimization skips queries with custom SQL, multiple states, and job deletion.

Why use = instead of ANY when filtering by one state?

A one-element ANY array selects the same rows as =, but PostgreSQL does not necessarily plan them the same way. There is also a distinction between state IN ('completed'), which PostgreSQL simplifies to state = 'completed', and state = ANY(ARRAY['completed']), which does not receive the same simplification in the measured PostgreSQL 17.11 plans.

Tom Lane explains the distinction in his response to PostgreSQL bug #17922. Equality lets the planner recognize that a column has one fixed value, making sorting by that column unnecessary. About applying that reasoning to ANY, he writes:

We don't make any attempt to make a similar deduction from =ANY clauses

His example involves a join and an ORDER BY, but the fixed-column reasoning also explains the behavior observed in River's query. River's existing index is:

CREATE INDEX river_job_state_and_finalized_at_index
ON river_job (state, finalized_at)
WHERE finalized_at IS NOT NULL;

The index orders entries by state first, then finalized time. With state = $2, the planner knows every matching row has the same state, so the remaining index order is useful for ORDER BY finalized_at DESC, id DESC. It can scan the index backward, sort jobs with equal timestamps by ID, and stop after enough jobs have been found. The measured plan calls this an Incremental Sort. With state = ANY($2), the measured plans instead scan the table and sort the matching jobs before applying the limit.

The explicit finalized_at IS NOT NULL condition solves a separate problem. PostgreSQL must recognize that the query only requests rows covered by the partial index. Although River's supported schemas enforce non-null finalized timestamps for finalized states, the measured planner does not infer the index condition from the state filter. Adding it explicitly makes the index eligible. This follows PostgreSQL's partial-index requirements.

Both changes are necessary for the measured query. Using the normal job-column projection, a parameterized limit of 100, and 888,000 jobs on local PostgreSQL 17.11:

State comparison	Explicit non-null condition	Custom plan	Generic plan
state = ANY($2)	No	102.426 ms	171.081 ms
state = ANY($2)	Yes	96.786 ms	109.454 ms
state = $2	No	94.184 ms	106.093 ms
state = $2	Yes	0.073 ms	0.062 ms

These were forced custom and generic prepared plans. A custom plan uses the supplied parameter values; a generic plan is reusable without depending on those values. In a generic plan, an array parameter could contain several states, whereas state = $2 still restricts the query to one state. The custom ANY plan was slow too, even though its array contained only completed. See PostgreSQL's prepared-plan documentation.

The successful plans read 103 rows through the existing index and perform a small incremental sort; the other three forms scan all 888,000 jobs. These are individual measurements, not averages. This does not mean ANY cannot use indexes or that replacing it always improves performance: the benefit here depends on one state, the index's column order, and the query's ordering and limit. Queries selecting multiple states retain ANY.

---

A separate commit fixes SQLite pagination skipping or repeating jobs. The SQLite driver now formats cursor timestamps consistently with stored timestamps, preventing incorrect comparisons.

Listing 100 completed jobs on a local PostgreSQL 17.11 database with 888,000 jobs:

Prepared plan	Before	After
Custom	102.426 ms	0.073 ms
Generic	171.081 ms	0.062 ms

The fixed query uses the existing index and reads 103 rows instead of scanning 888,000. These are individual measurements, not averages.