Skip to content

Fail open in consumer_has_unread when a consumer has no cursor rows - #429

Open
sergeivad wants to merge 1 commit into
ClickHouse:mainfrom
sergeivad:sergeivad/gate-fail-open-without-cursors
Open

sergeivad wants to merge 1 commit into
ClickHouse:mainfrom
sergeivad:sergeivad/gate-fail-open-without-cursors

Conversation

@sergeivad

Copy link
Copy Markdown

The bug

A MessagesGate configured without an explicit source list reads consumer_cursors only. The job it gates is the very job whose poll creates the first cursor row — so with zero rows the gate can never open, and the job never runs to create them.

A consumer reaches that state in two ordinary ways:

  • a fresh install, before the job has ever run;
  • any time cleanup_expired_consumer_cursors deletes every row after a quiet stretch longer than the cursor TTL.

The failure is silent: the job is simply skipped forever, and nothing in the logs says why.

The fix

With zero cursor rows, report unread whenever any messages exist at all. The job then runs once, its poll seeds cursors at MAX(rowid), and the check reverts to the normal cursor-based path.

Backlog is still skipped — "new consumers see only future messages" is delivery semantics and is unchanged. Consumers that already have at least one cursor row keep ignoring untracked sources exactly as before, so this only affects the state that was previously unrecoverable.

Tests

New coverage in tests/test_cron_gates.py and tests/test_db.py for both the zero-row and the established-consumer paths. Full suite green on top of current main.

A no-sources MessagesGate reads consumer_cursors only, and the job it
blocks is the very job whose poll creates the first cursor row — so with
zero rows the gate could never open. A consumer reaches that state on a
fresh install, and again whenever cleanup_expired_consumer_cursors
deletes every row after a quiet stretch longer than the cursor TTL.

With zero cursor rows, report unread whenever any messages exist at all:
the job runs once, its poll seeds cursors at MAX(rowid) (backlog still
skipped — "new consumers see only future messages" is delivery
semantics and unchanged), and the check reverts to cursor-based.
Established consumers (>=1 cursor row) still ignore untracked sources.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@CLAassistant

CLAassistant commented Sep 7, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@nerveai

nerveai commented Oct 21, 2026

Copy link
Copy Markdown

Nerve PR review · ✅ Ready to merge

This closes the gap that c0d9a1e left. That commit made the no-sources messages gate read expired cursors. But the 03:00 cleanup (nerve/cron/service.py:335) deletes those rows within a day of expiry, and a fresh install never has any. In both cases consumer_has_unread returned False forever, and nothing on the gated path seeds a row. The fix is narrow and sits in the right layer: it is read-only, the gate stays side-effect free, and consumers with at least one row behave exactly as before. It is a real fix for anyone gating an inbox job without listing sources. The branch is 543 commits behind but merges cleanly into main and into #537, whose forecast inherits it. CI has not run yet (fork PR).

Completeness: DB-level and gate-level tests cover four cases: fresh install, empty DB, after cleanup, and an established consumer. The fresh-install and after-cleanup cases would fail on main. docs/cron.md:246 still says "already tracks" and should mention the zero-row case.

Worth considering:

  • The run the gate opens still delivers nothing. poll_all_sources (nerve/agent/tools/handlers/sources.py:219) calls get_consumer_cursor, which re-seeds the cursor at MAX(rowid) (nerve/db/sources.py:351), so the mail that opened the gate is skipped. Main already behaves this way on its expired-row path, so this is not a regression. If the goal is that a quiet inbox's next message gets processed, a follow-up could resume from the expired row's cursor_seq, or stop deleting rows.

Checked: full suite in an offline sandbox at ec7a330: 3390 passed; read consumer_has_unread, get_consumer_cursor, poll_all_sources, the cleanup job, and a 3-way merge with #537 · automated review by nerveai — mention @nerveai to ask a follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants