Skip to content

perf(storage): allow a small SQLite connection pool - #354

Draft
tsan88 wants to merge 1 commit into
buggregator:masterfrom
tsan88:perf/sqlite-connection-pool
Draft

tsan88 wants to merge 1 commit into
buggregator:masterfrom
tsan88:perf/sqlite-connection-pool

Conversation

@tsan88

@tsan88 tsan88 commented Sep 4, 2026

Copy link
Copy Markdown

SetMaxOpenConns(1) is commented "SQLite single-writer", which is true for writing only. In WAL mode readers block neither each other nor the writer, but one shared connection puts ingestion and UI queries in the same queue: a heavy list request stalls incoming events, and a burst of events stalls the UI.

Changes

  • storage.OpenPooled(dsn, n); Open(dsn) stays as a single-connection wrapper, so existing callers are unaffected.
  • Pool size from database.max_open_conns / DATABASE_MAX_OPEN_CONNS, default 4.
  • Two DSN parameters are injected when a pool is requested and they are absent:
    • busy_timeout — otherwise a competing write fails instead of waiting;
    • _txlock=immediate — otherwise busy_timeout does not apply to writes at all, because a deferred transaction starts as a reader and SQLite refuses the lock upgrade immediately with SQLITE_BUSY (waiting would deadlock two readers that both want to write).

That second one was not theoretical: a four-connection pool without it produced upsert sentry_traces: database is locked (5) (SQLITE_BUSY) on our live stream within minutes. Every transaction in this code base is a writing one, so taking the write lock upfront costs nothing.

One caveat worth documenting: cache_size is per connection, so on a memory-constrained host the DSN value should be divided by the pool size. On our 1 GB host we went from cache_size(-65536) with one connection to cache_size(-16384) with four, and RSS dropped from 197 MB to 90 MB.

Testing

go vet ./..., go test ./... and go build pass. Added TestOpenPooled_ConcurrentWritesAndReads (4 writers × 25 writes with interleaved reads through the same pool — this is the test that fails with SQLITE_BUSY without _txlock=immediate) and TestOpenPooled_SingleConnection. Running in production on our instance with max_open_conns: 4 and zero busy errors since.

SetMaxOpenConns(1) is only required for writing. In WAL mode readers block
neither each other nor the writer, but one shared connection puts ingestion and
UI queries in the same queue: a heavy list request stalls incoming events, and a
burst of events stalls the UI.

storage.OpenPooled(dsn, n) keeps Open(dsn) as a single-connection wrapper, so
nothing changes for existing callers, and the pool size comes from
database.max_open_conns / DATABASE_MAX_OPEN_CONNS (default 4).

Two DSN parameters are required for a pool to be correct, and are injected when
absent:
  - busy_timeout — otherwise a competing write fails instead of waiting;
  - _txlock=immediate — otherwise busy_timeout does not apply to writes at all,
    because a deferred transaction starts as a reader and SQLite refuses the
    lock upgrade immediately with SQLITE_BUSY. This was not theoretical: a
    four-connection pool without it produced "upsert sentry_traces: database is
    locked (5)" on a live stream within minutes.

Note that a pool multiplies cache_size: it is per connection, so on a
memory-constrained host the DSN value should be divided by the pool size.

Measured on our instance (≈40k events/day, 1 GB RAM): with the pool the UI stops
waiting behind ingestion and the service RSS stays flat.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant