Skip to content

feat(trilean-sql): compile a real SQLite dialect alongside PostgreSQL - #20

Merged
Mearman merged 5 commits into
mainfrom
feat/sqlite-dialect
Sep 3, 2026
Merged

feat(trilean-sql): compile a real SQLite dialect alongside PostgreSQL#20
Mearman merged 5 commits into
mainfrom
feat/sqlite-dialect

Conversation

@Mearman

@Mearman Mearman commented Sep 3, 2026

Copy link
Copy Markdown
Member

Adds a real SQLite dialect to trilean-sql, so SqlCompileOptions.dialect becomes "postgres" | "sqlite" rather than a single literal. D1 is SQLite, which is the domain that motivated this package existing, so this is the second dialect the field's doc comment was always anticipating.

What actually differs between the two

Three things, and nothing else, which is why this is a small DialectConfig record in options.ts rather than a module per dialect or a Dialect interface with a method per node kind:

  • matches/notMatches emit REGEXP/NOT REGEXP instead of ~/!~
  • placeholders are a bare ? instead of $N::type — SQLite binds by position in emission order and has no type to cast to
  • the bare NULL in an empty memberOf loses its ::boolean annotation, since SQLite has no boolean type to annotate

Everything else the compiler emits is ANSI-standard and identical in both engines: the connectives, the six comparison operators, =/<>, IN/NOT IN, IS NOT NULL, and double-quoted identifiers with an embedded quote doubled. Branching anywhere else would be a branch that can never change the output.

PostgreSQL's compiled output is unchanged, byte for byte. Beyond the existing unit assertions (which assert exact SQL text) and the existing container-backed postgres.test.ts still passing untouched, this was checked differentially: 40,000 randomly generated trees compiled through main's built dist and this branch's, comparing both the compiled {sql, params} and every findUnpushableNodeKind result, with and without options — 80,000 comparisons, zero differences.

Why the guard refuses the same set for both

The refusal set carries over from PostgreSQL unchanged in structure, and that inheritance is the part that needed evidence rather than assertion — it would be worth nothing if SQLite happened to agree with trilean where PostgreSQL does not. Each one was measured against a real better-sqlite3 connection, and test/integration/sqlite.test.ts keeps measuring it:

  • Text under compare. A TEXT-affinity column compared against the number 5 is compared as text: '9' > 5 is true, '10' > 5 is not. SQLite's coercion is worse here than PostgreSQL's, not better — no error, no warning, just the wrong rows.
  • Ordered booleans. SQLite stores them as the integers 0 and 1 and orders them as integers, so gt answers definitely where trilean has no ordering at all.
  • Cross-kind comparison. 'abc' > 5, with no column or affinity involved, answers true rather than erroring.
  • NaN — the same refusal for a genuinely different reason. SQLite has no NaN, and a driver binding one substitutes SQL NULL: typeof(?) bound with NaN answers 'null', and NaN = NaN is therefore indeterminate rather than definitely false. That looks like agreement until you negate it, at which point trilean's definite true matches every row and SQLite's NULL still matches none. So the SQLite refusal carries its own wording rather than PostgreSQL's "NaN is equal to itself and greater than every other double", which would be simply false about SQLite.

The refusal structure is shared: guard.ts keeps one walk and one refusal per divergence, parameterised by a DialectDivergence record supplying only the reason text. There is no second copy of the walk.

Infinities are still deliberately not refused, and that was checked rather than assumed: better-sqlite3 binds them as real, Inf = Inf is 1, and ordering against finite values matches trilean's.

What a SQLite caller has to supply

Two things, both of which fail loudly rather than silently, and both documented in the README:

  • A REGEXP function, if the tree uses matches/notMatches. SQLite reserves REGEXP as syntax for a regexp(pattern, value) function it does not itself provide. An unregistered one is a query error — no such function: REGEXP — never a fragment that quietly matches nothing, which is what makes leaving it to the caller acceptable rather than a hole in the guarantee. The README gives the concrete db.function("regexp", ...) registration, including the two load-bearing details: return null for a NULL argument (SQLite does not propagate NULL through a user function on its own, and one answering 0 would make NOT REGEXP true for a row whose value is unknown — exactly the two-valued collapse this package exists to avoid), and return 1/0 rather than a JS boolean, which better-sqlite3 rejects from a user function.
  • Booleans bound as 0/1. params carries the tree's own literals unchanged in every dialect, and better-sqlite3 rejects a JS boolean outright.

Instants are the caveat to be deliberate about: SQLite has no timestamp type, so an instantLiteral is compared as text. Offset-bearing ISO-8601 in a single common offset sorts chronologically and compares correctly; mixed offsets do not. This is the SQLite counterpart of the existing PostgreSQL session-time-zone caveat, and the same advice ("pass offset-bearing ISO-8601") resolves both.

One fix on top of the dialect work

bbb4b58 fixes a defect the dialect dispatch introduced. Both entry points index a per-dialect table with options.dialect and neither checked the dialect has an entry, so a name outside the union — unreachable from TypeScript source naming a dialect literally, but entirely reachable from a dialect read from configuration and asserted into the union at the boundary — surfaced as TypeError: Cannot read properties of undefined (reading 'matches') from compilePredicateNode, and, worse, as the answer undefined from findUnpushableNodeKind for a tree the compiler then failed on. Reporting a tree as pushable is a promise that it will compile; under a dialect that does not exist it cannot. Both now refuse by name with UnknownDialectError, which carries the offending name and the implemented ones and takes its list from DIALECT_CONFIG's own keys, so implementing a dialect cannot leave the check behind. Before this branch there was nothing to get wrong here — main ignored the field entirely — so it is a regression this branch introduced rather than a pre-existing gap.

Out of scope, deliberately

No third dialect. No ORDER BY/JOIN/collection-quantifier support — some/every/fold stay refused for both. No bundled or auto-registered regexp function; that stays the caller's documented responsibility.

Verification

Run from a clean clone of this branch, not from cached state:

  • pnpm build, pnpm typecheck, pnpm lint clean
  • pnpm test — 89 unit tests in trilean-sql (the pre-existing PostgreSQL string assertions untouched and passing), 444 in trilean
  • pnpm test:integration — 42 in trilean-sql: postgres.test.ts unchanged and green against a real container, sqlite.test.ts green in memory
  • the differential main-vs-branch PostgreSQL comparison described above
  • a randomised parity fuzz outside the test suite: 3,000 generated trees compiled for SQLite, executed as real WHERE clauses against a seeded better-sqlite3 table, and compared row for row against evaluatePredicate over the same rows — zero disagreements
  • injection re-checked by execution rather than string assertion: a hostile value stays a bound parameter, and a hostile columnFor name (both an OR-injection and a statement-break attempt) becomes one quoted identifier the engine rejects as missing, with the table still standing

.tool-versions pins Node 22, which is what CI uses, so better-sqlite3 resolves a prebuilt binary rather than compiling from source. Its install script is allowed in pnpm-workspace.yaml for that reason — it is a native addon, and refusing the script would leave the package installed and unloadable.

@Mearman
Mearman marked this pull request as ready for review September 3, 2026 15:31
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review Completed 2026-09-03T15:38:47.622518Z bbb4b58 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

`SqlCompileOptions.dialect` widens from the literal `"postgres"` to
`SqlDialect`, and the three things that genuinely differ between the two
engines move behind a `DialectConfig` record read once per compilation:
the regular-expression operator `matches`/`notMatches` emits (`~`/`!~`
against `REGEXP`/`NOT REGEXP`), the placeholder form (`$N::type` against
a bare `?`), and the boolean annotation on the bare `NULL` an empty
`memberOf` compiles to (`::boolean` against nothing, SQLite having no
boolean type to annotate).

Everything else the compiler emits is ANSI-standard and stays
unbranched: the six comparison operators, `=`/`<>`, the connectives,
`IN`/`NOT IN`, `IS NOT NULL`, and double-quoted identifiers with an
embedded quote doubled.
The PostgreSQL dialect's compiled output is unchanged, byte for byte.

Which nodes the guard refuses is unchanged too, and identical for both
dialects: SQLite's type affinity produces the same silent definite
answers where trilean returns wrong-type, so every refusal carries over.
Only the reason text varies, and the NaN refusal needed its own wording
rather than PostgreSQL's reused: SQLite has no NaN at all, and a driver
binding one substitutes SQL NULL, so `NaN = NaN` is indeterminate there
rather than definitely false -- a divergence in the opposite direction
from PostgreSQL's NaN-equals-itself.
The engine the SQLite integration suite runs compiled fragments against,
in memory and in process, so that suite needs no Docker daemon the way
the PostgreSQL one does.
Pinned to 12.x rather than 13.x deliberately: 13 requires Node 22 or
newer, which is narrower than the `>=20` this package declares it
supports, and 12.11.1's own engines range still covers that floor.

It is a native addon, so its install script has to be allowed in
`pnpm-workspace.yaml` -- that script is what fetches the binding the
module cannot load without, unlike the three refused entries beside it,
whose scripts are optional to how this workspace uses them.
The same dual-execution parity harness the PostgreSQL suite uses, over
the same fixture: compile a tree, run the fragment as a real `WHERE`
clause, separately evaluate the same tree through `evaluatePredicate`
once per row, and assert the two agree on which rows match and which do
not.

It is more than a second run of an already-proven suite because SQLite
reaches its three-valued behaviour from a different starting point --
no boolean type, no timestamp type, no NaN, and affinity that coerces
where PostgreSQL rejects. Three cases measure the divergences the
guard's inherited refusals exist to prevent rather than only asserting
that each fires: NaN binding as SQL NULL (so the negated comparison
matches nothing where trilean matches every row), a TEXT-affinity column
compared lexicographically against a number (`'9' > 5` true, `'10' > 5`
false), and a boolean ordered as the integer it is stored as.

Two properties of the connection are load-bearing and asserted rather
than assumed: an unregistered `REGEXP` fails as a query error naming the
missing function rather than answering wrongly, and a registered one
must return NULL for a NULL argument, since SQLite does not propagate
NULL through a user function and one answering 0 would make
`NOT REGEXP` true for a row whose value is unknown.
…ment

A Dialects section covering what differs (the two regex operators, the
uncast `?` placeholder, the missing boolean annotation) and what does
not, plus the two things a SQLite caller has to supply that a PostgreSQL
caller does not: a `regexp` function, and booleans bound as 0/1, since
drivers do not agree that a JS boolean is bindable at all.

The Regular expressions section now carries the concrete better-sqlite3
registration, with both of its non-obvious details spelled out -- return
NULL for a NULL argument or `NOT REGEXP` answers TRUE for a row whose
value is unknown, and return 1/0 rather than a boolean, which the driver
rejects from a user function. An unregistered `REGEXP` is a query error
naming the missing function, which is what puts this in the same class
as the regex-dialect and instant-parsing caveats already documented for
PostgreSQL rather than making it a hole in the compile-time guarantee.

Refusal now states that the whole refusal set applies to both dialects
and only the reason text varies, with the NaN entry describing each
engine's own mechanism.
Both public entry points index a per-dialect table with `options.dialect`,
and neither checked that the dialect has an entry.
A name outside the union reached `compilePredicateNode` as
`TypeError: Cannot read properties of undefined (reading 'matches')`,
naming an internal field rather than the dialect,
and reached `findUnpushableNodeKind` as the answer `undefined`
for a tree the compiler then failed on --
so the one question that function exists to answer,
whether a tree can be pushed down, was answered wrongly.

`SqlDialect` is a closed union, so this is unreachable
from TypeScript source naming a dialect literally.
It is reachable from a dialect read from configuration
and asserted into the union at the boundary,
which is how a dialect is realistically supplied,
and is the same class of input the package already
validates by name elsewhere: a `columnFor` result that is
not a usable identifier, and an unrecognised node kind.

`UnknownDialectError` joins the two existing error classes,
carrying the offending name and the implemented ones as fields,
and `DIALECT_CONFIG`'s own keys are the list,
so implementing a dialect cannot leave the check behind.
@Mearman
Mearman force-pushed the feat/sqlite-dialect branch from bbb4b58 to 3993184 Compare September 3, 2026 15:50
@Mearman
Mearman merged commit 563eb0a into main Sep 3, 2026
13 checks passed
@Mearman
Mearman deleted the feat/sqlite-dialect branch September 3, 2026 15:51
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 1.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant