diff --git a/packages/trilean-sql/README.md b/packages/trilean-sql/README.md index 5f385bf..5be5e28 100644 --- a/packages/trilean-sql/README.md +++ b/packages/trilean-sql/README.md @@ -65,13 +65,13 @@ The fragment is a self-contained boolean expression with no `WHERE` keyword of i Returns `{ sql, params }`. Throws rather than approximating; see [Refusal](#refusal). -`options.dialect` is `"postgres"`. It is required rather than defaulted so that a second dialect is a new value here, not a change of behaviour for callers who never said which one they meant. +`options.dialect` is `"postgres"` or `"sqlite"`. It is required rather than defaulted so that a caller states the engine it is compiling for, instead of inheriting whichever one this package happened to implement first. See [Dialects](#dialects) for what differs between them; the tree, the refusals, and the three-valued guarantee do not. `options.columnFor(referenceKey)` maps a `reference` node's key onto `{ column, paramType? }`. Only string keys reach it — trilean permits any JSON value as a key, and a non-string one is refused before the call. Throwing from `columnFor` is how you reject a key you have no column for; the exception propagates out unchanged rather than being wrapped. It is memoised for the duration of one compilation, so it is called once per distinct key however many times that key appears. `column` may be qualified with dots (`"members.age"`); each segment is emitted as its own double-quoted identifier. Quoting is unconditional, which is what makes an arbitrary column name safe — an embedded quote is doubled, so a name carrying `"; DROP TABLE ...` becomes one inert identifier that simply does not exist. It also means the name is taken literally rather than case-folded, so return the column's real, case-exact name. -`paramType` declares the column's value kind: `"text"`, `"number"`, `"boolean"`, or `"timestamp"` (trilean's `instant`). It is optional and worth supplying — it does not change the emitted SQL, but it is the only thing that lets the compiler detect the operand-kind mismatches described under [Refusal](#refusal). Without it, a comparison PostgreSQL would coerce into a definite answer where trilean returns `indeterminate` compiles silently. +`paramType` declares the column's value kind: `"text"`, `"number"`, `"boolean"`, or `"timestamp"` (trilean's `instant`). It is optional and worth supplying — it does not change the emitted SQL, but it is the only thing that lets the compiler detect the operand-kind mismatches described under [Refusal](#refusal). Without it, a comparison the database would coerce into a definite answer where trilean returns `indeterminate` compiles silently. ### `findUnpushableNodeKind(node, options?)` @@ -92,22 +92,43 @@ Passing `options` widens the check: without them the walk is purely structural; ## What compiles -| Node | PostgreSQL | -| --- | --- | -| `and`, `or`, `not` | `AND`, `OR`, `NOT` | -| `allOf`, `anyOf` | n-ary `AND`, `OR`; empty operands become each connective's identity, `TRUE` and `FALSE`, matching the evaluator's own fold | -| `compare` | `>`, `>=`, `<`, `<=`, `=`, `<>` | -| `textCompare` | `=`, `<>`, and `~` / `!~` for `matches` / `notMatches` | -| `memberOf` | `IN` / `NOT IN`, one parameter per candidate | -| `exists` | `IS NOT NULL` | -| `reference` | the mapped column, as a quoted identifier | -| `textLiteral`, `numberLiteral`, `booleanLiteral`, `instantLiteral` | a bind parameter, cast to `text`, `double precision`, `boolean`, `timestamptz` | +| Node | PostgreSQL | SQLite | +| --- | --- | --- | +| `and`, `or`, `not` | `AND`, `OR`, `NOT` | same | +| `allOf`, `anyOf` | n-ary `AND`, `OR`; empty operands become each connective's identity, `TRUE` and `FALSE`, matching the evaluator's own fold | same | +| `compare` | `>`, `>=`, `<`, `<=`, `=`, `<>` | same | +| `textCompare` | `=`, `<>`, and `~` / `!~` for `matches` / `notMatches` | `=`, `<>`, and `REGEXP` / `NOT REGEXP` | +| `memberOf` | `IN` / `NOT IN`, one parameter per candidate | same | +| `exists` | `IS NOT NULL` | same | +| `reference` | the mapped column, as a quoted identifier | same | +| `textLiteral`, `numberLiteral`, `booleanLiteral`, `instantLiteral` | a bind parameter, cast to `text`, `double precision`, `boolean`, `timestamptz` | a bare `?`, uncast | -Every literal in the tree becomes a parameter. Nothing but structure, operators and quoted identifiers is ever written into the returned `sql`, so a literal's content cannot alter the statement. +Every literal in the tree becomes a parameter. Nothing but structure, operators and quoted identifiers is ever written into the returned `sql`, so a literal's content cannot alter the statement. Identifier quoting is the same in both: each dot-separated segment double-quoted with any embedded quote doubled, which is ANSI-standard and not a per-dialect rule. -Placeholders are always cast. That is not decoration: PostgreSQL rejects `$1 < $2` outright because it cannot determine either parameter's type, and casting each placeholder to the type its own literal kind implies is what makes a fragment's meaning independent of how a particular driver decided to infer an untyped parameter. `timestamptz` rather than `timestamp`, because trilean's `instant` is an ISO-8601 string that may carry an offset and parsing one as a naive timestamp would silently discard it. +PostgreSQL placeholders are always cast. That is not decoration: PostgreSQL rejects `$1 < $2` outright because it cannot determine either parameter's type, and casting each placeholder to the type its own literal kind implies is what makes a fragment's meaning independent of how a particular driver decided to infer an untyped parameter. `timestamptz` rather than `timestamp`, because trilean's `instant` is an ISO-8601 string that may carry an offset and parsing one as a naive timestamp would silently discard it. SQLite placeholders carry neither a number nor a cast, because there is nothing to write: parameters bind by the order they appear, and there is no type to annotate. It answers a comparison between two bare `?` from the bound values themselves. -An empty `memberOf` candidate list is worth a note, because `IN ()` is a syntax error and the two constants it is tempting to fold to are both wrong. An empty `in` is false and an empty `notIn` is true only once the operand itself is known, and both stay unknown while it is `NULL`. The compiled forms — `(x IS NULL AND NULL::boolean)` and `(x IS NOT NULL OR NULL::boolean)` — reproduce that exactly, which a bare `FALSE`/`TRUE` would not, most visibly under a surrounding `NOT`. +An empty `memberOf` candidate list is worth a note, because `IN ()` is a syntax error and the two constants it is tempting to fold to are both wrong. An empty `in` is false and an empty `notIn` is true only once the operand itself is known, and both stay unknown while it is `NULL`. The compiled forms — `(x IS NULL AND NULL::boolean)` and `(x IS NOT NULL OR NULL::boolean)`, and the same two without the cast under SQLite, which has no boolean type to annotate — reproduce that exactly, which a bare `FALSE`/`TRUE` would not, most visibly under a surrounding `NOT`. + +## Dialects + +The dialects differ in three places, and nowhere else. `matches`/`notMatches` compile to each engine's own regular-expression operator; placeholders are `$N::type` under PostgreSQL and a bare `?` under SQLite; and the bare `NULL` in an empty `memberOf` carries a `::boolean` annotation only where there is a boolean type to annotate. Everything else the compiler emits — the connectives, the six comparison operators, `=`/`<>`, `IN`/`NOT IN`, `IS NOT NULL`, quoted identifiers — is ANSI-standard and identical. + +```ts +const { sql, params } = compilePredicateNode(rule, { dialect: "sqlite", columnFor }); +// sql: (("age" >= ?) AND ("email" REGEXP ?)) +// params: [18, "@example[.]com$"] +``` + +What compiles, what is refused, and the row-for-row agreement with `evaluatePredicate` are the same in both. The refusals in particular are not PostgreSQL caution carried over untested: SQLite's type affinity produces the same silent definite answers where trilean returns `wrong-type`, and `test/integration/sqlite.test.ts` measures each one against a real connection. A `TEXT`-affinity column compared to the number `5` is compared *as text*, so `'9' > 5` is true there while `'10' > 5` is not; a boolean is an integer, so `gt` on one answers by integer ordering; and `'abc' > 5`, with no column involved at all, answers `true` rather than erroring. + +Two things a SQLite caller has to supply that a PostgreSQL caller does not, both of which fail loudly rather than silently: + +- **A `REGEXP` function**, if the tree uses `matches` or `notMatches`. See [Regular expressions](#regular-expressions). +- **Booleans bound as `0`/`1`.** SQLite has no boolean type, and drivers do not agree on whether a JS boolean is bindable at all — better-sqlite3 rejects one outright (*"SQLite3 can only bind numbers, strings, bigints, buffers, and null"*). `params` carries the tree's own literals unchanged in every dialect, so converting them is the binding caller's job: `params.map((v) => (typeof v === "boolean" ? Number(v) : v))`. + +A dialect this version does not implement is refused by name, from `compilePredicateNode` and `findUnpushableNodeKind` alike, with `UnknownDialectError`. `SqlDialect` is a closed union so TypeScript source cannot reach that, but a dialect read from configuration and asserted into the union at the boundary can, and reporting such a tree as pushable would promise a compilation that cannot happen. + +Instants are the one place to be deliberate rather than merely careful. SQLite has no timestamp type either, so an `instantLiteral` is compared as text against whatever text the column holds. Offset-bearing ISO-8601 in a single common offset (`2020-01-01T00:00:00Z`) sorts chronologically as a string and compares correctly; mixed offsets, or a format that is not ISO-8601, do not. This is the SQLite counterpart of the PostgreSQL session-time-zone caveat under [Refusal](#refusal), and the same advice resolves both. ## Refusal @@ -117,30 +138,48 @@ The check is an allow-list walk rather than a deny-list, so a node kind added to That guarantee is about the tree's structure, and it has two limits, both about the *content* of a string operand rather than any node's kind, and neither reachable by a walk over kinds. A `matches`/`notMatches` pattern is matched by the server in PostgreSQL's own regular-expression language, not ECMAScript's — see [Regular expressions](#regular-expressions), which is where the row-for-row claim actually stops. An `instantLiteral` is parsed by PostgreSQL rather than by `Date`, so one carrying no UTC offset is read in the *database session's* time zone where trilean reads it in the Node process's, and one PostgreSQL cannot parse at all raises a query error at execution time rather than this exception (trilean answers `indeterminate` for the same string). Pass instants as offset-bearing ISO-8601, which both read identically. -**Kinds this version does not translate.** `some`, `every`, `fold`: these range over a collection the caller's resolvers supply, which is not the query's row set. `lookup`, `call`, `delegate`, `treeReference`: each is resolved by something the database has no access to — the caller's resolvers, its function registry, an external system. `conditional`: not implemented here. `accumulator`: only meaningful inside a `reduce` fold. `arithmetic` and `negate`: these carry and combine units, and pushing them down would drop that dimensional analysis without saying so. `durationLiteral`: trilean compares durations by normalising both operands to milliseconds, with no column-level equivalent to normalise against. `complexLiteral`: PostgreSQL has no complex type. +Every refusal below applies to both dialects. What changes with the dialect is the `reason` text, which names the mechanism that actually applies to the engine you are compiling for — a `findUnpushableNodeKind` call given no `options` has no dialect to read and describes PostgreSQL, the one these refusals were first derived against. + +**Kinds this version does not translate.** `some`, `every`, `fold`: these range over a collection the caller's resolvers supply, which is not the query's row set. `lookup`, `call`, `delegate`, `treeReference`: each is resolved by something the database has no access to — the caller's resolvers, its function registry, an external system. `conditional`: not implemented here. `accumulator`: only meaningful inside a `reduce` fold. `arithmetic` and `negate`: these carry and combine units, and pushing them down would drop that dimensional analysis without saying so. `durationLiteral`: trilean compares durations by normalising both operands to milliseconds, with no column-level equivalent to normalise against. `complexLiteral`: neither engine has a complex type. -**Shapes refused despite a supported kind.** A `reference` whose key is not a string, since there is nothing to map. A `reference` or `numberLiteral` carrying a `unit`: a unit on a reference asserts that the resolved value carries the same one, and a column has no unit for that assertion to be checked against. A `numberLiteral` of `NaN`: trilean compares numbers with `===`, under which NaN equals nothing including itself, while PostgreSQL defines NaN as equal to itself and greater than every other double — so `NaN = NaN` selects every row there and none here. Infinities are not refused alongside it; both engines order them identically. +**Shapes refused despite a supported kind.** A `reference` whose key is not a string, since there is nothing to map. A `reference` or `numberLiteral` carrying a `unit`: a unit on a reference asserts that the resolved value carries the same one, and a column has no unit for that assertion to be checked against. A `numberLiteral` of `NaN`, which trilean compares with `===` — under which NaN equals nothing including itself — and neither engine reproduces, for opposite reasons. PostgreSQL defines NaN as equal to itself and greater than every other double, so `NaN = NaN` selects every row there and none here. SQLite has no NaN at all and a driver binding one substitutes SQL `NULL`, so the same comparison is *indeterminate* there and matches nothing — which looks like agreement until you negate it, at which point trilean's definite `true` matches every row and SQLite's `NULL` still matches none. Infinities are not refused alongside it; every engine here orders them identically. -**Operand pairings the two engines would answer differently.** These are the reason to declare `paramType`, and they are only detectable where it is declared: +**Operand pairings the engines would answer differently.** These are the reason to declare `paramType`, and they are only detectable where it is declared: -- A `compare` against text. trilean returns `wrong-type` and directs you to `textCompare`; PostgreSQL would order the operand by collation and answer definitely. -- An ordering `compare` (`gt`/`gte`/`lt`/`lte`) against a boolean. trilean has no order for booleans; PostgreSQL orders `false` before `true`. +- A `compare` against text. trilean returns `wrong-type` and directs you to `textCompare`. PostgreSQL orders the operand by collation and answers definitely; SQLite applies the text operand's own affinity to the other side and compares lexicographically, which is worse rather than better — `'9' > 5` is true and `'10' > 5` is not. +- An ordering `compare` (`gt`/`gte`/`lt`/`lte`) against a boolean. trilean has no order for booleans; PostgreSQL orders `false` before `true`, and SQLite orders the integers 0 and 1 it stores them as. - A `textCompare` against a non-text operand, which trilean treats as `wrong-type`. -- Any comparison whose operands are of different declared kinds — a number against an instant, say. trilean calls that `wrong-type`; PostgreSQL may coerce one to the other and answer definitely. +- Any comparison whose operands are of different declared kinds — a number against an instant, say. trilean calls that `wrong-type`; both engines may coerce one to the other and answer definitely. Left undeclared, these compile, and the divergence is real but invisible. That is the whole argument for supplying `paramType`. ## Regular expressions -`matches` and `notMatches` compile to PostgreSQL's `~` and `!~`, so the pattern is matched by the server. PostgreSQL's advanced regular expressions and ECMAScript's `RegExp` are close but not the same language: shorthand classes and lookahead exist in both, and much everyday pattern syntax is portable, but they are separate implementations with their own escapes, quantifier subtleties and matching rules. A pattern that relies on ECMAScript-specific behaviour may match differently once pushed down. Keep patterns to portable syntax, or evaluate them in process. +Under PostgreSQL, `matches` and `notMatches` compile to `~` and `!~`, so the pattern is matched by the server. PostgreSQL's advanced regular expressions and ECMAScript's `RegExp` are close but not the same language: shorthand classes and lookahead exist in both, and much everyday pattern syntax is portable, but they are separate implementations with their own escapes, quantifier subtleties and matching rules. A pattern that relies on ECMAScript-specific behaviour may match differently once pushed down. Keep patterns to portable syntax, or evaluate them in process. + +Under SQLite there is no built-in regular-expression support at all. `REGEXP` is reserved syntax for a `regexp(pattern, value)` function the connection has to register itself — `X REGEXP Y` is exactly `regexp(Y, X)`, pattern first — and the dialect emits `REGEXP` / `NOT REGEXP` for the same two operators. An unregistered one is a query error, `no such function: REGEXP`, rather than a fragment that quietly matches nothing, so this is a documented environment requirement in the same class as the caveats above and not a hole in the compile-time guarantee. The upside is that the pattern is then matched by your own `RegExp`, so the ECMAScript-versus-server-dialect divergence above does not arise. + +With better-sqlite3: + +```ts +db.function("regexp", (pattern, text) => + typeof pattern !== "string" || typeof text !== "string" + ? null + : new RegExp(pattern).test(text) + ? 1 + : 0, +); +``` + +Both details are load-bearing rather than stylistic. Returning `null` for a NULL argument is what keeps the third value intact: SQLite does not propagate NULL through a user function on its own, so one answering `0` for a NULL value would make `NOT REGEXP` answer `TRUE` for a row whose value is unknown — the two-valued collapse this package exists to avoid. Returning `1`/`0` rather than a JS boolean is what better-sqlite3 accepts; a boolean is rejected from a user function (*"returned an invalid value"*) for the same reason it is rejected as a bound parameter. ## Tests The unit suite asserts compiled SQL text and parameter arrays per node kind. It cannot, on its own, establish anything about three-valued behaviour: `("age" > $1)` is only indeterminate-preserving because of what PostgreSQL's planner does with a `NULL` age, which is a fact about PostgreSQL rather than about the string. -So the integration suite (`pnpm test:integration`) seeds a table whose rows carry real `NULL`s, and for every case executes the compiled fragment as a `WHERE` clause *and* evaluates the same tree through trilean's own `evaluatePredicate` once per row, asserting the two agree on which rows match and which do not. Agreement on absence matters as much as on presence: the case that distinguishes three-valued logic from two-valued is the row that appears in neither a predicate nor its negation. +So the integration suite (`pnpm test:integration`) runs the same fixture against a real engine, once per implementation: PostgreSQL in an ephemeral container, PGlite in process, and SQLite in memory through better-sqlite3. Each seeds a table whose rows carry real `NULL`s, and for every case executes the compiled fragment as a `WHERE` clause *and* evaluates the same tree through trilean's own `evaluatePredicate` once per row, asserting the two agree on which rows match and which do not. Agreement on absence matters as much as on presence: the case that distinguishes three-valued logic from two-valued is the row that appears in neither a predicate nor its negation. -That suite runs case for case against two engines. `postgres.test.ts` uses a real PostgreSQL server started as an ephemeral container, and needs a working Docker daemon. `pglite.test.ts` uses [PGlite](https://pglite.dev), which is PostgreSQL itself compiled to WebAssembly and run in process rather than a reimplementation of it, and needs nothing beyond Node — so the parity claim above is measurable on any machine, Docker or not, and PGlite is a verified target of the `postgres` dialect rather than an assumed one. +`postgres.test.ts` uses a real PostgreSQL server started as an ephemeral container, and needs a working Docker daemon. `pglite.test.ts` uses [PGlite](https://pglite.dev), which is PostgreSQL itself compiled to WebAssembly and run in process rather than a reimplementation of it, and needs nothing beyond Node — so the parity claim above is measurable on any machine, Docker or not, and PGlite is a verified target of the `postgres` dialect rather than an assumed one. `sqlite.test.ts` likewise needs nothing beyond Node, and also measures the divergences the guard's refusals exist to prevent — the NaN-to-NULL binding substitution, the lexicographic text comparison, the integer boolean ordering — rather than only asserting that each refusal fires, since inheriting PostgreSQL's refusal set would be worth nothing if SQLite happened to agree with trilean where PostgreSQL does not. ## Licence diff --git a/packages/trilean-sql/package.json b/packages/trilean-sql/package.json index e93ea8e..33c914c 100644 --- a/packages/trilean-sql/package.json +++ b/packages/trilean-sql/package.json @@ -70,6 +70,7 @@ "keywords": [ "sql", "postgres", + "sqlite", "predicate", "three-valued-logic", "query-builder", @@ -82,9 +83,11 @@ "@electric-sql/pglite": "^0.5.8", "@exadev/eslint-config": "^2.10.2", "@testcontainers/postgresql": "^12.1.0", + "@types/better-sqlite3": "^9.6.0", "@types/node": "^26.4.0", "@types/pg": "^8.15.6", "@vitest/coverage-v8": "^4.1.11", + "better-sqlite3": "^12.11.1", "eslint": "^10.9.1", "eslint-config-prettier": "^10.1.8", "eslint-plugin-prettier": "^5.5.6", diff --git a/packages/trilean-sql/src/compile.test.ts b/packages/trilean-sql/src/compile.test.ts index cc63b30..953f690 100644 --- a/packages/trilean-sql/src/compile.test.ts +++ b/packages/trilean-sql/src/compile.test.ts @@ -1,9 +1,14 @@ import type { PredicateNode } from "trilean"; import { describe, expect, it, vi } from "vitest"; import { compilePredicateNode } from "./compile"; -import { InvalidColumnError, UnsupportedNodeError } from "./errors"; -import type { SqlCompileOptions } from "./options"; -import { subjectOptions } from "./test-support/columns"; +import { + InvalidColumnError, + UnknownDialectError, + UnsupportedNodeError, +} from "./errors"; +import { findUnpushableNodeKind } from "./guard"; +import type { SqlCompileOptions, SqlDialect } from "./options"; +import { sqliteSubjectOptions, subjectOptions } from "./test-support/columns"; function compile( node: PredicateNode, @@ -459,3 +464,208 @@ describe("refusal", () => { }, ); }); + +describe("the sqlite dialect", () => { + function compileSqlite(node: PredicateNode) { + return compile(node, sqliteSubjectOptions); + } + + it("renders every placeholder as a bare '?', with no number and no cast", () => { + // SQLite binds by position in emission order rather than by an index written into the text, and it has no type to cast a parameter to. Asserted across a nested tree because the numbering is exactly what a bare '?' drops: the three parameters below are told apart only by the order they appear in. + expect( + compileSqlite({ + kind: "allOf", + operands: [ + { + kind: "compare", + op: "gt", + left: { kind: "reference", key: "age" }, + right: { kind: "numberLiteral", value: LOWER_BOUND }, + }, + { + kind: "memberOf", + op: "in", + operand: { kind: "reference", key: "name" }, + candidates: [ + { kind: "textLiteral", value: "b" }, + { kind: "textLiteral", value: "c" }, + ], + }, + ], + }), + ).toEqual({ + sql: '(("age" > ?) AND ("name" IN (?, ?)))', + params: [LOWER_BOUND, "b", "c"], + }); + }); + + it("compares two literals without either side needing a type", () => { + // The case PostgreSQL cannot execute uncast at all. SQLite answers it from the bound values themselves, so there is nothing to annotate. + expect( + compileSqlite({ + kind: "compare", + op: "lt", + left: { kind: "numberLiteral", value: 1 }, + right: { kind: "numberLiteral", value: 2 }, + }), + ).toEqual({ sql: "(? < ?)", params: [1, 2] }); + }); + + it("renders an instant literal as a plain parameter, with no timestamp type to cast to", () => { + expect( + compileSqlite({ + kind: "compare", + op: "gte", + left: { kind: "reference", key: "joined" }, + right: { kind: "instantLiteral", value: "2020-01-01T00:00:00+02:00" }, + }), + ).toEqual({ + sql: '("joined" >= ?)', + params: ["2020-01-01T00:00:00+02:00"], + }); + }); + + it.each([ + ["equals", "="], + ["notEquals", "<>"], + ["matches", "REGEXP"], + ["notMatches", "NOT REGEXP"], + ] as const)("compiles textCompare '%s' to '%s'", (op, sqlOperator) => { + // `=` and `<>` are ANSI-standard and identical to the PostgreSQL dialect's; only the two pattern operators differ, and SQLite's are the reserved REGEXP syntax for a function the connection registers itself. + expect( + compileSqlite({ + kind: "textCompare", + op, + left: { kind: "reference", key: "name" }, + right: { kind: "textLiteral", value: "^a" }, + }), + ).toEqual({ sql: `("name" ${sqlOperator} ?)`, params: ["^a"] }); + }); + + it("compiles an empty candidate list without a boolean annotation on the NULL", () => { + // SQLite has no boolean type to annotate, and the annotation is not what the encoding depends on: the integration suite executes both of these and gets the same three-valued answers the `::boolean` forms give PostgreSQL. + expect( + compileSqlite({ + kind: "memberOf", + op: "in", + operand: { kind: "reference", key: "name" }, + candidates: [], + }), + ).toEqual({ sql: '("name" IS NULL AND NULL)', params: [] }); + + expect( + compileSqlite({ + kind: "memberOf", + op: "notIn", + operand: { kind: "reference", key: "name" }, + candidates: [], + }), + ).toEqual({ sql: '("name" IS NOT NULL OR NULL)', params: [] }); + }); + + it("emits the dialect-neutral structure identically to PostgreSQL", () => { + // Everything the two dialects share, in one tree: the connectives, the six comparison operators, `IS NOT NULL`, `NOT IN`, and double-quoted identifiers. The only difference between this expectation and the PostgreSQL one is the placeholders. + const node: PredicateNode = { + kind: "not", + operand: { + kind: "and", + left: ageOver, + right: { + kind: "or", + left: { kind: "exists", operand: { kind: "reference", key: "note" } }, + right: { + kind: "memberOf", + op: "notIn", + operand: { kind: "reference", key: "age" }, + candidates: [{ kind: "numberLiteral", value: EXCLUDED_AGE }], + }, + }, + }, + }; + + expect(compileSqlite(node).sql).toBe( + '(NOT (("age" > ?) AND (("note" IS NOT NULL) OR ("age" NOT IN (?)))))', + ); + expect(compile(node).sql).toBe( + '(NOT (("age" > $1::double precision) AND (("note" IS NOT NULL) OR ("age" NOT IN ($2::double precision)))))', + ); + }); + + it("quotes and neutralises identifiers exactly as the PostgreSQL dialect does", () => { + // Double-quoting with an embedded quote doubled is ANSI-standard, so the injection defence is the same string in both dialects rather than a per-dialect rule. + const hostile: SqlCompileOptions = { + dialect: "sqlite", + columnFor: () => ({ column: 'note"; DROP TABLE subjects; --' }), + }; + expect( + compile( + { kind: "exists", operand: { kind: "reference", key: "anything" } }, + hostile, + ).sql, + ).toBe('("note""; DROP TABLE subjects; --" IS NOT NULL)'); + }); + + it("compiles an empty allOf and anyOf to the same identities", () => { + expect(compileSqlite({ kind: "allOf", operands: [] }).sql).toBe("(TRUE)"); + expect(compileSqlite({ kind: "anyOf", operands: [] }).sql).toBe("(FALSE)"); + }); +}); + +describe("a dialect this version does not implement", () => { + // `SqlDialect` is closed, so this is what a caller reading the name from configuration and asserting it into the union at the boundary reaches -- the only way an unimplemented name gets this far, and the reason the assertion is here rather than in the source under test. + const unimplemented = "mysql" as SqlDialect; + const mysqlOptions: SqlCompileOptions = { + dialect: unimplemented, + columnFor: () => ({ column: "age", paramType: "number" }), + }; + + const anyTree: PredicateNode = { + kind: "compare", + op: "gt", + left: { kind: "reference", key: "age" }, + right: { kind: "numberLiteral", value: ADULT_AGE }, + }; + + it("is refused by name, not as an internal error from an empty table lookup", () => { + expect(() => compilePredicateNode(anyTree, mysqlOptions)).toThrow( + UnknownDialectError, + ); + expect(() => compilePredicateNode(anyTree, mysqlOptions)).toThrow( + /unknown dialect "mysql": this version compiles "postgres", "sqlite"/, + ); + }); + + it("carries the offending name and the implemented ones as fields", () => { + try { + compilePredicateNode(anyTree, mysqlOptions); + expect.unreachable("compiling an unimplemented dialect must throw"); + } catch (error) { + expect(error).toBeInstanceOf(UnknownDialectError); + expect(error).toMatchObject({ + dialect: "mysql", + implemented: ["postgres", "sqlite"], + }); + } + }); + + it("is refused before the tree is walked, so the dialect is what gets reported", () => { + // A tree the guard would object to on its own. The dialect is the earlier problem and has to be the one named, since every refusal reason the walk could produce describes an engine that is not the one asked for. + expect(() => + compilePredicateNode( + { + kind: "compare", + op: "eq", + left: { kind: "reference", key: "age" }, + right: { kind: "numberLiteral", value: Number.NaN }, + }, + mysqlOptions, + ), + ).toThrow(UnknownDialectError); + }); + + it("never reports such a tree as pushable, which would promise a compilation that cannot happen", () => { + expect(() => findUnpushableNodeKind(anyTree, mysqlOptions)).toThrow( + UnknownDialectError, + ); + }); +}); diff --git a/packages/trilean-sql/src/compile.ts b/packages/trilean-sql/src/compile.ts index 8c61c13..79e9f5c 100644 --- a/packages/trilean-sql/src/compile.ts +++ b/packages/trilean-sql/src/compile.ts @@ -8,11 +8,12 @@ import { InvalidColumnError, UnsupportedNodeError } from "./errors"; import { findUnpushableNodeKind } from "./guard"; import type { CompiledSql, + DialectConfig, SqlColumnBinding, SqlCompileOptions, SqlParamType, } from "./options"; -import { POSTGRES_TYPE_NAME } from "./options"; +import { assertImplementedDialect, DIALECT_CONFIG } from "./options"; const COMPARISON_SQL: Readonly> = { gt: ">", @@ -23,20 +24,26 @@ const COMPARISON_SQL: Readonly> = { neq: "<>", }; -/** `~` and `!~` are PostgreSQL's own regular-expression match operators, so a pattern is matched by the database rather than shipped back to be matched in process. See the "Regular expressions" caveat in README.md: PostgreSQL's advanced regular expressions and ECMAScript's are close but not the same language. */ -const TEXT_COMPARISON_SQL: Readonly> = { - equals: "=", - notEquals: "<>", - matches: "~", - notMatches: "!~", -}; - /** - * The SQL type a literal placeholder is cast to: the one implied by the literal's own trilean kind. + * The operator each `textCompare` op emits, for one dialect. * - * Casting every placeholder means a fragment's meaning never depends on how a particular driver decided to infer an untyped parameter, and it is what makes a comparison between two literals (`$1 < $2`, which PostgreSQL rejects outright as having undeterminable parameter types) compile to something executable at all. + * `=` and `<>` are ANSI-standard equality and identical everywhere; only the two pattern operators are the dialect's own. Built as a complete record rather than resolved per node so that a `TextComparisonOperator` added to trilean later is a compile error here, instead of an undefined operator spliced into the emitted SQL. + */ +function textComparisonSqlFor( + dialect: Readonly, +): Readonly> { + return { + equals: "=", + notEquals: "<>", + matches: dialect.matches, + notMatches: dialect.notMatches, + }; +} + +/** + * The value kind a literal placeholder is rendered for: the one implied by the literal's own trilean kind. What a dialect does with it -- PostgreSQL casts the placeholder to the corresponding type, SQLite ignores it -- is `DialectConfig.placeholder`'s business. * - * A mapped column's declared `paramType` deliberately does not override this. It cannot differ: the guard has already refused any comparison whose operand kinds disagree, and each declared kind maps to the same PostgreSQL type as the literal kind it must then match. Consulting it here would be a branch that can never change the output. + * A mapped column's declared `paramType` deliberately does not override this. It cannot differ: the guard has already refused any comparison whose operand kinds disagree, and each declared kind implies the same SQL type as the literal kind it must then match. Consulting it here would be a branch that can never change the output. */ const PARAM_TYPE_OF_LITERAL: Readonly< Record< @@ -52,6 +59,9 @@ const PARAM_TYPE_OF_LITERAL: Readonly< interface CompileContext { readonly options: SqlCompileOptions; + /** Resolved once per compilation rather than looked up per node, alongside the `columnFor` memoisation, since the dialect cannot change mid-tree. */ + readonly dialect: Readonly; + readonly textComparison: Readonly>; readonly params: unknown[]; } @@ -61,11 +71,11 @@ function placeholder( castTo: SqlParamType, ): string { context.params.push(value); - return `$${String(context.params.length)}::${POSTGRES_TYPE_NAME[castTo]}`; + return context.dialect.placeholder(context.params.length, castTo); } /** - * Renders a column as a PostgreSQL identifier: each dot-separated segment double-quoted, with any embedded double quote doubled. + * Renders a column as a SQL identifier: each dot-separated segment double-quoted, with any embedded double quote doubled. Double-quoting is ANSI-standard and means the same thing in both dialects, so this needs no per-dialect branch. * * A column name cannot be a bind parameter -- it is part of the statement's structure, not its data -- so it is the one caller-supplied string that reaches the SQL text. Quoting it unconditionally is what keeps that safe: the doubling makes even a name containing `"; DROP TABLE ...` a single, inert identifier that simply does not exist. Quoting also means a name is taken literally rather than case-folded, so `columnFor` must return the column's real, case-exact name. */ @@ -173,15 +183,16 @@ function compilePredicate( case "textCompare": { const left = compileExpression(node.left, context); const right = compileExpression(node.right, context); - return `(${left} ${TEXT_COMPARISON_SQL[node.op]} ${right})`; + return `(${left} ${context.textComparison[node.op]} ${right})`; } case "memberOf": { const operand = compileExpression(node.operand, context); if (node.candidates.length === 0) { - // `IN ()` is a syntax error, and the two constants it would be tempting to fold to are both wrong: an empty `in` is false and an empty `notIn` is true only once the operand itself is known, and stay unknown while it is NULL. These two forms reproduce that exactly -- `NULL IS NULL AND NULL` is NULL while ` IS NULL AND NULL` is FALSE, and the `notIn` form is its mirror image -- which a bare FALSE/TRUE would not, most visibly under a surrounding NOT. + // `IN ()` is a syntax error, and the two constants it would be tempting to fold to are both wrong: an empty `in` is false and an empty `notIn` is true only once the operand itself is known, and stay unknown while it is NULL. These two forms reproduce that exactly -- `NULL IS NULL AND NULL` is NULL while ` IS NULL AND NULL` is FALSE, and the `notIn` form is its mirror image -- which a bare FALSE/TRUE would not, most visibly under a surrounding NOT. The suffix is the dialect's own boolean annotation on that bare NULL, empty for a dialect with no boolean type to annotate. + const nullLiteral = `NULL${context.dialect.emptyMemberOfNullSuffix}`; return node.op === "in" - ? `(${operand} IS NULL AND NULL::boolean)` - : `(${operand} IS NOT NULL OR NULL::boolean)`; + ? `(${operand} IS NULL AND ${nullLiteral})` + : `(${operand} IS NOT NULL OR ${nullLiteral})`; } const candidates = node.candidates .map((candidate) => compileExpression(candidate, context)) @@ -200,12 +211,13 @@ function compilePredicate( } /** - * Compiles a trilean predicate tree into a parameterised PostgreSQL boolean expression. + * Compiles a trilean predicate tree into a parameterised boolean expression in the dialect `options` names. * * Three-valued logic is not reimplemented on top of SQL; it is delegated to it. SQL's `AND`, `OR` and `NOT` over `TRUE`/`FALSE`/`NULL` are Kleene's strong three-valued tables, which are the same tables trilean's own `combineAnd`, `combineOr` and `not` implement, and a comparison against a NULL column yields `NULL` exactly where the evaluator would have returned `indeterminate` from an unresolved reference. A row excluded by `WHERE` because its condition was unknown is therefore excluded for the same reason, and by the same rule, as a subject the evaluator declines to judge. No indeterminacy column, sentinel value or `CASE` scaffolding is emitted, because none is needed. * * Every caller-supplied literal becomes a bind parameter. Nothing but structure, operators, and quoted column identifiers is ever written into the returned `sql`. * + * @throws {UnknownDialectError} if `options.dialect` names a dialect this version does not implement. * @throws {UnsupportedNodeError} if any node in the tree is one this compiler will not translate -- see `findUnpushableNodeKind`, which this runs first and which a caller can run itself to choose between pushdown and in-process evaluation without provoking an exception. * @throws {InvalidColumnError} if `columnFor` returns a column that cannot be rendered as an identifier. */ @@ -213,6 +225,9 @@ export function compilePredicateNode( node: PredicateNode, options: Readonly, ): CompiledSql { + // Before the walk rather than after it, so an unimplemented dialect is reported as itself rather than as whichever node the guard happened to object to first under another dialect's rules. + assertImplementedDialect(options.dialect); + // `columnFor` is called by the guard walk and again while compiling, so it is memoised for the duration of one compilation -- a caller's mapping may be a lookup of real cost, and it must not matter how many times the compiler happens to ask. const bindings = new Map(); const memoised: SqlCompileOptions = { @@ -229,6 +244,12 @@ export function compilePredicateNode( const unpushable = findUnpushableNodeKind(node, memoised); if (unpushable !== undefined) throw new UnsupportedNodeError(unpushable); - const context: CompileContext = { options: memoised, params: [] }; + const dialect = DIALECT_CONFIG[options.dialect]; + const context: CompileContext = { + options: memoised, + dialect, + textComparison: textComparisonSqlFor(dialect), + params: [], + }; return { sql: compilePredicate(node, context), params: context.params }; } diff --git a/packages/trilean-sql/src/errors.ts b/packages/trilean-sql/src/errors.ts index 4818a60..2f6469a 100644 --- a/packages/trilean-sql/src/errors.ts +++ b/packages/trilean-sql/src/errors.ts @@ -9,7 +9,7 @@ export class TrileanSqlError extends Error { /** * A node the compiler refuses to translate. The compiler never degrades: there is no "best effort" fragment, no silently dropped conjunct, and no approximation that answers differently from the in-process evaluator. Either the whole tree compiles to SQL that agrees with `evaluatePredicate` on every row, or this is thrown and the caller evaluates in process instead. * - * That guarantee is about the tree's *structure*, and it stops at two things whose meaning is a property of the operand's own content rather than of any node kind: a `matches`/`notMatches` pattern, which the server matches under PostgreSQL's regular-expression language rather than ECMAScript's, and an `instantLiteral` string the two engines parse differently or that PostgreSQL cannot parse at all (which fails as a query error at execution time, not as this exception). Both are covered in README.md; neither is detectable by a walk over node kinds. + * That guarantee is about the tree's *structure*, and it stops at two things whose meaning is a property of the operand's own content rather than of any node kind: a `matches`/`notMatches` pattern, which the server matches under its own regular-expression language rather than ECMAScript's, and an `instantLiteral` string the database reads differently from trilean or cannot read at all (which fails as a query error at execution time, not as this exception). Both are covered in README.md; neither is detectable by a walk over node kinds. * * `nodeKind` is the offending node's own `kind`, `path` locates it inside the tree, and `reason` says why it is not pushable -- which is not always about the kind alone (a `reference` carrying a `unit`, or a `compare` whose operands are statically of different kinds, are both refused despite `reference` and `compare` being supported kinds). */ @@ -36,7 +36,28 @@ export class UnsupportedNodeError extends TrileanSqlError { } /** - * A `columnFor` result whose `column` cannot be rendered as a PostgreSQL identifier. + * A dialect this version does not implement. + * + * `SqlDialect` is a closed union, so a caller who names a dialect in TypeScript source cannot reach this. A dialect read from configuration and asserted into the type at the boundary can, which is the case it exists for: without it the unimplemented name surfaces later as an internal `TypeError` from a table lookup that found nothing, naming neither the dialect nor the field it was reached from. Refusing by name at the entry point is the same convention `InvalidColumnError` applies to a `columnFor` result and the node walk applies to an unrecognised kind -- an input the type system describes but cannot enforce is checked once, where it arrives. + * + * `dialect` is the offending name and `implemented` lists the ones this version does compile. + */ +export class UnknownDialectError extends TrileanSqlError { + readonly dialect: string; + readonly implemented: readonly string[]; + + constructor(dialect: string, implemented: readonly string[]) { + super( + `unknown dialect ${JSON.stringify(dialect)}: this version compiles ${implemented.map((name) => JSON.stringify(name)).join(", ")}`, + ); + this.name = "UnknownDialectError"; + this.dialect = dialect; + this.implemented = implemented; + } +} + +/** + * A `columnFor` result whose `column` cannot be rendered as a SQL identifier. * * A column name is an identifier, not a value, so it is the one part of the emitted SQL that cannot be parameterised -- it has to be written into the statement text. It is always emitted double-quoted with any embedded quote doubled, which makes an arbitrary string safe, so this is not the injection defence; it rejects the two shapes that quoting cannot rescue into a valid identifier, an empty name and an empty dot-separated segment. */ diff --git a/packages/trilean-sql/src/guard.test.ts b/packages/trilean-sql/src/guard.test.ts index 0d6a6af..bc8d536 100644 --- a/packages/trilean-sql/src/guard.test.ts +++ b/packages/trilean-sql/src/guard.test.ts @@ -1,7 +1,7 @@ import type { ExpressionNode, PredicateNode } from "trilean"; import { describe, expect, it, vi } from "vitest"; import { findUnpushableNodeKind } from "./guard"; -import { subjectOptions } from "./test-support/columns"; +import { sqliteSubjectOptions, subjectOptions } from "./test-support/columns"; const ageOver: PredicateNode = { kind: "compare", @@ -331,3 +331,112 @@ describe("operand kinds trilean and PostgreSQL would answer differently", () => ).toBeUndefined(); }); }); + +describe("refusal reasons are worded for the dialect they describe", () => { + // Which trees are refused is a property of the divergence, not of the dialect: every pairing below is answered definitely by both engines and wrong-typed by trilean, so both dialects refuse all of them. What changes is the explanation, and each dialect's has to name the mechanism that actually applies to it -- a reason describing PostgreSQL's NaN ordering would be simply false about SQLite, which has no NaN at all. + + const nanEquality: PredicateNode = { + kind: "compare", + op: "eq", + left: { kind: "reference", key: "age" }, + right: { kind: "numberLiteral", value: Number.NaN }, + }; + + const orderedText: PredicateNode = { + kind: "compare", + op: "gt", + left: { kind: "reference", key: "name" }, + right: { kind: "textLiteral", value: "ada" }, + }; + + const orderedBoolean: PredicateNode = { + kind: "compare", + op: "gt", + left: { kind: "reference", key: "active" }, + right: { kind: "booleanLiteral", value: false }, + }; + + const crossKind: PredicateNode = { + kind: "compare", + op: "eq", + left: { kind: "reference", key: "age" }, + right: { kind: "instantLiteral", value: "2020-01-01T00:00:00Z" }, + }; + + it.each([nanEquality, orderedText, orderedBoolean, crossKind])( + "refuses the same node at the same path in either dialect", + (node) => { + const viaPostgres = findUnpushableNodeKind(node, subjectOptions); + const viaSqlite = findUnpushableNodeKind(node, sqliteSubjectOptions); + expect(viaPostgres).toBeDefined(); + expect(viaSqlite).toMatchObject({ + kind: viaPostgres?.kind, + path: viaPostgres?.path, + }); + }, + ); + + it("explains a refused NaN by the substitution SQLite's drivers actually make", () => { + // The integration suite proves this one against a real connection: better-sqlite3 binds NaN as SQL NULL, so `NaN = NaN` is indeterminate rather than definitely false, and the negation trilean answers definitely true matches nothing at all. + expect(findUnpushableNodeKind(nanEquality, sqliteSubjectOptions)).toEqual({ + kind: "numberLiteral", + path: "$.right", + reason: + "NaN is equal to nothing in trilean, not even itself, whereas SQLite has no NaN at all and a driver binding one substitutes SQL NULL -- so 'NaN = NaN' is indeterminate there rather than definitely false, and its negation matches every row instead of none", + }); + }); + + it("explains a refused NaN by PostgreSQL's own definition of it in the other dialect", () => { + expect(findUnpushableNodeKind(nanEquality, subjectOptions)).toEqual({ + kind: "numberLiteral", + path: "$.right", + reason: + "NaN is equal to nothing in trilean, not even itself, whereas PostgreSQL defines NaN as equal to itself and greater than every other double", + }); + }); + + it("explains an ordered text operand by the coercion each engine performs", () => { + expect( + findUnpushableNodeKind(orderedText, sqliteSubjectOptions)?.reason, + ).toBe( + "'compare' never compares text in trilean -- it returns wrong-type and directs the caller to 'textCompare' -- but SQLite would answer definitely for the text operand at $.left, comparing it under the text affinity it applies to the other side ('9' > 5 is true there, while '10' > 5 is not)", + ); + expect(findUnpushableNodeKind(orderedText, subjectOptions)?.reason).toBe( + "'compare' never compares text in trilean -- it returns wrong-type and directs the caller to 'textCompare' -- but PostgreSQL would order the text operand at $.left collation-wise and answer definitely", + ); + }); + + it("explains an ordered boolean by how each engine represents one", () => { + expect( + findUnpushableNodeKind(orderedBoolean, sqliteSubjectOptions)?.reason, + ).toBe( + "booleans have no ordering in trilean, so 'gt' against the boolean operand at $.left is wrong-type there, whereas SQLite stores booleans as the integers 0 and 1 and orders them as integers", + ); + expect(findUnpushableNodeKind(orderedBoolean, subjectOptions)?.reason).toBe( + "booleans have no ordering in trilean, so 'gt' against the boolean operand at $.left is wrong-type there, whereas PostgreSQL orders false before true", + ); + }); + + it("explains a cross-kind comparison by the coercion each engine reaches for", () => { + expect( + findUnpushableNodeKind(crossKind, sqliteSubjectOptions)?.reason, + ).toContain( + "whereas SQLite's type affinity may coerce one to the other and answer definitely", + ); + expect(findUnpushableNodeKind(crossKind, subjectOptions)?.reason).toContain( + "whereas PostgreSQL may coerce one to the other and answer definitely", + ); + }); + + it("describes PostgreSQL when no options name a dialect at all", () => { + // A structural walk has no dialect to read. It refuses exactly what either dialect refuses -- the point of the walk is unchanged -- and names the dialect these refusals were first derived against rather than inventing a dialect-free phrasing that describes neither engine. + expect( + findUnpushableNodeKind({ + kind: "compare", + op: "eq", + left: { kind: "numberLiteral", value: Number.NaN }, + right: { kind: "numberLiteral", value: Number.NaN }, + })?.reason, + ).toContain("PostgreSQL defines NaN as equal to itself"); + }); +}); diff --git a/packages/trilean-sql/src/guard.ts b/packages/trilean-sql/src/guard.ts index 0c138b8..2cb57b2 100644 --- a/packages/trilean-sql/src/guard.ts +++ b/packages/trilean-sql/src/guard.ts @@ -1,5 +1,6 @@ import type { ExpressionNode, PredicateNode } from "trilean"; -import type { SqlCompileOptions, SqlParamType } from "./options"; +import type { SqlCompileOptions, SqlDialect, SqlParamType } from "./options"; +import { assertImplementedDialect } from "./options"; /** Where the walk stopped, and why. `kind` is the offending node's own `kind` even when the objection is not to the kind itself. */ export interface UnpushableNode { @@ -12,7 +13,7 @@ export interface UnpushableNode { /** * trilean's own kind for a value, as far as it can be determined without running anything: from a literal's own node kind, or from a mapped column's declared `paramType`. `undefined` means the value's kind is only knowable at execution time, which is the normal case for a column the caller chose not to describe. * - * This exists because several of trilean's comparison rules are about the operands' kinds rather than their values, and PostgreSQL's rules for the same operand pair are different. Where both kinds are knowable, the difference is detectable at compile time and refused; where one is not, it is not detectable at all, which is the honest reason to declare `paramType`. + * This exists because several of trilean's comparison rules are about the operands' kinds rather than their values, and a SQL engine's rules for the same operand pair are different. Where both kinds are knowable, the difference is detectable at compile time and refused; where one is not, it is not detectable at all, which is the honest reason to declare `paramType`. */ type StaticValueKind = "text" | "number" | "boolean" | "instant"; @@ -25,7 +26,7 @@ const STATIC_KIND_OF_PARAM_TYPE: Readonly< timestamp: "instant", }; -/** The comparison operators that impose an ordering, as opposed to testing equality. trilean refuses these on booleans -- `true > false` is not a fact about the domain -- while PostgreSQL answers them, so a boolean operand under one of these is a real divergence rather than a stylistic one. */ +/** The comparison operators that impose an ordering, as opposed to testing equality. trilean refuses these on booleans -- `true > false` is not a fact about the domain -- while both dialects answer them, so a boolean operand under one of these is a real divergence rather than a stylistic one. */ const ORDERING_OPERATORS: ReadonlySet = new Set([ "gt", "gte", @@ -41,6 +42,47 @@ const STATIC_KIND_OF_LITERAL: Readonly> = { instantLiteral: "instant", }; +/** + * How each refusal reads for a given dialect. + * + * Which nodes and operand pairings get refused is identical in both -- every divergence the PostgreSQL dialect refuses is equally real under SQLite, reached by a different mechanism -- so only the explanation varies, and only where an explanation names the engine's own behaviour at all. A refusal whose reason is purely about trilean (`textCompare` requiring text operands, a unit-tagged literal, a collection the resolvers own) reads the same either way and is not here. + */ +interface DialectDivergence { + /** Why a `complexLiteral` has nothing to compile to. */ + complexNumbers: string; + /** Why a NaN literal cannot be pushed down: a whole reason rather than a clause, since the two dialects disagree with trilean for genuinely different reasons and in different directions. */ + nan: string; + /** How the engine reaches a definite answer for operands of different kinds, as the trailing clause of a `whereas ...`. */ + crossKindCoercion: string; + /** How the engine orders a text operand under a `compare`, as the trailing clause of a `but ...`. */ + textOrdering: (operandPath: string) => string; + /** How the engine orders booleans, as the trailing clause of a `whereas ...`. */ + booleanOrdering: string; +} + +const DIALECT_DIVERGENCE: Readonly> = { + postgres: { + complexNumbers: "PostgreSQL has no complex number type", + nan: "NaN is equal to nothing in trilean, not even itself, whereas PostgreSQL defines NaN as equal to itself and greater than every other double", + crossKindCoercion: + "PostgreSQL may coerce one to the other and answer definitely", + textOrdering: (operandPath) => + `PostgreSQL would order the text operand at ${operandPath} collation-wise and answer definitely`, + booleanOrdering: "PostgreSQL orders false before true", + }, + sqlite: { + complexNumbers: "SQLite has no complex number type", + // A different mechanism from PostgreSQL's, and a divergence in the opposite direction: SQLite has no NaN of its own, and a driver binding a JS NaN substitutes SQL NULL for it -- confirmed directly against better-sqlite3, where `typeof(?)` bound with NaN answers 'null'. So `NaN = NaN` is indeterminate there and the tree matches nothing, while its negation matches everything; trilean's `===` makes the same comparison definitely false and its negation definitely true. Refused for the same reason, worded for the mechanism that actually applies. + nan: "NaN is equal to nothing in trilean, not even itself, whereas SQLite has no NaN at all and a driver binding one substitutes SQL NULL -- so 'NaN = NaN' is indeterminate there rather than definitely false, and its negation matches every row instead of none", + crossKindCoercion: + "SQLite's type affinity may coerce one to the other and answer definitely", + textOrdering: (operandPath) => + `SQLite would answer definitely for the text operand at ${operandPath}, comparing it under the text affinity it applies to the other side ('9' > 5 is true there, while '10' > 5 is not)`, + booleanOrdering: + "SQLite stores booleans as the integers 0 and 1 and orders them as integers", + }, +}; + function staticValueKindOf( node: ExpressionNode, options: SqlCompileOptions | undefined, @@ -56,6 +98,7 @@ function staticValueKindOf( function findUnpushableExpression( node: ExpressionNode, path: string, + divergence: Readonly, ): UnpushableNode | undefined { // Read before the switch narrows `node` to `never` in its default branch, where the kind is still what the report needs to name. const unrecognisedKind: string = node.kind; @@ -88,14 +131,9 @@ function findUnpushableExpression( "a unit-tagged number is only comparable with an operand of the same unit, and SQL has no unit to compare", }; } - // NaN is the one double-precision value the two engines disagree about. trilean compares numbers with `===`, under which NaN is equal to nothing including itself; PostgreSQL defines NaN as equal to itself and greater than every other double, so `NaN = NaN` there is TRUE and selects every row of a table this predicate should have matched none of. Infinities are deliberately not refused alongside it: both engines order them identically and compare them equal to themselves, so they translate faithfully. Reachable despite `NumberLiteralNodeSchema` rejecting NaN, because this compiler's input is the inferred `PredicateNode` type -- TypeScript's `number` includes NaN -- and a tree built in code rather than parsed never meets that schema. + // NaN is the one double-precision value no dialect agrees with trilean about. trilean compares numbers with `===`, under which NaN is equal to nothing including itself; neither dialect reproduces that, and they fail to for different reasons -- see each `nan` reason in DIALECT_DIVERGENCE. Infinities are deliberately not refused alongside it: every engine here orders them identically and compares them equal to themselves, so they translate faithfully. Reachable despite `NumberLiteralNodeSchema` rejecting NaN, because this compiler's input is the inferred `PredicateNode` type -- TypeScript's `number` includes NaN -- and a tree built in code rather than parsed never meets that schema. if (Number.isNaN(node.value)) { - return { - kind: node.kind, - path, - reason: - "NaN is equal to nothing in trilean, not even itself, whereas PostgreSQL defines NaN as equal to itself and greater than every other double", - }; + return { kind: node.kind, path, reason: divergence.nan }; } return undefined; case "textLiteral": @@ -110,11 +148,7 @@ function findUnpushableExpression( "trilean compares durations by normalising both operands to milliseconds, which has no column-level equivalent to normalise against", }; case "complexLiteral": - return { - kind: node.kind, - path, - reason: "PostgreSQL has no complex number type", - }; + return { kind: node.kind, path, reason: divergence.complexNumbers }; case "arithmetic": case "negate": return { @@ -181,12 +215,13 @@ function findUnpushableExpression( } } -/** Checks the operands of a comparison-shaped predicate for a kind pairing trilean and PostgreSQL would answer differently. Returns the objection against `path` itself, since the divergence is a property of the pairing rather than of either operand alone. */ +/** Checks the operands of a comparison-shaped predicate for a kind pairing trilean and the target engine would answer differently. Returns the objection against `path` itself, since the divergence is a property of the pairing rather than of either operand alone. */ function findKindDivergence( kind: string, path: string, operands: readonly { node: ExpressionNode; path: string }[], options: SqlCompileOptions | undefined, + divergence: Readonly, ): UnpushableNode | undefined { const kinds = operands.map((operand) => ({ path: operand.path, @@ -205,7 +240,7 @@ function findKindDivergence( return { kind, path, - reason: `trilean treats a comparison between a '${first.staticKind}' value (${first.path}) and a '${mismatched.staticKind}' value (${mismatched.path}) as wrong-type, whereas PostgreSQL may coerce one to the other and answer definitely`, + reason: `trilean treats a comparison between a '${first.staticKind}' value (${first.path}) and a '${mismatched.staticKind}' value (${mismatched.path}) as wrong-type, whereas ${divergence.crossKindCoercion}`, }; } return undefined; @@ -215,16 +250,32 @@ function findUnpushablePredicate( node: PredicateNode, path: string, options: SqlCompileOptions | undefined, + divergence: Readonly, ): UnpushableNode | undefined { const unrecognisedKind: string = node.kind; switch (node.kind) { case "not": - return findUnpushablePredicate(node.operand, `${path}.operand`, options); + return findUnpushablePredicate( + node.operand, + `${path}.operand`, + options, + divergence, + ); case "and": case "or": return ( - findUnpushablePredicate(node.left, `${path}.left`, options) ?? - findUnpushablePredicate(node.right, `${path}.right`, options) + findUnpushablePredicate( + node.left, + `${path}.left`, + options, + divergence, + ) ?? + findUnpushablePredicate( + node.right, + `${path}.right`, + options, + divergence, + ) ); case "allOf": case "anyOf": { @@ -233,6 +284,7 @@ function findUnpushablePredicate( operand, `${path}.operands[${String(index)}]`, options, + divergence, ); if (unpushable !== undefined) return unpushable; } @@ -244,25 +296,35 @@ function findUnpushablePredicate( { node: node.right, path: `${path}.right` }, ]; for (const operand of operands) { - const unpushable = findUnpushableExpression(operand.node, operand.path); + const unpushable = findUnpushableExpression( + operand.node, + operand.path, + divergence, + ); if (unpushable !== undefined) return unpushable; } - const divergence = findKindDivergence(node.kind, path, operands, options); - if (divergence !== undefined) return divergence; + const mismatch = findKindDivergence( + node.kind, + path, + operands, + options, + divergence, + ); + if (mismatch !== undefined) return mismatch; for (const operand of operands) { const staticKind = staticValueKindOf(operand.node, options); if (staticKind === "text") { return { kind: node.kind, path, - reason: `'compare' never compares text in trilean -- it returns wrong-type and directs the caller to 'textCompare' -- but PostgreSQL would order the text operand at ${operand.path} collation-wise and answer definitely`, + reason: `'compare' never compares text in trilean -- it returns wrong-type and directs the caller to 'textCompare' -- but ${divergence.textOrdering(operand.path)}`, }; } if (staticKind === "boolean" && ORDERING_OPERATORS.has(node.op)) { return { kind: node.kind, path, - reason: `booleans have no ordering in trilean, so '${node.op}' against the boolean operand at ${operand.path} is wrong-type there, whereas PostgreSQL orders false before true`, + reason: `booleans have no ordering in trilean, so '${node.op}' against the boolean operand at ${operand.path} is wrong-type there, whereas ${divergence.booleanOrdering}`, }; } } @@ -274,7 +336,11 @@ function findUnpushablePredicate( { node: node.right, path: `${path}.right` }, ]; for (const operand of operands) { - const unpushable = findUnpushableExpression(operand.node, operand.path); + const unpushable = findUnpushableExpression( + operand.node, + operand.path, + divergence, + ); if (unpushable !== undefined) return unpushable; const staticKind = staticValueKindOf(operand.node, options); if (staticKind !== undefined && staticKind !== "text") { @@ -296,13 +362,21 @@ function findUnpushablePredicate( })), ]; for (const operand of operands) { - const unpushable = findUnpushableExpression(operand.node, operand.path); + const unpushable = findUnpushableExpression( + operand.node, + operand.path, + divergence, + ); if (unpushable !== undefined) return unpushable; } - return findKindDivergence(node.kind, path, operands, options); + return findKindDivergence(node.kind, path, operands, options, divergence); } case "exists": - return findUnpushableExpression(node.operand, `${path}.operand`); + return findUnpushableExpression( + node.operand, + `${path}.operand`, + divergence, + ); case "some": case "every": return { @@ -337,10 +411,21 @@ function findUnpushablePredicate( * `compilePredicateNode` runs this first and throws `UnsupportedNodeError` on any result, so calling it separately is only necessary to *decide* between pushdown and in-process evaluation without provoking an exception. * * Passing `options` widens the check: without them the walk is purely structural, and with them it also applies the operand-kind rules that depend on each mapped column's declared `paramType` (and therefore calls `columnFor`). + * + * Which nodes are refused does not depend on the dialect -- every divergence refused here is real in both -- so a structural walk given no `options` refuses exactly what a walk given them would. What `options` also settles is which engine each `reason` describes; with none to read a dialect from, the reasons describe PostgreSQL, the dialect these refusals were first derived against. + * + * @throws {UnknownDialectError} if `options` names a dialect this version does not implement. Reporting a tree as pushable is a promise that `compilePredicateNode` will compile it, and under a dialect that does not exist it cannot -- so this is refused here rather than left to surface from the compiler, which is the one caller this function's answer is for. */ export function findUnpushableNodeKind( node: PredicateNode, options?: SqlCompileOptions, ): UnpushableNode | undefined { - return findUnpushablePredicate(node, "$", options); + const dialect = options?.dialect ?? "postgres"; + assertImplementedDialect(dialect); + return findUnpushablePredicate( + node, + "$", + options, + DIALECT_DIVERGENCE[dialect], + ); } diff --git a/packages/trilean-sql/src/index.ts b/packages/trilean-sql/src/index.ts index 6fbd956..cfc82b9 100644 --- a/packages/trilean-sql/src/index.ts +++ b/packages/trilean-sql/src/index.ts @@ -2,6 +2,7 @@ export type { CompiledSql, SqlColumnBinding, SqlCompileOptions, + SqlDialect, SqlParamType, } from "./options"; @@ -13,5 +14,6 @@ export { compilePredicateNode } from "./compile"; export { InvalidColumnError, TrileanSqlError, + UnknownDialectError, UnsupportedNodeError, } from "./errors"; diff --git a/packages/trilean-sql/src/options.ts b/packages/trilean-sql/src/options.ts index c8725df..eeb1ee5 100644 --- a/packages/trilean-sql/src/options.ts +++ b/packages/trilean-sql/src/options.ts @@ -1,21 +1,26 @@ +import { UnknownDialectError } from "./errors"; + /** The declared SQL value kind of a mapped column. `"timestamp"` names trilean's `instant` kind, whose values are ISO-8601 strings. */ export type SqlParamType = "text" | "number" | "boolean" | "timestamp"; +/** The SQL dialects this compiler emits. A dialect is named rather than inferred: the same tree compiles to different text for each, and a caller that never said which one it meant would be relying on whichever happened to be the default. */ +export type SqlDialect = "postgres" | "sqlite"; + /** What a `reference` node's key maps onto in the target schema. */ export interface SqlColumnBinding { /** The column, optionally qualified with a table or schema by dots (`"orders.total"`). Each dot-separated segment is emitted as its own double-quoted identifier, so a segment may contain any character except a dot itself. */ column: string; /** - * The column's value kind, if the caller knows it. Optional, and worth supplying: it is the only thing that lets the compiler check a comparison against the column for the kind mismatches trilean itself treats as `wrong-type` -- comparing a text column with `compare` rather than `textCompare`, ordering a boolean, comparing a number against an instant. Left undeclared, those comparisons compile, and PostgreSQL may coerce its way to a definite answer where trilean would have returned indeterminate. + * The column's value kind, if the caller knows it. Optional, and worth supplying: it is the only thing that lets the compiler check a comparison against the column for the kind mismatches trilean itself treats as `wrong-type` -- comparing a text column with `compare` rather than `textCompare`, ordering a boolean, comparing a number against an instant. Left undeclared, those comparisons compile, and the database may coerce its way to a definite answer where trilean would have returned indeterminate. * - * It does not affect the emitted SQL, only whether the comparison is emitted at all. Every literal placeholder is already cast to the type its own trilean kind implies, and a comparison that survives the check is one whose operand kinds agree, so there is nothing left for the column's declared kind to change. + * It does not affect the emitted SQL, only whether the comparison is emitted at all. Every literal placeholder is already cast to the type its own trilean kind implies (in the dialects that cast at all), and a comparison that survives the check is one whose operand kinds agree, so there is nothing left for the column's declared kind to change. */ paramType?: SqlParamType; } export interface SqlCompileOptions { - /** PostgreSQL is the only dialect implemented. It is a required field rather than a default so that adding a second dialect later is a new value here, not a change of behaviour for callers who never said which one they meant. */ - dialect: "postgres"; + /** Which dialect to emit. It is a required field rather than a default so that a caller states the engine it is compiling for, instead of inheriting whichever one this package happened to implement first. */ + dialect: SqlDialect; /** * Maps a `reference` node's key onto a column. Called once per reference occurrence. * @@ -27,7 +32,7 @@ export interface SqlCompileOptions { export interface CompiledSql { /** A self-contained boolean expression, always parenthesised, suitable for dropping straight into a `WHERE` clause (or a `CHECK`, a `HAVING`, or a filtered index predicate). It carries no `WHERE` keyword of its own. */ sql: string; - /** Positional parameters, in `$1`-first order, to pass alongside `sql`. Every caller-supplied literal in the tree is here; none is ever written into `sql`. */ + /** Positional parameters, in emission order, to pass alongside `sql`. Every caller-supplied literal in the tree is here; none is ever written into `sql`. */ params: unknown[]; } @@ -38,3 +43,53 @@ export const POSTGRES_TYPE_NAME: Readonly> = { boolean: "boolean", timestamp: "timestamptz", }; + +/** + * Everything about the emitted SQL that is a property of the dialect rather than of the tree. + * + * It is deliberately this small. Most of what the compiler emits is ANSI-standard and identical in both engines -- the six comparison operators, `=` and `<>` for `textCompare`'s `equals`/`notEquals`, `AND`/`OR`/`NOT`, `IN`/`NOT IN`, `IS NOT NULL`, and double-quoted identifiers with an embedded quote doubled -- so branching on the dialect anywhere else would be a branch that can never change the output. Three things genuinely differ, and they are the three fields below. + */ +export interface DialectConfig { + /** The regular-expression match operator `textCompare`'s `matches` compiles to. */ + matches: string; + /** The negated regular-expression match operator `textCompare`'s `notMatches` compiles to. */ + notMatches: string; + /** Renders the `index`-th (1-based) bind placeholder, for a literal whose own trilean kind implies `castTo`. A dialect that does not need the cast ignores both arguments. */ + placeholder: (index: number, castTo: SqlParamType) => string; + /** Appended to the bare `NULL` in the two forms an empty `memberOf` candidate list compiles to, for a dialect that needs the resulting expression annotated with a boolean type. */ + emptyMemberOfNullSuffix: string; +} + +export const DIALECT_CONFIG: Readonly> = { + postgres: { + // PostgreSQL's own regular-expression match operators, so a pattern is matched by the server rather than shipped back to be matched in process. See the "Regular expressions" caveat in README.md: PostgreSQL's advanced regular expressions and ECMAScript's are close but not the same language. + matches: "~", + notMatches: "!~", + // Casting every placeholder means a fragment's meaning never depends on how a particular driver decided to infer an untyped parameter, and it is what makes a comparison between two literals (`$1 < $2`, which PostgreSQL rejects outright as having undeterminable parameter types) compile to something executable at all. + placeholder: (index, castTo) => + `$${String(index)}::${POSTGRES_TYPE_NAME[castTo]}`, + emptyMemberOfNullSuffix: "::boolean", + }, + sqlite: { + // SQLite has no built-in regular-expression support: `REGEXP` is reserved syntax for a `regexp(pattern, value)` function the connection must register itself, and an unregistered one fails loudly at query time with "no such function: REGEXP" rather than answering wrongly. See the "Regular expressions" section in README.md for the registration the SQLite dialect therefore requires of its caller. + matches: "REGEXP", + notMatches: "NOT REGEXP", + // SQLite parameters are dynamically typed and positional-by-order, so there is neither a number to write nor a type to cast to. A bare `?` is correct for every literal kind this compiler emits, including a comparison between two literals, which SQLite answers without needing either side annotated. + placeholder: () => "?", + // SQLite has no boolean type to annotate: `NULL` alone already carries the three-valued behaviour the empty-`memberOf` forms depend on. + emptyMemberOfNullSuffix: "", + }, +}; + +/** + * Refuses a dialect this version does not implement, before anything indexes a per-dialect table with it. + * + * Both entry points check their own argument, because both are reachable with a name the type describes but cannot enforce -- a dialect is exactly the sort of value that arrives as a configuration string asserted into the union at the boundary. Left unchecked, that name is not caught anywhere: the compiler reads an operator off `undefined` and throws a `TypeError` naming an internal field, and the guard, whose tables are only consulted for a node it actually objects to, answers "pushable" for a tree the compiler then fails on -- the one thing `findUnpushableNodeKind` exists to decide, decided wrongly. + * + * `DIALECT_CONFIG` is the list, rather than a second constant, so implementing a dialect cannot leave this behind. + */ +export function assertImplementedDialect(dialect: SqlDialect): void { + if (!Object.hasOwn(DIALECT_CONFIG, dialect)) { + throw new UnknownDialectError(dialect, Object.keys(DIALECT_CONFIG)); + } +} diff --git a/packages/trilean-sql/src/test-support/columns.ts b/packages/trilean-sql/src/test-support/columns.ts index 5d4f2f0..5c3ee23 100644 --- a/packages/trilean-sql/src/test-support/columns.ts +++ b/packages/trilean-sql/src/test-support/columns.ts @@ -1,9 +1,9 @@ import type { SqlColumnBinding, SqlCompileOptions } from "../options"; /** - * The schema the unit tests and the integration suite both compile against, so a fragment asserted as a string in one is the same fragment executed against a real server in the other. + * The schema the unit tests and every integration suite compile against, so a fragment asserted as a string in one is the same fragment executed against a real engine in the others. * - * `age`, `name`, `active` and `joined` each declare a `paramType`; `note` deliberately does not, which is what exercises the compiler's undeclared-column path (no operand-kind checking, and literal placeholders cast by their own kind instead of the column's). + * `age`, `name`, `active` and `joined` each declare a `paramType`; `note` deliberately does not, which is what exercises the compiler's undeclared-column path (no operand-kind checking, and literal placeholders rendered by their own kind instead of the column's). */ export const SUBJECT_COLUMNS: Readonly> = { age: { column: "age", paramType: "number" }, @@ -13,13 +13,21 @@ export const SUBJECT_COLUMNS: Readonly> = { note: { column: "note" }, }; +function columnForSubject(referenceKey: string): SqlColumnBinding { + const binding = SUBJECT_COLUMNS[referenceKey]; + if (binding === undefined) { + throw new Error(`no column mapped for reference key '${referenceKey}'`); + } + return binding; +} + export const subjectOptions: SqlCompileOptions = { dialect: "postgres", - columnFor: (referenceKey) => { - const binding = SUBJECT_COLUMNS[referenceKey]; - if (binding === undefined) { - throw new Error(`no column mapped for reference key '${referenceKey}'`); - } - return binding; - }, + columnFor: columnForSubject, +}; + +/** The same mapping compiled for SQLite. Sharing `columnFor` is the point: a column mapping is a property of the schema, not of the dialect, so the only difference between the two suites' options is the dialect they name. */ +export const sqliteSubjectOptions: SqlCompileOptions = { + dialect: "sqlite", + columnFor: columnForSubject, }; diff --git a/packages/trilean-sql/test/integration/sqlite.test.ts b/packages/trilean-sql/test/integration/sqlite.test.ts new file mode 100644 index 0000000..2e17e2f --- /dev/null +++ b/packages/trilean-sql/test/integration/sqlite.test.ts @@ -0,0 +1,646 @@ +import Database from "better-sqlite3"; +import type { + ComputedValue, + JsonValue, + PredicateNode, + Resolution, + Resolvers, +} from "trilean"; +import { evaluatePredicate } from "trilean"; +import { afterAll, beforeAll, describe, expect, it } from "vitest"; +import { compilePredicateNode } from "../../src/compile"; +import type { SqlCompileOptions } from "../../src/options"; +import { sqliteSubjectOptions } from "../../src/test-support/columns"; + +/** + * The SQLite counterpart of `postgres.test.ts`, and the same claim measured rather than asserted: every case compiles a tree, executes the fragment as a real `WHERE` clause against a real SQLite connection, and compares the rows it returns against the rows trilean's own evaluator judges `definite(true)` for the same tree. + * + * Two things make this more than a copy. SQLite reaches its three-valued behaviour from a different starting point -- no boolean type, no timestamp type, no NaN, and a type-affinity system that coerces rather than rejects -- so agreement here is evidence about the dialect rather than a second run of an already-proven one. And the same affinity system is why the guard's refusals carry over unchanged: the last two describe blocks measure the divergences those refusals exist to prevent, against this connection, rather than asserting that a refusal fires. + * + * Unlike the PostgreSQL suite this needs no Docker: better-sqlite3 runs the engine in process against an in-memory database. + */ + +const SCHEMA = ` + CREATE TABLE subjects ( + id TEXT PRIMARY KEY, + age REAL, + name TEXT, + active INTEGER, + joined TEXT, + note TEXT + ); +`; + +/** + * A second, deliberately tiny table whose only purpose is the coercion proofs at the end of this file. + * + * They need a column whose declared affinity does the coercing -- affinity is a property of a column, and two bound parameters compared against each other have none -- and they need values chosen so that the coerced answer and the honest one differ. Keeping them out of `subjects` leaves that fixture identical in shape to the PostgreSQL suite's, so a case comparing the two suites is comparing like with like. + */ +const COERCION_SCHEMA = ` + CREATE TABLE coercion ( + label TEXT PRIMARY KEY, + numeric_text TEXT, + flag INTEGER + ); +`; + +interface SubjectRow { + id: string; + age: number | null; + name: string | null; + active: boolean | null; + joined: string | null; + note: string | null; +} + +/** `null` in a column means the same thing as a reference that resolves to nothing: the value is not known. Every row below carries at least one, because a table of fully-populated rows would exercise none of what this suite exists to check. */ +const SUBJECTS: readonly SubjectRow[] = [ + { + id: "ada", + age: 30, + name: "ada", + active: true, + joined: "2020-01-01T00:00:00Z", + note: "hello", + }, + { + id: "grace", + age: 12, + name: "grace", + active: false, + joined: "2024-06-01T12:00:00Z", + note: "hi", + }, + { + id: "lin", + age: null, + name: "lin", + active: true, + joined: "2021-03-03T00:00:00Z", + note: null, + }, + { + id: "unknown", + age: 45, + name: null, + active: null, + joined: null, + note: null, + }, +]; + +/** + * Resolves a reference key against one row, mapping a NULL column to `found: false`. + * + * That mapping is the correspondence the whole design rests on, and stating it in one place here is what makes the parity assertions below meaningful: the evaluator is being given exactly the knowledge SQLite has about the same row, so any disagreement between them is the compiler's, not the fixture's. + */ +function resolversFor(row: Readonly): Resolvers { + const known: Record = { + ...(row.age !== null && { + age: { kind: "number", value: row.age }, + }), + ...(row.name !== null && { name: { kind: "text", value: row.name } }), + ...(row.note !== null && { note: { kind: "text", value: row.note } }), + ...(row.active !== null && { + active: { kind: "boolean", value: row.active }, + }), + ...(row.joined !== null && { + joined: { kind: "instant", value: row.joined }, + }), + }; + + return { + resolveValue: async (key: JsonValue) => { + const value = typeof key === "string" ? known[key] : undefined; + return Promise.resolve( + value === undefined ? { found: false } : { found: true, value }, + ); + }, + resolveLookup: () => { + throw new Error("no tree in this suite uses a lookup"); + }, + resolveCollection: () => { + throw new Error("no tree in this suite uses a collection"); + }, + }; +} + +/** The threshold the coercion proofs compare against: greater than the text '9' sorts, and less than the number 9 is, so a coerced comparison and an honest one disagree about it. */ +const COERCION_THRESHOLD = 5; + +/** The lower of the two integers SQLite stores a boolean as, so `flag > FALSE_AS_INTEGER` is the ordering comparison trilean has no answer for. */ +const FALSE_AS_INTEGER = 0; + +/** + * Maps a compiled parameter onto something SQLite can bind. + * + * The one value kind that needs it is `boolean`: SQLite has no boolean type, and better-sqlite3 refuses a JS boolean outright ("SQLite3 can only bind numbers, strings, bigints, buffers, and null") rather than coercing it. That is a property of the driver and the engine, not of the compiled fragment -- `compilePredicateNode` hands back the tree's own literals unchanged in every dialect -- so the conversion belongs to the caller binding them, which is what this suite is standing in for. It is a loud failure rather than a silent one, which is why the compiler leaves it to the caller; README.md documents it alongside the `REGEXP` registration. + */ +function bindable(value: unknown): unknown { + return typeof value === "boolean" ? Number(value) : value; +} + +let db: Database.Database; + +beforeAll(() => { + db = new Database(":memory:"); + /** + * SQLite reserves `REGEXP` as syntax for a `regexp(pattern, value)` function it does not itself provide, so the dialect's `matches`/`notMatches` only run on a connection that has registered one. Two details of this registration are load-bearing rather than incidental, and README.md documents both: + * + * It returns `null` when either argument is NULL. SQLite does not propagate NULL through a user function on its own, and a function that answered 0 for a NULL value would make `NOT REGEXP` answer TRUE for a row whose value is unknown -- exactly the two-valued collapse this package exists to avoid. + * + * It returns 1/0 rather than a JS boolean, which better-sqlite3 rejects from a user function ("returned an invalid value") for the same reason it rejects one as a bound parameter. + */ + db.function("regexp", (pattern: unknown, text: unknown) => + typeof pattern !== "string" || typeof text !== "string" + ? null + : new RegExp(pattern).test(text) + ? 1 + : 0, + ); + + db.exec(SCHEMA); + db.exec(COERCION_SCHEMA); + + const insert = db.prepare( + "INSERT INTO subjects (id, age, name, active, joined, note) VALUES (?, ?, ?, ?, ?, ?)", + ); + for (const row of SUBJECTS) { + insert.run( + row.id, + row.age, + row.name, + row.active === null ? null : Number(row.active), + row.joined, + row.note, + ); + } + + // '9' and '10' straddle 5 differently as text than as numbers, and 1 and 0 are what SQLite stores a boolean as. Both pairs are chosen so a coerced comparison and an honest one disagree. + const insertCoercion = db.prepare( + "INSERT INTO coercion (label, numeric_text, flag) VALUES (?, ?, ?)", + ); + insertCoercion.run("nine", "9", 1); + insertCoercion.run("ten", "10", FALSE_AS_INTEGER); +}); + +afterAll(() => { + db.close(); +}); + +function selectMatching(node: PredicateNode): string[] { + const compiled = compilePredicateNode(node, sqliteSubjectOptions); + const rows = db + .prepare( + `SELECT id FROM subjects WHERE ${compiled.sql} ORDER BY id`, + ) + .all(...compiled.params.map(bindable)); + return rows.map((row) => row.id); +} + +async function evaluatorMatching(node: PredicateNode): Promise { + const matched: string[] = []; + for (const row of SUBJECTS) { + const evaluation = await evaluatePredicate( + node, + undefined, + resolversFor(row), + ); + if (evaluation.status === "definite" && evaluation.value) { + matched.push(row.id); + } + } + return matched.sort(); +} + +/** Runs the tree both ways and asserts they agree, then hands back the row set so a case can also state what that set should be. Agreement alone would be satisfied by both being wrong in the same way, so every caller asserts the expected ids too. */ +async function agreeingRows(node: PredicateNode): Promise { + const viaSql = selectMatching(node); + const viaEvaluator = await evaluatorMatching(node); + expect(viaSql).toEqual(viaEvaluator); + return viaSql; +} + +describe("comparisons against a column that can be NULL", () => { + const olderThan18: PredicateNode = { + kind: "compare", + op: "gt", + left: { kind: "reference", key: "age" }, + right: { kind: "numberLiteral", value: 18 }, + }; + + it("excludes the row whose age is unknown", async () => { + await expect(agreeingRows(olderThan18)).resolves.toEqual([ + "ada", + "unknown", + ]); + }); + + it("still excludes it under negation, which two-valued logic could not do", async () => { + // The load-bearing case. Under two-valued logic every row appears in exactly one of a predicate and its negation, so `lin` would have to turn up here. It does not, in either engine: NOT UNKNOWN is UNKNOWN in SQLite exactly as `not(indeterminate)` is indeterminate in trilean. + await expect( + agreeingRows({ kind: "not", operand: olderThan18 }), + ).resolves.toEqual(["grace"]); + }); + + it("keeps a row whose unknown comparison is absorbed by a true disjunct", async () => { + await expect( + agreeingRows({ + kind: "anyOf", + operands: [ + olderThan18, + { + kind: "textCompare", + op: "equals", + left: { kind: "reference", key: "name" }, + right: { kind: "textLiteral", value: "lin" }, + }, + ], + }), + ).resolves.toEqual(["ada", "lin", "unknown"]); + }); + + it("collapses an unknown conjunct absorbed by a false one, observably under negation", async () => { + // `unknown AND false` has to be FALSE rather than UNKNOWN, and the difference only shows through a NOT: a genuinely FALSE conjunction negates to TRUE and the row appears, whereas an UNKNOWN one would negate to UNKNOWN and it would not. `lin` appearing here is that absorption being exercised. + await expect( + agreeingRows({ + kind: "not", + operand: { + kind: "allOf", + operands: [ + olderThan18, + { + kind: "textCompare", + op: "equals", + left: { kind: "reference", key: "name" }, + right: { kind: "textLiteral", value: "nobody" }, + }, + ], + }, + }), + ).resolves.toEqual(["ada", "grace", "lin"]); + }); + + it("compares instants across a NULL, as the ISO-8601 text SQLite stores them as", async () => { + // SQLite has no timestamp type: an instant is stored and compared as text. Offset-bearing ISO-8601 in a common offset sorts chronologically as a string, which is what makes this agree with the evaluator's own instant comparison rather than merely happening to. + await expect( + agreeingRows({ + kind: "compare", + op: "lt", + left: { kind: "reference", key: "joined" }, + right: { kind: "instantLiteral", value: "2022-01-01T00:00:00Z" }, + }), + ).resolves.toEqual(["ada", "lin"]); + }); + + it("compares booleans for equality across a NULL, as the integers SQLite stores them as", async () => { + await expect( + agreeingRows({ + kind: "compare", + op: "eq", + left: { kind: "reference", key: "active" }, + right: { kind: "booleanLiteral", value: true }, + }), + ).resolves.toEqual(["ada", "lin"]); + }); +}); + +describe("exists", () => { + const hasNote: PredicateNode = { + kind: "exists", + operand: { kind: "reference", key: "note" }, + }; + + it("partitions the table, because it is the one predicate neither engine leaves unknown", async () => { + const present = await agreeingRows(hasNote); + const absent = await agreeingRows({ kind: "not", operand: hasNote }); + expect(present).toEqual(["ada", "grace"]); + expect(absent).toEqual(["lin", "unknown"]); + expect([...present, ...absent].sort()).toEqual( + SUBJECTS.map((row) => row.id).sort(), + ); + }); +}); + +describe("textCompare", () => { + it("matches a pattern through the registered REGEXP function", async () => { + await expect( + agreeingRows({ + kind: "textCompare", + op: "matches", + left: { kind: "reference", key: "name" }, + right: { kind: "textLiteral", value: "^(a|g)" }, + }), + ).resolves.toEqual(["ada", "grace"]); + }); + + it("leaves a NULL operand unknown under a negated match", async () => { + // What a registered function has to get right, and the reason README.md spells the registration out rather than leaving it to the reader: `lin` is here because its name does not match, and `unknown` is absent because its name is not known. A regexp function that answered 0 for a NULL value instead of NULL would put `unknown` here too. + await expect( + agreeingRows({ + kind: "textCompare", + op: "notMatches", + left: { kind: "reference", key: "name" }, + right: { kind: "textLiteral", value: "^a" }, + }), + ).resolves.toEqual(["grace", "lin"]); + }); + + it("fails loudly rather than answering wrongly when REGEXP is not registered", () => { + // The one thing the SQLite dialect asks of its caller, and the reason asking is acceptable: an unregistered REGEXP is a query error naming the missing function, not a fragment that quietly matches nothing. + const bare = new Database(":memory:"); + try { + bare.exec(SCHEMA); + const compiled = compilePredicateNode( + { + kind: "textCompare", + op: "matches", + left: { kind: "reference", key: "name" }, + right: { kind: "textLiteral", value: "^a" }, + }, + sqliteSubjectOptions, + ); + expect(() => + bare + .prepare(`SELECT id FROM subjects WHERE ${compiled.sql}`) + .all(...compiled.params.map(bindable)), + ).toThrow(/no such function: REGEXP/i); + } finally { + bare.close(); + } + }); +}); + +describe("memberOf", () => { + it("matches a candidate list", async () => { + await expect( + agreeingRows({ + kind: "memberOf", + op: "in", + operand: { kind: "reference", key: "name" }, + candidates: [ + { kind: "textLiteral", value: "ada" }, + { kind: "textLiteral", value: "nobody" }, + ], + }), + ).resolves.toEqual(["ada"]); + }); + + it("leaves NOT IN unknown for a NULL operand", async () => { + await expect( + agreeingRows({ + kind: "memberOf", + op: "notIn", + operand: { kind: "reference", key: "name" }, + candidates: [{ kind: "textLiteral", value: "ada" }], + }), + ).resolves.toEqual(["grace", "lin"]); + }); + + it("compiles an empty 'in' to something that is false, not unknown, for a known operand", async () => { + // The encoding PostgreSQL needs a `::boolean` on and SQLite does not, so this is where the missing annotation is shown not to matter. Executed rather than asserted as a string, because `(x IS NULL AND NULL)` is only the right encoding if SQLite really does evaluate it to FALSE for a known operand and NULL for an unknown one. Negating it is what separates those two outcomes: only the rows with a known name come back. + const node: PredicateNode = { + kind: "memberOf", + op: "in", + operand: { kind: "reference", key: "name" }, + candidates: [], + }; + await expect(agreeingRows(node)).resolves.toEqual([]); + await expect(agreeingRows({ kind: "not", operand: node })).resolves.toEqual( + ["ada", "grace", "lin"], + ); + }); + + it("compiles an empty 'notIn' to something that is true, not unknown, for a known operand", async () => { + const node: PredicateNode = { + kind: "memberOf", + op: "notIn", + operand: { kind: "reference", key: "name" }, + candidates: [], + }; + await expect(agreeingRows(node)).resolves.toEqual(["ada", "grace", "lin"]); + await expect(agreeingRows({ kind: "not", operand: node })).resolves.toEqual( + [], + ); + }); +}); + +describe("degenerate and adversarial fragments", () => { + it("executes a comparison between two literals, which needs no placeholder typed", async () => { + // The mirror of the PostgreSQL case: there, both placeholders must carry a cast or the server rejects the statement outright; here, two bare `?` are enough, which is why the SQLite dialect emits no cast at all. + await expect( + agreeingRows({ + kind: "compare", + op: "lt", + left: { kind: "numberLiteral", value: 1 }, + right: { kind: "numberLiteral", value: 2 }, + }), + ).resolves.toEqual(["ada", "grace", "lin", "unknown"]); + }); + + it("executes an empty allOf and anyOf as their identities", async () => { + await expect( + agreeingRows({ kind: "allOf", operands: [] }), + ).resolves.toEqual(["ada", "grace", "lin", "unknown"]); + await expect( + agreeingRows({ kind: "anyOf", operands: [] }), + ).resolves.toEqual([]); + }); + + it("treats an injection attempt as data and leaves the table standing", async () => { + await expect( + agreeingRows({ + kind: "textCompare", + op: "equals", + left: { kind: "reference", key: "name" }, + right: { + kind: "textLiteral", + value: "ada'; DROP TABLE subjects; --", + }, + }), + ).resolves.toEqual([]); + + const surviving = db + .prepare<[], { count: number }>("SELECT count(*) AS count FROM subjects") + .get(); + expect(surviving?.count).toBe(SUBJECTS.length); + }); + + it("neutralises a hostile column name into one identifier the engine rejects", () => { + // A column name is the one caller-supplied string that has to reach the SQL text, so quoting is what makes it safe rather than parameterisation. Asserting the quoted string is not the same as establishing that SQLite reads it as a single inert identifier: this executes it, and the engine refusing it as a column that does not exist is the proof. The failure it rules out is the opposite outcome -- the injected `OR` taking effect and the fragment matching every row. + const hostile: SqlCompileOptions = { + dialect: "sqlite", + columnFor: () => ({ column: `name" = name OR "1` }), + }; + const compiled = compilePredicateNode( + { + kind: "textCompare", + op: "equals", + left: { kind: "reference", key: "name" }, + right: { kind: "textLiteral", value: "ada" }, + }, + hostile, + ); + expect(compiled.sql).toBe(`("name"" = name OR ""1" = ?)`); + + expect(() => + db + .prepare(`SELECT id FROM subjects WHERE ${compiled.sql}`) + .all(...compiled.params.map(bindable)), + ).toThrow(/no such column/i); + }); +}); + +describe("the divergences the guard's refusals exist to prevent", () => { + /** + * Each case here refuses a tree and then measures, against this connection, the wrong answer the refusal avoided. The refusals themselves are inherited unchanged from the PostgreSQL dialect, and that inheritance is exactly what needs evidence: it would be worth nothing if SQLite's affinity system happened to agree with trilean where PostgreSQL's coercion does not. + */ + + it("refuses NaN, and measures the driver substitution that refusal exists to prevent", async () => { + // A divergence in the opposite direction from PostgreSQL's, which is why the reason text is the dialect's own rather than a shared one. SQLite has no NaN: better-sqlite3 binds one as SQL NULL, so `NaN = NaN` is indeterminate there and matches nothing -- which happens to look like agreement -- while its negation matches nothing either, where trilean's `not(definite(false))` is definitely true and matches every row. The negation is the case that makes the divergence visible, so both are measured. + const equality: PredicateNode = { + kind: "compare", + op: "eq", + left: { kind: "numberLiteral", value: Number.NaN }, + right: { kind: "numberLiteral", value: Number.NaN }, + }; + expect(() => compilePredicateNode(equality, sqliteSubjectOptions)).toThrow( + /cannot compile 'numberLiteral'/, + ); + + const bound = db + .prepare<[number], { storedType: string }>( + "SELECT typeof(?) AS storedType", + ) + .get(Number.NaN); + expect(bound?.storedType).toBe("null"); + + const equalityRows = db + .prepare<[number, number], { id: string }>( + "SELECT id FROM subjects WHERE (? = ?) ORDER BY id", + ) + .all(Number.NaN, Number.NaN); + expect(equalityRows.map((row) => row.id)).toEqual([]); + await expect(evaluatorMatching(equality)).resolves.toEqual([]); + + const negation: PredicateNode = { kind: "not", operand: equality }; + const negatedRows = db + .prepare<[number, number], { id: string }>( + "SELECT id FROM subjects WHERE (NOT (? = ?)) ORDER BY id", + ) + .all(Number.NaN, Number.NaN); + expect(negatedRows.map((row) => row.id)).toEqual([]); + await expect(evaluatorMatching(negation)).resolves.toEqual( + SUBJECTS.map((row) => row.id).sort(), + ); + }); + + it("refuses an ordered text operand, and measures the lexicographic answer that refusal exists to prevent", () => { + // 9 and 10 are both greater than 5. Compared under the column's own TEXT affinity, which SQLite applies to the numeric side rather than the other way round, '9' > '5' and '10' > '5' disagree -- so the row that comes back is the wrong one, with no error and no warning. trilean returns wrong-type for the same comparison and directs the caller to `textCompare`. + const orderedText: PredicateNode = { + kind: "compare", + op: "gt", + left: { kind: "reference", key: "name" }, + right: { kind: "textLiteral", value: "ada" }, + }; + expect(() => + compilePredicateNode(orderedText, sqliteSubjectOptions), + ).toThrow(/cannot compile 'compare'/); + + const coerced = db + .prepare<[number], { label: string }>( + "SELECT label FROM coercion WHERE numeric_text > ? ORDER BY label", + ) + .all(COERCION_THRESHOLD); + expect(coerced.map((row) => row.label)).toEqual(["nine"]); + }); + + it("refuses an ordered boolean, and measures the integer ordering that refusal exists to prevent", () => { + // SQLite has no boolean type, so `active > false` is an ordering over the integers 0 and 1 and answers definitely. trilean has no ordering for booleans at all. + const orderedBoolean: PredicateNode = { + kind: "compare", + op: "gt", + left: { kind: "reference", key: "active" }, + right: { kind: "booleanLiteral", value: false }, + }; + expect(() => + compilePredicateNode(orderedBoolean, sqliteSubjectOptions), + ).toThrow(/cannot compile 'compare'/); + + const ordered = db + .prepare<[number], { label: string }>( + "SELECT label FROM coercion WHERE flag > ? ORDER BY label", + ) + .all(FALSE_AS_INTEGER); + expect(ordered.map((row) => row.label)).toEqual(["nine"]); + }); + + it("refuses a cross-kind comparison, and measures the coercion that refusal exists to prevent", () => { + // No column and no affinity involved: SQLite still answers, ordering every text value above every numeric one by storage class rather than reporting a type error. trilean calls the same comparison wrong-type. + const crossKindTree: PredicateNode = { + kind: "compare", + op: "gt", + left: { kind: "reference", key: "name" }, + right: { kind: "numberLiteral", value: COERCION_THRESHOLD }, + }; + expect(() => + compilePredicateNode(crossKindTree, sqliteSubjectOptions), + ).toThrow(/cannot compile 'compare'/); + + const crossKind = db + .prepare<[string, number], { answer: number }>("SELECT (? > ?) AS answer") + .get("abc", COERCION_THRESHOLD); + expect(crossKind?.answer).toBe(1); + }); +}); + +describe("a tree deep enough to mix every supported kind", () => { + it("agrees with the evaluator row for row", async () => { + const node: PredicateNode = { + kind: "anyOf", + operands: [ + { + kind: "and", + left: { + kind: "compare", + op: "gte", + left: { kind: "reference", key: "age" }, + right: { kind: "numberLiteral", value: 18 }, + }, + right: { + kind: "not", + operand: { + kind: "memberOf", + op: "in", + operand: { kind: "reference", key: "name" }, + candidates: [{ kind: "textLiteral", value: "grace" }], + }, + }, + }, + { + kind: "allOf", + operands: [ + { kind: "exists", operand: { kind: "reference", key: "note" } }, + { + kind: "or", + left: { + kind: "textCompare", + op: "matches", + left: { kind: "reference", key: "note" }, + right: { kind: "textLiteral", value: "^h" }, + }, + right: { + kind: "compare", + op: "eq", + left: { kind: "reference", key: "active" }, + right: { kind: "booleanLiteral", value: false }, + }, + }, + ], + }, + ], + }; + + await expect(agreeingRows(node)).resolves.toEqual(["ada", "grace"]); + }); +}); diff --git a/packages/trilean-sql/vitest.config.ts b/packages/trilean-sql/vitest.config.ts index cd60a12..b9020c3 100644 --- a/packages/trilean-sql/vitest.config.ts +++ b/packages/trilean-sql/vitest.config.ts @@ -1,10 +1,10 @@ import { defineConfig } from "vitest/config"; -// Two projects, mirroring packages/trilean's own split: "unit" covers the compiler itself (pure string and parameter production, no I/O), "integration" runs the compiled fragments against a real PostgreSQL engine. +// Two projects, mirroring packages/trilean's own split: "unit" covers the compiler itself (pure string and parameter production, no I/O), "integration" runs the compiled fragments against a real engine. // -// The integration project is where the three-valued-logic claim is actually tested rather than asserted: a compiled fragment's truth table is only meaningful once PostgreSQL's own planner has evaluated it against real NULLs, so those tests execute SQL rather than comparing strings. Every file under test/integration/ belongs to it, and the two there today run the same parity suite against two ways of reaching that planner: postgres.test.ts against a server started as an ephemeral container, which needs a working Docker daemon (testcontainers), and pglite.test.ts against PGlite, the same PostgreSQL compiled to WebAssembly and run in process, which needs nothing beyond Node. The Docker requirement of the first is why the project as a whole is separate from the default `pnpm test` and has its own CI job. +// The integration project is where the three-valued-logic claim is actually tested rather than asserted: a compiled fragment's truth table is only meaningful once the engine's own planner has evaluated it against real NULLs, so those tests execute SQL rather than comparing strings. Every file under test/integration/ belongs to it, and the three there today run the same parity suite against three ways of reaching that planner: postgres.test.ts against a server started as an ephemeral container, which needs a working Docker daemon (testcontainers); pglite.test.ts against PGlite, the same PostgreSQL compiled to WebAssembly and run in process, which needs nothing beyond Node; and sqlite.test.ts against a real SQLite engine in memory, which likewise needs nothing beyond Node. The Docker requirement of the first is why the project as a whole is separate from the default `pnpm test` and has its own CI job. // -// One project rather than one per engine, because they are the same tests: `vitest run --project integration` is the whole integration surface, and a third engine is a new file rather than new configuration. +// One project rather than one per engine, because they are the same tests: `vitest run --project integration` is the whole integration surface, and a further engine is a new file rather than new configuration. export default defineConfig({ test: { coverage: { @@ -24,7 +24,7 @@ export default defineConfig({ test: { name: "integration", include: ["test/integration/**/*.test.ts"], - // Starting the engine, applying the schema and seeding it happens once for the whole file; a cold `docker pull` on a runner with no cached image dominates that. The default 5s would fail on image pull alone. PGlite needs a fraction of it -- instantiating a WASM module rather than pulling an image -- but the timeout is the project's, and a generous one costs a suite that passes nothing. + // Starting the engine, applying the schema and seeding it happens once for the whole file; a cold `docker pull` on a runner with no cached image dominates that. The default 5s would fail on image pull alone. PGlite and SQLite need a fraction of it -- instantiating in process rather than pulling an image -- but the timeouts are set on the project rather than the PostgreSQL file alone, and a generous ceiling costs an in-memory suite that never approaches it nothing. testTimeout: 120_000, hookTimeout: 300_000, }, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index ed2d1cf..7c52ab1 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -163,6 +163,9 @@ importers: '@testcontainers/postgresql': specifier: ^12.1.0 version: 12.1.0 + '@types/better-sqlite3': + specifier: ^9.6.0 + version: 9.6.0 '@types/node': specifier: ^26.4.0 version: 26.4.1 @@ -172,6 +175,9 @@ importers: '@vitest/coverage-v8': specifier: ^4.1.11 version: 4.1.11(vitest@4.1.11) + better-sqlite3: + specifier: ^12.11.1 + version: 12.11.1 eslint: specifier: ^10.9.1 version: 10.9.1(jiti@2.6.1) @@ -1353,6 +1359,9 @@ packages: cpu: [arm64] os: [win32] + '@types/better-sqlite3@9.6.0': + resolution: {integrity: sha512-ZEEwBSgMu7GYJOynoagg5X9JbxfL6dTJDsgViJIqh67jV44kyOr9RXfmFjLK5rzC4MWssP06t9hu/JwGDnUbCg==} + '@types/chai@5.2.3': resolution: {integrity: sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==} @@ -1784,6 +1793,13 @@ packages: before-after-hook@4.0.0: resolution: {integrity: sha512-q6tR3RPqIB1pMiTRMFcZwuG5T8vwp+vUvEG0vuI6B+Rikh5BfPp2fQ82c925FOs+b0lcFQ8CFrL+KbilfZFhOQ==} + better-sqlite3@12.11.1: + resolution: {integrity: sha512-dq9AtApgg5PGFtBzPFSBl3HZQjHok5gaQCM6zh2Yk0aSmDCs1CbnVI8/HgASQkNKsWFpseIO9beg5xxpYhbIfA==} + engines: {node: 20.x || 22.x || 23.x || 24.x || 25.x || 26.x} + + bindings@1.5.0: + resolution: {integrity: sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==} + bl@4.1.0: resolution: {integrity: sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==} @@ -2024,6 +2040,10 @@ packages: supports-color: optional: true + decompress-response@6.0.0: + resolution: {integrity: sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==} + engines: {node: '>=10'} + deep-extend@0.6.0: resolution: {integrity: sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==} engines: {node: '>=4.0.0'} @@ -2230,6 +2250,10 @@ packages: resolution: {integrity: sha512-9Be3ZoN4LmYR90tUoVu2te2BsbzHfhJyfEiAVfz7N5/zv+jduIfLrV2xdQXOHbaD6KgpGdO9PRPM1Y4Q9QkPkA==} engines: {node: ^18.19.0 || >=20.5.0} + expand-template@2.0.3: + resolution: {integrity: sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==} + engines: {node: '>=6'} + expect-type@1.4.0: resolution: {integrity: sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==} engines: {node: '>=12.0.0'} @@ -2276,6 +2300,9 @@ packages: resolution: {integrity: sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ==} engines: {node: '>=16.0.0'} + file-uri-to-path@1.0.0: + resolution: {integrity: sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==} + fill-range@7.1.1: resolution: {integrity: sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg==} engines: {node: '>=8'} @@ -2354,6 +2381,9 @@ packages: git-log-parser@1.2.1: resolution: {integrity: sha512-PI+sPDvHXNPl5WNOErAK05s3j0lgwUzMN6o8cyQrDaKfT3qd7TmNJKeXX+SknI5I0QhG5fVPAEwSY4tRGDtYoQ==} + github-from-package@0.0.0: + resolution: {integrity: sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==} + glob-parent@6.0.2: resolution: {integrity: sha512-XxwI8EOhVQgWp6iDL+3b0r86f4d6AX6zSU55HfB4ydCEuXLXc5FcYeOu+nnGftS4TEju/11rt4KJPTMgbfmv4A==} engines: {node: '>=10.13.0'} @@ -2783,6 +2813,10 @@ packages: resolution: {integrity: sha512-vqiC06CuhBTUdZH+RYl8sFrL096vA45Ok5ISO6sE/Mr1jRbGH4Csnhi8f3wKVl7x8mO4Au7Ir9D3Oyv1VYMFJw==} engines: {node: '>=12'} + mimic-response@3.1.0: + resolution: {integrity: sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==} + engines: {node: '>=10'} + miniflare@5.20260815.0-alpha: resolution: {integrity: sha512-YAaGj4Sh5f4fqHKiMQ8zRHDOOM5IGUVtMhnLIeyjuQfU+9P6hcOTrHUVtbfj/ZPay9Kzik4pWELB39pGgefjiQ==} engines: {node: '>=22.0.0'} @@ -2832,6 +2866,9 @@ packages: engines: {node: ^10 || ^12 || ^13.7 || ^14 || >=15.0.1} hasBin: true + napi-build-utils@2.0.0: + resolution: {integrity: sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==} + natural-compare@1.4.0: resolution: {integrity: sha512-OWND8ei3VtNC9h7V60qff3SVobHr996CTwgxubgyQYEpg290h9J0buyECNNJexkFm5sOajh5G116RYA1c8ZMSw==} @@ -2841,6 +2878,10 @@ packages: nerf-dart@1.0.0: resolution: {integrity: sha512-EZSPZB70jiVsivaBLYDCyntd5eH8NTSMOn3rB+HxwdmKThGELLdYv8qVIMWvZEFy9w8ZZpW9h9OB32l1rGtj7g==} + node-abi@3.96.0: + resolution: {integrity: sha512-rebQ/lz7i0EkoLzUVSrKRzA69zMkwLp95kKMWoMDkkM00Suxz0D7zEQPwRml5fQum24mj7bPvmlgLAmu2JCiYg==} + engines: {node: '>=10'} + node-emoji@2.2.0: resolution: {integrity: sha512-Z3lTE9pLaJF47NyMhd4ww1yFTAP8YhYI8SleJiHzM46Fgpm5cnNzSl9XfzFNqbaz+VlJrIj3fXQ4DeN1Rjm6cw==} engines: {node: '>=18'} @@ -3141,6 +3182,12 @@ packages: resolution: {integrity: sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==} engines: {node: '>=0.10.0'} + prebuild-install@7.1.3: + resolution: {integrity: sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==} + engines: {node: '>=10'} + deprecated: No longer maintained. Please contact the author of the relevant native addon; alternatives are available. + hasBin: true + prelude-ls@1.2.1: resolution: {integrity: sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g==} engines: {node: '>= 0.8.0'} @@ -3341,6 +3388,12 @@ packages: resolution: {integrity: sha512-iuh+gPf28RkltuJC7W5MRi6XAjTDCAPC/prJUpQoG4vIP3MJZ+GTydVnodXA7pwvTKb2cA0m9OFZW/cdWy/I/w==} engines: {node: '>=6'} + simple-concat@1.0.1: + resolution: {integrity: sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==} + + simple-get@4.0.1: + resolution: {integrity: sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==} + skin-tone@2.0.0: resolution: {integrity: sha512-kUMbT1oBJCpgrnKoSr0o6wPtvRWT9W9UKvGLwfJYO2WuahZRHOpEyL1ckyMGgMWh0UdpmaoFqKKD29WTomNEGA==} engines: {node: '>=8'} @@ -3599,6 +3652,9 @@ packages: engines: {node: '>=18.0.0'} hasBin: true + tunnel-agent@0.6.0: + resolution: {integrity: sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==} + tunnel@0.0.6: resolution: {integrity: sha512-1h/Lnq9yajKY2PEbBadPXj3VxsDDu844OnaAo52UVmIzIvwwtBPIuNvkjuzBlTWpfJyUbG3ez0KSBibQkj4ojg==} engines: {node: '>=0.6.11 <=0.7.0 || >=0.7.3'} @@ -4900,6 +4956,10 @@ snapshots: '@turbo/windows-arm64@2.10.12': optional: true + '@types/better-sqlite3@9.6.0': + dependencies: + '@types/node': 26.4.1 + '@types/chai@5.2.3': dependencies: '@types/deep-eql': 4.0.2 @@ -5317,6 +5377,15 @@ snapshots: before-after-hook@4.0.0: {} + better-sqlite3@12.11.1: + dependencies: + bindings: 1.5.0 + prebuild-install: 7.1.3 + + bindings@1.5.0: + dependencies: + file-uri-to-path: 1.0.0 + bl@4.1.0: dependencies: buffer: 5.7.1 @@ -5551,6 +5620,10 @@ snapshots: dependencies: ms: 2.1.3 + decompress-response@6.0.0: + dependencies: + mimic-response: 3.1.0 + deep-extend@0.6.0: {} deep-is@0.1.4: {} @@ -5833,6 +5906,8 @@ snapshots: strip-final-newline: 4.0.0 yoctocolors: 2.2.0 + expand-template@2.0.3: {} + expect-type@1.4.0: {} fast-deep-equal@3.1.3: {} @@ -5865,6 +5940,8 @@ snapshots: dependencies: flat-cache: 4.0.1 + file-uri-to-path@1.0.0: {} + fill-range@7.1.1: dependencies: to-regex-range: 5.0.1 @@ -5938,6 +6015,8 @@ snapshots: through2: 2.0.5 traverse: 0.6.8 + github-from-package@0.0.0: {} + glob-parent@6.0.2: dependencies: is-glob: 4.0.3 @@ -6306,6 +6385,8 @@ snapshots: mimic-fn@4.0.0: {} + mimic-response@3.1.0: {} + miniflare@5.20260815.0-alpha: dependencies: '@cspotcode/source-map-support': 0.8.1 @@ -6353,12 +6434,18 @@ snapshots: nanoid@3.3.18: {} + napi-build-utils@2.0.0: {} + natural-compare@1.4.0: {} neo-async@2.6.2: {} nerf-dart@1.0.0: {} + node-abi@3.96.0: + dependencies: + semver: 7.8.5 + node-emoji@2.2.0: dependencies: '@sindresorhus/is': 4.6.0 @@ -6567,6 +6654,21 @@ snapshots: dependencies: xtend: 4.0.2 + prebuild-install@7.1.3: + dependencies: + detect-libc: 2.1.2 + expand-template: 2.0.3 + github-from-package: 0.0.0 + minimist: 1.2.8 + mkdirp-classic: 0.5.3 + napi-build-utils: 2.0.0 + node-abi: 3.96.0 + pump: 3.0.4 + rc: 1.2.8 + simple-get: 4.0.1 + tar-fs: 2.1.5 + tunnel-agent: 0.6.0 + prelude-ls@1.2.1: {} prettier-linter-helpers@1.0.1: @@ -6843,6 +6945,14 @@ snapshots: figures: 2.0.0 pkg-conf: 2.1.0 + simple-concat@1.0.1: {} + + simple-get@4.0.1: + dependencies: + decompress-response: 6.0.0 + once: 1.4.0 + simple-concat: 1.0.1 + skin-tone@2.0.0: dependencies: unicode-emoji-modifier-base: 1.0.0 @@ -7144,6 +7254,10 @@ snapshots: optionalDependencies: fsevents: 2.3.3 + tunnel-agent@0.6.0: + dependencies: + safe-buffer: 5.2.1 + tunnel@0.0.6: {} turbo@2.10.12: diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index f9e8aac..1e78a5a 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -9,7 +9,10 @@ linkWorkspacePackages: true # workerd (the Cloudflare Workers runtime @cloudflare/vitest-pool-workers drives the workers test suite through) and esbuild (its bundler) both need theirs: the script is how each fetches its own platform's binary, and neither runs at all without it. # # The three refused below arrive under testcontainers, which packages/trilean-sql's integration suite uses to start PostgreSQL. Each script is optional to the way this workspace uses the package: ssh2 and its cpu-features helper build native acceleration for SSH transports, and testcontainers reaches Docker over a local socket rather than SSH; protobufjs's script only regenerates bundled artefacts the published tarball already contains. Refusing them keeps a test-only dependency chain from executing arbitrary code at install time. +# +# better-sqlite3 is the engine packages/trilean-sql's SQLite integration suite executes its compiled fragments against, and it is a native addon: its install script is what fetches, or failing that builds, the binding the module cannot load without. Refusing it would leave the package installed and unloadable rather than merely unoptimised. allowBuilds: + better-sqlite3: true cpu-features: false esbuild: true protobufjs: false