Skip to content

Add temporal resource support - #233

Draft
C-Sinclair wants to merge 3 commits into
ash-project:mainfrom
C-Sinclair:feat/temporal-support
Draft

C-Sinclair wants to merge 3 commits into
ash-project:mainfrom
C-Sinclair:feat/temporal-support

Conversation

@C-Sinclair

@C-Sinclair C-Sinclair commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Contributor checklist

Leave anything that you believe does not apply unchecked.

  • I accept the AI Policy, or AI was not used in the creation of this PR.
  • Bug fixes include regression tests
  • Chores
  • Documentation changes
  • Features include unit/acceptance tests
  • Refactoring
  • Update dependencies

What

Temporal resource support for ash_sqlite, matching the feature on ash's temporal branch.

Postgres splits a validity period with one statement: UPDATE ... FOR PORTION OF valid_at FROM $as_of TO NULL, over a PRIMARY KEY (id, valid_at WITHOUT OVERLAPS). The key shipped in PostgreSQL 18; FOR PORTION OF is PG19.

SQLite has none of it, and no data-modifying CTE to build it from. So the period arithmetic happens in the data layer, the shape Ash.DataLayer.Ets already proves sufficient, with real SQLite constraints underneath.

The whole feature is compile-gated on Code.ensure_loaded?(Ash.Temporal), so it is inert on any released ash.

How a write works

AshSqlite.Temporal splits a period in three statements inside a transaction. A subscription created in January on the free tier is one row:

Jan ─────────────────────────────────▶  free

Moving it to pro in March gives the same two rows Postgres reaches in one statement:

Jan ──────────▶ Mar                     free   (1. closed at Mar)
                Mar ────────────────▶   pro    (2. copied forward, 3. updated)

Statement 3 is the ordinary non-temporal update path, which is why atomics, RETURNING and the changeset's filters need no temporal branch inside them.

Closing before inserting means the two periods never overlap between statements, so the non-overlap trigger sees no transient violation. The reverse order trips both the trigger and the partial unique index.

The transaction is opened in the data layer rather than by Ash. Ash.Actions.Update.Bulk wraps an atomic update only when the resource has after-batch hooks or the data layer asks for prefer_transaction_for_atomic_updates?, and this one answers false because a non-temporal atomic update is a single statement. A split is three, so AshSqlite.Temporal.transactionally/2 opens one with mode: :immediate, and AshSqlite.Verifiers.VerifyTemporal refuses a temporal resource whose repo leaves write_transactions? off.

Row identity on a temporal table is the primary key plus the period, so pkey_filter/2 carries the period, and so does the join bulk_updatable_query/6 builds when a query needs one.

What the database enforces

Postgres gets uniqueness and the access path from one declaration. SQLite needs three pieces, which AshSqlite.Temporal.Migration generates for the primary key and for every identity:

  • An index on (key..., json_extract(period, '$.lower')). SQLite can index an expression, so a keyed point-in-time read is a seek.
  • A partial unique index over the open-ended period. UNIQUE (key...) WHERE json_extract(period, '$.upper') IS NULL refuses a second current version for a key, which is what a split that forgets to close the prior version would produce.
  • A BEFORE INSERT / BEFORE UPDATE trigger pair rejecting overlapping closed periods with RAISE(ABORT).

The insert trigger omits the NEW.rowid exclusion the update trigger carries. An inserted row has no rowid yet, so that comparison is NULL and the trigger would never fire.

For a temporal resource the generator also stops emitting a PRIMARY KEY, plain unique indexes for identities, and any foreign key to a temporal destination. A temporal table has no unique key to reference; SQLite creates the child table anyway, then fails every insert into it with foreign key mismatch.

Reads

set_as_of/3 renders the containment test against the same json_extract expression the index is built on:

json_extract(valid_at, '$.lower') <= ?
  AND (json_extract(valid_at, '$.upper') IS NULL
       OR json_extract(valid_at, '$.upper') > ?)

Both bounds stay in the JSON rather than becoming generated columns. Generated columns reach an identical plan and cost every temporal resource two extra columns.

Compatibility

Built against Tests Warnings
Released ash 3.33.6, Elixir 1.18 / OTP 27 198 0
ash temporal branch, Elixir 1.20 / OTP 29 256 0

On released ash can?(_, :temporal) is false, set_as_of/3 is not defined, the test resources and their suite do not compile, and the compiler removes the rest. Generated migrations for a non-temporal resource are byte-identical to main, snapshot hashes included.

CI runs against released ash, so it proves nothing regressed but never executes a line of the temporal path or any of its 58 tests, which is the position ash_postgres is in too.

The "temporal" arms added to ash_version/1 and ash_sql_version/1 run the suite against the unreleased branches. Happy to drop them for ASH_VERSION=main once temporal merges.

Performance

200k rows, 20k keys × 10 versions, both sides at the driver rather than through Ash. Postgres is 19beta3 in a container, running PRIMARY KEY (id, valid_at WITHOUT OVERLAPS) and UPDATE ... FOR PORTION OF.

Operation PostgreSQL 19beta3 SQLite 3.53
Point-in-time read, one key 0.337 ms 0.010 ms
Every current version 0.886 ms 0.323 ms
Every version valid at T, unkeyed 7.070 ms 60.728 ms
Split one period 0.291 ms 0.079 ms
Split 100 periods 1.971 ms 10.470 ms

Read the first two rows with the transport in mind. Postgres is over TCP into a container and SQLite in process. The same PostgreSQL 18 answers the keyed read in 0.039 ms over a local Unix socket and 0.365 ms in a container, so roughly 0.3 ms of that gap is transport. Subtract it and SQLite is about four times faster on a keyed read rather than thirty.

The unkeyed scan is not transport-bound, and it is where Postgres wins outright:

Rows PostgreSQL 19 (GiST) SQLite (scan)
50k 6.41 ms 16.39 ms
200k 7.11 ms 56.09 ms
800k 13.43 ms 207.86 ms
2M 31.17 ms 511.87 ms

Where it stops

None of these work, and each needs a change somewhere.

Not supported What it would take
A relationship between two temporal resources ash_sql rendering parent/1 outside a lateral join. Ash.Resource.Verifiers.ValidateTemporalKeys requires temporal_keys, which bakes range_overlaps(parent(valid_at), valid_at) into the relationship filter; can?(_, {:lateral_join, _}) is false here, so AshSql.Expr.default_dynamic_expr/6 raises on the missing :parent_bindings. A relationship to a temporal resource works, via temporal_keys {nil, :valid_at}.
PERIOD foreign keys Nothing will fix this. SQLite has no such feature.
update_timestamp resolving to as_of Threading the write instant into update defaults, as AshPostgres.DataLayer.update_defaults/2 does. Today it reads the wall clock.
Bounds other than [) Honouring the attribute's inclusivity in the containment test, the triggers and the partial index, or refusing it in VerifyTemporal.
Polymorphic resources, attribute source:, attribute multitenancy Reading the table and column names from the resource rather than assuming the attribute name, and carrying the tenant attribute into identity key sets.

Two things found in ash while building this

  • Ash.Temporal.raw_instant/2 accepts only %DateTime{} or :now, so a :date resource cannot be given an explicit Date as as_of, even though Ash.Temporal.now_for/1 returns a Date for that same resource.
  • parent/1 in a relationship filter needs a lateral join. When a related query runs standalone the parent's value is already known, so substituting it as a literal, or rewriting the overlap as a correlated EXISTS, would serve every data layer without lateral joins.

Conor Sinclair added 3 commits September 20, 2026 18:59
Temporal lives on the `temporal` branches of ash and ash_sql and is in no
released version, so ASH_VERSION and ASH_SQL_VERSION gain a "temporal" case
alongside the existing "main" and "local" ones.

    ASH_VERSION=temporal ASH_SQL_VERSION=temporal mix test

Without it the feature compiles out entirely and the suite runs as before.
SQLite has no range type, so a range attribute is stored as a single JSON
text column and every range predicate compiles to comparisons on
json_extract of its keys.

AshSqlite.Type.RangeBound encodes both sides of every comparison at one
fixed ISO8601 precision. The comparison a range predicate performs is the
inner type's stored comparison rather than the adapter's, and the two do not
agree: DateTime.to_iso8601/1 keeps whatever precision the value carries, so
one instant has two spellings that compare unequal and in the wrong
direction. An absent bound is JSON null, so an unbounded side is tested with
IS NULL rather than against a sentinel of the inner type.

AshSqlite.SqlImplementation renders range_overlaps, range_contains over a
range and over a point, range_adjacent, range_lower and range_upper.
attribute_ecto_type/2 covers the write path, which the expression seam does
not reach. The migration generator maps Ash.Type.Range to :text.

Each test asserts agreement with the runtime Ash.Range function it mirrors.
A temporal resource stores one row per validity period. Postgres splits a
period with one statement, `UPDATE ... FOR PORTION OF valid_at FROM $as_of
TO NULL`, over a `PRIMARY KEY (id, valid_at WITHOUT OVERLAPS)` exclusion
constraint. SQLite has none of that, and no data-modifying CTE to build it
out of, so AshSqlite.Temporal reaches the same outcome in three statements:
close the version whose period contains as_of, copy that row forward under
[as_of, prior_upper), then apply the caller's update to the copy. Step three
is the ordinary non-temporal update path, which is why atomics, RETURNING
and the changeset's filters need no temporal branch inside them.

Closing before inserting means the two periods never overlap at any point
between statements, so the non-overlap trigger sees no transient violation.
The reverse order trips both the trigger and the partial unique index.

The transaction is opened here rather than by Ash. Ash wraps an atomic
update only when the resource has after-batch hooks or the data layer asks
for prefer_transaction_for_atomic_updates?, and this one answers false
because a non-temporal atomic update is a single statement. Without it a
failure in the third statement leaves the history rewritten with no current
version, and two concurrent writers interleave and lose a write.
AshSqlite.Verifiers.VerifyTemporal refuses a temporal resource whose repo
leaves write transactions off.

The row identity on a temporal table is the primary key plus the period, so
pkey_filter/2 carries the period and so does the join
bulk_updatable_query/6 builds when a query needs one. The key alone names
every version, and joining on it rewrites closed history.

set_as_of/3 renders the containment test against the same json_extract
expression the index is built on, so a keyed point-in-time read is an index
seek rather than a scan.

The upsert resolves its own match. SQLite will not use a partial unique
index as an ON CONFLICT target and the only unique index on a temporal
table is the partial one; Postgres cannot use ON CONFLICT against its
exclusion constraint either.

AshSqlite.Temporal.Migration generates what SQLite will not give from one
declaration: an index on each key set plus the period's lower bound, a
partial unique index refusing a second current version per key set, and a
non-overlap trigger pair. Every identity gets the same treatment, because an
identity on a temporal resource is unique per period rather than per table.
The insert trigger deliberately omits the NEW.rowid exclusion the update
trigger carries: an inserted row has no rowid yet, so the comparison is NULL
and the trigger would silently never fire.

The generator also stops emitting three things for a temporal resource. A
PRIMARY KEY, because it makes a second version impossible and on an integer
key makes the column SQLite's rowid alias. Identities as plain unique
indexes, for the same reason. And a foreign key to a temporal destination:
there is no unique key to point at, and SQLite creates the table anyway and
then fails every insert into the child with "foreign key mismatch".
@C-Sinclair
C-Sinclair force-pushed the feat/temporal-support branch from bfe88db to 55262c9 Compare September 21, 2026 15:30
@zachdaniel

Copy link
Copy Markdown
Contributor

This will be a bit of a monster to review, so give me some time 😄

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants