From 3dd55a77d8f957b2faa41eb03fc9cf2fab43f987 Mon Sep 17 00:00:00 2001 From: Lucas Carlson Date: Sat, 15 Aug 2026 18:51:53 -0700 Subject: [PATCH 1/8] feat: prove first-run and recovery paths --- .github/workflows/ci.yml | 2 + CHANGELOG.md | 10 + CONTRIBUTING.md | 46 + README.md | 1243 ++++----------------------- SECURITY.md | 24 + benchmarks/run.ts | 314 +++++++ benchmarks/shared.ts | 66 ++ benchmarks/worker.ts | 28 + docs/api.md | 4 + docs/benchmarks.md | 123 +++ docs/comparisons.md | 36 + docs/correctness.md | 23 + docs/fit.md | 58 ++ docs/parity.md | 10 +- docs/releasing.md | 6 +- docs/support.md | 39 + examples/failure-recovery/actor.ts | 51 ++ examples/failure-recovery/demo.ts | 188 ++++ examples/failure-recovery/worker.ts | 46 + examples/sqlite-quickstart.ts | 109 +++ package.json | 17 +- scripts/check-documentation.mjs | 29 +- scripts/release-artifact-smoke.mjs | 109 +++ src/cli.ts | 21 +- tsconfig.examples.json | 14 + tsconfig.quickstart-build.json | 12 + 26 files changed, 1554 insertions(+), 1074 deletions(-) create mode 100644 CONTRIBUTING.md create mode 100644 SECURITY.md create mode 100644 benchmarks/run.ts create mode 100644 benchmarks/shared.ts create mode 100644 benchmarks/worker.ts create mode 100644 docs/benchmarks.md create mode 100644 docs/comparisons.md create mode 100644 docs/fit.md create mode 100644 docs/support.md create mode 100644 examples/failure-recovery/actor.ts create mode 100644 examples/failure-recovery/demo.ts create mode 100644 examples/failure-recovery/worker.ts create mode 100644 examples/sqlite-quickstart.ts create mode 100644 scripts/release-artifact-smoke.mjs create mode 100644 tsconfig.examples.json create mode 100644 tsconfig.quickstart-build.json diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f3fc586..b1c3e35 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -23,6 +23,8 @@ jobs: - run: pnpm run test:coverage - run: pnpm run build - run: pnpm run pack:check + - run: pnpm run test:package + - run: pnpm run test:recovery - run: pnpm audit --audit-level=high postgresql: diff --git a/CHANGELOG.md b/CHANGELOG.md index 4d53a6f..f938ad8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,16 @@ ## 0.13.0 - 2026-08-15 +- Replace the exhaustive README with an outcome-first introduction, explicit + fit and correctness boundaries, sourced comparisons, and factual design + provenance. +- Add a packaged SQLite quickstart, a clean-install tarball smoke test, and a + deterministic multi-process crash and fencing demonstration. +- Add a reproducible benchmark harness, record locally observed SQLite, + PostgreSQL 18, and MySQL 8.4 measurements, and label other configurations as + unmeasured. +- Add support, contribution, and security policies plus stronger local + Markdown link validation. - **Breaking:** make unwrapped observables invalidation-only by default. They continue to detect changes and refresh dependent components without storing or sending their values. Wrap a projection in `broadcastValue()` to share diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..6a4917c --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,46 @@ +# Contributing + +Solid Objects changes can affect durable state and recovery. A contribution +should explain the invariant it changes and include a regression test at the +lowest layer that can prove it. + +## Setup + +Install Node.js 24.15 or newer, enable Corepack, and install the locked +dependencies: + +```bash +corepack enable +pnpm install --frozen-lockfile +``` + +## Validation + +Run the local quality gates before opening a pull request: + +```bash +pnpm run format:check +pnpm run check +pnpm run test:coverage +pnpm run build +pnpm run pack:check +pnpm run test:package +pnpm run test:recovery +``` + +Run `pnpm run test:browser` after installing Playwright Chromium. The database +and wake-up suites use these variables: + +- `SOLID_OBJECTS_DATABASE_URL` for PostgreSQL or MySQL integration tests; +- `SOLID_OBJECTS_REDIS_URL` for the optional Redis wake-up suite. + +Use disposable databases. Do not include credentials, production data, +customer identifiers, or other personal information in fixtures or reports. + +## Correctness changes + +For mailbox, lease, fencing, retry, effect, reminder, or migration changes, +include the failure sequence the test exercises. Prefer deterministic clocks, +explicit process coordination, and durable assertions over sleeps or mock-only +proof. Update [Correctness and delivery semantics](docs/correctness.md) when a +guarantee or limitation changes. diff --git a/README.md b/README.md index 6febff6..92016a3 100644 --- a/README.md +++ b/README.md @@ -1,1116 +1,253 @@ # Solid Objects JS -**Stateful, realtime TypeScript objects for Node.js, powered by the relational -database you already run.** +[![CI](https://github.com/cardmagic/solid-objects-js/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/cardmagic/solid-objects-js/actions/workflows/ci.yml) +[![npm](https://img.shields.io/npm/v/solid-objects)](https://www.npmjs.com/package/solid-objects) -Build multiplayer rooms, collaborative tools, carts, account workflows, device -sessions, and AI agents without assembling Redis locks, a message broker, a -timer service, and custom recovery code. Solid Objects serializes concurrent -changes to the same object, persists the result, and sends ordered realtime -updates from your existing SQLite, PostgreSQL, or MySQL database. +Solid Objects gives each logical application identity—such as a room, cart, +account, device, document, or agent session—durable state and a sequential +mailbox. Concurrent calls for one identity cannot overwrite each other. Calls +for different identities can run at the same time. -Keep the stack boring: ordinary TypeScript classes, ordinary SQL, and the Node -server and WebSocket transport you already use. Redis is optional, not a source -of truth. +Define ordinary TypeScript classes and run them in ordinary Node.js processes. +State, queued operations, retries, reminders, effects, and realtime +invalidations are stored in the SQLite, PostgreSQL, or MySQL database the +application already operates. -> SQLite uses Node's built-in `node:sqlite` module. PostgreSQL 14 or newer and -> MySQL 8.0 or newer use optional driver peer dependencies. +> **Early release:** the correctness core has automated coverage across the +> supported databases, browsers, process recovery, and packaged artifacts, but +> the TypeScript implementation is new. Read the [delivery boundaries](#delivery-boundaries) +> before using it for important data. -## Installation - -```bash -pnpm add solid-objects -``` - -## Stop race conditions at the object boundary - -Define the state and the operations allowed to change it: - -```typescript -import { Actor, broadcastValue } from "solid-objects" - -export class ClickerRoom extends Actor { - static override readonly actorType = "ClickerRoom" - - clicks = 0 - - click(): number { - this.clicks += 1 - return this.clicks - } - - override observables(): Record { - return { clicks: broadcastValue(this.clicks) } - } -} -``` - -Call it like an ordinary async object: +## The programming model ```typescript -const room = ClickerRoom.ref("launch-party") +import { Actor } from "solid-objects" -await Promise.all([room.click(), room.click(), room.click()]) - -console.log(await room.clicks) // 3 -``` +class Cart extends Actor { + static override readonly actorType = "Cart" -Those calls may arrive from different requests or, with PostgreSQL or MySQL, -different Node processes. They enter one durable mailbox for `launch-party`, -execute one at a time, and commit without a Redlock or application-level retry -loop. Calls to different room IDs can execute concurrently. - -The same committed change can update connected browsers: - -```typescript -import { SolidObjectsBrowserClient } from "solid-objects/browser" - -const client = new SolidObjectsBrowserClient({ - url: new URL("/solid-objects", window.location.href), - onInvalidation: ({ observables }) => renderClickCount(observables.clicks), -}) - -client.subscribe({ actorType: "ClickerRoom", actorId: "launch-party" }) -client.connect() -``` - -You provide the authenticated WebSocket handler and rendering function. Solid -Objects handles committed-state replay, ordered invalidations, resubscription -after your application reconnects, and stale-update fencing. Typed -per-subscriber payloads and a component registry cover private views and -targeted server-rendered fragment refreshes. - -## The boring stack for stateful applications - -Most stateful features eventually need the same machinery: load one entity, -serialize concurrent changes, persist the result, schedule follow-up work, and -recover after a process dies. Applications often assemble that machinery from -a database, Redis, a queue, distributed locks, and a pile of retry code. - -Solid Objects keeps that coordination in one place: the relational database. - -- An actor is addressed by its TypeScript class and ID. -- Public fields are JSON state. -- Public methods are durable operations. -- Public getters are ordered, read-only queries. -- Every actor has a durable, sequential mailbox. -- State, results, actor-to-actor delivery, effects, reminders, and observable - invalidations commit together. -- Renewable leases and fencing prevent stale workers from committing. - -## What you stop building - -For workloads organized around durable identities—accounts, carts, game rooms, -workflows, devices, collaborative documents, or agent sessions—Solid Objects -replaces a recurring layer of infrastructure and application code: - -- per-entity locking and race-condition handling; -- bespoke queue consumers that must preserve ordering; -- retry bookkeeping and poison-message handling; -- timer tables and scheduler claim logic; -- transactional outboxes for follow-up work; and -- durable invalidation bookkeeping. - -This is not an in-memory actor library. A call is complete only after its state -and durable consequences commit to the database. - -## Requirements - -- Node.js 24.15 or newer -- TypeScript 5.9 or newer for TypeScript applications -- SQLite, PostgreSQL 14 or newer, or MySQL 8.0 or newer with InnoDB - -## Write an ordinary TypeScript class - -```typescript -import { Actor, broadcastValue } from "solid-objects" - -export class Counter extends Actor { - static override readonly actorType = "Counter" - - count = 0 - - get doubled(): number { - return this.count * 2 - } - - increment({ amount = 1 }: { amount?: number } = {}): number { - this.count += amount - return this.count - } - - override observables(): Record { - return { count: broadcastValue(this.count) } - } -} -``` - -No wrapper, state interface, decorator, or operation union is required. Native -`#private` fields remain private and are not persisted. Persisted fields, -operation arguments, results, and observable values must be JSON-compatible. - -`observables()` is deliberately explicit. State fields and getters do not -become realtime data automatically. Use `broadcastValue(value)` when clients -need the value, or `broadcastInvalidation(value)` when clients only need to -know that a named dependency changed: - -```typescript -import { broadcastInvalidation, broadcastValue } from "solid-objects" + items: string[] = [] -override observables(): Record { - return { - version: broadcastValue(this.room?.version ?? 0), - playerOne: broadcastInvalidation(this.playerInSeat(1)), + add({ sku }: { sku: string }): number { + this.items.push(sku) + return this.items.length } } -``` - -Solid Objects computes and compares both values. Unwrapped values and values -wrapped in `broadcastInvalidation()` persist and send only their names when -they change. This keeps private component data out of shared invalidation -envelopes without application-maintained revision counters. Use -`broadcastValue()` only for a scalar deliberately shared with every authorized -actor subscriber. - -## Evolve state with explicit migrations - -Increase `stateVersion` and retain every adjacent migration when persisted -state changes shape: -```typescript -import { Actor, type JsonObject } from "solid-objects" - -export class ShoppingCart extends Actor { - static override readonly actorType = "ShoppingCart" - static override readonly stateVersion = 2 - static override readonly migrations = [ - { - from: 1, - to: 2, - migrate: (state: JsonObject): JsonObject => ({ - ...state, - currency: "USD", - }), - }, - ] - - items: string[] = [] - currency = "USD" -} +const cart = Cart.ref("cart-123") +await Promise.all([cart.add({ sku: "blue-shirt" }), cart.add({ sku: "green-hat" })]) ``` -Migrations run in order when an actor is next hydrated. They must be -deterministic, synchronous, JSON-compatible transformations and cannot write -through a guarded application database. New field defaults are filled from a -fresh actor after migration. Do not run application processes with different -`stateVersion` values at the same time: once new code persists a newer state, -old code rejects it. - -See [`docs/state-and-lifecycle.md`](docs/state-and-lifecycle.md) for field -discovery, rolling deployment, activation hooks, snapshots, rejection, and -destruction. - -## Point it at SQLite, PostgreSQL, or MySQL - -SQLite needs no database driver package: - -```typescript -import { configure } from "solid-objects" -import { sqlite } from "solid-objects/database/sqlite" -import { Counter } from "./counter.js" - -const runtime = configure({ - database: sqlite({ - path: "storage/solid-objects.sqlite3", - timeoutMilliseconds: 5_000, - lockRetryAttempts: 10, - }), - authorizeMessage: ({ authorizationContext }) => authorizationContext !== undefined, - authorizeQuery: ({ authorizationContext }) => authorizationContext !== undefined, - authorizeDestroy: ({ authorizationContext }) => authorizationContext !== undefined, - authorizeAdministration: ({ authorizationContext }) => isOperator(authorizationContext), - authorizeSubscription: ({ actorId, authorizationContext }) => - authorizationContext?.canViewCounter(actorId) === true, -}) - -runtime.register(Counter) -await runtime.install() - -const shutdown = new AbortController() -process.once("SIGTERM", () => shutdown.abort()) -process.once("SIGINT", () => shutdown.abort()) -await runtime.run(shutdown.signal) -``` +Both calls enter the durable mailbox for `cart-123`. They execute in order and +commit one state transition at a time, even when different requests or Node.js +processes submit them concurrently. -SQLite serializes access inside one Node process. Across processes, it uses the -native busy timeout and retries transient `BEGIN IMMEDIATE` contention with -short capped backoff. `lockRetryAttempts` bounds those retries; synchronous -invocations remain bounded by their end-to-end `timeoutMilliseconds` deadline. +## Run it now with SQLite -`configure()` installs this runtime as the default used by `Actor.ref()`. Use -`createRuntime()` when an application needs an isolated runtime and address its -actors through `runtime.ref(ActorClass, actorId)`. Actor code executing in that -runtime resolves its own actor references without changing the global default. - -For PostgreSQL, install the optional driver and replace the database value: +Node.js 24.15 or newer is required. The `0.13.0` release includes a +packaged quickstart: ```bash -pnpm add pg +npm exec --yes --package=solid-objects@0.13.0 -- solid-objects quickstart ``` -```typescript -import { postgresql } from "solid-objects/database/postgresql" +The command needs no repository checkout, database server, Redis, container, or +application configuration. It uses Node's built-in SQLite module and removes +its scoped temporary database before exiting. -const connectionString = process.env.DATABASE_URL -if (!connectionString) throw new Error("DATABASE_URL is required") +The executable asserts rather than merely printing a plausible result. It +proves that: -const database = postgresql({ - connectionString, - maximumConnections: 10, -}) +- 25 concurrent calls to one identity produce the exact committed state `25`; +- their return values are the complete sequence from `1` through `25`; +- operations for two different identities overlap in time; and +- the runtime closes and temporary state is removed. -const runtime = configure({ - database, - wakeUp: database.wakeUp(), -}) -``` +Before `0.13.0` reaches the registry, maintainers can run the identical +executable from a generated package tarball with `pnpm run test:package`. -PostgreSQL uses a bounded `pg` pool, 64-bit database timestamps and sequences, -row-locked sequence allocation, and the same durable polling contract as -SQLite. Keep `pg` at 8.23 or newer within the supported major. Portable -`DatabaseConnection` SQL uses `?` parameters; write `??` when a PostgreSQL query -needs the literal JSON existence operator. +## How it works -`database.wakeUp()` is opt-in. It uses one event-driven PostgreSQL client per -runtime to listen on role-specific channels and wake every matching local -waiter. Create it in every process that should send or receive notifications. -Polling remains the fallback if a notification is missed or the listener -reconnects. Because `LISTEN` is session-scoped, use a direct connection or -session pooling rather than transaction pooling for this client. +An object is addressed by its TypeScript class and application-defined ID. +Public fields are JSON state, public methods are durable operations, and public +getters are ordered queries. -For MySQL, install `mysql2` and configure its bounded promise pool: - -```bash -pnpm add mysql2 -``` +For each identity, Solid Objects: -```typescript -import { mysql } from "solid-objects/database/mysql" +1. commits calls to a durable per-ID mailbox; +2. claims one activation with a renewable lease; +3. executes one operation at a time outside the database transaction; +4. commits state, completion, and staged work in a short fenced transaction; +5. retries recoverable failures and exposes terminal failures as dead letters; +6. publishes committed realtime invalidations in revision order. -const connectionString = process.env.DATABASE_URL -if (!connectionString) throw new Error("DATABASE_URL is required") +The fence includes the activation owner, token, generation, expiration, and +claimed message. A worker that finishes JavaScript after losing its lease +cannot commit. See the executable [failure-recovery demonstration](examples/failure-recovery/demo.ts) +and the full [architecture](docs/architecture.md). -const database = mysql({ - connectionString, - maximumConnections: 10, -}) -``` +Redis is optional wake-up infrastructure. It can reduce notification latency +for a multi-process MySQL deployment, but the relational database remains the +durable source of truth and polling remains the recovery path. -MySQL uses InnoDB tables, 64-bit database timestamps and sequences, row-locked -sequence allocation, and bounded retries around the side-effect-free enqueue -transaction when InnoDB selects it as a deadlock victim. Use the Redis wake-up -adapter when a MySQL deployment wants cross-process notification latency; -durable polling remains sufficient for correctness. +## Good and poor fits -Authorization is deny-by-default. Actor IDs identify actors; they are not -capabilities. +| Good fit | Poor fit | +| --------------------------------------------------------- | --------------------------------------------------------- | +| Multiplayer rooms and collaborative sessions | A single-row update already solved by one SQL transaction | +| Shopping carts, accounts, devices, and per-user workflows | Bulk ingestion and data-parallel pipelines | +| Stateful agent sessions with ordered tool results | Very high-throughput global counters | +| Per-document or per-device reminders | Large JSON documents that should remain normalized rows | +| Realtime projections of committed state | Globally placed edge state or managed elastic placement | -`runtime.run()` supervises every built-in role and registered component. An -unexpected exit is cleaned up and rebuilt through its original factory with -capped exponential backoff. Shutdown stops replacement before asking the live -instances to finish, so no replacement can outlive the runtime. The shared -shutdown budget defaults to 15 seconds and can be changed with -`shutdownTimeoutMilliseconds`. Components and actor operations must cooperate -with cancellation where they receive an `AbortSignal`; actor operations and -other JavaScript code already running cannot be forcibly terminated. +One hot identity is intentionally serialized. Split an identity only when the +domain can tolerate independent ordering and transactions. Solid Objects does +not provide a transaction across object identities. -The default in-process wake-up adapter interrupts role polling as soon as this -runtime commits new work. Polling remains the correctness fallback, so a missed -or failed signal costs latency rather than losing work. Multi-process hosts not -using PostgreSQL notifications can provide a `WakeUpAdapter` backed by their -existing notification system without adding a required broker to the default -SQLite stack. +The longer decision guide is in [Choosing Solid Objects](docs/fit.md). -Applications that already operate Redis can use its optional Pub/Sub adapter: +## Delivery boundaries -```bash -pnpm add redis -``` - -```typescript -import { redisWakeUp } from "solid-objects/wake-up/redis" - -const runtime = configure({ - database, - wakeUp: redisWakeUp({ url: process.env.REDIS_URL ?? "redis://127.0.0.1:6379" }), -}) -``` - -The adapter lazily opens separate publisher and subscriber connections because -a subscribed Redis client cannot issue ordinary commands. Role-specific -channels wake every matching waiter in the process. Redis Pub/Sub is transient; -the relational database remains durable truth and bounded polling covers a -missed notification or unavailable Redis server. - -## Call it like a local object - -`await` is the committed call boundary. - -```typescript -const counter = Counter.ref("primary") +- Operations are ordered per identity and execute **at least once**. +- A crash after arbitrary external I/O but before the database commit can cause + that I/O to repeat. Use the stable effect ID or another durable idempotency + key at the external system. +- Fencing protects the Solid Objects database commit. It cannot undo an HTTP + request, email, payment, file write, or other external side effect. +- Different identities can execute concurrently; one hot identity cannot. +- State, result, actor-to-actor delivery, reminders, effects, commit actions, + and realtime invalidations commit together for one operation. +- Cross-object transactions are not provided. +- Application processes with incompatible `stateVersion` values must not run + together. Older code rejects state written by a newer version. +- Direct application-database writes are guarded only when the application + uses the supplied database facade. Unwrapped clients cannot be intercepted. +- Realtime sessions are process-local. A multi-process application must bridge + committed broadcast events to the processes holding live connections. -const count = await counter.increment({ amount: 2 }) -const doubled = await counter.doubled -``` +See [Correctness and delivery semantics](docs/correctness.md) and +[Errors and recovery](docs/errors-and-recovery.md) for the complete contract. -The method call is still a durable database operation: it enters the actor's -mailbox, waits its turn, and resolves with the committed, deeply frozen result. -If the enqueue transaction commits, a later wait timeout does not cancel the -durable message. If enqueue itself cannot commit within the timeout, -`SyncEnqueueTimeout` is raised and no message exists to recover. +## Realtime committed state -Use `with()` when invocation behavior needs configuration: +Actors opt into browser-visible dependencies. In `0.13`, an unwrapped +observable triggers invalidation without storing or sending its value. Use +`broadcastValue()` only for a scalar that every authorized subscriber may see: ```typescript -await counter - .with({ - authorizationContext: currentUser, - timeoutMilliseconds: 2_000, - idempotencyKey: "increment-123", - }) - .increment({ amount: 2 }) -``` - -Invocation options stay separate from actor arguments, so an actor may safely -use argument names such as `timeoutMilliseconds` or `authorizationContext`. -Every invocation receives a generated `requestId`; `idempotencyKey` remains the -caller's deduplication key and is never reused as request identity. During an -operation, `this.currentMessage` exposes both values along with `id`, -`enqueuedAt`, `actorType`, `actorId`, `sequence`, and `attempt`. - -Use `this.reject()` for an expected domain refusal that should roll back the -turn without retrying or blocking later mailbox work: +import { Actor, broadcastValue } from "solid-objects" -```typescript -class Reservation extends Actor { - static override readonly actorType = "Reservation" +class Room extends Actor { + static override readonly actorType = "Room" - available = 0 + version = 0 + privateHands: Record = {} - reserve({ quantity }: { quantity: number }): void { - if (quantity > this.available) { - this.reject("insufficientInventory", { - message: "Not enough inventory is available", - details: { available: this.available }, - }) + override observables(): Record { + return { + version: broadcastValue(this.version), + hands: this.privateHands, } - this.available -= quantity - } -} -``` - -Callers receive `Rejected` with `code`, frozen `details`, and the durable -`messageId`. Unexpected exceptions are retried and eventually surface as -`MessageFailed`. Rejection codes follow the same identifier rule as actor -members: a letter or underscore followed by letters, digits, or underscores. - -Do not make a committed actor call or wait on a message from inside -`database.transaction(...)` on the Solid Objects database. The runtime raises -`SyncInsideTransaction` before enqueue or waiting, avoiding a self-deadlock on -the transaction's checked-out connection. Send background work outside the -transaction, or let the actor coordinate same-database changes through a commit -action. - -## Read snapshots and destroy actors - -An authorized snapshot reads all persisted fields and getters from one -committed state image without entering the mailbox: - -```typescript -const snapshot = await Counter.ref("primary").snapshot({ - authorizationContext: currentUser, -}) - -console.log(snapshot.count, snapshot.doubled) -``` - -Snapshots are deeply frozen. Getters must not mutate state or stage durable -work. Because snapshots do not enter the mailbox, use an ordinary query when -the read must be ordered behind earlier messages. - -Destroy an actor through its separate deny-by-default policy: - -```typescript -const destroyed = await Counter.ref("primary").destroy({ - authorizationContext: currentUser, -}) -``` - -Destruction is idempotent and cascades through the current incarnation's -state, mailbox history, effects, reminders, broadcasts, and dead letters. A -later message creates a new incarnation, and an authorized waiter on the old -one receives `ActorDestroyed`. - -## Send background work without a queue service - -Use the typed `send` dispatcher when the caller should not wait for execution: - -```typescript -const message = await counter.send.increment({ amount: 2 }) - -const delayed = await counter.send - .with({ - availableAt: new Date(Date.now() + 60_000), - idempotencyKey: "increment-later", - authorizationContext: currentUser, - }) - .increment({ amount: 2 }) - -await message.status() -await message.result() -await message.wait({ timeoutMilliseconds: 2_000 }) -``` - -`MessageReference` stores durable identity, not authorization context. Pass -`authorizationContext` again to `status()`, `result()`, or `wait()`; every read -reauthorizes the stored operation. Operations that return `undefined`, -including ordinary `void` methods, are normalized to JSON `null` when their -durable result is read or awaited. - -Actor code must not call another reference directly or through `send`. Use -`sendTo()` so outbound delivery commits atomically with the source actor turn—no -separate broker or hand-built transactional outbox required: - -```typescript -class Account extends Actor { - static override readonly actorType = "Account" - - disable({ auditLogId }: { auditLogId: string }): void { - this.sendTo(AuditLog.ref(auditLogId)).record({ - eventName: "account_disabled", - }) - } -} -``` - -If `disable` fails or is rejected, the staged audit message is discarded. - -## Use database-backed timers - -Reminders are actor-owned durable alarms. The operation name is also the -reminder identity, so scheduling it again moves the existing reminder. - -```typescript -class Trial extends Actor { - static override readonly actorType = "Trial" - - expired = false - - armExpiration(): void { - this.schedule({ at: new Date(Date.now() + 86_400_000) }).expire!() - } - - reconcile(): void { - if (!this.expired) this.armExpiration() - } - - expire(): void { - this.expired = true - } -} -``` - -The non-null assertion is only needed by projects using -`noUncheckedIndexedAccess`; runtime registration still rejects unknown reminder -operations before persistence. Recurring reminders accept `everyMilliseconds` -and a `missed` policy of `"latest"` or `"all"`. - -Authorized operators can inspect alarm metadata and resume a reminder that was -paused after a scheduler error: - -```typescript -const paused = await runtime.reminders.all({ - status: "paused", - authorizationContext: currentUser, -}) - -const reminder = paused.items[0] -if (reminder) { - await runtime.reminders.resume(reminder.id, { - runAt: new Date(Date.now() + 60_000), - authorizationContext: currentUser, - }) -} -``` - -Inspection omits reminder arguments and error messages. Resume is idempotent; -completed reminders must be scheduled again by their owning actor. - -## Keep external I/O outside the transaction - -Effects run outside the actor turn through a transactional outbox. Handlers -must deduplicate external work using `context.id` because delivery is at least -once. - -```typescript -class Checkout extends Actor { - static override readonly actorType = "Checkout" - - status = "open" - - checkout({ paymentId }: { paymentId: string }): void { - this.status = "pending" - this.emit("chargePayment", { - arguments: { paymentId }, - onSuccess: "paymentSucceeded", - onFailure: "paymentFailed", - }) } - - paymentSucceeded(options: { - arguments: { paymentId: string } - result: { receiptId: string } - }): void { - this.status = "paid" - } - - paymentFailed(options: { arguments: { paymentId: string }; error: JsonObject }): void { - this.status = "failed" - } -} - -runtime.registerEffect("chargePayment", async ({ paymentId }, context) => { - return payments.charge({ paymentId, idempotencyKey: context.id }) -}) -``` - -Effect context also exposes `attempt`, `sourceMessageId`, `actorType`, and -`actorId`. The effect `id` is stable across retries and remains the external -idempotency key. - -Success callbacks receive `{ effectId, arguments, result }`. Failure callbacks -receive `{ effectId, arguments, error }`. The JSON `arguments` are the -values originally staged by `emit()`, so actors can correlate concurrent -effects without coupling the external handler to actor state. - -## Commit actions - -Commit actions make a short database-only write in the same fenced transaction -as actor state: - -```typescript -runtime.registerCommitAction("completeAttempt", async ({ attemptId }, context) => { - await context.connection.run("UPDATE attempts SET completed = 1 WHERE id = ?", [attemptId]) -}) -``` - -Database row generics are assertions, not runtime conversions. In particular, -SQLite integer columns are returned as `bigint`; type rows accordingly or -convert deliberately. See the [adapter value mapping](docs/configuration.md#database-value-mapping). - -Commit-action context includes the source message and request IDs, actor -identity, mailbox sequence, activation generation, and the active transaction -connection. - -Do not perform network I/O in a commit action. Use an effect when work cannot -share the Solid Objects database transaction. - -When actors also read an application database, wrap that database with the -guarded facade and use the same facade everywhere: - -```typescript -import { guardApplicationDatabase } from "solid-objects" -import { sqlite } from "solid-objects/database/sqlite" - -const applicationDatabase = guardApplicationDatabase(sqlite({ path: "application.sqlite3" })) -``` - -During actor execution, observable and payload projection, and state migration, -the facade permits `SELECT` through `get()` and `all()` and rejects `run()` or -row-returning write statements. A commit action stays inside the same read-only -context and writes only through its supplied fenced `context.connection`. This -boundary is opt-in: Solid Objects cannot intercept a separate ORM pool or an -unwrapped database client. - -## Inspect and retry terminal failures - -A committed invocation that exhausts its attempts raises `MessageFailed`. -The exception carries the durable `messageId` and the persisted error record in -`details`, so callers can correlate the failure without parsing its message. -If an already-authorized actor is destroyed while a caller is waiting, -`ActorDestroyed` is raised instead. - -Messages that exhaust their attempts remain available as dead letters. Access -is deny-by-default and goes through the administration policy: - -```typescript -const deadLetters = await runtime.deadLetters.all({ - authorizationContext: currentUser, -}) - -const deadLetter = deadLetters[0] -if (deadLetter) { - await runtime.deadLetters.retry(deadLetter.id, { - authorizationContext: currentUser, - }) } ``` -Retry creates one durable replacement message and records that link. Repeating -the retry returns the same `MessageReference` instead of enqueueing duplicate -work. +`version` crosses the shared invalidation channel. `hands` contributes only its +name when its real value changes, allowing a reauthorized component endpoint to +render subscriber-specific state without a manual revision counter. -## Reconcile application-owned actors +The browser package handles replay, reconnection, incarnation/revision fences, +personalized payloads, and framework-neutral component refresh. Applications +provide authentication, WebSocket transport, and rendering. See the +[browser protocol](docs/browser-protocol.md) and [authorization guide](docs/authorization.md). -Self-scheduling actors should have a low-frequency application reconciler for -lost alarms and lifecycle drift. The read side is bounded, immutable, and -administration-authorized: +## Comparison -```typescript -const page = await runtime.reconciliation.withoutPendingWork({ - actorType: Trial.actorType, - quietForMilliseconds: 24 * 60 * 60 * 1_000, - authorizationContext: currentUser, -}) - -for (const instance of page.items) { - await Trial.ref(instance.actorId).send.reconcile() -} -``` - -`active()`, `statesFor()`, and `orphaned()` cover the other reconciliation -views. State batches are migrated to the registered actor's current version -before they are returned. Reconciliation never writes actor state directly; -repairs enter the ordinary durable mailbox. - -## Retain history deliberately - -Message history defaults to 30 days, stopped process history to 7 days, and -actor instances never expire unless their actor type opts in: - -```typescript -const runtime = configure({ - database, - messageRetentionMilliseconds: 30 * 24 * 60 * 60 * 1_000, - messageRetentionByActorType: { - [AuditEvent.actorType]: 365 * 24 * 60 * 60 * 1_000, - }, - instanceRetentionByActorType: { - [EphemeralSession.actorType]: 7 * 24 * 60 * 60 * 1_000, - }, -}) -``` - -Preview each resource before pruning it: - -```typescript -const preview = await runtime.retention.preview({ - target: "messages", - authorizationContext: currentUser, -}) - -const pruned = await runtime.retention.prune({ - target: "messages", - authorizationContext: currentUser, -}) -``` - -`preview.count` is the number of rows currently eligible; `pruned.count` is the -number actually deleted after candidates are rechecked. - -Pruning rechecks every candidate in bounded transactions. Live mailbox work, -unfinished outboxes, scheduled reminders, dead letters, retry links, active -leases, and running processes are retained. - -## Verify an installation - -The doctor returns a structured report for startup checks, deployment probes, -or an application-owned CLI: - -```typescript -const report = await runtime.doctor.run() - -for (const check of report.checks) { - console.log(check.status, check.name, check.message) -} - -if (!report.healthy) process.exitCode = 1 -``` - -It checks configuration, schema migrations and required columns, the database -server version and MySQL table engines, authorization-policy configuration and -neutral-context posture, live runtime roles, and a targeted durable actor round -trip. Pass `{ roundTrip: "skip" }` for a read-only report. - -Inspect role liveness with `runtime.processes.all()`. Each record exposes -`shutdownState` (`"running"`, `"draining"`, or `"stopped"`) plus a current -`stale` calculation based on the configured heartbeat threshold, hostname, -host process ID, Node version, and Solid Objects version. -`runtime.processes.cleanup()` atomically marks stale owners stopped, releases -their actor activations, returns claimed messages to ready membership, and -releases their effect, reminder, and broadcast claims. - -For application-owned long-running roles, call `runtime.registerComponent()` -before `run()`. Each factory-created component implements `run(signal)`, -`requestShutdown()`, `stopped()`, and `stop()`. The runtime supervises and -replaces failed components just like built-in roles. See -[`docs/api.md`](docs/api.md#runtime-extensions-and-manual-workers) for the full -contract. - -## Operate it from the command line - -Export the configured runtime from an application module: - -```javascript -import { configure } from "solid-objects" -import { sqlite } from "solid-objects/database/sqlite" -import { Counter } from "./dist/counter.js" - -const runtime = configure({ - database: sqlite({ path: "storage/solid-objects.sqlite3" }), - authorizeAdministration: ({ authorizationContext }) => authorizationContext?.source === "cli", -}) - -runtime.register(Counter) -export default runtime -``` - -The CLI loads `solid-objects.config.js` by default; use `--config` for another -compiled module: - -```bash -pnpm exec solid-objects start -pnpm exec solid-objects doctor -pnpm exec solid-objects status -pnpm exec solid-objects cleanup -pnpm exec solid-objects dead-letters -pnpm exec solid-objects retry-dead-letter DEAD_LETTER_ID -pnpm exec solid-objects reminders --status paused -pnpm exec solid-objects resume-reminder REMINDER_ID -pnpm exec solid-objects prune messages -pnpm exec solid-objects prune messages --execute -``` +These systems solve different coordination problems. The table describes their +default unit and deployment model, not a quality ranking. -Pruning is preview-only unless `--execute` is present. Administrative commands -use `{ source: "cli" }` as their authorization context and emit JSON for shell -automation. +| Approach | Serialization and state unit | Durable substrate | Additional runtime | Recovery model | Placement | +| --------------------------- | ----------------------------------------------------- | ------------------------------------- | ------------------------------------------------------- | ---------------------------------------------- | ---------------------------- | +| SQL transaction or row lock | Selected rows in one transaction | Application database | None | Application retries the transaction | Application deployment | +| Traditional job queue | Job or queue; ordering depends on queue configuration | Broker or queue database | Queue workers and usually a broker | Retry the job | Application deployment | +| Solid Objects | TypeScript class plus object ID | Existing SQLite, PostgreSQL, or MySQL | Library in application processes | Retry the per-ID operation from durable state | Application deployment | +| Cloudflare Durable Objects | Object class plus globally unique ID | Per-object managed storage | Cloudflare Workers platform | Managed object activation | Cloudflare-selected location | +| Rivet Actors | Addressable actor | Actor state, KV, or per-actor SQLite | Rivet Engine or managed compute | Actor sleep, wake, and persistence | Configured Rivet deployment | +| DBOS | Workflow ID and checkpointed steps | PostgreSQL system database | Library; Conductor recommended for distributed recovery | Deterministic workflow replay from checkpoints | Application deployment | +| Restate | Service handler or keyed virtual object | Restate log and state store | Restate server or cloud service | Durable handler execution and journal replay | Restate deployment | -Every command accepts `--config PATH` or `-c PATH`. `doctor` accepts -`--skip-round-trip`; `reminders` accepts `--actor-type TYPE` and `--status -scheduled|paused|completed`; `resume-reminder` accepts an ISO `--run-at DATE`; -and `prune` accepts `--execute`. Run `solid-objects --help` for the command -summary. +The sourced, dimension-by-dimension comparison—including realtime projections, +edge placement, cross-identity transactions, and operational data access—is in +[docs/comparisons.md](docs/comparisons.md). -## Mount the operator dashboard +## Requirements and supported systems -The optional `solid-objects/web` entry point serves runtime statistics, -instances and committed state, ready and claimed messages, reminders, effects, -broadcasts, dead letters, and processes. It is not imported by -`solid-objects`, so workers that do not mount it carry no dashboard code. - -```typescript -import { createDashboard, createNodeDashboardHandler } from "solid-objects/web" - -const dashboard = createDashboard({ - runtime, - mountPath: "/solid-objects/dashboard", -}) - -const dashboardHandler = createNodeDashboardHandler({ - dashboard, - resolveContext: async (request) => ({ - authorizationContext: await currentOperator(request), - session: dashboardSession(request), - }), -}) - -server.on("request", (request, response) => dashboardHandler(request, response)) -``` - -For synthetic demos that intentionally need no authentication or session, use -`access: "public-read-only"`. It hides mutation controls and rejects every POST, -but it exposes all dashboard data, so never point it at private production state. - -Every data route calls `authorizeAdministration` with its own action and -resource before reading runtime tables. The policy denies by default. The host -session adapter stores the dashboard's masked CSRF token; state-changing -requests without a token from that session receive 403. - -The dashboard adds only two actions: instance pause/resume and idempotent -dead-letter retry. Configure immutable custom tabs, routes, renderer overrides, -and middleware through `extensions`. See -[`docs/dashboard.md`](docs/dashboard.md) for mounting, policy, security, and -extension contracts. - -## Test durable workflows without sleeps - -`runtime.testing.drain()` runs configured roles in deterministic passes until -they are idle. It does not advance reminder schedules or effect retry backoff; -retryable effects rescheduled into the future remain pending. Throw -`NonRetryableError` in a test handler when the scenario is terminal failure. -Select roles when a test needs a narrower boundary: - -```typescript -const message = await Counter.ref("test").send.increment() - -await runtime.testing.drain({ roles: ["actors"] }) - -expect(await message.status()).toBe("completed") -``` - -Run reminders against an explicit future instant without changing their stored -schedules or sleeping: - -```typescript -await runtime.testing.runDueReminders({ now: fiveMinutesFromNow }) -await runtime.testing.drain({ roles: ["actors"] }) -``` - -`runtime.testing.reset()` stops and discards the cached caller worker, then -deletes every actor-owned table and process row in dependency order. Use it in -test setup and teardown; it does not rely on transactional tests or foreign-key -cascades. - -## Connect observability without coupling the runtime - -Provide a synchronous instrumentation sink and forward events to the -observability system already used by the application: - -```typescript -const runtime = configure({ - database, - instrumentation: (event) => diagnosticsChannel.publish(event), -}) -``` - -Events use names such as `solid_objects.message.enqueued`, -`solid_objects.message.completed`, `solid_objects.effect.failed`, -`solid_objects.dead_letter.created`, and `solid_objects.actor.destroyed`. -Records are immutable and contain operational metadata only. Arguments, actor -state, results, rejection messages and details, error messages, and broadcast -payloads never enter the instrumentation API. A sink failure is logged and -cannot fail durable work. - -Moving an existing alarm to another time emits -`solid_objects.reminder.replaced` only after the actor turn commits. The event -contains the actor identity, operation, reminder ID, and previous and next run -times without reminder arguments. - -## Add realtime updates without exposing all state - -Connect an authenticated socket to the transport-neutral subscription manager. -The application owns the WebSocket server and decides what object represents -the authenticated connection: - -```typescript -server.on("connection", (socket, request) => { - const session = runtime.realtime.connect({ - authorizationContext: request.user, - send: (envelope) => socket.send(JSON.stringify(envelope)), - }) - - socket.on("message", (data) => { - session.receive(data).catch(() => socket.close(1008, "subscription rejected")) - }) - socket.on("close", () => session.close()) -}) -``` - -Every subscribe request calls `authorizeSubscription` before actor lookup. An -accepted subscription immediately receives the latest committed observable -projection with its actor incarnation and revision, without adding a mailbox -message. Later invalidations come from the durable outbox in actor revision -order. Duplicate and stale revisions are fenced, and one broken connection -cannot interrupt delivery to another. - -Invalidation envelopes deliver value-broadcast observables to every authorized -subscriber. An observable wrapped in `broadcastInvalidation()` contributes -only its name to the envelope's `invalidations` array. Use invalidation-only -observables with reauthorized component endpoints when the underlying value is -private; use typed payloads when the browser needs subscriber-specific data. -Never put credentials or secrets in value-broadcast observables. - -Direct session delivery is process-local. When WebSocket connections and -workers run in several Node processes, configure `broadcast` to publish each -durable event through the application's shared transport, and have every -process feed received events to `runtime.realtime.publish(event)`. Polling and -the durable outbox remain the correctness fallback; the shared transport fans -a committed event out to the processes that own live connections. - -The browser entry contains no Node imports. It validates versioned invalidation -envelopes, tracks actor incarnations and revisions, and ignores stale delivery: - -```typescript -import { SolidObjectsBrowserClient } from "solid-objects/browser" - -const client = new SolidObjectsBrowserClient({ - url: new URL("/solid-objects", window.location.href), - onInvalidation: ({ observables }) => render(observables), -}) - -client.subscribe({ actorType: "Counter", actorId: "primary" }) -client.connect() -``` - -For server-rendered or framework-owned UI fragments, register their observable -dependencies and let one invalidation refresh only the affected targets: - -```typescript -import { SolidObjectsBrowserClient, SolidObjectsComponentRegistry } from "solid-objects/browser" - -const componentRegistry = new SolidObjectsComponentRegistry({ - refresh: async ({ actorType, actorId, instanceId, revision, batch, components, signal }) => { - const response = await fetch("/components/refresh", { - method: "POST", - signal, - headers: { "content-type": "application/json" }, - body: JSON.stringify({ actorType, actorId, instanceId, revision, batch, components }), - }) - if (!response.ok) throw new Error(`component refresh failed with ${response.status}`) - return response.json() - }, - apply: ({ component, rendered }) => { - updateComponent(component.target, rendered, { strategy: component.strategy }) - }, -}) - -componentRegistry.register({ - actorType: "GameRoom", - actorId: "table-1", - target: "player-one", - name: "player", - key: 1, - observes: ["playerOne"], - batch: "playmat", - strategy: "morph", -}) - -const client = new SolidObjectsBrowserClient({ - url: new URL("/solid-objects", window.location.href), - onInvalidation: (envelope) => componentRegistry.invalidate(envelope), -}) -``` - -Registrations sharing a batch are refreshed in one request. Same-revision -invalidations merge in a microtask, a strictly newer request aborts the older -one, and per-target incarnation/revision fences prevent a late response from -overwriting current UI. `replace` and `morph` are strategies passed to the -application's synchronous `apply` callback; the library does not assume a DOM -framework. The refresh endpoint must authenticate the request and reauthorize -every requested component and dependency. - -Run `pnpm run test:browser` after installing Playwright's Chromium build to -exercise the browser entry through native WebSocket and browser APIs. - -For subscriber-specific views, declare a static payload map with a TypeScript -`satisfies` check: - -```typescript -import { Actor, type PayloadBroadcasts } from "solid-objects" - -type Viewer = { - accountId: string -} - -class GameRoom extends Actor { - static override readonly actorType = "GameRoom" - static override readonly payloads = { - playmat: (room, viewer) => ({ - turn: room.turn, - hand: room.hands[viewer.accountId] ?? [], - }), - } satisfies PayloadBroadcasts - - turn = 1 - hands: Record = {} -} -``` - -Declare named payload return shapes with `type`, not `interface`. -`PayloadBroadcastValue` is a JSON object or array, and TypeScript interfaces do -not implicitly provide the JSON object's string index signature. - -Request payloads by name and render them separately from observable -invalidations: - -```typescript -const client = new SolidObjectsBrowserClient({ - url: new URL("/solid-objects", window.location.href), - onInvalidation: ({ observables }) => renderScalars(observables), - onPayload: ({ name, payload }) => renderPayload(name, payload), -}) - -client.subscribe({ - actorType: "GameRoom", - actorId: "primary", - payloads: ["playmat"], -}) -``` - -Each payload runs against committed state and the subscribing session's fresh -authorization context. `authorizeQuery` is called with the payload name before -projection. A denied or failing payload is omitted without stopping sibling -payloads or observable invalidations. - -`broadcast` remains available when an application also needs to forward the -same durable events through another transport or broker. Browser-visible actor -IDs and observable values are not authorization. - -## Delivery contract - -- Messages are ordered per actor identity and delivered at least once. -- Different actor identities may execute concurrently. -- A worker drains at most `maxMessagesPerActivationPass` turns from one actor, - then yields its still-due work behind actors that were already waiting. -- A global claim scans at most `claimScanLimit` ordered candidates, continuing - to another ready actor after a lost lease race. -- Long-running workers reuse hydrated actors for - `idleDeactivationTimeoutMilliseconds` while renewing the same fenced lease. -- State, completion, staged messages, effects, reminders, commit actions, and - observable broadcasts share one fenced commit. -- A lost or expired activation lease cannot commit. -- Failed turns roll state and staged intents back and block later work until - retry or dead-letter completion. -- Effects can execute more than once. -- Results and snapshots are deeply frozen copies. A snapshot contains every - persisted field and getter from one committed state image; snapshot getters - must not mutate state or stage durable work. - -Override protected `onActivate()` and `onDeactivate()` methods when an actor -needs a process-local resource during that window. Hooks may be asynchronous, -cannot write through a guarded application database, and are nondurable; -`onDeactivate()` is best effort and must not carry correctness work. - -## Current scope - -The current runtime supports Node.js 24, SQLite through built-in `node:sqlite`, -PostgreSQL 14 or newer through `pg` 8.23, and MySQL 8.0 or newer through -`mysql2` 3.23 with InnoDB. Applications own their HTTP server, WebSocket -authentication, and rendering integration. +- Node.js 24.15 or newer +- TypeScript 5.9 or newer for TypeScript applications +- SQLite through `node:sqlite`, PostgreSQL 14 or newer, or MySQL 8.0 or newer + with InnoDB +- optional `pg`, `mysql2`, or `redis` peer dependency only for the selected + adapter + +The exact CI matrix and boundaries are documented in +[Supported versions](docs/support.md). + +## Operations + +`runtime.run(signal)` supervises actor, effect, reminder, broadcast, retention, +and stale-process recovery roles. The database-backed operator dashboard is an +optional `solid-objects/web` export with deny-by-default administration policy, +session-backed CSRF protection, and Fetch or Node/Connect mounting. + +The dashboard defaults to authorized read/write access. An authorized +read-only mode removes mutations, while an explicitly public read-only mode is +appropriate only for synthetic demo data because it exposes stored arguments, +results, errors, identifiers, and operational metadata. + +Administration remains available through the JSON CLI and typed runtime +managers. See [Operations](docs/operations.md), the [dashboard guide](docs/dashboard.md), +and [Configuration](docs/configuration.md). + +## Design provenance + +Solid Objects JS is a Node.js and TypeScript implementation informed by the +Ruby [`solid_objects`](https://github.com/cardmagic/solid_objects) design. It +began at the `0.12` capability generation because the initial implementation +targeted the Ruby `0.12` contract; the number does not represent twelve earlier +JavaScript release generations. + +The TypeScript implementation is not a source translation. It redesigned the +API around inferred TypeScript references, Node runtime supervision, +`node:sqlite`/`pg`/`mysql2` adapters, transport-neutral realtime sessions, +Web Components, and browser-safe package exports. The +[parity ledger](docs/parity.md) records capability relationships and deliberate +runtime differences. + +The Ruby project first appeared publicly on August 6, 2026, and this TypeScript +repository on August 13, 2026. Both remain early releases. The +[`mtg-playmat`](https://github.com/cardmagic/mtg-playmat) application uses the +Ruby actor and realtime design as current dogfood; that is not evidence of a +TypeScript deployment. ## Documentation -- [`docs/state-and-lifecycle.md`](docs/state-and-lifecycle.md) covers actor - discovery, migrations, lifecycle hooks, rejection, snapshots, and - destruction. -- [`docs/configuration.md`](docs/configuration.md) lists every runtime and - adapter option with its default and constraint. -- [`docs/errors-and-recovery.md`](docs/errors-and-recovery.md) maps public - errors to retry and recovery behavior. -- [`docs/api.md`](docs/api.md) indexes every supported public export and runtime - manager. -- [`docs/operations.md`](docs/operations.md), - [`docs/architecture.md`](docs/architecture.md), and - [`docs/correctness.md`](docs/correctness.md) define the operating and delivery - contracts. -- [`docs/authorization.md`](docs/authorization.md) and - [`docs/browser-protocol.md`](docs/browser-protocol.md) cover security and the - transport-neutral realtime protocol. -- [`docs/dashboard.md`](docs/dashboard.md) covers the optional operator - dashboard, Fetch and Node mounting, policies, CSRF sessions, and extensions. -- [`docs/releasing.md`](docs/releasing.md) documents the tag-driven npm release - workflow for maintainers. +- [Getting the architecture right](docs/architecture.md) +- [Correctness and delivery semantics](docs/correctness.md) +- [Choosing Solid Objects](docs/fit.md) +- [Benchmarks and methodology](docs/benchmarks.md) +- [Supported versions and test matrix](docs/support.md) +- [Public API](docs/api.md) +- [State and lifecycle](docs/state-and-lifecycle.md) +- [Operations](docs/operations.md) +- [Configuration](docs/configuration.md) +- [Authorization](docs/authorization.md) +- [Browser protocol](docs/browser-protocol.md) +- [Operator dashboard](docs/dashboard.md) +- [Errors and recovery](docs/errors-and-recovery.md) +- [Design parity](docs/parity.md) +- [Contributing](CONTRIBUTING.md) +- [Security policy](SECURITY.md) ## License -Solid Objects is released under the MIT License. +Solid Objects is released under the [MIT License](MIT-LICENSE). diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..ad81f27 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,24 @@ +# Security policy + +## Private reports + +Report vulnerabilities through GitHub's +[private vulnerability reporting](https://github.com/cardmagic/solid-objects-js/security/advisories/new). +Do not open a public issue for a vulnerability that could expose data, bypass +authorization, corrupt durable state, or help an attacker. + +Include the affected package version, adapter, deployment shape, impact, and a +minimal reproduction using synthetic data. Never include credentials, database +dumps, access tokens, or real application data. + +## Correctness and data-safety reports + +A non-sensitive correctness bug may use a +[GitHub issue](https://github.com/cardmagic/solid-objects-js/issues). Describe +the expected invariant, the observed state transition, process or retry +sequence, and database adapter. If the report reveals an exploitable condition +or private data, use private vulnerability reporting instead. + +Only the latest released version receives fixes. Older releases may be useful +for reproducing a regression, but users should verify the fix on the latest +release. diff --git a/benchmarks/run.ts b/benchmarks/run.ts new file mode 100644 index 0000000..41642ce --- /dev/null +++ b/benchmarks/run.ts @@ -0,0 +1,314 @@ +import assert from "node:assert/strict" +import { mkdtemp, rm } from "node:fs/promises" +import { cpus, freemem, platform, release, tmpdir, totalmem } from "node:os" +import { join } from "node:path" +import { fileURLToPath } from "node:url" +import { fork, type ChildProcess } from "node:child_process" +import { performance } from "node:perf_hooks" +import type { MessageReference } from "solid-objects" +import { BenchmarkCounter, benchmarkRuntime, type BenchmarkDatabase } from "./shared.ts" + +type Shape = "warm-hot" | "warm-many" | "cold-many" +type Handler = "synchronous" | "asynchronous" +type Topology = "one-process" | "four-processes" + +interface Result { + shape: Shape + handler: Handler + topology: Topology + operations: number + concurrency: number + throughputPerSecond: number + latencyMilliseconds: { p50: number; p95: number; p99: number } +} + +const database = option("database", "sqlite") as BenchmarkDatabase +if (!(["sqlite", "postgresql", "mysql"] as const).includes(database)) { + throw new TypeError(`unsupported database ${database}`) +} +const operations = positiveIntegerOption("operations", 250) +const warmupOperations = positiveIntegerOption("warmup", 25) +const concurrency = positiveIntegerOption("concurrency", 16) +const temporaryDirectory = await mkdtemp(join(tmpdir(), "solid-objects-benchmark-")) +const databasePath = join(temporaryDirectory, "benchmark.sqlite3") +const databaseLocation = + database === "postgresql" + ? requiredEnvironment("SOLID_OBJECTS_POSTGRESQL_BENCHMARK_URL") + : database === "mysql" + ? requiredEnvironment("SOLID_OBJECTS_MYSQL_BENCHMARK_URL") + : databasePath +const tableNamePrefix = `solid_objects_benchmark_${process.pid}_` +const results: Result[] = [] + +try { + for (const topology of ["one-process", "four-processes"] as const) { + const runtime = benchmarkRuntime({ + database, + ...(database === "sqlite" + ? { databasePath: databaseLocation } + : { databaseUrl: databaseLocation }), + tableNamePrefix, + workerCount: topology === "one-process" ? 4 : 1, + }) + const shutdown = new AbortController() + let running: Promise | undefined + let workers: ChildProcess[] = [] + try { + await runtime.install() + if (topology === "one-process") { + running = runtime.run(shutdown.signal) + } else { + workers = await spawnWorkers({ + database, + location: databaseLocation, + tableNamePrefix, + }) + } + for (const shape of ["warm-hot", "warm-many", "cold-many"] as const) { + for (const handler of ["synchronous", "asynchronous"] as const) { + results.push( + await measure({ + runtime, + shape, + handler, + topology, + operations, + warmupOperations, + concurrency, + }), + ) + } + } + } finally { + shutdown.abort() + await running + const workerExits = workers.map(waitForExit) + for (const worker of workers) worker.send("stop") + await Promise.all(workerExits) + await runtime.testing.reset() + await runtime.close() + } + } + + const runtime = benchmarkRuntime({ + database, + ...(database === "sqlite" + ? { databasePath: databaseLocation } + : { databaseUrl: databaseLocation }), + tableNamePrefix, + workerCount: 1, + }) + await runtime.install() + const databaseVersion = await readDatabaseVersion(runtime) + await runtime.close() + process.stdout.write( + `${JSON.stringify( + { + measuredAt: new Date().toISOString(), + packageVersion: "0.13.0", + runtime: { + node: process.version, + platform: `${platform()} ${release()}`, + cpu: cpus()[0]?.model ?? "unknown", + logicalCpus: cpus().length, + totalMemoryBytes: totalmem(), + freeMemoryBytesAtReport: freemem(), + }, + database: { adapter: database, version: databaseVersion }, + methodology: { + operations, + warmupOperations, + concurrency, + warmIdentityCount: 100, + workerCount: 4, + latencyBoundary: "durable enqueue through committed result", + percentile: "nearest rank", + }, + results, + }, + null, + 2, + )}\n`, + ) +} finally { + await rm(temporaryDirectory, { recursive: true }) +} + +async function measure(options: { + runtime: ReturnType + shape: Shape + handler: Handler + topology: Topology + operations: number + warmupOperations: number + concurrency: number +}): Promise { + const runId = `${options.topology}-${options.shape}-${options.handler}-${Date.now()}` + const actorIds = Array.from({ length: 100 }, (_value, index) => `${runId}-warm-${index}`) + if (options.shape !== "cold-many") { + const ids = options.shape === "warm-hot" ? [actorIds[0] as string] : actorIds + await runBatch({ ...options, operations: ids.length, actorId: (index) => ids[index] as string }) + } + await runBatch({ + ...options, + operations: options.warmupOperations, + actorId: identitySelector(options.shape, actorIds, `${runId}-warmup`), + }) + const startedAt = performance.now() + const latencies = await runBatch({ + ...options, + actorId: identitySelector(options.shape, actorIds, `${runId}-measure`), + }) + const elapsedMilliseconds = performance.now() - startedAt + return { + shape: options.shape, + handler: options.handler, + topology: options.topology, + operations: options.operations, + concurrency: options.concurrency, + throughputPerSecond: round((options.operations * 1_000) / elapsedMilliseconds), + latencyMilliseconds: { + p50: percentile(latencies, 50), + p95: percentile(latencies, 95), + p99: percentile(latencies, 99), + }, + } +} + +async function runBatch(options: { + runtime: ReturnType + handler: Handler + operations: number + concurrency: number + actorId(index: number): string +}): Promise { + let nextIndex = 0 + const latencies: number[] = [] + await Promise.all( + Array.from({ length: Math.min(options.concurrency, options.operations) }, async () => { + for (;;) { + const index = nextIndex + nextIndex += 1 + if (index >= options.operations) return + const reference = options.runtime.ref(BenchmarkCounter, options.actorId(index)) + const startedAt = performance.now() + const message = + options.handler === "synchronous" + ? await reference.send.increment() + : await reference.send.incrementAfterYield() + await waitForResult(message) + latencies.push(performance.now() - startedAt) + } + }), + ) + assert.equal(latencies.length, options.operations) + return latencies +} + +async function waitForResult(message: MessageReference): Promise { + const deadline = performance.now() + 60_000 + while (performance.now() < deadline) { + if ((await message.result()) !== undefined) return + await new Promise((resolvePromise) => setTimeout(resolvePromise, 1)) + } + throw new Error(`message ${message.id} did not complete within 60 seconds`) +} + +function identitySelector(shape: Shape, warmIds: string[], prefix: string) { + if (shape === "warm-hot") return () => warmIds[0] as string + if (shape === "warm-many") return (index: number) => warmIds[index % warmIds.length] as string + return (index: number) => `${prefix}-${index}` +} + +async function spawnWorkers(options: { + database: BenchmarkDatabase + location: string + tableNamePrefix: string +}): Promise { + const workers = Array.from({ length: 4 }, () => + fork( + fileURLToPath(new URL("./worker.ts", import.meta.url)), + [options.database, options.location, options.tableNamePrefix], + { stdio: ["ignore", "ignore", "inherit", "ipc"] }, + ), + ) + await Promise.all( + workers.map( + (worker) => + new Promise((resolvePromise, reject) => { + worker.once("error", reject) + worker.on("message", (message) => { + if (message === "ready") resolvePromise() + }) + }), + ), + ) + return workers +} + +async function readDatabaseVersion(runtime: ReturnType): Promise { + return runtime.settings.database.connection(async (connection) => { + if (database === "sqlite") { + const row = await connection.get<{ version: string }>("SELECT sqlite_version() AS version") + return row?.version ?? "unknown" + } + if (database === "postgresql") { + const row = await connection.get<{ version: string }>( + "SELECT current_setting('server_version') AS version", + ) + return row?.version ?? "unknown" + } + const row = await connection.get<{ version: string }>("SELECT VERSION() AS version") + return row?.version ?? "unknown" + }) +} + +function waitForExit(child: ChildProcess): Promise { + if (child.exitCode !== null) { + return child.exitCode === 0 + ? Promise.resolve() + : Promise.reject(new Error(`benchmark worker exited ${child.exitCode}`)) + } + if (child.signalCode !== null) { + return Promise.reject(new Error(`benchmark worker exited with ${child.signalCode}`)) + } + return new Promise((resolvePromise, reject) => { + child.once("error", reject) + child.once("exit", (code) => { + if (code === 0) resolvePromise() + else reject(new Error(`benchmark worker exited ${code}`)) + }) + }) +} + +function percentile(values: number[], percent: number): number { + const ordered = [...values].sort((left, right) => left - right) + const index = Math.max(0, Math.ceil((percent / 100) * ordered.length) - 1) + return round(ordered[index] ?? 0) +} + +function round(value: number): number { + return Math.round(value * 100) / 100 +} + +function option(name: string, fallback: string): string { + const index = process.argv.indexOf(`--${name}`) + if (index === -1) return fallback + const value = process.argv[index + 1] + if (!value) throw new TypeError(`--${name} requires a value`) + return value +} + +function positiveIntegerOption(name: string, fallback: number): number { + const value = Number(option(name, String(fallback))) + if (!Number.isSafeInteger(value) || value < 1) { + throw new TypeError(`--${name} must be a positive integer`) + } + return value +} + +function requiredEnvironment(name: string): string { + const value = process.env[name] + if (!value) throw new TypeError(`${name} is required`) + return value +} diff --git a/benchmarks/shared.ts b/benchmarks/shared.ts new file mode 100644 index 0000000..72d0fe1 --- /dev/null +++ b/benchmarks/shared.ts @@ -0,0 +1,66 @@ +import { Actor, createRuntime, type SolidObjectsRuntime } from "solid-objects" +import { mysql } from "solid-objects/database/mysql" +import { postgresql } from "solid-objects/database/postgresql" +import { sqlite } from "solid-objects/database/sqlite" + +export type BenchmarkDatabase = "sqlite" | "postgresql" | "mysql" + +export class BenchmarkCounter extends Actor { + static override readonly actorType = "BenchmarkCounter" + + count = 0 + + increment(): number { + this.count += 1 + return this.count + } + + async incrementAfterYield(): Promise { + await new Promise((resolve) => setImmediate(resolve)) + this.count += 1 + return this.count + } +} + +export function benchmarkRuntime(options: { + database: BenchmarkDatabase + databasePath?: string + databaseUrl?: string + tableNamePrefix: string + workerCount: number +}): SolidObjectsRuntime { + const runtime = createRuntime({ + database: benchmarkDatabase(options), + tableNamePrefix: options.tableNamePrefix, + pollingIntervalMilliseconds: 5, + workerCount: options.workerCount, + effectWorkerCount: 0, + reminderSchedulerCount: 0, + retentionIntervalMilliseconds: 0, + deadProcessCleanupIntervalMilliseconds: 0, + authorizeMessage: () => true, + authorizeQuery: () => true, + }) + runtime.register(BenchmarkCounter) + return runtime +} + +function benchmarkDatabase(options: { + database: BenchmarkDatabase + databasePath?: string + databaseUrl?: string +}) { + if (options.database === "sqlite") { + if (!options.databasePath) throw new TypeError("databasePath is required for SQLite") + return sqlite({ + path: options.databasePath, + timeoutMilliseconds: 10_000, + lockRetryAttempts: 50, + }) + } + if (!options.databaseUrl) throw new TypeError("databaseUrl is required") + if (options.database === "postgresql") { + return postgresql({ connectionString: options.databaseUrl, maximumConnections: 20 }) + } + return mysql({ connectionString: options.databaseUrl, maximumConnections: 20 }) +} diff --git a/benchmarks/worker.ts b/benchmarks/worker.ts new file mode 100644 index 0000000..c5e49c4 --- /dev/null +++ b/benchmarks/worker.ts @@ -0,0 +1,28 @@ +import { benchmarkRuntime, type BenchmarkDatabase } from "./shared.ts" + +const database = requiredArgument(2) as BenchmarkDatabase +const location = requiredArgument(3) +const tableNamePrefix = requiredArgument(4) +const runtime = benchmarkRuntime({ + database, + ...(database === "sqlite" ? { databasePath: location } : { databaseUrl: location }), + tableNamePrefix, + workerCount: 1, +}) +const shutdown = new AbortController() + +process.on("message", (message) => { + if (message === "stop") shutdown.abort() +}) + +await runtime.install() +process.send?.("ready") +await runtime.run(shutdown.signal) +await runtime.close() +process.disconnect?.() + +function requiredArgument(index: number): string { + const value = process.argv[index] + if (!value) throw new TypeError(`argument ${index - 1} is required`) + return value +} diff --git a/docs/api.md b/docs/api.md index f103359..e5bd92f 100644 --- a/docs/api.md +++ b/docs/api.md @@ -114,6 +114,10 @@ The manager types are `DeadLetter`; `ReminderPage`, `ReminderPageOptions`, `AdministrationOptions` carries the application-owned authorization context for administration calls. +The packaged `solid-objects quickstart` command is config-free and runs the +SQLite example shipped in the npm artifact. Every other CLI command loads the +application runtime configured through `--config`. + `ProcessRecord.shutdownState` is `"running"`, `"draining"`, or `"stopped"`; there is no separate `running` field. `RetentionResult.count` means eligible rows for `preview()` and rows actually deleted for `prune()`. diff --git a/docs/benchmarks.md b/docs/benchmarks.md new file mode 100644 index 0000000..6548631 --- /dev/null +++ b/docs/benchmarks.md @@ -0,0 +1,123 @@ +# Benchmarks + +The benchmark harness measures committed actor operations. It is intended to +show tradeoffs and catch large regressions, not to predict application capacity. + +## Scenarios + +- `warm-hot`: all operations target one previously created identity. +- `warm-many`: operations rotate across 100 previously created identities. +- `cold-many`: every measured operation creates a new identity. +- `synchronous`: the actor method mutates state without yielding. +- `asynchronous`: the actor method yields through `setImmediate()` before the + same mutation. +- `one-process`: four actor workers run in the caller's Node process. +- `four-processes`: four Node worker processes share the database. + +Latency begins before durable enqueue and ends when the committed result is +available. Throughput uses the wall time for the measured batch. Percentiles +use nearest rank. Defaults are 25 warmup operations, 250 measured operations, +and client concurrency 16. + +## Run the harness + +SQLite needs no service: + +```bash +pnpm run benchmark -- --database sqlite +``` + +PostgreSQL and MySQL runs require a disposable database. The harness deletes +benchmark rows but leaves its uniquely prefixed empty tables for inspection. + +```bash +SOLID_OBJECTS_POSTGRESQL_BENCHMARK_URL=postgresql://... \ + pnpm run benchmark -- --database postgresql + +SOLID_OBJECTS_MYSQL_BENCHMARK_URL=mysql://... \ + pnpm run benchmark -- --database mysql +``` + +Use `--operations`, `--warmup`, and `--concurrency` to change the recorded +dataset. Redirect stdout to retain the JSON result. + +## Observed results + +Measured on August 15, 2026 with the prepared `0.13.0` source tree: + +- Apple M5, 10 logical CPUs, 24 GiB memory +- macOS 26.6 (`darwin 25.6.0`) +- Node.js 26.7.0 +- SQLite 3.53.4 on the internal SSD, PostgreSQL 18.4 and MySQL 8.4.11 in + Docker Desktop +- 25 warmup operations, 250 measured operations, concurrency 16 + +### SQLite 3.53.4 + +| Topology | Shape | Handler | ops/s | p50 ms | p95 ms | p99 ms | +| -------------- | --------- | ------------ | -----: | -----: | ------: | ------: | +| one process | warm hot | synchronous | 95.51 | 35.34 | 1159.07 | 1711.76 | +| one process | warm hot | asynchronous | 448.63 | 35.96 | 38.95 | 41.47 | +| one process | warm many | synchronous | 44.35 | 141.55 | 1886.35 | 2543.15 | +| one process | warm many | asynchronous | 240.64 | 48.62 | 75.10 | 671.64 | +| one process | cold many | synchronous | 30.91 | 436.98 | 1502.26 | 1729.10 | +| one process | cold many | asynchronous | 57.15 | 167.36 | 967.74 | 1066.49 | +| four processes | warm hot | synchronous | 453.10 | 27.24 | 72.09 | 77.88 | +| four processes | warm hot | asynchronous | 487.39 | 29.27 | 60.66 | 68.28 | +| four processes | warm many | synchronous | 78.88 | 86.33 | 890.29 | 1381.65 | +| four processes | warm many | asynchronous | 47.56 | 220.37 | 1130.53 | 1341.86 | +| four processes | cold many | synchronous | 70.45 | 174.67 | 643.53 | 668.46 | +| four processes | cold many | asynchronous | 39.92 | 221.01 | 1309.13 | 1442.30 | + +### PostgreSQL 18.4 + +| Topology | Shape | Handler | ops/s | p50 ms | p95 ms | p99 ms | +| -------------- | --------- | ------------ | ----: | ------: | ------: | ------: | +| one process | warm hot | synchronous | 66.99 | 232.89 | 317.94 | 375.68 | +| one process | warm hot | asynchronous | 72.27 | 211.57 | 280.89 | 316.61 | +| one process | warm many | synchronous | 76.69 | 129.88 | 552.01 | 1963.27 | +| one process | warm many | asynchronous | 25.78 | 340.96 | 2326.38 | 6917.26 | +| one process | cold many | synchronous | 12.70 | 1262.08 | 1734.77 | 2199.20 | +| one process | cold many | asynchronous | 11.23 | 1254.46 | 2796.81 | 2905.99 | +| four processes | warm hot | synchronous | 83.71 | 191.83 | 220.87 | 231.84 | +| four processes | warm hot | asynchronous | 86.42 | 184.56 | 210.11 | 215.48 | +| four processes | warm many | synchronous | 95.47 | 100.31 | 330.27 | 1663.84 | +| four processes | warm many | asynchronous | 37.04 | 232.71 | 1601.29 | 4447.85 | +| four processes | cold many | synchronous | 14.79 | 1114.18 | 1263.33 | 1313.70 | +| four processes | cold many | asynchronous | 11.68 | 1231.94 | 2555.34 | 2848.80 | + +### MySQL 8.4.11 + +| Topology | Shape | Handler | ops/s | p50 ms | p95 ms | p99 ms | +| -------------- | --------- | ------------ | ----: | ------: | ------: | ------: | +| one process | warm hot | synchronous | 28.11 | 504.36 | 1648.10 | 2088.05 | +| one process | warm hot | asynchronous | 25.29 | 508.26 | 1750.17 | 2222.72 | +| one process | warm many | synchronous | 61.79 | 165.62 | 463.59 | 2036.31 | +| one process | warm many | asynchronous | 22.45 | 446.17 | 2420.92 | 7818.65 | +| one process | cold many | synchronous | 10.23 | 1353.37 | 3038.76 | 3185.15 | +| one process | cold many | asynchronous | 10.04 | 1338.31 | 3344.18 | 3477.06 | +| four processes | warm hot | synchronous | 29.89 | 427.18 | 1483.88 | 1997.56 | +| four processes | warm hot | asynchronous | 27.59 | 448.16 | 1518.00 | 1589.57 | +| four processes | warm many | synchronous | 69.77 | 157.79 | 410.83 | 2263.13 | +| four processes | warm many | asynchronous | 36.21 | 265.24 | 1579.43 | 4594.33 | +| four processes | cold many | synchronous | 13.68 | 1088.93 | 2186.30 | 2519.92 | +| four processes | cold many | asynchronous | 11.38 | 1209.84 | 2532.38 | 2730.74 | + +The poor throughput and tail latency in cold and asynchronous cases are +observed limitations, not capacity recommendations. The small asynchronous +yield changed scheduling enough to improve some cases and worsen others; +repeat runs on application-shaped payloads are required before drawing a +general conclusion. PostgreSQL 14, MySQL 8.0, and other database versions are +covered by integration tests but were not benchmarked. + +## Sources of bias + +- A developer laptop shares CPU, memory, and storage with unrelated processes. +- Loopback database connections exclude production network latency. +- Filesystem cache, SQLite WAL state, Node JIT warmup, and garbage collection + affect short runs. +- Docker Desktop adds virtualization overhead to containerized databases. +- The payload is a small counter, not a representative application state size. +- The harness measures default durability settings and one client concurrency. +- Hot-identity results deliberately include serialization and cannot be scaled + by adding workers. diff --git a/docs/comparisons.md b/docs/comparisons.md new file mode 100644 index 0000000..bbe9ce8 --- /dev/null +++ b/docs/comparisons.md @@ -0,0 +1,36 @@ +# System comparisons + +This guide compares coordination models so an application can choose the +smallest mechanism that meets its requirements. It does not rank the projects. + +| Approach | State and serialization unit | Deployment and durable substrate | Separate service | Replay versus state | Realtime and edge placement | Cross-identity transaction | Data access | +| --------------------------- | ---------------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------ | --------------------------------------------------------------------- | -------------------------------------------------------------- | --------------------------------------------------------------- | +| SQL transaction or row lock | Rows selected by one transaction | Application process and SQL database | No | The application retries a failed transaction | Application-owned | Yes, for rows in the same database transaction | Ordinary application tables and SQL tools | +| Traditional job queue | A job, queue, or configured grouping key | Workers plus broker or queue database | Usually | The job is retried; mutable entity state remains application-owned | Application-owned | Not supplied by the queue | Queue administration plus application data stores | +| Solid Objects | Actor class and application-defined ID | Node processes plus existing SQLite, PostgreSQL, or MySQL | No; Redis wake-up is optional | The operation retries against durable actor state | Committed projections; application-owned transport; no edge placement | No | Relational tables, typed administration, CLI, and dashboard | +| Cloudflare Durable Objects | Object class and globally unique ID | Cloudflare Workers plus per-object managed storage | Cloudflare platform | Object activation with durable state, not workflow-step replay | WebSockets and Cloudflare-selected object location | Storage transactions are scoped to one object | Object storage APIs and platform tooling | +| Rivet Actors | Addressable actor | Rivet Engine or managed compute with actor state, KV, or per-actor SQLite | Rivet Engine | Actor persistence and lifecycle; workflows add recorded steps | Actor events and deployment-dependent placement | No general transaction across actors | Actor APIs and selected persistence model | +| DBOS | Workflow ID and checkpointed steps | Application processes plus PostgreSQL system database | No orchestration server for the library; Conductor is recommended for distributed recovery | Deterministic workflow replay skips checkpointed steps | Workflow events; application placement | PostgreSQL transactions remain separate from workflow identity | PostgreSQL system database, client, CLI, and optional Conductor | +| Restate | Service handler or keyed virtual object | Application services plus Restate's durable log and state store | Yes | Durable execution journals handler progress and object state | Service protocol and Restate deployment | No shared SQL transaction across object keys | Restate APIs, state tools, snapshots, and backups | + +## Primary references + +- PostgreSQL documents row-lock behavior, transaction lifetime, and deadlock + handling in [Explicit Locking](https://www.postgresql.org/docs/18/explicit-locking.html). +- BullMQ is one representative traditional queue; its + [worker concurrency documentation](https://docs.bullmq.io/guide/workers/concurrency) + distinguishes local concurrency from multiple worker processes. +- Cloudflare documents global uniqueness, per-object storage, single-threaded + execution, and placement in [What are Durable Objects?](https://developers.cloudflare.com/durable-objects/concepts/what-are-durable-objects/). +- Rivet documents addressable actors and persistence in + [Actors](https://rivet.dev/docs/actors/) and + [Persistence](https://rivet.dev/docs/actors/persistence). +- DBOS documents its PostgreSQL checkpoints, recovery, distributed processes, + and optional control plane in [DBOS Architecture](https://docs.dbos.dev/architecture). +- Restate documents per-key write serialization in + [Virtual Objects](https://docs.restate.dev/foundations/services#virtual-object) + and its storage requirements in the + [self-hosted server overview](https://docs.restate.dev/server/overview). + +External systems evolve. Recheck these primary sources before relying on one +row as a procurement or architecture decision. diff --git a/docs/correctness.md b/docs/correctness.md index f4c9db6..9f32c27 100644 --- a/docs/correctness.md +++ b/docs/correctness.md @@ -1,5 +1,7 @@ # Correctness and delivery semantics +## Guarantees + - Delivery is ordered per actor identity and at least once. - Different identities may execute concurrently. - Sequence allocation and durable enqueue are one transaction. @@ -51,3 +53,24 @@ failing projection cannot stop its siblings or observable delivery. A state change on an actor declaring payloads creates a revision broadcast even when the actor declares no scalar observables. + +## Limitations and non-goals + +- At-least-once execution means actor code may begin more than once. State and + staged intents from a failed turn roll back, but arbitrary external work does + not. External systems need stable idempotency keys. +- The activation fence protects the Solid Objects commit. It cannot revoke or + undo network calls, files, emails, payments, or other external effects. +- One identity processes one write operation at a time. This is the ordering + guarantee and also the hot-identity throughput limit. +- A commit is scoped to one actor turn. There is no transaction spanning two + actor identities. +- Processes with incompatible `stateVersion` definitions cannot safely overlap. + Once newer code persists a state version, older code rejects that actor. +- The application owns HTTP, WebSocket authentication, rendering, process + placement, capacity, database backups, and database failover. +- Redis and PostgreSQL notifications reduce wake-up latency but do not replace + durable polling or become a source of truth. +- Large documents, bulk pipelines, globally placed edge state, and global + counters are outside the intended workload. Prefer an ordinary row + transaction when it completely enforces the invariant. diff --git a/docs/fit.md b/docs/fit.md new file mode 100644 index 0000000..99b9670 --- /dev/null +++ b/docs/fit.md @@ -0,0 +1,58 @@ +# Choosing Solid Objects + +Solid Objects is useful when an application has many independently addressed +entities and every entity needs ordered state changes, durable work, recovery, +or realtime projections. + +## Use it when + +- Concurrent requests can update the same room, cart, account, device, + document, or session. +- Each identity needs its own ordering boundary and durable mailbox. +- Operations must recover after a Node process exits. +- State changes stage reminders, effects, actor-to-actor messages, or realtime + invalidations atomically. +- The application already operates SQLite, PostgreSQL, or MySQL and should keep + durable coordination there. + +## Prefer a row transaction when + +One short transaction with an update, constraint, or row lock completely +enforces the invariant. A direct transaction has less machinery, less stored +history, and no actor-state migration contract. + +## Prefer another design when + +- Work is a bulk or data-parallel pipeline rather than per-identity state. +- One global identity must sustain more writes than one sequential mailbox can + commit. +- State is a large document or relational dataset that should be queried and + updated in smaller normalized pieces. +- The application needs a transaction spanning several independent object + identities. +- Compute and state must be automatically placed close to clients at the edge. +- The team wants a managed control plane to place, scale, and recover workers. +- Durable workflow replay across named steps is more important than a mutable + object with ordered operations. + +## Model identities deliberately + +One hot identity is intentionally serialized. An identity should correspond to +the smallest domain boundary that requires one total order. Splitting a room by +player or a cart by item may improve parallelism, but it also gives up atomic +ordering across the split. + +Different identities can run concurrently when worker capacity and the +database allow it. The [benchmark harness](benchmarks.md) measures both the hot +and independent-identity cases. + +## Operational cost + +The relational database stores actor instances, ready and claimed mailbox +membership, message history, leases, effects, reminders, broadcasts, dead +letters, and process records. Retention policies and the operator dashboard +make that state inspectable, but they do not remove the need to monitor and +back up the database. + +Redis is optional. It is a transient notification path rather than durable +state, so losing Redis increases polling latency without losing committed work. diff --git a/docs/parity.md b/docs/parity.md index 991a5ca..875291c 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -1,12 +1,14 @@ -# Ruby parity ledger +# Design parity ledger -This ledger tracks spiritual feature parity with the Ruby `solid_objects` gem. +This ledger tracks capability parity with the Ruby `solid_objects` gem. Parity means preserving a capability and its correctness or security boundary, not copying a Rails API into Node. -Reference: Ruby `solid_objects` 0.13.0. +Reference: Ruby `solid_objects` 0.13.0. The JavaScript package began at the +Ruby design's `0.12` capability generation; that version number did not imply +earlier JavaScript releases. -The Node `0.13.0` implementation has spiritual parity with that reference. Its +The Node `0.13.0` implementation has capability parity with that reference. Its relational runtime, correctness boundaries, administration, diagnostics, operator dashboard, realtime projections, browser behavior, and supported adapters have native equivalents. Rails-specific rendering surfaces are diff --git a/docs/releasing.md b/docs/releasing.md index 22a703b..14486fa 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -30,8 +30,10 @@ npm trust github solid-objects \ lockfile when needed, and move the release notes out of the Unreleased section in `CHANGELOG.md`. 2. Run `pnpm run format:check`, `pnpm run check`, `pnpm run test:coverage`, - `pnpm run build`, `pnpm run pack:check`, and - `pnpm audit --audit-level=high`. + `pnpm run build`, `pnpm run pack:check`, `pnpm run test:package`, + `pnpm run test:recovery`, `pnpm run test:browser`, and + `pnpm audit --audit-level=high`. Run the PostgreSQL, MySQL, and Redis jobs + against the versions in [the support matrix](support.md). 3. Commit and push `main`. 4. Create and push an annotated tag matching the package version: diff --git a/docs/support.md b/docs/support.md new file mode 100644 index 0000000..5e2c379 --- /dev/null +++ b/docs/support.md @@ -0,0 +1,39 @@ +# Supported versions and test matrix + +## Runtime support + +| Component | Supported or tested range | +| -------------- | ----------------------------------------------------------- | +| Node.js | 24.15 or newer; CI uses 24.15 | +| TypeScript | 5.9 or newer for TypeScript applications | +| SQLite | Node's built-in `node:sqlite` on the supported Node runtime | +| PostgreSQL | 14 or newer; CI runs 14 and 18 | +| MySQL | 8.0 or newer with InnoDB; CI runs 8.0 and 8.4 | +| Redis wake-up | Optional; CI runs Redis 7 | +| Browser client | Chromium through Playwright | + +The package is ESM-only. PostgreSQL, MySQL, and Redis require their optional +peer dependency. SQLite has no driver dependency beyond Node.js. + +## What the matrix covers + +The default suite exercises actor definitions, mailbox ordering, state +migrations, leases, fencing, retries, dead letters, effects, reminders, +realtime outboxes, administration, authorization, retention, lifecycle, +timeouts, and SQLite behavior. + +Database jobs run the real adapter suites against PostgreSQL and MySQL servers. +The Redis job runs wake-up behavior against a real Redis server. The browser +job uses native WebSocket connections and Chromium for replay, payload, +component, dashboard, and revision-fence behavior. + +The quality job also builds the ESM package, inspects `npm pack`, installs the +generated tarball in a clean temporary project, runs its packaged SQLite +quickstart, and executes the multi-process recovery demonstration. + +## Boundaries + +CI currently runs on Ubuntu. Local validation also occurs on macOS, but the +project does not claim a complete operating-system compatibility matrix. A +database version being accepted by configuration is not a substitute for its +listed integration job. diff --git a/examples/failure-recovery/actor.ts b/examples/failure-recovery/actor.ts new file mode 100644 index 0000000..a6235b1 --- /dev/null +++ b/examples/failure-recovery/actor.ts @@ -0,0 +1,51 @@ +import { appendFile, access, writeFile } from "node:fs/promises" +import { join } from "node:path" +import { Actor } from "solid-objects" + +export class RecoveryCounter extends Actor { + static override readonly actorType = "RecoveryCounter" + + count = 0 + + async recover({ controlDirectory }: { controlDirectory: string }): Promise { + const message = this.currentMessage + if (!message) throw new Error("recover requires a durable message") + const attempt = message.attempt + await appendFile( + join(controlDirectory, "external-effects.jsonl"), + `${JSON.stringify({ messageId: message.id, attempt, processId: process.pid })}\n`, + ) + await writeFile(join(controlDirectory, `started-${attempt}-${process.pid}`), "") + process.send?.({ event: "operation.started", attempt, processId: process.pid }) + if (attempt === 1) await waitForFile(join(controlDirectory, "release-first-attempt")) + this.count += 1 + return this.count + } + + async serialize({ controlDirectory }: { controlDirectory: string }): Promise { + const message = this.currentMessage + if (!message) throw new Error("serialize requires a durable message") + await appendFile( + join(controlDirectory, "serialization.jsonl"), + `${JSON.stringify({ event: "start", messageId: message.id, at: Date.now() })}\n`, + ) + await new Promise((resolve) => setTimeout(resolve, 100)) + this.count += 1 + await appendFile( + join(controlDirectory, "serialization.jsonl"), + `${JSON.stringify({ event: "finish", messageId: message.id, at: Date.now() })}\n`, + ) + return this.count + } +} + +async function waitForFile(path: string): Promise { + for (;;) { + try { + await access(path) + return + } catch { + await new Promise((resolve) => setTimeout(resolve, 10)) + } + } +} diff --git a/examples/failure-recovery/demo.ts b/examples/failure-recovery/demo.ts new file mode 100644 index 0000000..14b5bec --- /dev/null +++ b/examples/failure-recovery/demo.ts @@ -0,0 +1,188 @@ +import assert from "node:assert/strict" +import { existsSync } from "node:fs" +import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { fileURLToPath } from "node:url" +import { fork, type ChildProcess } from "node:child_process" +import { createRuntime, type ActorReference, type MessageReference } from "solid-objects" +import { sqlite } from "solid-objects/database/sqlite" +import { RecoveryCounter } from "./actor.ts" + +interface WorkerMessage { + event: string + attempt?: number + processed?: number +} + +const directory = await mkdtemp(join(tmpdir(), "solid-objects-recovery-")) +const databasePath = join(directory, "state.sqlite3") +const runtime = createRuntime({ + database: sqlite({ path: databasePath, timeoutMilliseconds: 2_000, lockRetryAttempts: 20 }), + leaseDurationMilliseconds: 250, + leaseRenewalIntervalMilliseconds: 50, + processHeartbeatIntervalMilliseconds: 75, + processAliveThresholdMilliseconds: 300, + workerCount: 1, + effectWorkerCount: 0, + reminderSchedulerCount: 0, + retentionIntervalMilliseconds: 0, + deadProcessCleanupIntervalMilliseconds: 0, + authorizeMessage: () => true, + authorizeQuery: () => true, + authorizeAdministration: () => true, +}) + +try { + runtime.register(RecoveryCounter) + await runtime.install() + const serialization = await proveSerialization() + const crash = await proveCrashRecovery() + const fencing = await proveFencing() + process.stdout.write(`${JSON.stringify({ serialization, crash, fencing }, null, 2)}\n`) +} finally { + await runtime.close() + await rm(directory, { recursive: true }) +} + +assert.equal(existsSync(directory), false) + +async function proveSerialization(): Promise<{ finalState: number; overlap: false }> { + const controlDirectory = join(directory, "serialization") + await mkdir(controlDirectory) + const reference = runtime.ref(RecoveryCounter, "serialized") + const messages = await Promise.all([ + reference.send.serialize({ controlDirectory }), + reference.send.serialize({ controlDirectory }), + ]) + const workers = [spawnWorker(), spawnWorker()] + await Promise.all(workers.map(({ finished }) => finished)) + await Promise.all(messages.map((message) => message.result())) + const events = await jsonLines(join(controlDirectory, "serialization.jsonl")) + assert.equal(events.length, 4) + const starts = events.filter((event) => event.event === "start") + const finishes = events.filter((event) => event.event === "finish") + assert.equal(starts.length, 2) + assert.equal(finishes.length, 2) + assert(Number(starts[1]?.at) >= Number(finishes[0]?.at)) + const snapshot = await reference.snapshot() + assert.equal(snapshot.count, 2) + return { finalState: snapshot.count, overlap: false } +} + +async function proveCrashRecovery(): Promise<{ + attempts: number + finalState: number + repeatedEffects: number +}> { + const controlDirectory = join(directory, "crash") + await mkdir(controlDirectory) + const reference = runtime.ref(RecoveryCounter, "crash") + const message = await reference.send.recover({ controlDirectory }) + const firstWorker = spawnWorker() + await firstWorker.waitFor((entry) => entry.event === "operation.started" && entry.attempt === 1) + firstWorker.child.kill("SIGKILL") + await firstWorker.finished.catch(() => undefined) + await wait(350) + const recoveryWorker = spawnWorker() + await recoveryWorker.finished + await message.result() + return recoveryResult({ reference, message, controlDirectory }) +} + +async function proveFencing(): Promise<{ + attempts: number + finalState: number + repeatedEffects: number +}> { + const controlDirectory = join(directory, "fencing") + await mkdir(controlDirectory) + const reference = runtime.ref(RecoveryCounter, "fencing") + const message = await reference.send.recover({ controlDirectory }) + const staleWorker = spawnWorker() + await staleWorker.waitFor((entry) => entry.event === "operation.started" && entry.attempt === 1) + staleWorker.child.kill("SIGSTOP") + await wait(350) + const recoveryWorker = spawnWorker() + await recoveryWorker.finished + await message.result() + staleWorker.child.kill("SIGCONT") + await writeFile(join(controlDirectory, "release-first-attempt"), "") + await staleWorker.waitFor((entry) => entry.event === "solid_objects.activation.lost") + await staleWorker.finished + return recoveryResult({ reference, message, controlDirectory }) +} + +async function recoveryResult(options: { + reference: ActorReference + message: MessageReference + controlDirectory: string +}): Promise<{ attempts: number; finalState: number; repeatedEffects: number }> { + const stored = await runtime.repository.findMessage(options.message.id) + const attempts = Number(stored?.attempt_count) + const snapshot = await options.reference.snapshot() + const effects = await jsonLines(join(options.controlDirectory, "external-effects.jsonl")) + assert.equal(attempts, 2) + assert.equal(snapshot.count, 1) + assert.equal(effects.length, 2) + return { attempts, finalState: snapshot.count, repeatedEffects: effects.length } +} + +function spawnWorker(): { + child: ChildProcess + finished: Promise + waitFor(predicate: (message: WorkerMessage) => boolean): Promise +} { + const child = fork(fileURLToPath(new URL("./worker.ts", import.meta.url)), [databasePath], { + cwd: fileURLToPath(new URL("../..", import.meta.url)), + stdio: ["ignore", "pipe", "pipe", "ipc"], + }) + const messages: WorkerMessage[] = [] + const listeners = new Set<(message: WorkerMessage) => void>() + let stderr = "" + child.stderr?.on("data", (chunk) => { + stderr += chunk + }) + child.on("message", (message: WorkerMessage) => { + messages.push(message) + for (const listener of listeners) listener(message) + }) + const finished = new Promise((resolvePromise, reject) => { + child.once("error", reject) + child.once("exit", (code, signal) => { + if (code === 0) { + resolvePromise() + return + } + reject(new Error(`worker exited with code ${code} and signal ${signal}\n${stderr}`)) + }) + }) + return { + child, + finished, + waitFor: (predicate) => { + const existing = messages.find(predicate) + if (existing) return Promise.resolve(existing) + return new Promise((resolvePromise) => { + const listener = (message: WorkerMessage) => { + if (!predicate(message)) return + listeners.delete(listener) + resolvePromise(message) + } + listeners.add(listener) + }) + }, + } +} + +async function jsonLines(path: string): Promise>> { + return (await readFile(path, "utf8")) + .trim() + .split("\n") + .filter(Boolean) + .map((line) => JSON.parse(line) as Record) +} + +async function wait(milliseconds: number): Promise { + await new Promise((resolve) => setTimeout(resolve, milliseconds)) +} diff --git a/examples/failure-recovery/worker.ts b/examples/failure-recovery/worker.ts new file mode 100644 index 0000000..44d8914 --- /dev/null +++ b/examples/failure-recovery/worker.ts @@ -0,0 +1,46 @@ +import { createRuntime } from "solid-objects" +import { sqlite } from "solid-objects/database/sqlite" +import { RecoveryCounter } from "./actor.ts" + +const databasePath = requiredArgument(2) +const runtime = createRuntime({ + database: sqlite({ path: databasePath, timeoutMilliseconds: 2_000, lockRetryAttempts: 20 }), + pollingIntervalMilliseconds: 10, + leaseDurationMilliseconds: 250, + leaseRenewalIntervalMilliseconds: 50, + processHeartbeatIntervalMilliseconds: 75, + processAliveThresholdMilliseconds: 300, + workerCount: 1, + effectWorkerCount: 0, + reminderSchedulerCount: 0, + retentionIntervalMilliseconds: 0, + deadProcessCleanupIntervalMilliseconds: 0, + authorizeMessage: () => true, + authorizeQuery: () => true, + authorizeAdministration: () => true, + instrumentation: ({ name, attributes }) => { + process.send?.({ event: name, attributes }) + }, +}) + +runtime.register(RecoveryCounter) +await runtime.install() +const worker = runtime.worker() + +try { + let processed = 0 + for (let attempt = 0; attempt < 200 && processed === 0; attempt += 1) { + processed = await worker.runOnce({ activationRetention: "release" }) + if (processed === 0) await new Promise((resolve) => setTimeout(resolve, 10)) + } + process.send?.({ event: "worker.finished", processed }) +} finally { + await worker.stop() + await runtime.close() +} + +function requiredArgument(index: number): string { + const value = process.argv[index] + if (!value) throw new TypeError(`argument ${index - 1} is required`) + return value +} diff --git a/examples/sqlite-quickstart.ts b/examples/sqlite-quickstart.ts new file mode 100644 index 0000000..819880c --- /dev/null +++ b/examples/sqlite-quickstart.ts @@ -0,0 +1,109 @@ +import assert from "node:assert/strict" +import { existsSync } from "node:fs" +import { mkdtemp, rm } from "node:fs/promises" +import { tmpdir } from "node:os" +import { join, resolve } from "node:path" +import { fileURLToPath } from "node:url" +import { Actor, createRuntime } from "solid-objects" +import { sqlite } from "solid-objects/database/sqlite" + +class Counter extends Actor { + static override readonly actorType = "QuickstartCounter" + + count = 0 + + increment(): number { + this.count += 1 + return this.count + } + + async pause({ milliseconds }: { milliseconds: number }): Promise<{ + startedAt: number + finishedAt: number + }> { + const startedAt = performance.now() + await new Promise((resolvePromise) => setTimeout(resolvePromise, milliseconds)) + return { startedAt, finishedAt: performance.now() } + } +} + +export async function runQuickstart( + options: { + signal?: AbortSignal + write?: (value: string) => void + } = {}, +): Promise { + const directory = await mkdtemp(join(tmpdir(), "solid-objects-quickstart-")) + const databasePath = join(directory, "state.sqlite3") + const runtime = createRuntime({ + database: sqlite({ path: databasePath }), + workerCount: 2, + effectWorkerCount: 0, + reminderSchedulerCount: 0, + retentionIntervalMilliseconds: 0, + deadProcessCleanupIntervalMilliseconds: 0, + authorizeMessage: () => true, + authorizeQuery: () => true, + }) + const shutdown = new AbortController() + const abort = () => shutdown.abort(options.signal?.reason) + options.signal?.addEventListener("abort", abort, { once: true }) + let running: Promise | undefined + let sameIdentityFinalState = 0 + let independentIdentitiesOverlapped = false + + try { + runtime.register(Counter) + await runtime.install() + running = runtime.run(shutdown.signal) + + const counter = runtime.ref(Counter, "room-1") + const results = await Promise.all(Array.from({ length: 25 }, () => counter.increment())) + assert.deepEqual( + [...results].sort((left, right) => left - right), + Array.from({ length: 25 }, (_value, index) => index + 1), + ) + sameIdentityFinalState = await counter.count + assert.equal(sameIdentityFinalState, 25) + + const pauses = await Promise.all([ + runtime.ref(Counter, "room-2").send.pause({ milliseconds: 100 }), + runtime.ref(Counter, "room-3").send.pause({ milliseconds: 100 }), + ]) + const windows = await Promise.all( + pauses.map((message) => message.wait({ timeoutMilliseconds: 5_000 })), + ) + const firstWindow = windows[0] + const secondWindow = windows[1] + if (!firstWindow || !secondWindow) throw new Error("two execution windows are required") + independentIdentitiesOverlapped = + firstWindow.startedAt < secondWindow.finishedAt && + secondWindow.startedAt < firstWindow.finishedAt + assert.equal(independentIdentitiesOverlapped, true) + } finally { + shutdown.abort() + await running + await runtime.close() + options.signal?.removeEventListener("abort", abort) + await rm(directory, { recursive: true }) + } + + assert.equal(existsSync(directory), false) + const write = options.write ?? ((value: string) => process.stdout.write(value)) + write( + `${JSON.stringify( + { + sameIdentityCalls: 25, + sameIdentityFinalState, + independentIdentitiesOverlapped, + temporaryStateRemoved: true, + }, + null, + 2, + )}\n`, + ) +} + +if (process.argv[1] && fileURLToPath(import.meta.url) === resolve(process.argv[1])) { + await runQuickstart() +} diff --git a/package.json b/package.json index ef900dd..db5b7f6 100644 --- a/package.json +++ b/package.json @@ -1,21 +1,22 @@ { "name": "solid-objects", "version": "0.13.0", - "description": "Stateful, realtime TypeScript objects backed by PostgreSQL, SQLite, or MySQL", + "description": "Race-free realtime state per application identity, backed by your SQL database", "type": "module", "license": "MIT", "author": "Lucas Carlson", "keywords": [ - "actors", "concurrency", - "durable-objects", + "durable-state", "mysql", "nodejs", "postgresql", "realtime", "sqlite", "state-management", - "typescript" + "typescript", + "actors", + "durable-objects" ], "repository": { "type": "git", @@ -31,6 +32,7 @@ "files": [ "dist", "docs", + "examples", "README.md", "CHANGELOG.md", "MIT-LICENSE" @@ -66,8 +68,8 @@ } }, "scripts": { - "build": "pnpm run clean && tsc -p tsconfig.build.json && node scripts/prepare-executable.mjs", - "check": "pnpm run check:parameters && pnpm run check:documentation && tsc -p tsconfig.json --noEmit", + "build": "pnpm run clean && tsc -p tsconfig.build.json && tsc -p tsconfig.quickstart-build.json && node scripts/prepare-executable.mjs", + "check": "pnpm run check:parameters && pnpm run check:documentation && tsc -p tsconfig.json --noEmit && tsc -p tsconfig.examples.json --noEmit", "check:documentation": "node scripts/check-documentation.mjs", "check:parameters": "node scripts/check-parameter-style.mjs", "clean": "node scripts/clean.mjs", @@ -78,8 +80,11 @@ "test:coverage": "vitest run --coverage", "test:postgresql": "vitest run test/postgresql.test.ts", "test:mysql": "vitest run test/mysql.test.ts", + "test:package": "node scripts/release-artifact-smoke.mjs", + "test:recovery": "pnpm run build && node examples/failure-recovery/demo.ts", "test:redis": "vitest run test/redis-wake-up.test.ts", "test:watch": "vitest", + "benchmark": "pnpm run build && node benchmarks/run.ts", "pack:check": "pnpm pack --dry-run && node scripts/check-package.mjs", "prepack": "pnpm run build" }, diff --git a/scripts/check-documentation.mjs b/scripts/check-documentation.mjs index 30204f0..ae8093c 100644 --- a/scripts/check-documentation.mjs +++ b/scripts/check-documentation.mjs @@ -1,9 +1,11 @@ -import { readFile } from "node:fs/promises" +import { readFile, stat } from "node:fs/promises" import { dirname, resolve } from "node:path" const repositoryRoot = resolve(import.meta.dirname, "..") const documentationPaths = [ "README.md", + "CONTRIBUTING.md", + "SECURITY.md", "docs/api.md", "docs/architecture.md", "docs/authorization.md", @@ -12,23 +14,34 @@ const documentationPaths = [ "docs/correctness.md", "docs/dashboard.md", "docs/errors-and-recovery.md", + "docs/fit.md", + "docs/benchmarks.md", + "docs/comparisons.md", "docs/operations.md", "docs/parity.md", + "docs/releasing.md", "docs/state-and-lifecycle.md", + "docs/support.md", ] for (const documentationPath of documentationPaths) { const source = await readFile(resolve(repositoryRoot, documentationPath), "utf8") - for (const match of source.matchAll(/\[[^\]]+\]\(([^)]+\.md(?:#[^)]+)?)\)/g)) { + for (const match of source.matchAll(/!?\[[^\]]*\]\(([^)\s]+)(?:\s+"[^"]*")?\)/g)) { const link = match[1] - if (!link) continue - const [target, anchor] = link.split("#", 2) - if (!target) continue - const targetPath = resolve(repositoryRoot, dirname(documentationPath), target) - const targetSource = await readFile(targetPath, "utf8").catch(() => { + if (!link || /^(?:https?:|mailto:)/.test(link)) continue + const [encodedTarget = "", encodedAnchor] = link.split("#", 2) + const target = decodeURIComponent(encodedTarget) + const anchor = encodedAnchor ? decodeURIComponent(encodedAnchor) : undefined + const targetPath = target + ? resolve(repositoryRoot, dirname(documentationPath), target) + : resolve(repositoryRoot, documentationPath) + await stat(targetPath).catch(() => { throw new Error(`${documentationPath} links to missing ${link}`) }) - if (anchor && !headingAnchors(targetSource).has(anchor)) { + if (!anchor) continue + if (target && !target.toLowerCase().endsWith(".md")) continue + const targetSource = target ? await readFile(targetPath, "utf8") : source + if (!headingAnchors(targetSource).has(anchor)) { throw new Error(`${documentationPath} links to missing heading ${link}`) } } diff --git a/scripts/release-artifact-smoke.mjs b/scripts/release-artifact-smoke.mjs new file mode 100644 index 0000000..c58912d --- /dev/null +++ b/scripts/release-artifact-smoke.mjs @@ -0,0 +1,109 @@ +import assert from "node:assert/strict" +import { mkdtemp, mkdir, readFile, rm } from "node:fs/promises" +import { tmpdir } from "node:os" +import { join, resolve } from "node:path" +import { spawn } from "node:child_process" + +const repositoryRoot = resolve(import.meta.dirname, "..") +const temporaryDirectory = await mkdtemp(join(tmpdir(), "solid-objects-package-")) +const artifactDirectory = join(temporaryDirectory, "artifact") +const projectDirectory = join(temporaryDirectory, "project") + +try { + await mkdir(artifactDirectory) + await mkdir(projectDirectory) + await run("pnpm", ["run", "build"], { cwd: repositoryRoot }) + const packed = JSON.parse( + await run( + "npm", + ["pack", "--json", "--ignore-scripts", "--pack-destination", artifactDirectory], + { cwd: repositoryRoot }, + ), + )[0] + assert.equal(packed.name, "solid-objects") + assert.equal(packed.version, "0.13.0") + + const packagedPaths = new Set(packed.files.map((file) => file.path)) + for (const expectedPath of [ + "dist/index.js", + "dist/executable.js", + "dist/examples/sqlite-quickstart.js", + "examples/sqlite-quickstart.ts", + "docs/correctness.md", + "README.md", + ]) { + assert(packagedPaths.has(expectedPath), `package is missing ${expectedPath}`) + } + assert.equal( + [...packagedPaths].some((path) => path.startsWith("src/")), + false, + ) + assert.equal( + [...packagedPaths].some((path) => path.startsWith("test/")), + false, + ) + + const tarballPath = join(artifactDirectory, packed.filename) + await run("npm", ["init", "--yes"], { cwd: projectDirectory }) + await run("npm", ["install", "--ignore-scripts", tarballPath], { cwd: projectDirectory }) + + const installedPackage = JSON.parse( + await readFile(join(projectDirectory, "node_modules/solid-objects/package.json"), "utf8"), + ) + assert.equal(installedPackage.version, "0.13.0") + + const resolvedModule = ( + await run( + process.execPath, + [ + "--input-type=module", + "--eval", + "process.stdout.write(import.meta.resolve('solid-objects'))", + ], + { cwd: projectDirectory }, + ) + ).trim() + assert(resolvedModule.includes("/node_modules/solid-objects/dist/index.js")) + assert.equal(resolvedModule.startsWith(`file://${repositoryRoot}`), false) + + const quickstart = await run( + join(projectDirectory, "node_modules/.bin/solid-objects"), + ["quickstart"], + { cwd: projectDirectory }, + ) + const result = JSON.parse(quickstart) + assert.deepEqual(result, { + sameIdentityCalls: 25, + sameIdentityFinalState: 25, + independentIdentitiesOverlapped: true, + temporaryStateRemoved: true, + }) +} finally { + await rm(temporaryDirectory, { recursive: true }) +} + +async function run(command, argumentsValue, options) { + return new Promise((resolvePromise, reject) => { + const child = spawn(command, argumentsValue, { + ...options, + env: { ...process.env, NO_COLOR: "1" }, + stdio: ["ignore", "pipe", "pipe"], + }) + let stdout = "" + let stderr = "" + child.stdout.on("data", (chunk) => { + stdout += chunk + }) + child.stderr.on("data", (chunk) => { + stderr += chunk + }) + child.once("error", reject) + child.once("exit", (code) => { + if (code === 0) { + resolvePromise(stdout) + return + } + reject(new Error(`${command} exited ${code}\n${stdout}${stderr}`)) + }) + }) +} diff --git a/src/cli.ts b/src/cli.ts index f93161c..877e78c 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -11,6 +11,7 @@ export interface CliRunOptions { } const COMMANDS = new Set([ + "quickstart", "start", "doctor", "status", @@ -36,10 +37,27 @@ export async function runCli( if (!COMMANDS.has(command)) throw new TypeError(`unknown command ${JSON.stringify(command)}`) const parsed = parseArguments(commandArguments) + const write = options.write ?? ((value: string) => process.stdout.write(value)) + if (command === "quickstart") { + assertOptions(parsed, { command }) + assertNoPositionals(parsed, command) + const module = (await import( + new URL("./examples/sqlite-quickstart.js", import.meta.url).href + )) as { + runQuickstart(options: { + signal?: AbortSignal + write: (value: string) => void + }): Promise + } + await module.runQuickstart({ + ...(options.signal === undefined ? {} : { signal: options.signal }), + write, + }) + return 0 + } const configurationPath = stringOption(parsed, "config") ?? "solid-objects.config.js" const loadRuntime = options.loadRuntime ?? loadRuntimeModule const runtime = await loadRuntime(configurationPath) - const write = options.write ?? ((value: string) => process.stdout.write(value)) await runtime.install() try { @@ -232,6 +250,7 @@ function help(): string { return `Usage: solid-objects [options] Commands: + quickstart start doctor [--skip-round-trip] status diff --git a/tsconfig.examples.json b/tsconfig.examples.json new file mode 100644 index 0000000..df29592 --- /dev/null +++ b/tsconfig.examples.json @@ -0,0 +1,14 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "allowImportingTsExtensions": true, + "baseUrl": ".", + "paths": { + "solid-objects": ["./src/index.ts"], + "solid-objects/database/sqlite": ["./src/database/sqlite.ts"], + "solid-objects/database/postgresql": ["./src/database/postgresql.ts"], + "solid-objects/database/mysql": ["./src/database/mysql.ts"] + } + }, + "include": ["examples/**/*.ts", "benchmarks/**/*.ts"] +} diff --git a/tsconfig.quickstart-build.json b/tsconfig.quickstart-build.json new file mode 100644 index 0000000..71b5001 --- /dev/null +++ b/tsconfig.quickstart-build.json @@ -0,0 +1,12 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "rootDir": "examples", + "outDir": "dist/examples", + "declaration": false, + "declarationMap": false, + "sourceMap": true, + "noEmit": false + }, + "include": ["examples/sqlite-quickstart.ts"] +} From 8cc66fed1d00fc539c9d88a1b523494b489c4306 Mon Sep 17 00:00:00 2001 From: Lucas Carlson Date: Sun, 16 Aug 2026 07:20:25 -0700 Subject: [PATCH 2/8] fix: fail fast when benchmark workers exit Reject worker readiness when startup exits and terminate sibling workers so benchmark failures cannot hang. Validate recovery event records against concrete schemas before assertions. See #2 --- benchmarks/processes.ts | 59 ++++++++++++++++++++++++ benchmarks/run.ts | 44 ++++++------------ examples/failure-recovery/demo.ts | 61 +++++++++++++++++++++---- test/benchmark-processes.test.ts | 17 +++++++ test/fixtures/benchmark-worker-exit.mjs | 1 + 5 files changed, 143 insertions(+), 39 deletions(-) create mode 100644 benchmarks/processes.ts create mode 100644 test/benchmark-processes.test.ts create mode 100644 test/fixtures/benchmark-worker-exit.mjs diff --git a/benchmarks/processes.ts b/benchmarks/processes.ts new file mode 100644 index 0000000..458f5ed --- /dev/null +++ b/benchmarks/processes.ts @@ -0,0 +1,59 @@ +import type { ChildProcess } from "node:child_process" + +export function waitForWorkerReady(worker: ChildProcess): Promise { + const priorExit = workerExitError(worker, "before ready") + if (priorExit) return Promise.reject(priorExit) + + return new Promise((resolvePromise, reject) => { + const cleanup = () => { + worker.off("error", onError) + worker.off("exit", onExit) + worker.off("message", onMessage) + } + const onError = (error: Error) => { + cleanup() + reject(error) + } + const onExit = (code: number | null, signal: NodeJS.Signals | null) => { + cleanup() + reject(workerExitError({ exitCode: code, signalCode: signal }, "before ready")) + } + const onMessage = (message: string) => { + if (message !== "ready") return + cleanup() + resolvePromise() + } + worker.once("error", onError) + worker.once("exit", onExit) + worker.on("message", onMessage) + }) +} + +export function waitForWorkerExit(worker: ChildProcess): Promise { + const priorExit = workerExitError(worker) + if (priorExit) { + return worker.exitCode === 0 ? Promise.resolve() : Promise.reject(priorExit) + } + + return new Promise((resolvePromise, reject) => { + worker.once("error", reject) + worker.once("exit", (code, signal) => { + if (code === 0) { + resolvePromise() + return + } + reject(workerExitError({ exitCode: code, signalCode: signal })) + }) + }) +} + +function workerExitError( + worker: Pick, + phase?: string, +): Error | undefined { + if (worker.exitCode === null && worker.signalCode === null) return undefined + const suffix = phase ? ` ${phase}` : "" + return new Error( + `benchmark worker exited${suffix} with code ${worker.exitCode} and signal ${worker.signalCode}`, + ) +} diff --git a/benchmarks/run.ts b/benchmarks/run.ts index 41642ce..529a5ef 100644 --- a/benchmarks/run.ts +++ b/benchmarks/run.ts @@ -6,6 +6,7 @@ import { fileURLToPath } from "node:url" import { fork, type ChildProcess } from "node:child_process" import { performance } from "node:perf_hooks" import type { MessageReference } from "solid-objects" +import { waitForWorkerExit, waitForWorkerReady } from "./processes.ts" import { BenchmarkCounter, benchmarkRuntime, type BenchmarkDatabase } from "./shared.ts" type Shape = "warm-hot" | "warm-many" | "cold-many" @@ -82,7 +83,7 @@ try { } finally { shutdown.abort() await running - const workerExits = workers.map(waitForExit) + const workerExits = workers.map(waitForWorkerExit) for (const worker of workers) worker.send("stop") await Promise.all(workerExits) await runtime.testing.reset() @@ -232,18 +233,17 @@ async function spawnWorkers(options: { { stdio: ["ignore", "ignore", "inherit", "ipc"] }, ), ) - await Promise.all( - workers.map( - (worker) => - new Promise((resolvePromise, reject) => { - worker.once("error", reject) - worker.on("message", (message) => { - if (message === "ready") resolvePromise() - }) - }), - ), - ) - return workers + try { + await Promise.all(workers.map(waitForWorkerReady)) + return workers + } catch (error) { + const workerExits = workers.map(waitForWorkerExit) + for (const worker of workers) { + if (worker.exitCode === null && worker.signalCode === null) worker.kill() + } + await Promise.allSettled(workerExits) + throw error + } } async function readDatabaseVersion(runtime: ReturnType): Promise { @@ -263,24 +263,6 @@ async function readDatabaseVersion(runtime: ReturnType) }) } -function waitForExit(child: ChildProcess): Promise { - if (child.exitCode !== null) { - return child.exitCode === 0 - ? Promise.resolve() - : Promise.reject(new Error(`benchmark worker exited ${child.exitCode}`)) - } - if (child.signalCode !== null) { - return Promise.reject(new Error(`benchmark worker exited with ${child.signalCode}`)) - } - return new Promise((resolvePromise, reject) => { - child.once("error", reject) - child.once("exit", (code) => { - if (code === 0) resolvePromise() - else reject(new Error(`benchmark worker exited ${code}`)) - }) - }) -} - function percentile(values: number[], percent: number): number { const ordered = [...values].sort((left, right) => left - right) const index = Math.max(0, Math.ceil((percent / 100) * ordered.length) - 1) diff --git a/examples/failure-recovery/demo.ts b/examples/failure-recovery/demo.ts index 14b5bec..9ab5edd 100644 --- a/examples/failure-recovery/demo.ts +++ b/examples/failure-recovery/demo.ts @@ -15,6 +15,18 @@ interface WorkerMessage { processed?: number } +interface SerializationEvent { + event: "start" | "finish" + messageId: string + at: number +} + +interface ExternalEffectEvent { + messageId: string + attempt: number + processId: number +} + const directory = await mkdtemp(join(tmpdir(), "solid-objects-recovery-")) const databasePath = join(directory, "state.sqlite3") const runtime = createRuntime({ @@ -58,7 +70,10 @@ async function proveSerialization(): Promise<{ finalState: number; overlap: fals const workers = [spawnWorker(), spawnWorker()] await Promise.all(workers.map(({ finished }) => finished)) await Promise.all(messages.map((message) => message.result())) - const events = await jsonLines(join(controlDirectory, "serialization.jsonl")) + const events = await readJsonLines( + join(controlDirectory, "serialization.jsonl"), + parseSerializationEvent, + ) assert.equal(events.length, 4) const starts = events.filter((event) => event.event === "start") const finishes = events.filter((event) => event.event === "finish") @@ -121,7 +136,10 @@ async function recoveryResult(options: { const stored = await runtime.repository.findMessage(options.message.id) const attempts = Number(stored?.attempt_count) const snapshot = await options.reference.snapshot() - const effects = await jsonLines(join(options.controlDirectory, "external-effects.jsonl")) + const effects = await readJsonLines( + join(options.controlDirectory, "external-effects.jsonl"), + parseExternalEffectEvent, + ) assert.equal(attempts, 2) assert.equal(snapshot.count, 1) assert.equal(effects.length, 2) @@ -175,12 +193,39 @@ function spawnWorker(): { } } -async function jsonLines(path: string): Promise>> { - return (await readFile(path, "utf8")) - .trim() - .split("\n") - .filter(Boolean) - .map((line) => JSON.parse(line) as Record) +async function readJsonLines( + path: string, + parse: (line: string) => Value, +): Promise { + return (await readFile(path, "utf8")).trim().split("\n").filter(Boolean).map(parse) +} + +function parseSerializationEvent(line: string): SerializationEvent { + const event = JSON.parse(line) as Partial + if ( + (event.event !== "start" && event.event !== "finish") || + typeof event.messageId !== "string" || + typeof event.at !== "number" + ) { + throw new TypeError("invalid serialization event") + } + return { event: event.event, messageId: event.messageId, at: event.at } +} + +function parseExternalEffectEvent(line: string): ExternalEffectEvent { + const event = JSON.parse(line) as Partial + if ( + typeof event.messageId !== "string" || + typeof event.attempt !== "number" || + typeof event.processId !== "number" + ) { + throw new TypeError("invalid external effect event") + } + return { + messageId: event.messageId, + attempt: event.attempt, + processId: event.processId, + } } async function wait(milliseconds: number): Promise { diff --git a/test/benchmark-processes.test.ts b/test/benchmark-processes.test.ts new file mode 100644 index 0000000..eff210b --- /dev/null +++ b/test/benchmark-processes.test.ts @@ -0,0 +1,17 @@ +import { fork } from "node:child_process" +import { fileURLToPath } from "node:url" +import { describe, expect, it } from "vitest" +import { waitForWorkerReady } from "../benchmarks/processes.js" + +describe("benchmark worker processes", () => { + it("rejects when a worker exits before signaling readiness", async () => { + const worker = fork( + fileURLToPath(new URL("./fixtures/benchmark-worker-exit.mjs", import.meta.url)), + { stdio: ["ignore", "ignore", "ignore", "ipc"] }, + ) + + await expect(waitForWorkerReady(worker)).rejects.toThrow( + "benchmark worker exited before ready with code 7 and signal null", + ) + }) +}) diff --git a/test/fixtures/benchmark-worker-exit.mjs b/test/fixtures/benchmark-worker-exit.mjs new file mode 100644 index 0000000..9c37421 --- /dev/null +++ b/test/fixtures/benchmark-worker-exit.mjs @@ -0,0 +1 @@ +process.exitCode = 7 From 6594af782d67fb37ffc47a8bf82cbb3c930d44c3 Mon Sep 17 00:00:00 2001 From: Lucas Carlson Date: Sun, 16 Aug 2026 07:49:11 -0700 Subject: [PATCH 3/8] docs: explain coordination use cases Show concrete stateful coordination patterns before the API example so readers can recognize when a per-identity mailbox fits their problem. Keep the hot-identity and SQL-transaction boundaries beside those examples. See #2 --- README.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/README.md b/README.md index 92016a3..688d1f3 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,28 @@ application already operates. > the TypeScript implementation is new. Read the [delivery boundaries](#delivery-boundaries) > before using it for important data. +## What Solid Objects is for + +Use Solid Objects when more than one request, job, or process can act on the +same logical thing and the next action must use its latest committed state. +These are the stateful coordination patterns for which people often reach for +Durable Objects: + +| Pattern | One identity per | What the object coordinates | +| --------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------- | +| Multiplayer, presence, or collaboration | Room, session, or document | Joins, moves, and edits commit in order; subscribers refresh from committed state | +| Reservations and expiring holds | Show, resource, or stock item | Availability checks and holds cannot interleave; a durable reminder can release an old hold | +| Checkout and account workflows | Cart, order, account, device | The current step, retries, and effect results return to the same ordered mailbox | +| Per-key rate limits | API key, account, or device | Token checks and decrements are serialized; a reminder can refill the bucket | +| Stateful agent sessions | Agent session | Messages and tool results apply in order and pending work survives a worker exit | + +The common shape is one durable coordination boundary with an application +defined identity. Work for that identity is serialized, while unrelated rooms, +carts, accounts, or sessions can progress concurrently. A single global rate +limiter or another very hot identity is a poor fit because it becomes an +intentional bottleneck. If one ordinary row transaction solves the problem, +prefer that. See [Choosing Solid Objects](docs/fit.md) for the longer guide. + ## The programming model ```typescript From 70293eb193e275c7332a6ed65511bb3ca5cf03bb Mon Sep 17 00:00:00 2001 From: Lucas Carlson Date: Sun, 16 Aug 2026 07:54:13 -0700 Subject: [PATCH 4/8] docs: reorder README introduction Lead with the programming model, follow with concrete coordination use cases, and keep the SQLite quickstart directly after both sections. See #2 --- README.md | 44 ++++++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 688d1f3..7bda777 100644 --- a/README.md +++ b/README.md @@ -18,28 +18,6 @@ application already operates. > the TypeScript implementation is new. Read the [delivery boundaries](#delivery-boundaries) > before using it for important data. -## What Solid Objects is for - -Use Solid Objects when more than one request, job, or process can act on the -same logical thing and the next action must use its latest committed state. -These are the stateful coordination patterns for which people often reach for -Durable Objects: - -| Pattern | One identity per | What the object coordinates | -| --------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------- | -| Multiplayer, presence, or collaboration | Room, session, or document | Joins, moves, and edits commit in order; subscribers refresh from committed state | -| Reservations and expiring holds | Show, resource, or stock item | Availability checks and holds cannot interleave; a durable reminder can release an old hold | -| Checkout and account workflows | Cart, order, account, device | The current step, retries, and effect results return to the same ordered mailbox | -| Per-key rate limits | API key, account, or device | Token checks and decrements are serialized; a reminder can refill the bucket | -| Stateful agent sessions | Agent session | Messages and tool results apply in order and pending work survives a worker exit | - -The common shape is one durable coordination boundary with an application -defined identity. Work for that identity is serialized, while unrelated rooms, -carts, accounts, or sessions can progress concurrently. A single global rate -limiter or another very hot identity is a poor fit because it becomes an -intentional bottleneck. If one ordinary row transaction solves the problem, -prefer that. See [Choosing Solid Objects](docs/fit.md) for the longer guide. - ## The programming model ```typescript @@ -64,6 +42,28 @@ Both calls enter the durable mailbox for `cart-123`. They execute in order and commit one state transition at a time, even when different requests or Node.js processes submit them concurrently. +## What Solid Objects is for + +Use Solid Objects when more than one request, job, or process can act on the +same logical thing and the next action must use its latest committed state. +These are the stateful coordination patterns for which people often reach for +Durable Objects: + +| Pattern | One identity per | What the object coordinates | +| --------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------- | +| Multiplayer, presence, or collaboration | Room, session, or document | Joins, moves, and edits commit in order; subscribers refresh from committed state | +| Reservations and expiring holds | Show, resource, or stock item | Availability checks and holds cannot interleave; a durable reminder can release an old hold | +| Checkout and account workflows | Cart, order, account, device | The current step, retries, and effect results return to the same ordered mailbox | +| Per-key rate limits | API key, account, or device | Token checks and decrements are serialized; a reminder can refill the bucket | +| Stateful agent sessions | Agent session | Messages and tool results apply in order and pending work survives a worker exit | + +The common shape is one durable coordination boundary with an application +defined identity. Work for that identity is serialized, while unrelated rooms, +carts, accounts, or sessions can progress concurrently. A single global rate +limiter or another very hot identity is a poor fit because it becomes an +intentional bottleneck. If one ordinary row transaction solves the problem, +prefer that. See [Choosing Solid Objects](docs/fit.md) for the longer guide. + ## Run it now with SQLite Node.js 24.15 or newer is required. The `0.13.0` release includes a From 03a62310efced7b8f37ab976ca9573476a41ad63 Mon Sep 17 00:00:00 2001 From: Lucas Carlson Date: Sun, 16 Aug 2026 08:01:28 -0700 Subject: [PATCH 5/8] docs: tighten README claims Classify the package immediately, describe the exact browser coverage, and scope the quickstart assertions to one local run. Put the runnable example before the broader use-case table. See #2 --- README.md | 65 +++++++++++++++++++++++++++---------------------------- 1 file changed, 32 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index 7bda777..2484df9 100644 --- a/README.md +++ b/README.md @@ -3,20 +3,19 @@ [![CI](https://github.com/cardmagic/solid-objects-js/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/cardmagic/solid-objects-js/actions/workflows/ci.yml) [![npm](https://img.shields.io/npm/v/solid-objects)](https://www.npmjs.com/package/solid-objects) -Solid Objects gives each logical application identity—such as a room, cart, -account, device, document, or agent session—durable state and a sequential -mailbox. Concurrent calls for one identity cannot overwrite each other. Calls -for different identities can run at the same time. +Solid Objects is a TypeScript actor library for Node.js that gives each +application-defined identity durable state and a sequential mailbox, backed by +SQLite, PostgreSQL, or MySQL. Concurrent calls for one identity cannot +overwrite each other. Calls for different identities can run at the same time. Define ordinary TypeScript classes and run them in ordinary Node.js processes. State, queued operations, retries, reminders, effects, and realtime -invalidations are stored in the SQLite, PostgreSQL, or MySQL database the -application already operates. +invalidations are stored in the database the application already operates. > **Early release:** the correctness core has automated coverage across the -> supported databases, browsers, process recovery, and packaged artifacts, but -> the TypeScript implementation is new. Read the [delivery boundaries](#delivery-boundaries) -> before using it for important data. +> supported databases, the Chromium browser client, process recovery, and +> packaged artifacts, but the TypeScript implementation is new. Read the +> [delivery boundaries](#delivery-boundaries) before using it for important data. ## The programming model @@ -42,28 +41,6 @@ Both calls enter the durable mailbox for `cart-123`. They execute in order and commit one state transition at a time, even when different requests or Node.js processes submit them concurrently. -## What Solid Objects is for - -Use Solid Objects when more than one request, job, or process can act on the -same logical thing and the next action must use its latest committed state. -These are the stateful coordination patterns for which people often reach for -Durable Objects: - -| Pattern | One identity per | What the object coordinates | -| --------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------- | -| Multiplayer, presence, or collaboration | Room, session, or document | Joins, moves, and edits commit in order; subscribers refresh from committed state | -| Reservations and expiring holds | Show, resource, or stock item | Availability checks and holds cannot interleave; a durable reminder can release an old hold | -| Checkout and account workflows | Cart, order, account, device | The current step, retries, and effect results return to the same ordered mailbox | -| Per-key rate limits | API key, account, or device | Token checks and decrements are serialized; a reminder can refill the bucket | -| Stateful agent sessions | Agent session | Messages and tool results apply in order and pending work survives a worker exit | - -The common shape is one durable coordination boundary with an application -defined identity. Work for that identity is serialized, while unrelated rooms, -carts, accounts, or sessions can progress concurrently. A single global rate -limiter or another very hot identity is a poor fit because it becomes an -intentional bottleneck. If one ordinary row transaction solves the problem, -prefer that. See [Choosing Solid Objects](docs/fit.md) for the longer guide. - ## Run it now with SQLite Node.js 24.15 or newer is required. The `0.13.0` release includes a @@ -77,8 +54,8 @@ The command needs no repository checkout, database server, Redis, container, or application configuration. It uses Node's built-in SQLite module and removes its scoped temporary database before exiting. -The executable asserts rather than merely printing a plausible result. It -proves that: +The executable asserts rather than merely printing a plausible result. In one +local run, it verifies that: - 25 concurrent calls to one identity produce the exact committed state `25`; - their return values are the complete sequence from `1` through `25`; @@ -88,6 +65,28 @@ proves that: Before `0.13.0` reaches the registry, maintainers can run the identical executable from a generated package tarball with `pnpm run test:package`. +## What Solid Objects is for + +Use Solid Objects when more than one request, job, or process can act on the +same logical thing and the next action must use its latest committed state. +These are the stateful coordination patterns for which people often reach for +Durable Objects: + +| Pattern | One identity per | What the object coordinates | +| --------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------- | +| Multiplayer, presence, or collaboration | Room, session, or document | Joins, moves, and edits commit in order; subscribers refresh from committed state | +| Reservations and expiring holds | Show, resource, or stock item | Availability checks and holds cannot interleave; a durable reminder can release an old hold | +| Checkout and account workflows | Cart, order, account, device | The current step, retries, and effect results return to the same ordered mailbox | +| Per-key rate limits | API key, account, or device | Token checks and decrements are serialized; a reminder can refill the bucket | +| Stateful agent sessions | Agent session | Messages and tool results apply in order and pending work survives a worker exit | + +The common shape is one durable coordination boundary with an application +defined identity. Work for that identity is serialized, while unrelated rooms, +carts, accounts, or sessions can progress concurrently. A single global rate +limiter or another very hot identity is a poor fit because it becomes an +intentional bottleneck. If one ordinary row transaction solves the problem, +prefer that. See [Choosing Solid Objects](docs/fit.md) for the longer guide. + ## How it works An object is addressed by its TypeScript class and application-defined ID. From 6f40eafaa417bd0a6404bb73f27d72919cad769e Mon Sep 17 00:00:00 2001 From: Lucas Carlson Date: Sun, 16 Aug 2026 08:17:57 -0700 Subject: [PATCH 6/8] docs: clarify README headline State the deployment distinction and existing-database requirement in the repository headline so readers can classify the project immediately. See #2 --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2484df9..5e1d097 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Solid Objects JS +# Solid Objects: Durable Objects without Cloudflare using just your existing database [![CI](https://github.com/cardmagic/solid-objects-js/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/cardmagic/solid-objects-js/actions/workflows/ci.yml) [![npm](https://img.shields.io/npm/v/solid-objects)](https://www.npmjs.com/package/solid-objects) From 6a3bf4c85074745743eb92f1055775d274a99134 Mon Sep 17 00:00:00 2001 From: Lucas Carlson Date: Sun, 16 Aug 2026 08:21:03 -0700 Subject: [PATCH 7/8] docs: refine README introduction Use the familiar Durable Objects category while stating the Node and SQL deployment boundaries in a concise headline and subheadline. See #2 --- README.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 5e1d097..6391477 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,13 @@ -# Solid Objects: Durable Objects without Cloudflare using just your existing database +# Durable Objects for Node, backed by your existing SQL database [![CI](https://github.com/cardmagic/solid-objects-js/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/cardmagic/solid-objects-js/actions/workflows/ci.yml) [![npm](https://img.shields.io/npm/v/solid-objects)](https://www.npmjs.com/package/solid-objects) -Solid Objects is a TypeScript actor library for Node.js that gives each -application-defined identity durable state and a sequential mailbox, backed by -SQLite, PostgreSQL, or MySQL. Concurrent calls for one identity cannot -overwrite each other. Calls for different identities can run at the same time. +Build addressable TypeScript objects with serialized calls and durable state +using SQLite, PostgreSQL, or MySQL, without deploying to Cloudflare. + +Concurrent calls for one identity cannot overwrite each other. Calls for +different identities can run at the same time. Define ordinary TypeScript classes and run them in ordinary Node.js processes. State, queued operations, retries, reminders, effects, and realtime From 9cdd7ae3f808eac603b2c9c9f3f5184bf9b7dde5 Mon Sep 17 00:00:00 2001 From: Lucas Carlson Date: Sun, 16 Aug 2026 08:27:35 -0700 Subject: [PATCH 8/8] docs: finalize 0.13.0 release text Remove the temporary registry note before packaging the README and record the intended August 16 publication date in the changelog. See #2 --- CHANGELOG.md | 2 +- README.md | 3 --- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f938ad8..99a86cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ ## Unreleased -## 0.13.0 - 2026-08-15 +## 0.13.0 - 2026-08-16 - Replace the exhaustive README with an outcome-first introduction, explicit fit and correctness boundaries, sourced comparisons, and factual design diff --git a/README.md b/README.md index 6391477..cb8fc34 100644 --- a/README.md +++ b/README.md @@ -63,9 +63,6 @@ local run, it verifies that: - operations for two different identities overlap in time; and - the runtime closes and temporary state is removed. -Before `0.13.0` reaches the registry, maintainers can run the identical -executable from a generated package tarball with `pnpm run test:package`. - ## What Solid Objects is for Use Solid Objects when more than one request, job, or process can act on the