diff --git a/content/docs/ai/actions-as-tools.mdx b/content/docs/ai/actions-as-tools.mdx index 40bd3ff0af..f440cd0d98 100644 --- a/content/docs/ai/actions-as-tools.mdx +++ b/content/docs/ai/actions-as-tools.mdx @@ -41,7 +41,7 @@ Three action types are supported: | `action.type` | Dispatch path | Wiring needed | |:---|:---|:---| | `script` | `IDataEngine.executeAction(object, target, ctx)` — the same call Studio makes | none beyond the metadata service | -| `api` | HTTP call to `action.target` via `ApiActionClient` (default: `fetch`) | `apiBaseUrl` (or a custom `apiClient`) | +| `api` | HTTP call to `action.target` via the configured `apiClient` (default: `fetch`) | `apiActionBaseUrl` (or a custom `apiClient`) | | `flow` | `IAutomationService.execute(target, { triggerData })` | `automation` service registered | Console-only types (`url`, `modal`, `form`) are always skipped. Dangerous @@ -158,7 +158,7 @@ automatically scope what an agent can see and do. There is no separate **How it works:** 1. The REST routes for `/api/v1/ai/assistant/chat` and - `/api/v1/ai/agents/:id/chat` pull the authenticated principal out of + `/api/v1/ai/agents/:agentName/chat` pull the authenticated principal out of `req.user` and forward it to `aiService.chatWithTools(...)` as `toolExecutionContext: { actor, conversationId, environmentId }`. Both **cookie session** (`better-auth.session_token`) and **Bearer @@ -197,8 +197,10 @@ Omit `toolExecutionContext` to keep the previous system-level behaviour ## LLM-generated conversation titles -Auto-titling is **opt-in** (disabled by default; enable it via the `ai` -settings namespace from Console → Settings → AI). Once enabled, after a +Auto-titling is **on by default** for any configured LLM provider (the +memory/echo provider never triggers it, since no real LLM call is made); +toggle `title_generation_enabled` off via the `ai` settings namespace from +Console → Settings → AI if you'd rather leave conversations unnamed. Once a conversation has at least one user + assistant exchange (≥ 2 messages) and still has no title, the AI service fires a short, out-of-band LLM call (default cap **16 characters**, single-line, no quotes) to summarise what the diff --git a/content/docs/ai/chatbot-integration.mdx b/content/docs/ai/chatbot-integration.mdx index f863efe123..05ee3433a6 100644 --- a/content/docs/ai/chatbot-integration.mdx +++ b/content/docs/ai/chatbot-integration.mdx @@ -30,7 +30,7 @@ contract. On a cloud / EE dev host (where `@objectstack/service-ai` is available — see the callout above), the `default` and `full` plugin tier presets both -include the `ai` capability, so `objectstack dev` boots the AI services +include the `ai` capability, so `os dev` boots the AI services unless you opt out with `--preset minimal`. Provide a Vercel AI Gateway model and key via env vars: @@ -106,7 +106,7 @@ const chat = useObjectChat({ ## 4. HITL (Human-in-the-Loop) flow When the agent picks a dangerous action (e.g. `delete_task`) the tool -handler enqueues an `ai_pending_action` row and the chat returns a +handler enqueues an `ai_pending_actions` row and the chat returns a `pending_approval` tool result. Your Console UI then: 1. Polls `GET /api/v1/ai/pending-actions?status=pending`. diff --git a/content/docs/ai/skills-reference.mdx b/content/docs/ai/skills-reference.mdx index fc134ce29d..928c3e5e10 100644 --- a/content/docs/ai/skills-reference.mdx +++ b/content/docs/ai/skills-reference.mdx @@ -46,7 +46,7 @@ ObjectStack ships **9 domain-specific skills**. Each is self-contained — an AI | # | Skill | Domain | Path | What it covers | | :--- | :--- | :--- | :--- | :--- | | 1 | [Platform](#platform) | `platform` | `skills/objectstack-platform/` | Bootstrap, configure, extend, and operate ObjectStack runtimes. Covers project setup (`defineStack`, drivers, adapters, scaffolding), plugin and service development (PluginContext, DI, kernel hooks like `kernel:ready` and `data:*`), and operations (CLI commands, migrations, deployment, test harnesses via LiteKernel). | -| 2 | [Data](#data) | `data` | `skills/objectstack-data/` | Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, row-level security — and the seed datasets (`defineDataset()`) that load fixtures and reference data alongside them. | +| 2 | [Data](#data) | `data` | `skills/objectstack-data/` | Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, row-level security — and the seed definitions (`defineSeed()`) that load fixtures and reference data alongside them. | | 3 | [Query](#query) | `query` | `skills/objectstack-query/` | Construct ObjectQL queries — filters, sorting, pagination, aggregation, joins/expansion, window functions, and full-text search. | | 4 | [UI](#ui) | `ui` | `skills/objectstack-ui/` | Author ObjectStack UI metadata — Views (list/form/kanban/calendar/gantt), Apps (navigation), Pages (structured plus the HTML and React source-authoring tiers, ADR-0080/0081), Dashboards, Reports, Charts, Actions, and package Docs (`src/docs/*.md`). | | 5 | [Automation](#automation) | `automation` | `skills/objectstack-automation/` | Design ObjectStack automation — Flows (visual logic), Workflows (declarative rules), Triggers, Approvals, scheduled jobs, and webhooks. | @@ -75,7 +75,7 @@ Do not use for data schema design (see objectstack-data) or query patterns (see **Domain** `data` · **Path** `skills/objectstack-data/` -Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, row-level security — and the seed datasets (`defineDataset()`) that load fixtures and reference data alongside them. +Design ObjectStack data schemas — objects, fields, field conditional rules, relationships, validations, indexes, lifecycle hooks, permissions, row-level security — and the seed definitions (`defineSeed()`) that load fixtures and reference data alongside them. Use when the user is creating or modifying `*.object.ts` / `*.seed.ts` files, picking field types, modelling relationships, writing `beforeInsert`/`afterUpdate` hooks, configuring per-object access control, or authoring bootstrap / demo data. Use for `visibleWhen` / `readonlyWhen` / `requiredWhen` rules that belong on fields. @@ -147,7 +147,7 @@ Do not use for general LLM prompting questions unrelated to ObjectStack metadata Design the server-side API surface that an ObjectStack runtime exposes — REST/GraphQL endpoints, auth providers, realtime channels, error envelopes, batch/versioning contracts. -Use when the user is adding `*.endpoint.ts`, configuring auth providers, defining custom routes, or extending the REST/GraphQL generator. +Use when the user is adding `ApiEndpoint` entries to the `apis:` array in `defineStack()`, configuring auth providers, defining custom routes, or extending the REST/GraphQL generator. Do not use for: consuming an ObjectStack API from a client (that is just standard HTTP — no skill needed); the auto-generated CRUD endpoints (those follow from objectstack-data); request-side query syntax (see objectstack-query). CEL expressions in route guards or auth predicates: load objectstack-formula alongside. diff --git a/content/docs/ai/skills.mdx b/content/docs/ai/skills.mdx index 942c2882e7..7e1ec554d8 100644 --- a/content/docs/ai/skills.mdx +++ b/content/docs/ai/skills.mdx @@ -67,7 +67,7 @@ Each skill ships `SKILL.md` plus a generated `references/` index, and richer ski | **Overview** | `SKILL.md` | Yes | High-level guide with decision trees and quick-start examples | | **References** | `references/_index.md` | Generated | Pointers into the published `@objectstack/spec` Zod sources (present in every skill except Formula) | | **Rules** | `rules/*.md` | Optional | Detailed implementation rules with ✅ correct / ❌ incorrect code examples (today: Data, Platform, Query) | -| **Evaluations** | `evals/*.md` | Optional | Test cases to validate AI assistant understanding | +| **Evaluations** | `evals/*.md` | Optional | Test cases to validate AI assistant understanding (mostly planned — `objectstack-automation` ships the first worked example; other skills' `evals/` directories are still placeholders) | ### SKILL.md — The Entry Point @@ -144,11 +144,11 @@ Each skill has **clear boundaries** — it knows what it's responsible for and e ▼ ▼ ▼ ┌───────┐ ┌──────┐ ┌────────────┐ │ Query │ │ UI │ │ Automation │ - └───────┘ └──────┘ └─────┬──────┘ - │ │ - ┌────▼────┐ ┌────▼────┐ - │ API │ │ AI │ - └─────────┘ └─────────┘ + └───┬───┘ └──────┘ └─────┬──────┘ + │ │ + ┌────▼────┐ ┌────▼────┐ + │ API │ │ AI │ + └─────────┘ └─────────┘ Formula and i18n are cross-cutting — load them alongside any host skill. ``` @@ -215,7 +215,7 @@ Each skill maps directly to an ObjectStack protocol domain. The schema skill tea ### 4. Testable -The `evals/` directory in each skill allows teams to validate that their AI assistant correctly understands the protocol. This is analogous to test suites for runtime code — but for AI comprehension. +The `evals/` directory in each skill is reserved for test cases that validate an AI assistant correctly understands the protocol — analogous to test suites for runtime code, but for AI comprehension. Most skills' `evals/` directories are still placeholders; `objectstack-automation` ships the first worked example (`evals/approvals/test-revise-loop.md`). --- diff --git a/content/docs/api/client-sdk.mdx b/content/docs/api/client-sdk.mdx index 7f091c683f..7bcd2d2d59 100644 --- a/content/docs/api/client-sdk.mdx +++ b/content/docs/api/client-sdk.mdx @@ -35,11 +35,11 @@ const client = new ObjectStackClient({ }); async function main() { - // 1. Connect — fetches discovery manifest - await client.connect(); + // 1. Connect — resolves with the discovery manifest + const discovery = await client.connect(); // 2. Check available services - console.log('Services:', client.discovery?.services); + console.log('Services:', discovery.services); // → { metadata: { enabled: true, status: 'degraded' }, data: { enabled: true, status: 'available' }, auth: { enabled: false, ... } } // 3. Query data @@ -80,17 +80,16 @@ When you call `client.connect()`, the client: 3. Configures all API route paths dynamically ```typescript -await client.connect(); +// connect() resolves with the discovery manifest — capture the return value +const discovery = await client.connect(); -// Discovery result is available -const discovery = client.discovery; console.log(discovery.version); // "1.0.0" console.log(discovery.environment); // "development" // Check if a service is available before using it if (discovery.services?.auth?.enabled) { // Auth plugin is installed — login is available - await client.auth.login({ username: 'admin', password: 'secret' }); + await client.auth.login({ email: 'admin@example.com', password: 'secret' }); } else { console.log(discovery.services?.auth?.message); // → "Install an auth plugin to enable" @@ -105,7 +104,7 @@ if (discovery.services?.auth?.enabled) { ## Protocol Coverage -The `@objectstack/client` SDK aims to implement the ObjectStack API protocol specification. The core namespaces are listed below; the client also exposes additional namespaces (`approvals`, `feed`, `organizations`, `projects`/environments) — see [`index.ts`](https://github.com/objectstack-ai/framework/blob/main/packages/client/src/index.ts) for the full surface: +The `@objectstack/client` SDK aims to implement the ObjectStack API protocol specification. The core namespaces are listed below; the client also exposes additional namespaces (`approvals`, `feed`, `organizations`, `oauth`, `projects`/environments) — see [`index.ts`](https://github.com/objectstack-ai/framework/blob/main/packages/client/src/index.ts) for the full surface: | Namespace | Status | Methods | Purpose | |:----------|:------:|:--------|:--------| @@ -116,7 +115,7 @@ The `@objectstack/client` SDK aims to implement the ObjectStack API protocol spe | **permissions** | ✅ | 3 | Access control checks | | **packages** | ✅ | 6 | Plugin/package lifecycle management | | **views** | ✅ | 5 | UI view definitions | -| **workflow** | ✅ | 5 | Workflow state transitions | +| **workflow** | ✅ | 3 | Workflow state transitions | | **analytics** | ✅ | 3 | Analytics queries | | **automation** | ✅ | 1 | Automation triggers | | **storage** | ✅ | 2 | File upload & download | @@ -217,7 +216,7 @@ const result = await client.analytics.query({ cube: 'account', measures: ['revenue.sum', 'count'], dimensions: ['industry'], - filters: [{ member: 'status', operator: 'equals', values: ['active'] }], + where: { status: 'active' }, limit: 100, }); @@ -235,10 +234,10 @@ const explained = await client.analytics.explain({ ```typescript const packages = await client.packages.list(); -await client.packages.install({ name: 'plugin-auth', version: '1.0.0' }); -await client.packages.enable('plugin-auth'); -await client.packages.disable('plugin-auth'); -await client.packages.uninstall('plugin-auth'); +await client.packages.install({ id: 'com.objectstack.plugin-auth', version: '1.0.0' }); +await client.packages.enable('com.objectstack.plugin-auth'); +await client.packages.disable('com.objectstack.plugin-auth'); +await client.packages.uninstall('com.objectstack.plugin-auth'); ``` ### Additional Namespaces @@ -248,7 +247,7 @@ The client also provides full implementations for: ```typescript // Auth — User authentication and session management await client.auth.login({ email: 'user@example.com', password: 'pass' }); -await client.auth.register({ email: 'new@example.com', password: 'pass' }); +await client.auth.register({ email: 'new@example.com', password: 'pass', name: 'New User' }); await client.auth.me(); await client.auth.logout(); await client.auth.refreshToken('refresh-token-string'); @@ -272,10 +271,10 @@ await client.approvals.reject(requestId, { comment: 'Incomplete' }); await client.approvals.listActions(requestId); // audit trail // Realtime — WebSocket subscriptions -await client.realtime.connect({ protocol: 'websocket' }); -await client.realtime.subscribe({ channel: 'account', event: 'update' }); +await client.realtime.connect({ transport: 'websocket' }); +await client.realtime.subscribe({ channel: 'account', events: ['record.updated'] }); await client.realtime.unsubscribe('subscription-id'); -await client.realtime.setPresence('account', { status: 'online' }); +await client.realtime.setPresence('account', { userId: 'user-123', status: 'online', lastSeen: new Date().toISOString() }); await client.realtime.getPresence('account'); await client.realtime.disconnect(); @@ -320,7 +319,7 @@ await client.views.delete('account', viewId); ``` -**Service availability**: Optional services (workflow, ai, etc.) are only available when the corresponding plugin is installed on the server. Always check `client.discovery?.services` to verify service availability before calling these methods. +**Service availability**: Optional services (workflow, ai, etc.) are only available when the corresponding plugin is installed on the server. Always check the `services` map on the discovery result returned by `client.connect()` (cache it yourself — the client has no `discovery` getter) to verify service availability before calling these methods. --- @@ -381,8 +380,8 @@ The `find` method accepts an options object with **canonical** (recommended) fie | `offset` | `number` | Records to skip (OFFSET) | `0` | | `expand` | `Record` or `string[]` | Relation loading (JOIN) | `{ owner: {} }` | - -**Removed in 11:** the legacy query-field aliases were removed — use the canonical names: `fields` (was `select`), `where` (was `filter`/`filters`), `orderBy` (was `sort`), `limit` (was `top`), `offset` (was `skip`). + +**Canonical names recommended:** `find()` accepts the canonical names above; the legacy aliases (`select`, `filter`/`filters`, `sort`, `top`, `skip`) still work for backward compatibility, but new code should use the canonical names. (`@objectstack/client-react`'s `useQuery`/`useInfiniteQuery` hooks are stricter — the legacy aliases were removed there in v11; see `docs/upgrading-to-11.md`.) ### Batch Options @@ -418,9 +417,9 @@ try { |:-----|:-----|:---------|:----------|:------------| | `validation_error` | 400 | validation | No | Input validation failed | | `invalid_query` | 400 | validation | No | Malformed query expression | -| `unauthenticated` | 401 | auth | No | Authentication required | -| `permission_denied` | 403 | auth | No | Insufficient permissions | -| `resource_not_found` | 404 | request | No | Resource does not exist | +| `unauthenticated` | 401 | authentication | No | Authentication required | +| `permission_denied` | 403 | authorization | No | Insufficient permissions | +| `resource_not_found` | 404 | not_found | No | Resource does not exist | | `rate_limit_exceeded` | 429 | rate_limit | Yes | Too many requests | | `internal_error` | 500 | server | Yes | Unexpected server error | | `service_unavailable` | 503 | server | Yes | Service temporarily unavailable | @@ -543,7 +542,7 @@ Integration tests verify end-to-end communication with a live ObjectStack server For detailed information about the client's protocol implementation: -- **[Protocol Compliance Matrix](https://github.com/objectstack-ai/framework/blob/main/packages/client/CLIENT_SPEC_COMPLIANCE.md)** — Method-by-method verification of all API methods across 15 namespaces +- **[Protocol Compliance Matrix](https://github.com/objectstack-ai/framework/blob/main/packages/client/CLIENT_SPEC_COMPLIANCE.md)** — Method-by-method verification of all API methods across 13 namespaces - **[Integration Test Specifications](https://github.com/objectstack-ai/framework/blob/main/packages/client/CLIENT_SERVER_INTEGRATION_TESTS.md)** — Comprehensive test cases for client-server communication - **[Package README](https://github.com/objectstack-ai/framework/blob/main/packages/client/README.md)** — Developer navigation and API reference diff --git a/content/docs/api/data-api.mdx b/content/docs/api/data-api.mdx index 381ab792eb..48ac11845f 100644 --- a/content/docs/api/data-api.mdx +++ b/content/docs/api/data-api.mdx @@ -21,12 +21,12 @@ Query records with filtering, sorting, selection, and pagination. | `select` | query | Comma-separated field names | | `filter` | query | Filter expression (JSON). `filters` also accepted for backward compatibility. | | `sort` | query | Sort expression (e.g. `name asc` or `-created_at`) | -| `top` | query | Limit (default: 20) | +| `top` | query | Max records to return. No default — omitting it returns all matching records. | | `skip` | query | Offset | | `expand` | query | Comma-separated list of relations to eager-load | | `search` | query | Full-text search query | -> **Note:** OData-style `$`-prefixed parameters (`$filter`, `$select`, `$orderby`, `$top`, `$skip`) are supported via the [OData endpoint](/docs/references/api/odata). The standard REST API uses unprefixed parameter names. +> **Note:** OData-style `$`-prefixed parameters (`$filter`, `$select`, `$orderby`, `$top`, `$skip`, `$expand`, `$count`, `$search`) are also accepted directly on this same endpoint as aliases — they're normalized internally to the parameter names above. There is no separate standalone OData endpoint. **Response**: ```json @@ -192,13 +192,21 @@ Filtering uses the canonical Query DSL `where` object (the same MongoDB-style `F ### `GET /analytics/meta` -Get auto-generated cube metadata for all objects. +Get metadata for all registered cubes. Cubes are explicitly defined (via `defineCube` +or the analytics service's `cubes` config) — a cube referenced by a query that isn't +yet registered is lazily auto-inferred from that query's shape, but metadata isn't +proactively generated for every object. -**Response**: Array of cube definitions with measures, dimensions, and time dimensions. +**Response**: Array of cube definitions with measures and dimensions (time-based +dimensions are `dimensions` entries with `type: "time"`). ### `POST /analytics/sql` -Execute a raw SQL analytics query (if supported by driver). +Generate the SQL for a given analytics query **without executing it** (dry-run/debug). +Accepts the same body shape as `/analytics/query`; support depends on the underlying +driver/strategy. + +**Response**: `{ success: true, data: { sql: string, params: unknown[] } }` --- diff --git a/content/docs/api/data-flow.mdx b/content/docs/api/data-flow.mdx index 96c6e9ab58..1e6ea50e81 100644 --- a/content/docs/api/data-flow.mdx +++ b/content/docs/api/data-flow.mdx @@ -103,7 +103,7 @@ sequenceDiagram Sec-->>K: Fields filtered by profile K-->>API: Final record - API-->>C: 200 OK { data: { ... } } + API-->>C: 200 OK { object: 'task', id: 'tsk_123', record: { ... } } ``` @@ -129,15 +129,15 @@ sequenceDiagram participant EB as Event Bus participant WS as WebSocket - C->>API: POST /api/v1/data/task { data: {...} } + C->>API: POST /api/v1/data/task { title: '...', ... } API->>K: create('task', data, context) K->>V: validate(schema, data) alt Validation Failed - V-->>K: ZodError - K-->>API: 400 VALIDATION_ERROR - API-->>C: { error: { details: [...] } } + V-->>K: ValidationError { fields: [...] } + K-->>API: 400 VALIDATION_FAILED + API-->>C: { error, code: 'VALIDATION_FAILED', fields: [...] } end V-->>K: ✓ Valid data @@ -165,7 +165,7 @@ sequenceDiagram EB->>WS: Broadcast to subscribers K-->>API: Created record - API-->>C: 201 Created { data: { id: 'tsk_456', ... } } + API-->>C: 201 Created { object: 'task', id: 'tsk_456', record: { ... } } ``` --- @@ -199,7 +199,7 @@ flowchart LR | Output | Used By | Purpose | |:---|:---|:---| -| JSON Schema | VS Code, IntelliJ | Autocomplete and validation in `*.object.ts` files | +| JSON Schema | VS Code, IntelliJ | Autocomplete and validation for `objectstack.config.ts` (via `os generate schema`) | | TypeScript Types | Plugin developers | Type-safe access to object definitions | | Manifest | Kernel | Runtime metadata for query validation and execution | | Metadata API | Client SDK | Dynamic object/field discovery | @@ -211,6 +211,10 @@ flowchart LR How data changes propagate to connected clients in real time. + +**Implementation status:** This diagram (and the `emit(...)` → `EB->>WS` broadcast step in the Write Flow diagram above) shows the target design. The realtime service that ships today is an **in-memory pub/sub adapter** (`InMemoryRealtimeAdapter` in `@objectstack/service-realtime`), delivered to JS clients via **long-polling** (`RealtimeAPI` in `@objectstack/client`) — no WebSocket transport is wired up yet. The adapter also only matches subscriptions by `object` and `eventTypes`; per-field `filter` conditions (e.g. `{ project: 'prj_1' }`) are accepted in the request shape but are not yet enforced server-side. See [Real-Time Protocols](/docs/protocol/objectos/realtime-protocol) for the full implementation-status breakdown. + + ```mermaid sequenceDiagram participant W as Writer Client @@ -246,7 +250,7 @@ sequenceDiagram Sub->>R2: { event: 'task.updated', data: { id: 'tsk_123', ... } } K-->>API: Updated record - API-->>W: 200 OK { data: { ... } } + API-->>W: 200 OK { object: 'task', id: 'tsk_123', record: { ... } } ``` diff --git a/content/docs/api/environment-routing.mdx b/content/docs/api/environment-routing.mdx index 49640173e1..f79fddeab1 100644 --- a/content/docs/api/environment-routing.mdx +++ b/content/docs/api/environment-routing.mdx @@ -27,25 +27,22 @@ Enable scoped route registration in `objectstack.config.ts`: ```typescript import { defineStack } from '@objectstack/spec'; -const stack = defineStack({ +export default defineStack({ // ...your manifest, objects, apis, etc. -}); - -export default { - ...stack, api: { enableProjectScoping: true, projectResolution: 'auto', }, -}; +}); ``` - -`defineStack` validates against a schema that has no top-level `api` field, so -unknown keys are stripped — passing `api` *inside* `defineStack({ ... })` is -silently dropped and scoping stays disabled. Attach the `api` block to the -exported config object instead, as shown above. The CLI reads it from the -exported config (`config.api`) when registering the REST and dispatcher plugins. + +`api` is a declared top-level field on `ObjectStackDefinitionSchema`, so it +survives `defineStack`'s strict parsing — you can pass it directly inside the +`defineStack({ ... })` call as shown above. (Older stacks that instead spread +it onto the exported config object, e.g. `export default { ...stack, api: {...} }`, +still work the same way.) The CLI reads the resolved value from the exported +config (`config.api`) when registering the REST and dispatcher plugins. The option names are historical for compatibility with existing config files; @@ -73,7 +70,7 @@ const client = new ObjectStackClient({ const env = client.project('env_prod'); -await env.data.find('customer', { limit: 20 }); +await env.data.find('customer', { top: 20 }); await env.meta.getItems('object'); await env.packages.list(); ``` diff --git a/content/docs/api/error-catalog.mdx b/content/docs/api/error-catalog.mdx index 58837f70e4..dba8b340e9 100644 --- a/content/docs/api/error-catalog.mdx +++ b/content/docs/api/error-catalog.mdx @@ -12,6 +12,10 @@ ObjectStack uses a structured error system with **9 error categories** and **51 **Import:** `import { StandardErrorCode, ErrorCategory, ErrorResponseSchema } from '@objectstack/spec/api'` + +**Spec vs. wire format:** The codes below are `StandardErrorCode` — the standardized, lowercase snake_case contract that plugin/hook authors should throw with (see [Server-Side Error Handling](/docs/api/error-handling-server)). The kernel REST server (`@objectstack/rest`) that serves `/api/v1/data/*` today emits a flatter envelope with SCREAMING_SNAKE_CASE codes instead — e.g. `VALIDATION_FAILED`, `PERMISSION_DENIED`, `RECORD_NOT_FOUND`, `CONCURRENT_UPDATE` — not the codes in this catalog. See the [API Overview](/docs/api#error-handling) for both wire formats in use and [Wire Format](/docs/api/wire-format#7-error-response-format) for JSON examples. + + --- ## Error Categories @@ -384,6 +388,15 @@ interface FieldError { ## Client-Side Error Handling + +The `@objectstack/client` SDK's built-in fetch error handling attaches only +`code`, `category`, `httpStatus`, `retryable`, and `details` to the thrown +error — not `fieldErrors`, `retryAfter`, or `requestId` directly. If your +server populates those on the response body, read them from +`apiError.details` (e.g. `apiError.details?.fieldErrors`) until the client +surfaces them at the top level. + + ### TypeScript Example ```typescript diff --git a/content/docs/api/error-handling-client.mdx b/content/docs/api/error-handling-client.mdx index 597186f04b..b406d64fa6 100644 --- a/content/docs/api/error-handling-client.mdx +++ b/content/docs/api/error-handling-client.mdx @@ -138,7 +138,7 @@ function TaskForm() { const response = await fetch('/api/v1/data/task', { method: 'POST', headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ data }), + body: JSON.stringify(data), }); if (!response.ok) { diff --git a/content/docs/api/error-handling-server.mdx b/content/docs/api/error-handling-server.mdx index e51de23aa1..df4325096e 100644 --- a/content/docs/api/error-handling-server.mdx +++ b/content/docs/api/error-handling-server.mdx @@ -8,7 +8,7 @@ description: Best practices for handling and throwing errors in ObjectStack plug This guide covers best practices for handling and throwing errors within ObjectStack plugins and server-side code — from creating custom errors with proper codes to transaction rollback and integration with `safeParsePretty()`. -**Error Contract:** ObjectStack defines a standardized error envelope, `ErrorResponseSchema` (exported from `@objectstack/spec/api`). It wraps an `EnhancedApiError` with a machine-readable `code`, `httpStatus`, optional `category`, and `fieldErrors`. Errors you throw on the server should carry these fields so the API layer can serialize them into that envelope. +**Error Contract:** ObjectStack defines a standardized error envelope, `ErrorResponseSchema` (exported from `@objectstack/spec/api`). It wraps an `EnhancedApiError` with a machine-readable `code`, `httpStatus`, optional `category`, and `fieldErrors`. This is the spec's target contract for plugin/hook authors to throw against — but the kernel REST server (`@objectstack/rest`) that serves `/api/v1/data/*` does not auto-translate an arbitrary thrown error into this envelope today. It emits a flatter `{ error, code }` shape and only recognizes a small, fixed set of SCREAMING_SNAKE_CASE codes (`VALIDATION_FAILED`, `PERMISSION_DENIED`, `RECORD_NOT_FOUND`, `CONCURRENT_UPDATE`, `DELETE_RESTRICTED`, …). See the [API Overview](/docs/api#error-handling) and [Wire Format](/docs/api/wire-format#7-error-response-format) for both wire formats in use, and the [Error Catalog](/docs/api/error-catalog) for the full `StandardErrorCode` list. --- diff --git a/content/docs/api/index.mdx b/content/docs/api/index.mdx index 075e2f9d71..7ea6622d36 100644 --- a/content/docs/api/index.mdx +++ b/content/docs/api/index.mdx @@ -15,7 +15,7 @@ ObjectStack exposes a fully typed REST API. All endpoints use JSON request/respo | :--- | :--- | | **REST** | ✅ Auto-generated from the protocol (`@objectstack/rest`) — CRUD, query, batch, metadata, packages | | **Realtime** | ⚠️ In-process pub/sub service (`@objectstack/service-realtime`, single-instance); the `/realtime/*` REST routes and WebSocket/SSE transport are plugin-provided — none ships in the open framework | -| **MCP** | ✅ Objects and opted-in actions exposed as Model Context Protocol tools ([AI module](/docs/ai)) | +| **MCP** | ✅ Non-system objects and their actions exposed as Model Context Protocol tools, gated by the caller's permissions/RLS rather than a per-action opt-in flag ([AI module](/docs/ai)) | | **GraphQL** | ⚠️ Route is wired but **bring-your-own service**: `/graphql` returns 501 unless an implementation of the `IGraphQLService` contract is registered — none ships in the open framework | | **OData** | ⚠️ Vocabulary only: REST list endpoints accept OData-style operators (e.g. `$top`), but there is no standalone OData endpoint | @@ -53,50 +53,37 @@ Returns the full discovery manifest. **Response**: ```json { - "name": "ObjectOS", "version": "1.0.0", - "environment": "development", + "apiName": "ObjectStack API", "routes": { "data": "/api/v1/data", "metadata": "/api/v1/meta", - "analytics": "/api/v1/analytics", - "auth": null, - "workflow": null - }, - "features": { - "graphql": false, - "search": false, - "websockets": false, - "files": false, - "analytics": true, - "ai": false, - "workflow": false, - "notifications": false, - "i18n": false + "analytics": "/api/v1/analytics" }, "services": { "metadata": { "enabled": true, - "status": "degraded", + "status": "available", "route": "/api/v1/meta", - "provider": "kernel", - "message": "In-memory registry; DB persistence pending" + "provider": "objectql" }, - "data": { "enabled": true, "status": "available", "route": "/api/v1/data", "provider": "kernel" }, - "analytics": { "enabled": true, "status": "available", "route": "/api/v1/analytics" }, - "auth": { "enabled": false, "status": "unavailable", "message": "Install an auth plugin to enable" } + "data": { "enabled": true, "status": "available", "route": "/api/v1/data", "provider": "objectql" }, + "analytics": { "enabled": true, "status": "available", "route": "/api/v1/analytics", "provider": "objectql" }, + "auth": { "enabled": false, "status": "unavailable", "message": "Install plugin-auth to enable" } }, - "locale": { - "default": "en", - "supported": ["en", "zh-CN"], - "timezone": "UTC" + "capabilities": { + "feed": { "enabled": false }, + "automation": { "enabled": false }, + "search": { "enabled": false } } } ``` +Disabled/uninstalled route keys (e.g. `auth`, `workflow`) are omitted from `routes` entirely rather than set to `null`; check `services` to tell "not installed" apart from "installed but not yet mounted here." + ### `GET /.well-known/objectstack` -Alias for the discovery endpoint used by auto-discovery in the client SDK. +Served by the runtime dispatcher (`@objectstack/runtime`), not `@objectstack/rest` — its body is wrapped as `{ "data": { ... } }` and includes fields (`name`, `environment`, `features`, `locale`) that the `@objectstack/rest`-served `/api/v1` response above does not. The client SDK's `connect()` tries `/api/v1/discovery` first and falls back to this endpoint, unwrapping either `body.data` or the bare `body`. **Service Status Values**: `available` (fully operational), `registered` (route declared but handler unverified — may return 501), `degraded` (partial functionality), `unavailable` (not installed), `stub` (placeholder that throws errors) @@ -117,11 +104,11 @@ Error responses depend on which HTTP server is in front of the kernel. There are } ``` -Validation failures additionally include an `issues` array. Common codes emitted by the kernel REST server: +Validation failures additionally include a `fields` array (one entry per invalid field). Common codes emitted by the kernel REST server: | Code | HTTP | Description | |:-----|:-----|:------------| -| `VALIDATION_FAILED` | 400 | Input validation failed (includes `issues`) | +| `VALIDATION_FAILED` | 400 | Input validation failed (includes `fields`) | | `PERMISSION_DENIED` | 403 | Insufficient permissions | | `RECORD_NOT_FOUND` | 404 | Resource does not exist | | `CONCURRENT_UPDATE` | 409 | Record was modified by another user | diff --git a/content/docs/api/metadata-api.mdx b/content/docs/api/metadata-api.mdx index 3d6dbda821..d36e7d1be5 100644 --- a/content/docs/api/metadata-api.mdx +++ b/content/docs/api/metadata-api.mdx @@ -70,9 +70,10 @@ List installed packages. ### `POST /packages` -Install a package from manifest. +Publish a package (manifest + metadata) to the package registry. -**Body**: `{ manifest: { name: "plugin-auth", version: "1.0.0", ... } }` +**Body**: `{ manifest: { id: "plugin-auth", name: "Plugin Auth", version: "1.0.0", ... }, metadata: { objects: [...], views: [...], ... } }` +**Response**: `{ success: true, message: "...", package: { id: "plugin-auth", version: "1.0.0" } }` ### `GET /packages/:id` diff --git a/content/docs/api/plugin-endpoints.mdx b/content/docs/api/plugin-endpoints.mdx index 55ee4d12dd..50e8c06094 100644 --- a/content/docs/api/plugin-endpoints.mdx +++ b/content/docs/api/plugin-endpoints.mdx @@ -13,12 +13,12 @@ These REST endpoints are only available when the corresponding plugin is install These endpoints are only available when an auth plugin is installed. Check `discovery.services.auth.enabled` first. -### `POST /auth/login` +### `POST /auth/sign-in/email` -Authenticate and receive a session token. +Authenticate with email and password (better-auth's email sign-in route, mounted under the auth plugin's `/auth` prefix) and receive a session token. There is no `/auth/login` route. -**Body**: `{ username: "admin", password: "secret" }` -**Response**: `{ token: "jwt...", expiresAt: "...", user: { ... } }` +**Body**: `{ email: "admin@example.com", password: "secret" }` +**Response**: `{ token: "jwt...", user: { ... } }` --- @@ -48,11 +48,15 @@ The automation dispatcher also exposes flow CRUD (`GET`/`POST /automation`, `GET | Method | Endpoint | Description | |:-------|:---------|:------------| -| GET | `/ui/views?object=:object` | List views for an object | -| GET | `/ui/views/:viewId` | Get a view definition | -| POST | `/ui/views` | Create a new view | -| PATCH | `/ui/views/:viewId` | Update a view | -| DELETE | `/ui/views/:viewId` | Delete a view | +| GET | `/ui/views/:object` | List views for an object | +| GET | `/ui/views/:object/:viewId` | Get a view definition | +| POST | `/ui/views/:object` | Create a new view | +| PATCH | `/ui/views/:object/:viewId` | Update a view | +| DELETE | `/ui/views/:object/:viewId` | Delete a view | + + +The auto-generated (non-CRUD) view resolver `GET /ui/view/:object/:type` is always available and provided by the kernel — see [Metadata API](/docs/api/metadata-api). It is a separate route from the `/ui/views` CRUD above. + ### Realtime (`/realtime`) — Plugin Required @@ -82,10 +86,13 @@ The core dispatcher implements only the list / read / read-all routes above. Dev | Method | Endpoint | Description | |:-------|:---------|:------------| | POST | `/ai/nlq` | Natural language → query | -| POST | `/ai/chat` | AI chat conversation | | POST | `/ai/suggest` | Get value suggestions | | POST | `/ai/insights` | Get data insights | + +There is no `/ai/chat` route — the AI chat route was removed so the wire protocol aligns with the Vercel AI SDK. Use `useChat()` (`@ai-sdk/react`) directly against the streaming chat endpoint rather than the client SDK's `ai` namespace, which intentionally does not expose a `chat` method. + + ### i18n (`/i18n`) — Plugin Required | Method | Endpoint | Description | diff --git a/content/docs/api/wire-format.mdx b/content/docs/api/wire-format.mdx index 92b17ab6f5..97ee9e64a7 100644 --- a/content/docs/api/wire-format.mdx +++ b/content/docs/api/wire-format.mdx @@ -61,7 +61,7 @@ The response is the `CreateDataResponse` envelope: `{ object, id, record }`. "created_at": "2025-01-20T10:30:00.000Z", "updated_at": "2025-01-20T10:30:00.000Z", "created_by": "usr_01HQ3V5K8N2M4P6R7T9W", - "owner": "usr_01HQ3V5K8N2M4P6R7T9W" + "owner_id": "usr_01HQ3V5K8N2M4P6R7T9W" } } ``` @@ -131,7 +131,7 @@ Filtering uses the MongoDB-style `where` clause: `{ field: value }` for equality }, "fields": ["id", "title", "status", "priority", "assigned_to", "due_date"], "expand": { - "assigned_to": { "object": "user", "fields": ["id", "name"] } + "assigned_to": { "object": "sys_user", "fields": ["id", "name"] } }, "orderBy": [ { "field": "due_date", "order": "asc" } @@ -225,7 +225,7 @@ The response is the `UpdateDataResponse` envelope: `{ object, id, record }`. "created_at": "2025-01-20T10:30:00.000Z", "updated_at": "2025-01-20T14:15:00.000Z", "created_by": "usr_01HQ3V5K8N2M4P6R7T9W", - "owner": "usr_01HQ3V5K8N2M4P6R7T9W" + "owner_id": "usr_01HQ3V5K8N2M4P6R7T9W" } } ``` @@ -302,7 +302,7 @@ Metadata is addressed by type name. `object` is the metadata type, so this route "name": "assigned_to", "label": "Assigned To", "type": "lookup", - "reference": "user" + "reference": "sys_user" }, { "name": "due_date", @@ -365,7 +365,7 @@ Field-level failures carry a `fields` array. ```json { - "error": "Insufficient permissions to update task records", + "error": "[Security] Access denied: operation 'update' on object 'task' is not permitted for roles [standard_user]", "code": "PERMISSION_DENIED", "object": "task" } @@ -394,7 +394,7 @@ Returned when an `If-Match` / `expectedVersion` token no longer matches the stor **`POST /api/v1/data/task/batch`** -Process many records of a **single** operation type in one request. The body carries one `operation` (`create`, `update`, `upsert`, or `delete`) plus a `records` array. By default the batch is atomic (rolled back on any failure); set `options.atomic: false` to allow partial success. +Process many records of a **single** operation type in one request. The body carries one `operation` (`create`, `update`, `upsert`, or `delete`) plus a `records` array. By default (`options.atomic: true`) processing stops at the first failing record — records already written earlier in the same batch are **not** rolled back, since there is no wrapping database transaction. Set `options.atomic: false` (with `options.continueOnError: true`) to keep processing every record and collect a full partial-success report. ### Request @@ -415,7 +415,7 @@ Process many records of a **single** operation type in one request. The body car ### Response — `200 OK` -The response is the `BatchUpdateResponse` envelope: a top-level `success` flag plus `total` / `succeeded` / `failed` counts and a per-record `results` array. +The response is the `BatchUpdateResponse` envelope: a top-level `success` flag plus `total` / `succeeded` / `failed` counts and a per-record `results` array. Each successful entry echoes the written `record`; pass `options.returnRecords: false` to get back just `{ id, success }` per result. ```json { @@ -425,15 +425,15 @@ The response is the `BatchUpdateResponse` envelope: a top-level `success` flag p "succeeded": 2, "failed": 0, "results": [ - { "id": "tsk_01HQ4A7B9D3F5G8J2K4L", "success": true, "index": 0 }, - { "id": "tsk_01HQ4B8C0E4G6H9K3L5M", "success": true, "index": 1 } + { "id": "tsk_01HQ4A7B9D3F5G8J2K4L", "success": true, "record": { "id": "tsk_01HQ4A7B9D3F5G8J2K4L", "status": "done" } }, + { "id": "tsk_01HQ4B8C0E4G6H9K3L5M", "success": true, "record": { "id": "tsk_01HQ4B8C0E4G6H9K3L5M", "status": "done" } } ] } ``` ### Partial Failure Response -When `options.atomic: false` and some records fail, the failing entries carry an `errors` array: +When `options.atomic: false` and some records fail, the failing entries carry a single `error` message string (not an array): ```json { @@ -443,14 +443,8 @@ When `options.atomic: false` and some records fail, the failing entries carry an "succeeded": 1, "failed": 1, "results": [ - { "id": "tsk_01HQ4A7B9D3F5G8J2K4L", "success": true, "index": 0 }, - { - "success": false, - "index": 1, - "errors": [ - { "code": "record_not_found", "message": "Record not found: task/tsk_invalid_id" } - ] - } + { "id": "tsk_01HQ4A7B9D3F5G8J2K4L", "success": true, "record": { "id": "tsk_01HQ4A7B9D3F5G8J2K4L", "status": "done" } }, + { "id": "tsk_invalid_id", "success": false, "error": "Record tsk_invalid_id not found in task" } ] } ``` @@ -479,7 +473,8 @@ When `options.atomic: false` and some records fail, the failing entries carry an | Header | Description | |:---|:---| | `X-Request-Id` | Server-assigned (or echoed) request ID | -| `X-RateLimit-Limit` | Maximum requests per window | -| `X-RateLimit-Remaining` | Remaining requests in current window | -| `X-RateLimit-Reset` | UTC timestamp when the window resets | | `ETag` | Entity tag for conditional requests | + + +**Rate limiting is opt-in:** ObjectStack ships a token-bucket `RateLimiter` primitive (`@objectstack/runtime`), but it is **not** wired into the default REST response path — a deployment must add it at the adapter layer. Only when a deployment does so will responses carry `X-RateLimit-Limit` / `X-RateLimit-Remaining` / `X-RateLimit-Reset` (and a `429` with `Retry-After` once the limit is hit). + diff --git a/content/docs/automation/approvals.mdx b/content/docs/automation/approvals.mdx index e538ee99ea..0c3435dc23 100644 --- a/content/docs/automation/approvals.mdx +++ b/content/docs/automation/approvals.mdx @@ -34,11 +34,20 @@ The node declares **who approves** (a named user, a role, the submitter's manage // Illustrative — see the Approval reference for the exact node schema. defineFlow({ name: 'invoice_approval', + type: 'record_change', runAs: 'user', // submitter's RLS unless a step needs more - trigger: { object: 'invoice', on: 'create' }, nodes: [ - { type: 'approval', approver: { type: 'role', role: 'finance_manager' }, - onApprove: 'mark_approved', onReject: 'mark_rejected' }, + { id: 'start', type: 'start', label: 'Start', + config: { objectName: 'invoice', triggerType: 'record-after-create' } }, + { id: 'approval', type: 'approval', label: 'Approval', + config: { approvers: [{ type: 'role', value: 'finance_manager' }] } }, + { id: 'mark_approved', type: 'update_record', label: 'Mark Approved' }, + { id: 'mark_rejected', type: 'update_record', label: 'Mark Rejected' }, + ], + edges: [ + { id: 'e1', source: 'start', target: 'approval' }, + { id: 'e2', source: 'approval', target: 'mark_approved', label: 'approve' }, + { id: 'e3', source: 'approval', target: 'mark_rejected', label: 'reject' }, ], }); ``` @@ -74,7 +83,7 @@ export const opportunityApproval: Flow = { type: 'approval', label: 'Manager Approval', config: { - approvers: [{ type: 'user', value: '${record.owner_manager_id}' }], + approvers: [{ type: 'field', value: 'owner_manager_id' }], behavior: 'unanimous', approvalStatusField: 'approval_status', lockRecord: true, @@ -118,7 +127,7 @@ Separating *who configures* from *who approves* from *as whom it runs* is the sa ## Runnable example -- Flows: [`examples/app-showcase/src/flows`](https://github.com/objectstack-ai/framework/tree/main/examples/app-showcase/src/flows). +- Flows: [`examples/app-showcase/src/automation/flows`](https://github.com/objectstack-ai/framework/tree/main/examples/app-showcase/src/automation/flows). - Approval schema: [`packages/spec/src/automation/approval.zod.ts`](https://github.com/objectstack-ai/framework/blob/main/packages/spec/src/automation/approval.zod.ts). ## Anti-patterns diff --git a/content/docs/automation/flows.mdx b/content/docs/automation/flows.mdx index 667e001d46..6c55f5705c 100644 --- a/content/docs/automation/flows.mdx +++ b/content/docs/automation/flows.mdx @@ -595,9 +595,9 @@ export const hotLeadFollowUp: Flow = { label: 'Create Follow-up Task', config: { object: 'task', - data: { + fields: { subject: 'Follow up on hot lead', - related_to: '${record.id}', + related_to: '{record.id}', priority: 'high', }, }, @@ -607,8 +607,10 @@ export const hotLeadFollowUp: Flow = { type: 'notify', label: 'Notify Owner', config: { - channel: 'in_app', - template: 'hot_lead_created', + recipients: '{record.owner}', + title: 'New hot lead', + message: 'Hot lead created: {record.name}', + channels: ['inbox'], }, }, { id: 'end', type: 'end', label: 'End' }, diff --git a/content/docs/automation/hook-bodies.mdx b/content/docs/automation/hook-bodies.mdx index 814043c8ba..ed1312fb9f 100644 --- a/content/docs/automation/hook-bodies.mdx +++ b/content/docs/automation/hook-bodies.mdx @@ -5,7 +5,7 @@ description: How hook handlers and script-action bodies travel through ObjectSta # Hook & Action Bodies -ObjectStack treats every hook handler and every `type: 'script'` action as **pure metadata**. There is no separate `.mjs` file shipped alongside the project artifact, no dynamic `import()` at runtime, and no filesystem dependency on the cloud. A body is either: +ObjectStack treats every hook handler and every `type: 'script'` action as **pure metadata**. In the self-contained (body-only) form there is no separate `.mjs` file shipped alongside the project artifact, no dynamic `import()` at runtime, and no filesystem dependency on the cloud — though today a legacy `objectstack-runtime.{hash}.mjs` back-compat bundle can still ship, and does get dynamically imported at boot, for any handler that hasn't been lowered to a metadata `body` yet (see [Migration](#migration) below). A body is either: - **L1 — Expression**    a formula-engine string, side-effect-free. - **L2 — Sandboxed JS**    a JavaScript source string executed inside an isolated VM with declared capabilities. @@ -33,7 +33,7 @@ export default defineStack({ ``` ```jsonc -// Build artifact (account.hook.json — what objectos actually loads) +// Build artifact (excerpt from dist/objectstack.json's "hooks" array — what objectos actually loads) { "name": "normalize_account", "object": "account", @@ -61,9 +61,8 @@ This is the same trade-off ServiceNow made (Business Rules), Salesforce made (Fo ## L1 — Expression bodies -Pure formula. No IO, no mutation. Used for: +Pure formula. No IO, no mutation. This is the `body.language: 'expression'` shape, used for: -- Hook `condition` (run-this-hook? predicate) - Action `body` for trivial computed values - Validation rules @@ -71,7 +70,9 @@ Pure formula. No IO, no mutation. Used for: { "language": "expression", "source": "input.amount > 1000 && input.status == 'open'" } ``` -Evaluated by the same formula engine that powers field formulas — see [Formula Reference](/docs/data-modeling/formulas). +Hook `condition` is a **separate field with a different envelope** — a bare CEL string, or `{ dialect: 'cel', source }` (`ExpressionInputSchema` in `packages/spec/src/shared/expression.zod.ts`), not a `{ language, source }` body. e.g. `condition: 'record.status == "open" && record.amount > 1000'`. + +Both forms are evaluated by the same formula engine that powers field formulas — see [Formula Reference](/docs/data-modeling/formulas). ## L2 — Sandboxed JS bodies @@ -99,7 +100,7 @@ The script sees only what the surrounding `ctx` object exposes: | `ctx.api.object(name).find\|count\|aggregate` | Cross-object reads, scoped to current tenant. | `api.read` | | `ctx.api.object(name).insert\|update\|delete` | Cross-object writes. | `api.write` | | `ctx.crypto.randomUUID()` | UUID generation. | `crypto.uuid` | -| `ctx.crypto.hash(algo, data)` | Sha-256/512 etc. | `crypto.hash` | +| `ctx.crypto.hash(algo, data)` _(not yet wired)_ | Sha-256/512 etc. The `crypto.hash` capability exists in the schema and is inferred by the build-time extractor, but the QuickJS sandbox currently only installs `ctx.crypto.randomUUID` — calling `ctx.crypto.hash(...)` throws inside the VM today. | `crypto.hash` | | `ctx.log.{info,warn,error}` | Structured logging. | `log` | | `ctx.connector(name).(...)` _(planned)_ | Outbound HTTP / SaaS calls. **Not yet wired into the sandbox** — ships with the separate Connector spec. | (separate Connector spec) | @@ -108,9 +109,9 @@ The script sees only what the surrounding `ctx` object exposes: The CLI builder **rejects** any source that uses: - `import` / `require` / dynamic `import()` -- `fetch`, `XMLHttpRequest`, `WebSocket` -- `process`, `globalThis`, `Buffer`, `setImmediate` -- `eval`, `new Function`, `Function` constructor +- `fetch` +- `process`, `globalThis` +- `eval`, `new Function` - references to identifiers from value-only top-level imports Need outbound HTTP? Define a **Connector recipe** as metadata and call it via `ctx.connector(...)`. (Connector spec is tracked separately and ships after L1+L2 stabilises.) @@ -148,7 +149,7 @@ If you have a body that genuinely cannot be expressed in L1+L2 (typically: it ne 2. For each inline handler, take its source via `String(fn)` (the callable is already loaded by tsx/esbuild). 3. Run a regex allow-list over the stringified body (see "What the sandbox forbids" above). 4. **Pass:** emit `body: { language: 'js', source: , capabilities: }`. -5. **Forbidden token (default):** extraction fails, a `bodyExtractionWarning` is recorded, and the callable still ships via the back-compat handler-ref bundle — the build does **not** abort. Pass `objectstack compile --strict-body` to turn extraction warnings into a hard build failure (exit 1) with a per-callable diagnostic, e.g. `hook 'normalize_account': fetch() is not allowed in hook bodies — declare a Connector recipe instead.` +5. **Forbidden token (default):** extraction fails, a `bodyExtractionWarning` is recorded, and the callable still ships via the back-compat handler-ref bundle — the build does **not** abort. Pass `objectstack compile --strict-body` to turn extraction warnings into a hard build failure (exit 1) with a per-callable diagnostic, e.g. `hook 'normalize_account': fetch() is not allowed in hook/action bodies — declare a Connector recipe instead.` Capabilities are inferred by matching known patterns in the body source (e.g. `ctx.api.object(...).insert(...)` ⇒ `api.write`). A fuller AST-based analysis is planned for a later version. You can override with a directive comment when the inference is wrong. diff --git a/content/docs/automation/hooks.mdx b/content/docs/automation/hooks.mdx index 25b204f933..4c715bf589 100644 --- a/content/docs/automation/hooks.mdx +++ b/content/docs/automation/hooks.mdx @@ -86,7 +86,8 @@ ctx = { result, // mutable operation result (after* events) previous, // record state before the operation (update/delete) session, // { userId, tenantId, roles, accessToken, isSystem } - user, // { id, name, email } convenience shortcut + user, // { id, name, email } convenience shortcut — reserved for future use; + // not currently set by the engine, use session.userId instead transaction, // active transaction handle, if any ql, // ObjectQL engine reference api, // scoped cross-object access: ctx.api.object('x') diff --git a/content/docs/automation/webhooks.mdx b/content/docs/automation/webhooks.mdx index 4b939355b3..b3791c6b1e 100644 --- a/content/docs/automation/webhooks.mdx +++ b/content/docs/automation/webhooks.mdx @@ -44,27 +44,33 @@ rely on a stable contract. ## 2. Design principles **P1 — At-least-once, never at-most-once.** Receivers MUST be prepared -for duplicates. We never silently drop a delivery, but we also never -guarantee uniqueness — that requires receiver-side idempotency keys, which -we provide but cannot enforce. +for duplicates. Once a delivery is enqueued as a `sys_http_delivery` row we +never silently drop it, but we also never guarantee uniqueness — that +requires receiver-side idempotency keys, which we provide but cannot enforce. +(The enqueue step itself is not yet this durable across nodes — see §4.1.) **P2 — Durable before fast.** Every delivery is persisted before it is attempted. A node crash mid-flight loses at most the in-flight HTTP attempt; the next node picks the row up from the queue. -**P3 — Signed by default.** Every outbound request carries a signature -the receiver can verify with a shared secret. Customers can detect -spoofing without writing custom auth. +**P3 — Signed when configured.** Whenever a webhook's `definition_json` +carries a `secret`, every outbound request for it carries a signature the +receiver can verify with that shared secret (see §6). The `secret` field is +optional, not auto-generated — a webhook created without one is delivered +unsigned. Customers can detect spoofing without writing custom auth, once +they set a secret. -**P4 — Safe egress.** The dispatcher refuses to send to localhost, -private IP ranges, or the metadata endpoints of common cloud providers -unless the operator explicitly allowlists them. Webhooks are a textbook -SSRF vector; the runtime defends against it. +**P4 — Safe egress.** Webhooks are a textbook SSRF vector, and the design +goal is to refuse localhost, private IP ranges, and cloud-provider metadata +endpoints by default. **This is not shipped yet** — see §7: the dispatcher +today POSTs to whatever URL is configured, with no blocklisting. Treat +webhook URL configuration as privileged until this lands. **P5 — Observable.** Every delivery attempt is visible to the owner of -the webhook: status, response code, latency, retry schedule, manual -redelivery. The Studio surfaces this without custom UI code by reusing -the standard object/view machinery. +the webhook: status, response code, retry schedule, manual redelivery. +(Per-attempt latency is computed in memory during the HTTP call but is not +persisted to `sys_http_delivery` or surfaced today.) The Studio surfaces this +without custom UI code by reusing the standard object/view machinery. ## 3. Data model @@ -80,7 +86,7 @@ CRUD, permissions, audit, and Studio UI without bespoke code. The subscription record. One row per "I want webhook X to fire for object Y". The full transport configuration (headers, secret, timeout, method) is carried in `definition_json`, a serialised `Webhook` JSON (canonical schema: -`WebhookSchema` in `@objectstack/spec/automation/webhook`). +`WebhookSchema`, exported from `@objectstack/spec/automation`). | Field | Type | Notes | |-------------------|-----------|------------------------------------------------------------------------------------| @@ -93,7 +99,7 @@ in `definition_json`, a serialised `Webhook` JSON (canonical schema: | `method` | text | HTTP method. Default `POST`. | | `description` | textarea | Free-text description. | | `active` | boolean | Inactive webhooks are skipped by the dispatcher. Default `true`. | -| `definition_json` | textarea | Serialised `Webhook` JSON (`@objectstack/spec/automation/webhook`) — carries the full headers / auth / retry / payload config, including the signing `secret`, custom `headers`, and `timeoutMs`. | +| `definition_json` | textarea | Serialised `Webhook` JSON (`WebhookSchema` from `@objectstack/spec/automation`) — carries the full headers / auth / retry / payload config, including the signing `secret`, custom `headers`, and `timeoutMs`. | | `created_at` | datetime | Standard audit columns. | | `updated_at` | datetime | | @@ -142,8 +148,8 @@ writable. | `response_code` | number | Last HTTP status code received. | | `response_body` | textarea | Truncated to the first 16 KB. | | `error` | textarea | Last transport-level error (DNS, connect, timeout). | -| `created_at` | number | Epoch ms. | -| `updated_at` | number | Epoch ms. | +| `created_at` | datetime | Native TIMESTAMP column — written as a `Date`, not an epoch-ms number. | +| `updated_at` | datetime | Native TIMESTAMP column — written as a `Date`, not an epoch-ms number. | > **Why store full payload?** Receivers may be down for hours; we must > retry the *exact* bytes we promised to send. Recomputing payload from @@ -155,9 +161,9 @@ Five stages, each implemented as a thin layer over an existing primitive. ``` ┌─────────────────────────────────────────────────────────────────┐ -│ 1. Event eventBus.emit('data:account:updated', payload, │ -│ { scope: 'cluster', deliverySemantics: 'at-least- │ -│ once', partitionKey: payload.id }) │ +│ 1. Event realtimeService.publish({ type, object, payload, │ +│ timestamp }) — plain in-process pub/sub; no │ +│ cluster / retry options are attached (see §4.1) │ └──────────────────────────┬──────────────────────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────────────┐ @@ -184,21 +190,42 @@ Five stages, each implemented as a thin layer over an existing primitive. ### 4.1 Stage 1 — Event emission -Producers (CRUD handlers, flow nodes, manual API triggers) emit a -**cluster-scoped, at-least-once** event with the record id as -`partitionKey`. These three settings together guarantee: - -- Every node sees the event (subscribers may live anywhere). -- Even on producer crash mid-emit, the event survives. -- Two rapid updates to the same record arrive at the subscriber - in order, so no "stale → fresh → stale" replay reaches the receiver. +Producers (the ObjectQL engine's insert/update/delete handlers) call +`IRealtimeService.publish(event)` after the write commits, where `event` is a +plain `{ type, object, payload, timestamp }` record — e.g. +`{ type: 'data.record.updated', object: 'account', payload: { recordId, +changes, after }, timestamp: }`. + +> **Not yet cluster-aware.** The only shipped `IRealtimeService` implementation +> is `InMemoryRealtimeAdapter`, an in-process, single-node pub/sub with no +> persistence — its own documentation calls this out as the "v1 deployment +> contract: single-instance only" (`packages/services/service-realtime/src/realtime-service-plugin.ts`). +> Concretely, today: +> - There is no `eventBus.emit()` API and no `scope` / `deliverySemantics` / +> `partitionKey` options on the call — `publish()` takes only the event. +> - Events published on one node are **not** delivered to subscribers +> (including the webhook auto-enqueuer) on another node; cross-node fan-out +> is future work. +> - Delivery is synchronous and unpersisted before Stage 3 — if the process +> crashes between the write and `publish()` returning, the event does not +> survive. +> - There is no explicit `partitionKey` ordering primitive; two updates to the +> same record only stay in order because a single node runs JavaScript +> single-threaded. +> +> Everything from Stage 3 onward (the `sys_http_delivery` outbox and +> `HttpDispatcher`) is durable and cluster-aware as described below — the gap +> is upstream, between the CRUD write and the enqueue in Stage 2. ### 4.2 Stage 2 — Matching -The subscriber is a single cluster-wide service -(`clusterScope: 'cluster', leaderStrategy: 'idempotent-broadcast'` — -running on every node is fine because the INSERT in stage 3 is keyed by -`(event_id, webhook_id)` UNIQUE, so duplicates collapse). +The subscriber (`AutoEnqueuer`) is a plain service registered per-node via +`ctx.registerService('webhook.autoEnqueuer', ...)` — it runs independently on +every node with no `clusterScope` / `leaderStrategy` annotation, subscribing to +its own node-local realtime events only (see §4.1). Duplicate enqueues (e.g. +from a periodic cache refresh re-matching the same event) still collapse +safely because the Stage 3 INSERT is keyed by the `(source, dedup_key)` +UNIQUE constraint. For each incoming event the subscriber: @@ -285,7 +312,7 @@ on top: "object": "account", "recordId": "acc_123", "action": "updated", - "timestamp": 1730000000000 + "timestamp": "2024-10-27T00:00:00.000Z" // ...remaining fields from the originating event payload } ``` @@ -295,7 +322,9 @@ Notes: - **`object`** — short object name the event came from. - **`recordId`** — id of the affected record. - **`action`** — `created` / `updated` / `deleted` / `undeleted`. -- **`timestamp`** — event timestamp (epoch ms). +- **`timestamp`** — event timestamp, an ISO 8601 string (passed through + unchanged from the originating realtime event's `timestamp` field — not + an epoch-ms number). - Any additional fields the event carried (e.g. record snapshot data) are spread in after these four. @@ -419,30 +448,29 @@ A precise table of what the runtime promises and what it does not. | Failure | Guarantee | |------------------------------------------|-------------------------------------------------------------| -| Producer node crashes mid-emit | Event durably in transport bus (at-least-once), redelivered on producer restart. | +| Producer node crashes mid-emit | **Not durable today.** The realtime bus (`InMemoryRealtimeAdapter`) is an unpersisted, in-process pub/sub — an event lost before Stage 3's INSERT is gone, not redelivered (see §4.1). | | Subscriber node crashes after persist | Row exists in `sys_http_delivery`, another node picks it up. | | Dispatcher node crashes mid-HTTP | Row stays `in_flight` with `claimed_by`; it reverts to `pending` after the claim TTL and is re-posted. The TTL derives from the dispatcher tick (`intervalMs`, default 500ms): `lockTtlMs = 5 × intervalMs`, `claimTtlMs = 2 × lockTtlMs` (so ~5s at defaults), all configurable via `HttpDispatcherOptions`. | -| Receiver returns 5xx | Retry per backoff schedule until `maxAttempts`. | +| Receiver returns 5xx | Retry per backoff schedule until the fixed 8-attempt budget is exhausted (§4.5). | | Receiver returns 4xx | Treated as terminal — no retry, status `dead` immediately. Exception: 408 / 429 are retried. | | Receiver returns 2xx | `status = success`, no more attempts. | | DNS / connect / TLS error | Treated as 5xx — retry per backoff. | | Timeout (per-attempt) | Treated as 5xx — retry per backoff. | | Network partition between dispatcher and DB | Worker pauses; row stays `pending`. On reconnect, normal claim resumes. | -| Two events for the same record arrive out of order | `partitionKey` ordering at the event bus prevents this. Receiver still sees deliveries in emit order. | +| Two events for the same record arrive out of order | No explicit ordering primitive; holds today only because a single node runs the emit → match → enqueue path synchronously (see §4.1). Not guaranteed across nodes. | | Duplicate delivery (at-least-once) | Receiver gets the same `X-Objectstack-Delivery` id twice. Must dedupe on that header — we provide the key, can't enforce. | ## 13. Plugin location & layering ``` packages/plugins/plugin-webhooks/ -├── src/ -│ ├── webhook-outbox-plugin.ts # plugin: registers sys_webhook + nav, -│ │ # starts the auto-enqueuer, mounts redeliver -│ ├── sys-webhook.object.ts # ObjectSchema.create() — sys_webhook config -│ ├── auto-enqueuer.ts # realtime data.record.* → enqueue onto outbox -│ └── schema.ts # shared types -└── test/ - └── … +└── src/ + ├── webhook-outbox-plugin.ts # plugin: registers sys_webhook + nav, + │ # starts the auto-enqueuer, mounts redeliver + ├── sys-webhook.object.ts # ObjectSchema.create() — sys_webhook config + ├── auto-enqueuer.ts # realtime data.record.* → enqueue onto outbox + ├── auto-enqueuer.test.ts # tests are colocated in src/, not a separate test/ dir + └── schema.ts # shared types ``` The delivery runtime — `sys_http_delivery`, `HttpDispatcher`, the HTTP sender, @@ -475,7 +503,8 @@ Every design choice matches one or more established systems: - **AWS EventBridge / SQS** — at-least-once durability, dead-letter queue model (our `status = dead`). - **Kubernetes admission webhooks** — egress validation and TLS-only - enforcement. + enforcement (design inspiration only — per §7 this is not yet enforced + by our dispatcher). ## 15. Migration plan diff --git a/content/docs/automation/workflows.mdx b/content/docs/automation/workflows.mdx index 11f06aeb7e..c4ea232e64 100644 --- a/content/docs/automation/workflows.mdx +++ b/content/docs/automation/workflows.mdx @@ -109,7 +109,7 @@ Approvals are Flow nodes: type: 'approval', label: 'Manager Approval', config: { - approvers: [{ type: 'user', value: '${record.owner_manager_id}' }], + approvers: [{ type: 'field', value: 'owner_manager_id' }], behavior: 'unanimous', approvalStatusField: 'approval_status', lockRecord: true, diff --git a/content/docs/concepts/architecture.mdx b/content/docs/concepts/architecture.mdx index 074094bc41..7d6127c9c0 100644 --- a/content/docs/concepts/architecture.mdx +++ b/content/docs/concepts/architecture.mdx @@ -195,7 +195,7 @@ export const HighValueCustomerFlow = defineFlow({ }); ``` -See the [Automation Protocol](/docs/protocol/objectos) for the full Flow node and edge reference. +See the [Flow Metadata reference](/docs/automation/flows) for the full Flow node and edge reference. ObjectOS **orchestrates** these rules at runtime, independent of the data structure or UI. @@ -456,9 +456,11 @@ export const OpportunityKanbanView = defineView({ { field: 'customer' }, ], // Kanban-specific config: group columns by the stage field. + // `columns` here lists the fields shown on each card (required). kanban: { groupByField: 'stage', summarizeField: 'amount', + columns: ['title', 'amount', 'customer'], }, }, }); diff --git a/content/docs/concepts/index.mdx b/content/docs/concepts/index.mdx index efd1b465fa..e6ab829667 100644 --- a/content/docs/concepts/index.mdx +++ b/content/docs/concepts/index.mdx @@ -133,11 +133,11 @@ This ensures ObjectStack apps can run on Node.js + PostgreSQL today, Python + SQ | Layer | Responsibility | Example | | :--- | :--- | :--- | | **Protocol** | Defines capabilities | A row-level-security policy slot (`operation` + `using` clause) | -| **App** | Defines business logic | `{ operation: 'select', using: "role = 'admin'" }` | -| **Engine** | Enforces the logic | Compiles the `using` condition into a SQL `WHERE` clause | +| **App** | Defines business logic | `{ operation: 'select', using: "role == 'admin'" }` | +| **Engine** | Enforces the logic | Compiles the `using` condition into a query filter | -CRUD permissions (`allowRead`, `allowEdit`, …) are simple booleans — they grant or deny an operation. Record-level *conditional* access is a separate mechanism: [Row-Level Security](/docs/concepts/architecture) policies, whose `using` clause is a SQL-like (PostgreSQL-compatible) predicate over context variables such as `current_user.roles` and `current_user.id`. +CRUD permissions (`allowRead`, `allowEdit`, …) are simple booleans — they grant or deny an operation. Record-level *conditional* access is a separate mechanism: [Row-Level Security](/docs/concepts/architecture) policies, whose `using` clause is a CEL predicate over context variables such as `current_user.roles` and `current_user.id`. ### Single Source of Truth diff --git a/content/docs/concepts/metadata-driven.mdx b/content/docs/concepts/metadata-driven.mdx index d0eeb92a0e..41b663f080 100644 --- a/content/docs/concepts/metadata-driven.mdx +++ b/content/docs/concepts/metadata-driven.mdx @@ -136,7 +136,7 @@ DELETE /api/v1/data/task/:id Your entire business logic lives in: - Object definitions (`.object.ts`) - View configurations (`.view.ts`) -- Workflow rules (`.workflow.ts`) +- Automation flows (`.flow.ts`) and lifecycle hooks (`.hook.ts`) The Kernel simply **interprets** these definitions. @@ -181,7 +181,7 @@ const FieldSchema = z.object({ type Field = z.infer; // Derived: JSON Schema (for IDE autocomplete) -const jsonSchema = zodToJsonSchema(FieldSchema); +const jsonSchema = z.toJSONSchema(FieldSchema); ``` ### 3. Technology Agnostic diff --git a/content/docs/concepts/metadata-lifecycle.mdx b/content/docs/concepts/metadata-lifecycle.mdx index 26d168225c..651b4785fd 100644 --- a/content/docs/concepts/metadata-lifecycle.mdx +++ b/content/docs/concepts/metadata-lifecycle.mdx @@ -49,10 +49,10 @@ Reads walk top-to-bottom: the first non-null layer wins. Writes always route to | Primitive | Package | Purpose | | :--- | :--- | :--- | -| **Repository** | `@objectstack/metadata-core` | CRUD + watch interface over a single metadata source. `InMemoryRepository` and `LayeredRepository` ship from `@objectstack/metadata-core`; `FileSystemRepository` from `@objectstack/metadata-fs`; `SysMetadataRepository` from `@objectstack/objectql`. | +| **Repository** | `@objectstack/metadata-core` | CRUD + watch interface over a single metadata source. `InMemoryRepository` and `LayeredRepository` ship from `@objectstack/metadata-core`; `FileSystemRepository` from `@objectstack/metadata-fs`; `SysMetadataRepository` from `@objectstack/metadata-protocol` (re-exported by `@objectstack/objectql` for back-compat, per [ADR-0076](https://github.com/objectstack-ai/framework/blob/main/docs/adr/0076-objectql-core-tiering.md)). | | **Change Log** | `@objectstack/metadata-core` | Append-only log of every mutation, tagged with a monotonic `seq`. Watchers can replay from any `since`. | -| **Cache** | `@objectstack/metadata` | In-memory snapshot of the registry, keyed by `MetaRef`. Invalidated by change-log events. | -| **Registry** | `@objectstack/metadata` | Typed registry the Kernel and plugins query. Built from the cache. | +| **Cache** | `@objectstack/metadata-core` | In-memory snapshot of the registry, keyed by `MetaRef`. Invalidated by change-log events. | +| **Registry** | *(per consumer)* | Not a single package — each consumer owns a typed projection over the cache (e.g. `SchemaRegistry` in `@objectstack/objectql`), per [ADR-0008 §2.9](https://github.com/objectstack-ai/framework/blob/main/docs/adr/0008-metadata-repository-and-change-log.md#29-registry-layer). | `MetaRef = (type, name, org)`. As of [ADR-0008 §0 amendment (2026-04-13)](https://github.com/objectstack-ai/framework/blob/main/docs/adr/0008-metadata-repository-and-change-log.md#0-2026-04-13-amendment--drop-project-and-branch-from-metaref), `project` and `branch` are removed from the runtime tuple. Project survives only as an artifact-packaging concept on the `objectstack.json` envelope; branching is left to Git. @@ -62,8 +62,8 @@ Reads walk top-to-bottom: the first non-null layer wins. Writes always route to When a user edits a view in Studio: -1. Studio calls `PUT /api/v1/metadata/views/case_grid` (REST). -2. `protocol.ts:saveMetaItem()` validates against `MetadataTypeRegistry.allowOrgOverride` (see [overlay whitelist](#overlay-whitelist)). +1. Studio calls `PUT /api/v1/meta/view/case_grid` (REST). +2. `protocol.ts:saveMetaItem()` validates against `MetadataTypeRegistryEntry.allowOrgOverride` (see [overlay whitelist](#overlay-whitelist)). 3. If allowed, the call lands on `MetadataRepository.put(ref, body, { parentVersion, actor })`. 4. The repository: - Verifies `parentVersion` matches the current head (`ConflictError` on mismatch). @@ -97,7 +97,8 @@ In shared-database multi-tenancy, **most metadata types must not be per-org cust | Type | `allowOrgOverride` | Rationale | | :--- | :---: | :--- | | `view`, `dashboard`, `report`, `email_template` | ✅ | Pure rendering. Per-org customization is safe. | -| `flow`, `agent` | ✅ | Per-org overlays are allowed for automation and agent definitions. | +| `flow` | ✅ | Per-org overlays are allowed for automation definitions. | +| `agent` | ❌ | Agents are platform-owned and closed to third parties (ADR-0063 §2) — no per-org agent fork. | | `permission`, `role`, `profile` | ✅ | Per-org overlays are allowed; tenant-level controls layer on top. | | `object`, `field` | ❌ | Defines the table schema. Overriding would break existing data. | | `datasource` | ❌ | Connection strings; multi-tenant isolation is enforced at a higher layer. | @@ -138,15 +139,16 @@ The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON s | Component | State | | :--- | :--- | -| `InMemoryRepository`, `FileSystemRepository`, `LayeredRepository` | ✅ Shipped (`@objectstack/metadata-core`) | +| `InMemoryRepository`, `LayeredRepository` | ✅ Shipped (`@objectstack/metadata-core`) | +| `FileSystemRepository` | ✅ Shipped (`@objectstack/metadata-fs`) | | Change log + `seq` (per-org, monotonic) | ✅ Shipped | | SSE bridge (`/api/v1/dev/metadata-events`, `event: metadata-change`) | ✅ Shipped | | Studio `useMetadataHmr` + `HmrStatusBadge` | ✅ Shipped | | Console dev-mode HMR reloader (`MetadataHmrReloader`) | ✅ Shipped | -| `SysMetadataRepository` (overlay over `sys_metadata`) | ✅ Shipped (`@objectstack/objectql`) | +| `SysMetadataRepository` (overlay over `sys_metadata`) | ✅ Shipped (`@objectstack/metadata-protocol`, re-exported by `@objectstack/objectql`) | | `LayeredRepository(SysMeta + artifact)` composition | ✅ Shipped | | `protocol.ts:saveMetaItem` routed through `SysMetadataRepository.put` | ✅ Shipped (PR-10d.6, flag removed) | -| `sys_metadata_history` table (durable, org-keyed change log) | ✅ Shipped (`@objectstack/objectql`) | +| `sys_metadata_history` table (durable, org-keyed change log) | ✅ Shipped (`@objectstack/metadata-protocol`) | | Cache + Registry refactor against `MetadataRepository` | ⏳ Post-M0 | > **Cross-replica overlay sync is out of scope.** Single-instance deployments @@ -165,7 +167,7 @@ The hash is `sha256:` + 64-hex of a canonical (sorted-keys, no-undefined) JSON s |:---|:---|:---| | `defineView(...)` / `defineFlow(...)` / any source file → compiled into `dist/objectstack.json` | ❌ Never. Loaded into the in-memory registry on boot; refreshed via HMR in dev. | ❌ The artifact's own version history *is* Git. The metadata layer does not duplicate it. | | Editing a `.json` under `//.json` (FS overlay, e.g. `/view/case_grid.json`) | ❌ FS layer is independent of DB. | ✅ Appended to the change log at `/.objectstack/.log/main.jsonl` by `FileSystemRepository`. | -| Studio inline edit, or `PUT /api/v1/metadata/...` (REST) on an `allowOrgOverride: true` type | ✅ Written by `SysMetadataRepository.put()` as an **overlay row** scoped to `organization_id`. | ✅ Appended to `sys_metadata_history` (per-org `event_seq`) in the **same transaction** as the `sys_metadata` write (single-instance scope; no cross-replica push). | +| Studio inline edit, or `PUT /api/v1/meta/...` (REST) on an `allowOrgOverride: true` type | ✅ Written by `SysMetadataRepository.put()` as an **overlay row** scoped to `organization_id`. | ✅ Appended to `sys_metadata_history` (per-org `event_seq`) in the **same transaction** as the `sys_metadata` write (single-instance scope; no cross-replica push). | | Deploying a new build (new `dist/objectstack.json`) | ❌ The artifact is loaded into memory, not synced into `sys_metadata`. | ❌ Use Git tags / your deployment platform's release log; that's where artifact "version history" lives. | ### Why artifact never enters the database diff --git a/content/docs/concepts/north-star.mdx b/content/docs/concepts/north-star.mdx index 599ddb18a2..16569a9d0f 100644 --- a/content/docs/concepts/north-star.mdx +++ b/content/docs/concepts/north-star.mdx @@ -50,7 +50,7 @@ This framework repo owns: - kernel/runtime/bootstrap in `packages/core` and `packages/runtime` - REST, ObjectQL, metadata, services, plugins, adapters, CLI, examples, docs - bundled console integration in `packages/console` -- docs site in `apps/docs` and account app in `apps/account` +- docs site in `apps/docs` and account app in `packages/apps/account` Cloud control-plane distribution, public SaaS operations, and ObjectUI source development live outside this repo. diff --git a/content/docs/data-modeling/drivers.mdx b/content/docs/data-modeling/drivers.mdx index 5f9f1476fd..5c5b6dfc53 100644 --- a/content/docs/data-modeling/drivers.mdx +++ b/content/docs/data-modeling/drivers.mdx @@ -223,7 +223,7 @@ export default defineStack({ datasources: { analytics: { driver: 'postgres', - config: { connection: process.env.ANALYTICS_URL }, + config: { url: process.env.ANALYTICS_URL }, }, cache: { driver: 'mongodb', diff --git a/content/docs/data-modeling/field-type-decision-tree.mdx b/content/docs/data-modeling/field-type-decision-tree.mdx index e59215374a..159bc0d314 100644 --- a/content/docs/data-modeling/field-type-decision-tree.mdx +++ b/content/docs/data-modeling/field-type-decision-tree.mdx @@ -253,5 +253,5 @@ Can't find your use case above? Check this table: | "Is it for AI/ML?" | `vector` | -**Still unsure?** Start with `text` for strings or `number` for numerics. You can change the field type later, but ObjectStack does not auto-convert existing data: compatible changes pass cleanly, while narrowing or incompatible changes surface a build-time warning that existing values may not convert cleanly. +**Still unsure?** Start with `text` for strings or `number` for numerics. You can change the field type later, but ObjectStack does not auto-convert existing data: a small set of compatible narrowings (e.g. `text` → `textarea`/`markdown`/`html`/`code`, `date` ↔ `datetime`) pass cleanly, while other type changes are rejected with a `destructive_change` error when you save the updated object — existing values may not convert cleanly. Re-submit the save with `?force=true` to proceed anyway. diff --git a/content/docs/data-modeling/field-types.mdx b/content/docs/data-modeling/field-types.mdx index c956eb5224..8c906eb9d1 100644 --- a/content/docs/data-modeling/field-types.mdx +++ b/content/docs/data-modeling/field-types.mdx @@ -324,7 +324,6 @@ Parent-child relationship (cascading delete by default). |:---|:---|:---|:---| | `reference` | `string` | **required** | Target (master) object name | | `referenceFilters` | `string[]` | — | Filters applied to lookup dialogs (e.g. `"active = true"`) | -| `writeRequiresMasterRead` | `boolean` | — | Require read access on master to write detail | | `deleteBehavior` | `'restrict' \| 'cascade' \| 'set_null'` | `'cascade'` | Behavior when parent is deleted (master-detail cascades unless set to `restrict`) | | `inlineEdit` | `boolean \| 'grid' \| 'form'` | — | Edit child records inline on the parent create/edit form (`true` = auto-pick, `'grid'`, or `'form'`) | | `inlineColumns` | `array` | — | Optional explicit inline grid columns | diff --git a/content/docs/data-modeling/formulas.mdx b/content/docs/data-modeling/formulas.mdx index 9a152b3a51..37e3e9bdc9 100644 --- a/content/docs/data-modeling/formulas.mdx +++ b/content/docs/data-modeling/formulas.mdx @@ -11,8 +11,8 @@ piece of metadata needs to compute a value or evaluate a condition: - **Formula fields** (`type: 'formula'`) - **Predicates** — validation `condition`, sharing `condition`, field conditional rules (`visibleWhen`, `readonlyWhen`, `requiredWhen`), - view section/column visibility (`visibleOn`), action `disabled`, view filter - `criteria`, hook `condition`, flow decisions + view section/column visibility (`visibleOn`), action `disabled`, + hook `condition`, flow decisions - **Dynamic seed values** — fixtures whose value depends on the install-time clock or identity context @@ -24,8 +24,9 @@ plus the ObjectStack standard library. > **Why CEL?** Formal grammar, abundant public training corpus (so AI authors > emit it natively), AST-first persistence, and sandboxed execution with > bounded cost. ObjectStack does **not** ship a Salesforce-flavor DSL — the -> previous custom 22-function engine was deleted in M9. See -> [north-star §8](/docs/concepts/north-star) "No private expression DSL". +> previous custom 22-function engine was deleted in M9. See the +> [north-star](/docs/concepts/north-star) principles on typed metadata over +> custom DSLs. --- @@ -86,8 +87,13 @@ import { ObjectSchema, Field } from '@objectstack/spec/data'; export const Invoice = ObjectSchema.create({ name: 'invoice', - titleFormat: tmpl`Invoice {{record.invoice_no}} – {{record.customer.name}}`, + nameField: 'display_title', // ADR-0079 — the record title is a designated field; composite titles migrate off the deprecated `titleFormat` to a text formula fields: { + // Composite record title as a text formula, surfaced via `nameField` above. + display_title: Field.formula({ + returnType: 'text', + expression: F`"Invoice " + string(record.invoice_no) + " – " + record.customer.name`, + }), total: Field.formula({ returnType: 'number', expression: F`record.subtotal + record.subtotal * record.tax_rate`, @@ -112,7 +118,8 @@ so the field silently never computes. To declare what the formula returns, set **`returnType`** (`'number' | 'text' | 'boolean' | 'date'`); it is inferred and stamped automatically by the AI build path, and consumers (dashboard measures, formatting) read it instead of re-parsing the expression. The CEL source is the -**`expression`** key (the `formula` alias is normalized to it on save). +**`expression`** key — it is the only key the schema and runtime read for a +formula's source; there is no separate `formula` source key. The three CEL tagged-template helpers — `cel`, `F` (formula alias), `P` @@ -493,4 +500,4 @@ validator backs `objectstack build` and metadata registration (and a planned - [Seed Data](/docs/data-modeling/seed-data) — using `cel\`...\`` for dynamic install-time values - [Hook Bodies](/docs/automation/hook-bodies) — when to use a JS hook vs a CEL `condition` - [`packages/formula/`](https://github.com/objectstack-ai/framework/tree/main/packages/formula) — engine + stdlib source -- [north-star §8](/docs/concepts/north-star) — anti-pattern: private DSLs +- [North Star](/docs/concepts/north-star) — architecture principles (typed metadata over custom DSLs) diff --git a/content/docs/data-modeling/queries.mdx b/content/docs/data-modeling/queries.mdx index b010f605df..11aee7471a 100644 --- a/content/docs/data-modeling/queries.mdx +++ b/content/docs/data-modeling/queries.mdx @@ -199,6 +199,14 @@ position from the previous page. There is no `keyset`/`after` query property. } ``` + +This object form of a field node isn't wired up for top-level `fields` projection. When the +object's schema is registered, the engine's unknown-field filter compares each entry against +the schema's field names via `String(f)`, so an object entry never matches and is silently +dropped from the projection — the aliased field is simply missing from results, no error. Use +`expand` to pull in a relationship's fields instead. + + --- ## Expand (Related Records) @@ -243,6 +251,13 @@ applied on this path. | `array_agg` | Collect into array | `{ function: 'array_agg', field: 'tag', alias: 'all_tags' }` | | `string_agg` | Concatenate strings | `{ function: 'string_agg', field: 'name', alias: 'names' }` | + +`count_distinct` / `array_agg` / `string_agg` are only fully supported on the MongoDB driver. +The SQL driver's aggregate function mapper throws `Unsupported aggregate function` for all +three (only `count`/`sum`/`avg`/`min`/`max` are mapped), and the in-memory driver's aggregator +silently returns `null` for them. Avoid these three on SQL- or memory-backed objects. + + ### Aggregation Example ```typescript @@ -258,10 +273,26 @@ applied on this path. } ``` + +`having` is accepted by `QuerySchema` but is not enforced by the query engine today. +`EngineAggregateOptions` (the type `ObjectQL.aggregate()` actually takes) has no `having` +field, and both the REST `findData()` dispatcher and `ObjectQL.aggregate()` build their +driver-facing query from only `where` / `groupBy` / `aggregations` — a `having` clause is +silently dropped before it reaches any driver. No driver (SQL, in-memory, MongoDB) filters +on it either. Post-filter grouped results on the client until this is wired up. + + --- ## Joins + +`joins` is defined in `QuerySchema` (`JoinNodeSchema`) and the SQL driver advertises +`supports.joins: true`, but no driver's `find()` actually executes a join today — the SQL, +in-memory, and MongoDB drivers all ignore the `joins` array (there is no `.join()`/`.leftJoin()` +call in the SQL driver's query builder). Use `expand` for relationship traversal instead. + + ### Join Types | Type | Description | @@ -328,10 +359,25 @@ applied on this path. | `language` | `string` | Language for stemming/stopwords | | `highlight` | `boolean` | Return highlighted matches | + +Only `query` and `fields` are implemented. The engine expands `search` into a driver-agnostic +`$and`-of-`$or`-of-`$contains` filter (ADR-0061) — `fuzzy`, `operator`, `boost`, `minScore`, +`language`, and `highlight` are accepted by `QuerySchema` but read nowhere in +`expandSearchToFilter()` / `normalizeSearch()`, so they have no effect. Multiple search terms +are always AND-ed regardless of `operator`. + + --- ## Window Functions + +`windowFunctions` is only reachable by calling the SQL driver's `findWithWindowFunctions()` +method directly. `ObjectQL.find()` / `.aggregate()` and the `POST /api/v1/data/:object/query` +route never call it, so sending `windowFunctions` through the standard client/REST query path +has no effect. + + | Function | Description | |:---|:---| | `row_number` | Sequential row number within partition | @@ -377,6 +423,13 @@ applied on this path. } ``` + +The top-level `distinct: true` flag is defined in `QuerySchema` but isn't applied by any +driver's `find()` (SQL, in-memory, and MongoDB all ignore it). A separate +`driver.distinct(object, field)` method exists on the SQL and in-memory drivers, but it isn't +called by `ObjectQL.find()`/`.aggregate()`, so it isn't reachable through a normal query. + + ### Group By with Having ```typescript @@ -391,6 +444,11 @@ applied on this path. } ``` + +As noted under [Aggregations](#aggregations) above, `having` is not currently enforced — +it's dropped before reaching the aggregation engine or any driver. + + --- ## Common Query Patterns diff --git a/content/docs/data-modeling/relationships.mdx b/content/docs/data-modeling/relationships.mdx index 632d61237c..0f5f2c1589 100644 --- a/content/docs/data-modeling/relationships.mdx +++ b/content/docs/data-modeling/relationships.mdx @@ -22,15 +22,25 @@ fields: { ### Filtered Lookups -Reference records that meet criteria: +Reference records that meet criteria, using `lookupFilters` for static conditions and +`dependsOn` to scope candidates by another field on the same record: ```typescript contact: Field.lookup('contact', { label: 'Contact', - referenceFilters: ['account = {account}', 'is_active = true'], + dependsOn: ['account'], + lookupFilters: [ + { field: 'is_active', operator: 'eq', value: true }, + ], }) ``` + + The legacy `referenceFilters: string[]` property (e.g. `['is_active = true']`) is accepted + by the schema but is **not** read by the record-picker UI — it filters nothing. Use the + structured `lookupFilters` (`{ field, operator, value }`) and `dependsOn` shown above instead. + + ### Self-Referencing Lookups Create hierarchies: diff --git a/content/docs/data-modeling/schema-design.mdx b/content/docs/data-modeling/schema-design.mdx index 16c7f9bcc6..ddbd41c89a 100644 --- a/content/docs/data-modeling/schema-design.mdx +++ b/content/docs/data-modeling/schema-design.mdx @@ -27,7 +27,7 @@ export const MyObject = ObjectSchema.create({ description: 'Description...', // Help text // Display configuration - titleFormat: '{{record.field1}} - {{record.field2}}', + nameField: 'field1', // Canonical title field (ADR-0079) highlightFields: ['field1', 'field2', 'field3'], // Fields definition @@ -55,7 +55,8 @@ export const MyObject = ObjectSchema.create({ | `pluralLabel` | string | Plural display name | `'Accounts'` | | `icon` | string | Icon identifier | `'building'` | | `description` | string | Help text | `'Companies...'` | -| `titleFormat` | string | Record title template (`{{record.field}}` interpolation) | `'{{record.name}} - {{record.id}}'` | +| `nameField` | string | Canonical primary title field — the stored field used as the record display name (ADR-0079); the deprecated alias `displayNameField` is still accepted | `'name'` | +| `titleFormat` | string | Deprecated (ADR-0079 → `nameField`). Render-only title template (`{{record.field}}` interpolation); an explicit `nameField` now takes precedence | `'{{record.name}} - {{record.id}}'` | | `highlightFields` | string[] | Most-important fields, in priority order (default columns, cards, previews, detail highlight strip; ADR-0085 — formerly `compactLayout`; the old spelling was retired and is now rejected) | `['name', 'status']` | ### Enable Features @@ -284,7 +285,7 @@ here (ADR-0085). ❌ **DON'T:** - Create too many indexes (slows writes) - Put indexes on low-cardinality fields -- Use SOQL in loops +- Run ObjectQL queries inside loops (N+1 queries) ### 5. Validation @@ -348,6 +349,11 @@ export const Account = ObjectSchema.create({ label: 'Account Owner', required: true, }), + + is_active: Field.boolean({ + label: 'Active', + defaultValue: true, + }), }, indexes: [ diff --git a/content/docs/data-modeling/seed-data.mdx b/content/docs/data-modeling/seed-data.mdx index f0e9ae1203..7787a09400 100644 --- a/content/docs/data-modeling/seed-data.mdx +++ b/content/docs/data-modeling/seed-data.mdx @@ -68,7 +68,7 @@ record (matched by `externalId`). | `insert` | Create only, throw on duplicate | Append-only tables, audit logs | | `update` | Update only, skip if not found | Migration patches on existing rows | | `ignore` | Create if new, silently skip duplicates | Bootstrap data that must not overwrite user edits | -| `replace` | Delete ALL records then insert | Cache / lookup tables rebuilt on each run | +| `replace` | Insert without checking for an existing match — the seed loader does **not** delete anything itself; the target table must already be empty (or cleared by the caller) | Cache / lookup tables rebuilt from an already-truncated table | ### `upsert` — Recommended Default @@ -97,10 +97,12 @@ defineSeed(SystemRole, { }); ``` -### `replace` — Full Table Rebuild +### `replace` — Insert Into an Already-Cleared Table ```typescript -// ⚠️ Deletes ALL records in the object before inserting. +// ⚠️ The seed loader does NOT delete existing records for you — `replace` +// always inserts. Truncate/clear the table yourself before this seed runs, +// or every load will attempt to insert duplicate rows. // Only use for cache or lookup tables with no user-generated data. defineSeed(ExchangeRateCache, { externalId: 'key', @@ -238,54 +240,69 @@ Available in the seed CEL context: `isBlank(v)`, `coalesce(v, fallback)` - **Scope:** `os.user`, `os.org`, `os.env` -### Binding records to a user (`os.user`) +### Owner-style fields (`owner_id`, `created_by`, `assigned_to`) -Many objects have a **required** owner lookup — `owner_id`, `created_by`, -`assigned_to`. To seed such a record, bind it to a user with `cel\`os.user.id\``. -This is the single canonical convention; there is no `currentUser()`, `@admin`, -or similar special syntax. +Many objects have an owner lookup — `owner_id`, `created_by`, `assigned_to`. +On a fresh boot there are no human users yet — seed data loads *before* the +first sign-up — so the platform does **not** mint a placeholder system user to +own these fields. The recommended pattern is to **leave owner-style fields +unset** in the record: ```typescript defineSeed(Project, { externalId: 'code', records: [{ - code: 'bootstrap', - name: 'Bootstrap Project', - owner_id: cel`os.user.id`, // ← bound to the seed identity + code: 'bootstrap', + name: 'Bootstrap Project', + // owner_id intentionally omitted — filled in by the first-admin handoff below }], }); ``` -**Where does `os.user` come from?** On a fresh boot there are no human users yet -— seeding runs *before* the first sign-up. So the runtime provisions a -deterministic, non-loginable **system user** (`usr_system`, role `system`) -*before* any seed runs and binds it to `os.user`. It owns seeded data the way -Salesforce's "Automated Process" user does — it has no credential and **cannot -sign in**. - -- The **human login admin** is created separately (CLI sign-up / first-signup - promotion) through better-auth and need **not** be the seed owner. +**What actually happens to `os.user`?** The seed loader binds `os.user` to +whatever identity the load run carries (`config.identity`). During the normal +boot sequence no identity is supplied, so `os.user` resolves to a null +identity and `cel\`os.user.id\`` evaluates to `null` — the record still seeds +successfully, with the field left `null` rather than the load failing. Once +the first human user is promoted to platform admin, a one-time ownership +handoff re-owns every orphaned row (`owner_id` `null`, or the legacy +`usr_system` value from older databases) to that admin. + +- Because `cel\`os.user.id\`` resolves to `null` before an admin exists, a + **required** (non-nullable) owner-style field must not depend on it — + either leave the field optional/unset (recommended) or supply a literal + value. +- `cel\`os.user.id\`` still resolves to a real user id when the loader is + invoked with an explicit identity (e.g. a re-seed run through tooling that + passes `config.identity`). - `os.org.id` resolves to the current organization; during a per-tenant replay it is that tenant's id, falling back to the load's `organizationId`. -This ordering guarantee means `cel\`os.user.id\`` / `cel\`os.org.id\`` always -resolve at boot — you never have to sequence seeds around user creation. - ### Failure is loud, not silent -If a record uses a CEL value that cannot be resolved — e.g. `cel\`os.user.id\`` -when the system identity could not be provisioned — the record is **not silently -dropped**. The loader counts it as an error, marks the load unsuccessful, and -logs an actionable message: +If a record's CEL expression cannot be evaluated at all (a malformed +expression, or a reference the CEL engine cannot compile), the record is +**not silently dropped**. The loader counts it as an error, marks the load +unsuccessful, and logs an actionable message: + +``` +[SeedLoader] Cannot resolve dynamic seed values for project record #0: . + `os.user.id` resolves to null at seed time (the owning admin does not exist yet) and + owner-style fields are assigned by the first-admin handoff — so a required, non-owner + field must not depend on it. Provide a literal value or make the field optional. +``` + +Separately, if the write itself fails after resolution (e.g. a `NOT NULL` +column rejects a `null` value produced by an unresolved owner field), that +surfaces as its own error: ``` -[SeedLoader] Cannot resolve dynamic seed values for project record #0: - ... Records using cel`os.user.id` / cel`os.org.id` require a seed identity — - ensure a system/admin user exists before seeding. +[SeedLoader] Failed to write project record #0 (code=bootstrap): ``` -Write failures (e.g. a required field still missing after resolution) are -surfaced the same way. Tooling should check `result.success` / `result.errors`. +Both cases increment `result.summary.totalErrored`, add an entry to +`result.errors`, and flip `result.success` to `false`. Tooling should check +`result.success` / `result.errors` rather than assuming a clean run. --- diff --git a/content/docs/data-modeling/validation-rules.mdx b/content/docs/data-modeling/validation-rules.mdx index 02d221e72f..a6d011e76c 100644 --- a/content/docs/data-modeling/validation-rules.mdx +++ b/content/docs/data-modeling/validation-rules.mdx @@ -64,9 +64,9 @@ These properties apply to **all** field types and are validated by the base `Fie | Property | Type | Default | Validation Behavior | |:---|:---|:---|:---| -| `format` | `string` | `email` | Validates RFC 5322 email format | +| `format` | `string` | `email` | Validates a basic `local@domain` shape | -**Default constraints:** Must conform to valid email format. +**Default constraints:** Must contain an `@` and a domain with a dot — a lightweight pattern check, not full RFC 5322 validation. ### `url` @@ -80,9 +80,9 @@ These properties apply to **all** field types and are validated by the base `Fie | Property | Type | Default | Validation Behavior | |:---|:---|:---|:---| -| `format` | `string` | `phone` | Validates phone number format | +| `format` | `string` | `phone` | Validates a permissive phone-number character set | -**Default constraints:** Accepts E.164 and common national formats. +**Default constraints:** Accepts digits, `+ ( ) - .` and spaces (minimum 5 characters) — a lenient character-set check, not strict E.164 structural validation. ### `password` @@ -91,7 +91,7 @@ These properties apply to **all** field types and are validated by the base `Fie | `maxLength` | `number` | — | Maximum password length | | `minLength` | `number` | — | Minimum password length | -**Default constraints:** Value is never returned in read operations. Stored as a one-way hash owned by the auth subsystem — for reversible encrypted-at-rest secrets, use the `secret` type instead. +**Default constraints:** Validated the same as `text` (`maxLength`/`minLength` only) — the engine does not automatically hash or mask a `password`-typed field's value on read; even the platform's own credential column (`sys_account.password`) is declared as `Field.text()`, with hashing and verification owned entirely by the auth subsystem (better-auth), not driven by this field type. For a reversible encrypted-at-rest value on your own objects, use the `secret` type instead. --- @@ -209,7 +209,7 @@ These properties apply to **all** field types and are validated by the base `Fie | `options` | `SelectOption[]` | — | **Required.** Static option list | | `defaultValue` | `string` | — | Must match an option `value` | -**Option validation:** Each option `value` must be lowercase (`^[a-z_][a-z0-9_]*$`). +**Option validation:** Each option `value` must be a lowercase system identifier — starts with a letter, then letters/digits/underscores/dots (`^[a-z][a-z0-9_.]*$`), minimum 2 characters. ### `multiselect` @@ -256,7 +256,6 @@ These properties apply to **all** field types and are validated by the base `Fie |:---|:---|:---|:---| | `reference` | `string` | — | **Required.** Parent object name | | `deleteBehavior` | `enum` | `cascade` | Master-detail cascades at runtime unless set to `restrict` | -| `writeRequiresMasterRead` | `boolean` | — | Require read access to master record | **Default constraints:** Enforces parent-child ownership. Child records cascade-delete with the parent by default. @@ -266,7 +265,7 @@ These properties apply to **all** field types and are validated by the base `Fie |:---|:---|:---|:---| | `reference` | `string` | — | **Required.** Self-referencing object name | -**Default constraints:** Self-referencing lookup for hierarchical structures. Prevents circular references. +**Default constraints:** Self-referencing lookup for hierarchical structures. Stored and expanded like a `lookup`; the engine does not run a cycle check on write, so a self-reference chain that loops back on itself is not automatically rejected. --- @@ -432,7 +431,7 @@ These properties apply to **all** field types and are validated by the base `Fie ### `tags` -**Default constraints:** Stored as array of strings. Each tag is trimmed and deduplicated. +**Default constraints:** Stored as array of strings. A lone scalar value is coerced into a single-element array; the engine does not trim or deduplicate entries. --- @@ -467,7 +466,9 @@ For sensitive data, use the properties and types the platform actually enforces: | `trackHistory` | `boolean` | — | Render the field's value changes as entries on the record activity timeline | For reversible encrypted-at-rest values (API keys, tokens, DB passwords), use the -`secret` field type; for credentials, use `password` (one-way hash). See the +`secret` field type — it is the type with an enforced masking/encryption code path. +`password` is validated like plain text and has no built-in hashing or read-masking +(see the `password` section above). See the [Field Type Gallery](/docs/data-modeling/field-types). --- @@ -478,10 +479,10 @@ For reversible encrypted-at-rest values (API keys, tokens, DB passwords), use th |:---|:---|:---| | `text` | — | `maxLength`, `minLength`, `format` | | `textarea` | — | `maxLength`, `minLength` | -| `email` | — | RFC 5322 email format | +| `email` | — | Basic `local@domain` shape (not full RFC 5322) | | `url` | — | Valid URL with protocol | -| `phone` | — | E.164 / national format | -| `password` | — | One-way hash, never returned | +| `phone` | — | Permissive character set, not strict E.164 | +| `password` | — | Validated like `text`; no built-in hashing/masking | | `markdown` | — | `maxLength` | | `html` | — | Sanitized, `maxLength` | | `richtext` | — | Sanitized, `maxLength` | @@ -493,13 +494,13 @@ For reversible encrypted-at-rest values (API keys, tokens, DB passwords), use th | `time` | — | ISO 8601 time, 24h | | `boolean` | — | `true` / `false` only | | `toggle` | — | `true` / `false` only | -| `select` | `options` | Option values must be lowercase snake_case | +| `select` | `options` | Option values must be lowercase system identifiers | | `multiselect` | `options` | Array of valid option values | | `radio` | `options` | Single value from options | | `checkboxes` | `options` | Array of valid option values | | `lookup` | `reference` | Foreign key integrity | | `master_detail` | `reference` | Cascade delete, ownership | -| `tree` | `reference` | Self-referencing, no circular refs | +| `tree` | `reference` | Self-referencing; no automatic cycle check | | `image` | — | `fileAttachmentConfig` for dimensions | | `file` | — | `fileAttachmentConfig` for restrictions | | `avatar` | — | Single image, typically square | @@ -518,5 +519,5 @@ For reversible encrypted-at-rest values (API keys, tokens, DB passwords), use th | `signature` | — | Base64 image, typically immutable | | `qrcode` | — | Format-specific validation | | `progress` | — | Numeric, typically 0–100 | -| `tags` | — | String array, trimmed, deduplicated | +| `tags` | — | String array; no automatic trim/dedup | | `vector` | `dimensions` | Numeric array of exact `dimensions` length | diff --git a/content/docs/deployment/cloud-artifact-api.mdx b/content/docs/deployment/cloud-artifact-api.mdx index 4f5bcf3edc..c45d099ff5 100644 --- a/content/docs/deployment/cloud-artifact-api.mdx +++ b/content/docs/deployment/cloud-artifact-api.mdx @@ -112,7 +112,7 @@ calls, not data-plane calls. |:---|:---| | `packages/cli/src/commands/package/publish.ts` | CLI `os package publish` command and endpoint construction (the legacy `publish.ts` / `rollback.ts` commands were removed — #2237). | | `packages/runtime/src/http-dispatcher.ts` | Kernel-resolution seam for per-request environment resolution. The concrete id/hostname registry ships in the host distribution `@objectstack/objectos-runtime` (not part of this open-source repo). | -| `packages/cloud-connection/src/runtime-config-plugin.ts` | Console runtime-config for active/default environment state. | +| `packages/cloud-connection/src/runtime-config-plugin.ts` | Console runtime-config for default (hostname-resolved) environment state. | | `packages/spec/src/system/environment-artifact.zod.ts` | Normative artifact envelope schema. | --- diff --git a/content/docs/deployment/environment-variables.mdx b/content/docs/deployment/environment-variables.mdx index 3945318e18..03034915ac 100644 --- a/content/docs/deployment/environment-variables.mdx +++ b/content/docs/deployment/environment-variables.mdx @@ -47,9 +47,9 @@ read at startup unless noted otherwise. Boolean variables accept `true` / `false | Variable | Type | Default | Description | |:---|:---|:---|:---| -| `OS_DATABASE_URL` | url | — | Database connection string (e.g. `file:./data.sqlite`, `postgres://…`, `libsql://…`). | -| `OS_DATABASE_DRIVER` | enum | inferred | Force a specific driver when the URL is ambiguous. `sqlite` \| `postgres` \| `libsql`. | -| `OS_STORAGE_ROOT` | path | `/storage` | Root directory for the local file storage adapter. | +| `OS_DATABASE_URL` | url | — | Database connection string (e.g. `file:./data.sqlite`, `postgres://…`, `mongodb://…`, `memory://`). `libsql://` (Turso) is not dispatched by the open-core runtime — that driver ships separately in the ObjectStack Cloud distribution. | +| `OS_DATABASE_DRIVER` | enum | inferred | Force a specific driver when the URL is ambiguous. `memory` \| `sqlite` \| `sqlite-wasm` \| `postgres` \| `mongodb`. | +| `OS_STORAGE_ROOT` | path | `./.objectstack/data/uploads` | Root directory for the local file storage adapter, relative to the process cwd (used by `os serve`'s default `storage` capability wiring). | | `OS_ARTIFACT_PATH` | path | — | Path or `http(s)://` URL to a compiled `objectstack.json` artifact to boot the kernel from. | --- @@ -229,7 +229,7 @@ Provider credentials keep their upstream names. ObjectStack does not rename |:---|:---|:---|:---| | `OS_MARKETPLACE_CACHE` | enum | `on` | `off` disables the in-memory marketplace listing cache. | | `OS_MARKETPLACE_PUBLIC_BASE_URL` | url | — | Public base URL of the marketplace registry (proxied from this runtime when set). | -| `OS_METADATA_WRITABLE` | boolean | `true` (dev) / `false` (prod) | Allow PATCH / PUT on `/api/v1/metadata/*` at runtime. | +| `OS_METADATA_WRITABLE` | csv | — (none) | Comma-separated metadata type names (e.g. `hook,validation`) granted a runtime escape hatch that treats them as `allowOrgOverride: true`, letting artifact-backed items of those protected types be overridden per-org outside their static registry declaration. See [ADR-0005](https://github.com/objectstack-ai/framework/blob/main/docs/adr/0005-metadata-customization-overlay.md). | --- @@ -277,10 +277,9 @@ the hosted ObjectStack Cloud control plane. | `OS_KERNEL_TTL_MS` | number | `900000` | Idle TTL for cached kernels in ms. | | `OS_ENV_CACHE_TTL_MS` | number | `300000` | Env-record cache TTL in ms. | | `OS_ARTIFACT_CACHE_TTL_MS` | number | `300000` | Artifact-record cache TTL in ms. | -| `OS_ARTIFACT_FETCH_TIMEOUT_MS` | number | `30000` | Timeout for remote artifact fetches. `0` disables the timeout. | +| `OS_ARTIFACT_FETCH_TIMEOUT_MS` | number | `60000` | Timeout for remote artifact fetches. Only a positive value is honored — an unset, non-numeric, or `0` value falls back to the 60s default (pass `fetchTimeoutMs: 0` in `artifactSource` config to actually disable the timeout). | | `OS_INLINE_SEED_BUDGET_MS` | number | `8000` | Time budget for synchronous seed execution at boot before deferring to a worker. | | `OS_TENANT_AUDIT` | flag | `1` | Set to `0` to silence the tenant-isolation audit warnings emitted by the SQL driver. | -| `OS_NODE_ID` | string | hostname | Stable node id used by the webhook outbox for distributed locking. | --- @@ -288,7 +287,7 @@ the hosted ObjectStack Cloud control plane. Some env vars accept a legacy alias for compatibility. **Ecosystem-standard names** (e.g. `DATABASE_URL`, `AUTH_SECRET`, `BETTER_AUTH_*`, `PORT`, `CORS_*`, `MCP_SERVER_*`) are permanently accepted and no longer warn. ObjectStack's own former names are deprecated — prefer the canonical `OS_*`. -> **Removed in 11** (rename required): `OS_MULTI_TENANT` → `OS_MULTI_ORG_ENABLED`, `OBJECTSTACK_METADATA_WRITABLE` → `OS_METADATA_WRITABLE`, `AUTH_BASE_URL`/`OS_AUTH_BASE_URL` → `OS_AUTH_URL`. +> **Removed in 11** (rename required): `OS_MULTI_TENANT` → `OS_MULTI_ORG_ENABLED`, `AUTH_BASE_URL`/`OS_AUTH_BASE_URL` → `OS_AUTH_URL`. | Canonical | Legacy | |:---|:---| @@ -305,6 +304,6 @@ Some env vars accept a legacy alias for compatibility. **Ecosystem-standard name | `OS_MCP_SERVER_ENABLED` | `MCP_SERVER_ENABLED` | | `OS_MCP_SERVER_NAME` | `MCP_SERVER_NAME` | | `OS_MCP_SERVER_TRANSPORT` | `MCP_SERVER_TRANSPORT` | -| `OS_NODE_ID` | `OBJECTSTACK_NODE_ID` | +| `OS_METADATA_WRITABLE` | `OBJECTSTACK_METADATA_WRITABLE` | | `OS_HOME` | `OBJECTSTACK_HOME` | | `OS_DEV_CRYPTO_KEY` | `OBJECTSTACK_DEV_CRYPTO_KEY` | diff --git a/content/docs/deployment/index.mdx b/content/docs/deployment/index.mdx index e68ace4afb..9c62b799fb 100644 --- a/content/docs/deployment/index.mdx +++ b/content/docs/deployment/index.mdx @@ -128,7 +128,10 @@ requests and do not run through a target environment kernel. | `OS_PORT` | HTTP listen port. | Third-party provider variables such as `OPENAI_API_KEY`, `TURSO_*`, -`SMTP_*`, `RESEND_API_KEY`, and OAuth client secrets keep their provider names. +`RESEND_API_KEY`, and OAuth client secrets keep their provider names. Mail +settings are the exception — they route through the settings env-override +convention as `OS_MAIL_` (e.g. `OS_MAIL_SMTP_HOST`, `OS_MAIL_SMTP_PORT`), +not a raw `SMTP_*` name. **Set `OS_SECRET_KEY` for any containerized or multi-node deployment.** The diff --git a/content/docs/deployment/single-project-mode.mdx b/content/docs/deployment/single-project-mode.mdx index a36f4560d6..a9479c17e8 100644 --- a/content/docs/deployment/single-project-mode.mdx +++ b/content/docs/deployment/single-project-mode.mdx @@ -88,12 +88,12 @@ needed, the `X-Environment-Id` request header. ```typescript import { createDefaultHostConfig, createStandaloneStack } from '@objectstack/runtime'; -const stack = createStandaloneStack({ +const stack = await createStandaloneStack({ environmentId: 'env_local', artifactPath: './dist/objectstack.json', }); -const host = createDefaultHostConfig({ +const host = await createDefaultHostConfig({ environmentId: 'env_local', artifactPath: './dist/objectstack.json', }); diff --git a/content/docs/deployment/troubleshooting.mdx b/content/docs/deployment/troubleshooting.mdx index dca477db9e..51b0438879 100644 --- a/content/docs/deployment/troubleshooting.mdx +++ b/content/docs/deployment/troubleshooting.mdx @@ -13,7 +13,7 @@ Solutions to the most common issues encountered when working with ObjectStack. ### "Invalid enum value" for field type -**Symptom:** Zod validation fails with `Invalid enum value. Expected 'text' | 'number' | ...` +**Symptom:** Zod validation fails with `Invalid option: expected one of "text"|"number"|...` **Cause:** The `type` value has a typo or uses an unsupported type name. @@ -296,10 +296,12 @@ console.log(field.maxLength?.toString() ?? 'no limit'); ### "Bundle size is too large" +**Cause:** `@objectstack/spec` does not re-export domain schemas (like `FieldSchema` or `QuerySchema`) from its root — only `define*` factory functions and a handful of top-level schemas are available there. Each domain (~400 Zod schema closures total) is only reachable through its subpath, so an import that assumes it's on the root will not resolve. + **Fix:** Import from subpaths instead of the root: ```typescript -// ❌ Imports everything +// ❌ Not exported from the root at all import { FieldSchema, QuerySchema } from '@objectstack/spec'; // ✅ Tree-shakeable subpath imports @@ -307,7 +309,7 @@ import { FieldSchema } from '@objectstack/spec/data'; import { ErrorResponseSchema } from '@objectstack/spec/api'; ``` -Available subpaths: `data`, `api`, `ui`, `system`, `kernel`, `ai`, `automation`, `contracts`, `integration`, `security`, `studio`. +Available subpaths: `data`, `api`, `ui`, `system`, `kernel`, `ai`, `automation`, `contracts`, `integration`, `security`, `studio`, `cloud`, `qa`, `identity`, `shared`. --- diff --git a/content/docs/deployment/vercel.mdx b/content/docs/deployment/vercel.mdx index 7145b59613..0fa55e5e0b 100644 --- a/content/docs/deployment/vercel.mdx +++ b/content/docs/deployment/vercel.mdx @@ -155,9 +155,10 @@ storage for artifacts. | `OS_S3_SECRET_ACCESS_KEY` | optional | Falls back to AWS SDK credential chain | | `OS_S3_FORCE_PATH_STYLE` | optional | `1` for MinIO / self-hosted | -> See [Publish, Versioning & Preview](/docs/deployment/publish-and-preview) for the full -> storage matrix (native S3, Cloudflare R2, MinIO, etc.) and what the cloud -> control plane stores in each bucket. +> The Cloud control plane's storage backend (S3, Cloudflare R2, MinIO, etc.) is +> configured and documented in that separate distribution, not in this framework +> repo. See [Publish, Versioning & Preview](/docs/deployment/publish-and-preview) +> for the package publish/install workflow the control plane exposes to the CLI. **Never commit secrets.** Use Vercel's environment variable UI or the Vercel CLI (`vercel env add`) to configure tokens and S3 credentials.