feat: Universal Content Engine mvp 6 - #741
Conversation
Stage 6 starts at the definition layer: three new field shapes, one canonical
field-path vocabulary, and the definition-time validation that makes every
generated table name and every flattened column known before a request runs.
- `field.relation({ multiple, ordered })` for to-many and ordered relations.
`target` was already a thunk, so a self-relation needs nothing special.
- `field.group({ fields })` for reusable structured groups. Localization is a
property of the group, never of a leaf: half a logical value on each table
would mean two revision histories and two permissions for one editor box.
- `field.repeatable({ fields })` for ordered child rows with stable identity.
- `content/paths.ts` is the single leaf-path <-> column mapping every
subsystem reads, so `seo.title` and `seo_title` cannot drift.
- `content/advanced.ts` resolves the generated junction and child table names
and refuses every advanced-field mistake at import time.
Nested create/update schemas, leaf-path public allowlisting and the
membership filter land with them. Stage 1-5 behaviour is unchanged: a content
type that declares no advanced field resolves to empty arrays everywhere.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The runtime half of Stage 6's storage model. Nothing here is JSONB, an EAV table or a comma-separated identifier list: a to-many relation is a junction table with two real foreign keys, a repeatable is a child table with a serial primary key of its own, and a group is flattened into ordinary columns. - `advanced-tables.ts` generates both shapes, with PRIMARY KEY (itemId, relatedItemId), UNIQUE (itemId, position) and a reverse index on the junction; UNIQUE (itemId, position) and ON DELETE CASCADE on the child. - `advanced-store.ts` is the one read/write layer both kinds share. Positions are settled in two passes through a negative parking space, so a reorder can never violate the unique index even for an instant - and child identity survives it, which is what makes "edit entry 11" and a revision restore mean anything. - Relation and repeatable mutations are wrappers over `update`, so each one inherits the no-op rule, the lock, `updatedAt`, and - on an editorial content type - the version guard, the revision and the event. The guard runs before a single junction row is touched, so a losing writer leaves nothing behind. - Revisions record collections as identity and groups as nested values, never flattened column names and never the expanded related records. - Public reads batch-load collections one query per field per page, so a list never issues one per row. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Search, cache and events all read the same vocabulary now - canonical paths - so none of them needed a Stage 6 code path of its own. - `search.contentFields` accepts group leaves and repeatable leaves. A repeatable leaf resolves to every child's value joined in **position** order, because position is what the page renders and an index that disagreed with the page would highlight the wrong entry. - The synchronizer matches `changedFields` against indexed paths *and* their containers, so a rewritten FAQ answer (reported as `faq`) still moves the document, while a relation change moves nothing unless the relation actually contributes to a configured projection. - Collections are read back only when the configuration names one, so every Stage 1-5 content type pays a boolean check and no query. - Cache and events needed no change at all: `changedFields` already carries `seo.description` and `categories`, and the existing locale fan-out already treats a shared field as reaching every locale - which a collection is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…uite `example.advanced-article` exercises every Stage 6 shape at once: an unordered to-many relation, an ordered **self**-relation, a localized group, a shared group and a repeatable - on a content type that is also localized, published, editorial, public and searchable. Migration 0031 is what the definition compiles to, generated rather than written: two junction tables, one child table, four flattened group columns across the base and translation tables, and the constraints that make ordering and integrity facts about Postgres rather than about service code. The new suite is the half a mock cannot show. A duplicate junction row is a 23505, a category still in use is a 23503, a cascade really does take the child rows, and five concurrency tests prove exactly one writer wins a race - relation vs relation, relation vs scalar, repeatable vs repeatable, reorder vs reorder - while two different records stay independent. Three real bugs fell out of running it: to-many relations were being asked for a base-table foreign key they do not have, the translation model was selecting a group rather than its leaf columns, and the localized public join was looking a canonical path up on an aliased table that only carries column keys. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`target: () => thisContentType` makes a definition's own inferred type circular, and TypeScript resolves that by widening the whole definition to `any` - silently. Every nested value type, every leaf-path allowlist check and every compile-time guarantee Stage 6 adds would disappear, with no error to say so. `self: true` carries no reference at all, and `defineContentType` rebinds the thunk on a *copy* of the field map once the definition exists - so a descriptor const shared by two content types cannot end up pointing both relations at whichever was declared last. Also adds the definition-time and type-level suites: 31 runtime cases covering every advanced-field rule, and 28 type assertions whose interesting half is negative - a group must not leak its flattened column names, a collection must not appear on a list row, a private leaf must not appear in a public response, an index must not accept a repeatable path. One of them pins the variance of `ContentModel<T>`, which a `UnionToIntersection` in the table types had already broken once. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three editors, all rendered through the AutoForm seam `ContentField` already uses - so a leaf inside a group is the same input a top-level field of that kind renders, and Stage 6 grows no second form system. - **Group** renders as a `fieldset`/`legend`, so a screen reader announces "SEO" with every leaf inside it - which is what tells `SEO / Title` from `Article / Title`. A nullable group gets a switch, because `seo: null` is a different stored state from every leaf happening to be empty. - **Repeatable** gets Add / Edit / Remove / Move up / Move down as labelled buttons. Drag-and-drop may be layered on later but can never be the only way to reorder. React keys come from a client-side key that is stable for a row's life, never from `id` - an unsaved row has no `id`, and keying off it would make React reuse one row's DOM state for another's value. - **To-many relation** wraps the existing async combobox as an "add one" control. Reorder buttons appear only for an `ordered: true` relation, since the engine sorts an unordered set by target id whatever the editor does. Each editor controls the nested value the API takes, so nothing is flattened on submit and nothing re-nested on load. The revision diff learned to render the three shapes too, rather than `[object Object]`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Seven pages under `dev/content-engine/`, registered in `meta.json` after the localization set: - **advanced-modeling** - the three shapes, the canonical-path vocabulary, and why a content type that declares none is unchanged. - **relations** - to-one, to-many, ordered, self, the junction table's constraints, `onDelete` per direction, and the membership filter. - **structured-fields** - leaves, canonical paths, partial updates, the four nullability states and the two rules that keep them distinguishable, and why localization belongs to the group. - **repeatable-fields** - the child table, identity vs position, the two-pass reorder, the atomic form save, and the one lock. - **advanced-modeling-public-api** - leaf-level allowlisting, relations as identifiers, and why there is no expansion to configure. - **advanced-modeling-migrations** - what is generated, and two existing-data migrations written so the destructive statement is in a *different file* from the copy, with a verification step between. - **advanced-modeling-limitations** - every refusal, with the reason and the thing to do instead. `fields.mdx` gains the three new rows and points at them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two pure-function suites, and both found a real bug. `paths.test.ts` pins the one leaf-path <-> column mapping every subsystem reads: what is and is not a path, the camelCase compilation, the flatten, the fold back, and the two directions of a nullable group (`seo: null` writes NULL to every leaf; every leaf NULL reads back as `seo: null`). `advanced-projection.test.ts` covers the diff, the column patch, the snapshot, the search document and the public projector. It caught: - `diffChangedPaths` reporting a collection as a changed *column* when handed a field map that still contained one. Harmless at the call sites, which pass the shared half - but a function that is only correct because of who calls it is a function waiting to be called by somebody else. - `contentSearchDocument` normalizing a repeatable's joined text a second time, collapsing the newlines *between* children into single spaces. The values were all there; the boundaries were not, so two adjacent answers ran together into a phrase neither of them contains. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…self
`references: { related: () => thisContent.table.id }` names the model inside
its own initializer, and TypeScript resolves that by widening the model to
`any` - silently taking every typed service, schema and column map with it.
The definition-level version of this trap was fixed by `self: true`; this is
the same trap one layer down.
`ContentReferences` now subtracts self-relations from the map it demands, and
the table generators resolve them from the table being built - the only place
that reference exists anyway. Which meant `self` had to become a literal type
parameter, which meant the "exactly one of `self` and `target`" rule had to
move out of the signature (a union there defeats literal inference) and into
`defineContentType`, before the rebind - the only moment a supplied target and
the placeholder are still distinguishable.
Also adds the route-level suite the example plugin was missing: mounting the
generated admin and public routes converts every request and response schema to
JSON Schema, so a nested group or repeatable array Zod could not describe fails
at `pnpm test` rather than at the first request.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
vitnode/packages/vitnode/src/content/server/search-sync.ts
Lines 372 to 375 in d3ca3c6
Merge advanced values into localized search documents
When a localized searchable type indexes a repeatable leaf such as the example's faq.answer, this call supplies only the base row and translation row. Unlike the non-localized path, syncContentLocalizedSearch never loads or merges model.service(c).advanced(itemId), so the repeatable container is always absent and its configured text is omitted from every localized search document—even after an FAQ change triggers an upsert.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const toRows = (value: unknown): EditorRow[] => | ||
| Array.isArray(value) |
There was a problem hiding this comment.
When editing an existing record, ContentTableView passes its findMany row directly to the form, while the detail route still uses findById; neither read includes collection tables. Consequently field.value is undefined here, so existing repeatables render as empty (and relation sets have the same problem); interacting with such a field then submits a replacement derived from an empty list and can delete previously stored entries. Load the newly added findDetail/advanced data before opening the editor.
Useful? React with 👍 / 👎.
| service.update( | ||
| itemId, | ||
| { [field]: [...next] } as ContentUpdateInput<TDefinition>, | ||
| options, | ||
| ); |
There was a problem hiding this comment.
For editorial content types, the documented service.relations.* shortcuts accept expectedVersion, but this dispatches to the plain createContentService.update implementation, which ignores that option and does not create a revision or emit editorial effects. A stale shortcut write therefore succeeds without advancing the editorial version/history or notifying listeners; bind these shortcuts to the editorial mutation path or avoid exposing them for editorial definitions.
AGENTS.md reference: AGENTS.md:L34-L36
Useful? React with 👍 / 👎.
| add: async (itemId, relatedItemId, options) => { | ||
| const current = await read(itemId, options); | ||
|
|
||
| return await write( |
There was a problem hiding this comment.
For non-editorial records, concurrent granular mutations can lose data because add reads the collection before service.update starts its transaction and acquires the row lock. If two calls add different targets from the same initial state, both compute stale replacement arrays; the writes serialize, but the second replacement removes the first addition. Perform the read-modify-write inside the locked transaction (the repeatable create, delete, and update helpers have the same race).
Useful? React with 👍 / 👎.
| const repeatableSchema = (fieldValue: ContentFieldDescriptor): z.ZodType => | ||
| z | ||
| .array(repeatableRowSchema(fieldValue)) | ||
| .min(contentRepeatableMin(fieldValue)) | ||
| .max(contentRepeatableMax(fieldValue)); |
There was a problem hiding this comment.
An update payload may contain the same existing child id more than once because this array schema enforces only length bounds. The writer then updates that single database row repeatedly and settles it into only the last requested position, returning success with fewer children than the accepted payload described. Add a refinement requiring all present child IDs to be distinct, analogous to relationSetSchema.
Useful? React with 👍 / 👎.
**Translation revisions lost every localized group.** Stage 6 taught the base snapshot about groups and left the translation one running each localized field through the scalar coercion - which returns `null` for an object. So a translation revision recorded `seo: null` for every record that had SEO, and restoring one blanked it. Both halves now share `toFieldSnapshot` and one `projectSnapshotFields`, and the leaf read that made that possible - `readContentLeaf` - is also what lets the diff work against a logical row, so the restore no longer reports an unchanged group as changed. **Localized search documents were missing their collections.** Three paths build one - live sync, translation effects, base editorial effects - and none supplied the shared collections, so a document made of `faq.answer` contained none of it, and editing one localized leaf silently rewrote the document without it. The rebuild was worse: it classified indexed names through the top-level field maps, where neither `seo.description` nor `faq.question` is a key, and dropped both. `contentSearchAdvancedValues` is now the single place that decides whether a document depends on collections, and `resolveSearchSources` resolves each path to the column or child table it actually lives in. Collections are batched per page and deduplicated across locales. **Collection helpers read before taking the lock.** `add` read the current list, then called `update`, which opened the transaction - so two concurrent additions each computed from the same empty list and one disappeared. The read now happens after `SELECT ... FOR UPDATE`, inside the transaction that writes. The plain service merges; the editorial service - which now has its own `relations` and `repeatable` - arbitrates on `expectedVersion`. `ContentWriteOptions` no longer offers a version the plain service would have ignored. **The typed collection API was `Record<string, …>`.** `service.relations.typo` compiled. Both maps are now keyed by the content type's actual field names, and a repeatable's create/update/list values are inferred from its own leaves. Also: the localized preview read had the same top-level classification bug as the rebuild, and the public projection now loads only the collections it exposes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
vitnode/packages/vitnode/src/content/schemas.ts
Lines 834 to 835 in 02c677a
Include group leaves in public filter schemas
resolvePublicApi accepts filterable group paths such as syndication.priority, but filterShape(fields) only emits top-level field names and therefore has no entry for that path. The generated HTTP schema silently strips the requested filter, returning an unfiltered list, while direct service calls later reject it as unknown. Resolve canonical paths to their leaf descriptors when constructing this schema.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // A to-many relation has no foreign key *here*: its two are on the | ||
| // generated junction table, and its picker resolves through | ||
| // `model.advancedTables` rather than through a column on this row. | ||
| if (isContentRelationCollection(fieldValue)) continue; |
There was a problem hiding this comment.
Skipping collection relations removes them from the references map consumed by service.options, while the new relation-set editor calls that method through /options/{field}. Consequently, opening the picker for any multiple: true relation reaches the !target branch and returns an error, so editors cannot add targets. Resolve these entries from the junction table as the comment describes, or provide a separate collection-options path.
Useful? React with 👍 / 👎.
| const relatedReference: ColumnReferenceThunk | undefined = fieldValue.self | ||
| ? itemReference | ||
| : referenceThunks[entry.field]; |
There was a problem hiding this comment.
For a non-self collection relation, this accepts the supplied reference thunk without checking that its table matches fieldValue.target().tableName. A mistaken references.categories thunk therefore creates a valid foreign key to the wrong content table; target validation then follows that actual foreign key, so writes succeed with IDs from the wrong model despite the definition and generated types claiming otherwise. Apply the same runtime target-table check used for to-one relations before building the junction.
Useful? React with 👍 / 👎.
| typeof values.id === "number" | ||
| ? `saved-${values.id}` | ||
| : `draft-${index}`, |
There was a problem hiding this comment.
When two newly added rows are reordered, their keys are regenerated from their new array positions, so React keeps component instances attached to positions rather than to the logical rows. Stateful leaf controls such as AutoFormNullableNumber retain their old local input text while the form values move, displaying or subsequently writing values for the wrong child. Assign each draft row an identity that persists across renders and reorders instead of deriving it from index.
Useful? React with 👍 / 👎.
1 participant
Improving Documentation
pnpm lint:fixto fix formatting issues before opening the PR.Description
What?
Why?