feat: Add content engine Stage 3 — Search - #735
Open
aXenDeveloper wants to merge 18 commits into
Open
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
`SearchModel` derived a document's owner from `c.get("plugin")`, and a rebuild
runs inside the core cron request - so the same record was stored as
`@vitnode/example` when a mutation route indexed it and `core` when a rebuild
did. The Elasticsearch adapter made it worse by hardcoding `pluginId: "core"`
in `toSource`, so the mirrored document could disagree with the canonical row.
`SearchDocument` gains an optional `pluginId`, and ownership is resolved in one
place - `SearchModel.resolveOwner` - as document, then request, then `"core"`.
An explicit owner now wins over the request, `pluginId` joins the conflict
`set` so a rebuild can repair a row written before its indexer declared one,
and the adapter serializes what it was given.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`SearchIndexer.load` returned documents only, and the rebuild treated an empty
array as end-of-source. A page can read rows and project none of them - every
row on it published with an unusable title, say - so the rebuild stopped there
and never reached the valid records behind it.
`load` now returns `{ documents, itemsRead }`. The rebuild advances its cursor
by `itemsRead` and terminates only on `itemsRead === 0`, which also keeps
multi-document-per-item indexers correct: a document count was never a source
count. Every implementation in the repository is updated, so there is no
transitional union to remove later.
The rebuild also stamps the registering plugin on any document that names no
owner, and generated indexers carry their plugin id into the mapper - the queue
drains inside the core cron request, where the request's plugin is not the
owner.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The status route raised a collection's source total to `Math.max(total, indexed)` and the UI called anything with `indexed >= total` healthy. So 10 documents for 9 published records was rewritten to 10/10, 100%, "Indexed" - the one state that most needs attention was the one guaranteed to be hidden. Both counts are now reported as measured, and a collection is "Indexed" only when they match exactly; any mismatch in either direction is stale. Coverage can read past 100% because that is the signal, while the progress bar is clamped so it cannot draw past its track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`ResolvedContentSearchConfig.enabled` was `boolean`, so a searchable definition only satisfied `SearchableContentTypeDefinition` after an `as` - and the type test asserted the cast rather than the behaviour. `defineContentType` now infers the whole `search` argument as one type parameter and reads the literal back off it, so `enabled` resolves to `true` or `false` and the public type needs no assertion. Inferring the object rather than its parts is what makes this work: an intersection member is not an inference site, and the field rules stay in the parameter's *constraint*, which is checked once `TPublicField` is resolved. `titleField` also has to be non-nullable now, at compile time and at runtime: a `null` heading is not a search result, and a record without one is skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`syncContentSearch` caught the search engine's error and then awaited
`c.get("log").error(...)`, which writes to the database - so a logger that was
down for the same reason the engine was turned a committed content write into an
HTTP 500. That is exactly the guarantee the feature documents it keeps.
Logging is now wrapped the way `LocalEventsAdapter` wraps its own, falling back
to the console, and the returned outcome still carries the original search error
rather than the logger's.
The Postgres suite also gains a case for a published row the mapper cannot
project: page one reads it and yields nothing, and `itemsRead` is what carries
the rebuild through to the valid rows behind it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Documents plugin ownership across live sync and both rebuild shapes, the difference between source rows and produced documents, why an empty document page does not end a rebuild, the real meaning of the AdminCP counts (including over-indexed as stale, and malformed data as under-indexed), and that a logger failure is as harmless to the mutation as a search engine failure. States plainly that there is no durable retry mechanism. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Stage 3 changed `load` to return `SearchIndexerPage`, which is the right contract but broke every external plugin using the documented public import - a bare `SearchDocument[]` no longer compiled. `load` now returns `SearchIndexerLoadResult`, and one helper decides what a result means. A page passes through; a non-empty array reports the requested limit, because that is what the old rebuild advanced by and an array carries no source count - `documents.length` would skip rows for any indexer that emits several documents per item. An empty array is the only end signal it has, which is exactly why the array form cannot express "rows read, none projected" and is deprecated rather than merely older. `ContentSearchIndexer` pins the generated adapter to the page result, so the widened contract does not weaken what the engine itself guarantees. A blank `pluginId` is now treated as absent everywhere ownership is resolved, through a shared `searchDocumentOwner`, since it became public input. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A collection with documents but no registered indexer was reported as owned by core, because `indexer?.pluginId ?? "core"` had no other source to consult. The rows themselves know better: they carry the plugin that wrote them. The coverage query now selects `pluginId` alongside the counts, and ownership resolves as registered indexer, then stored owner, then `"unknown"`. The registered indexer stays canonical - it is what the next rebuild will stamp on the rows - so a disagreement resolves in its favour rather than reporting a mismatch nobody can act on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Covers the preferred page result and what `itemsRead` counts, the deprecated array result and the one guarantee it cannot make, and how the AdminCP names the owner of a collection whose indexer is gone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`total` fell back to the indexed count when no indexer was registered, so a collection nothing can rebuild reported 11/11, 100%, "Indexed" - the fallback matching itself read as full coverage. The status response now carries `hasIndexer`, taken from whether an indexer was found and from nothing else: rows knowing which plugin wrote them says nothing about whether anything can write them again. `total` is `null` when there is no indexer to ask, so coverage is absent rather than invented, and the UI gains an `orphaned` status that is decided before the counts are compared. An indexer without the optional `count` still falls back to the indexed count - that is the documented meaning of leaving `count` out, and it is a registered collection either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`Reindex` on an orphaned collection deleted it: the task cleared the item type and then rebuilt nothing, because the filtered indexer list was empty. The rebuild route now answers 404 for an item type no indexer claims, and the task refuses it before `search.clear` runs. Both checks are needed - the route gives the AdminCP an immediate, explainable failure, while the task also covers direct queue dispatches and a job queued while the indexer was still registered. A full rebuild is untouched: it clears the whole index and refills every registered indexer, so orphaned documents are still removed by it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With a scoped rebuild now refused for a collection that has no indexer, the only way left to remove its stale documents was a full rebuild of everything. That is too blunt, so orphaned rows get their own action. `POST /search/clear` deletes one collection's documents and requires a non-empty `itemType`, so no payload clears the index by omission. It refuses a collection that still has an indexer with a 409: that one has a rebuild, which reaches the same freshness without deleting anything. In the AdminCP an orphaned row shows `Remove documents` behind a confirmation that says the documents cannot be rebuilt, instead of `Reindex`; its coverage cell explains that no indexer is registered rather than drawing a bar, and the status reads destructive rather than muted. Registered rows keep `Reindex` exactly as before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Defines what makes a collection orphaned, what the AdminCP shows for one, and the distinction that matters: a scoped rebuild requires a registered indexer and never clears what it cannot rebuild, while a full rebuild clears everything and so does remove orphaned documents. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Registering a `SearchIndexer` is optional - `searchIndexers?` on the plugin API, and `search.index()` needs nothing else. A plugin that indexes live and registers no indexer is valid and may be keeping its collection perfectly current, but the AdminCP labelled it "Orphaned" and told the administrator its plugin had been uninstalled and that nothing could rebuild its documents. The first claim was unfounded; the second was misleading, because the next live write recreates them. The status is now `unmanaged`, and every label, comment and description says only what is known: no rebuild indexer is registered, so a rebuild cannot reproduce the collection. It no longer speculates about the plugin. The model is unchanged - `hasIndexer` from indexer lookup alone, `total: null` with no indexer, `11 / —` and no percentage, and the status decided before the counts are compared. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`POST /search/clear` awaited `c.get("log").warn` after the documents were already
gone. The logger writes to the database, so a logger outage turned a cleanup that
had happened into a failed request - sending an administrator to look for
documents that were no longer there.
Logging is now wrapped with a console fallback, the same shape the events adapter
and the content search sync use, and the response stays `{ cleared: true }`. A
failed `clear` still propagates, and it writes no success audit.
The route's copy also stops calling the collection orphaned: it removes the
current indexed state of a collection with no rebuild indexer, and a live-writing
plugin may recreate those documents.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`search.index()` and a registered `SearchIndexer` are two independent capabilities: the first makes a collection searchable, the second makes it rebuildable. Documenting only the pair left live-only plugins looking broken. Defines "unmanaged by the rebuild system" as what it is - documents with no registered indexer - states plainly that this does not prove the plugin is gone, and spells out the operational consequence: a full rebuild recreates only collections that have an indexer, so live-only ones are removed by it and return only when their plugin writes again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
aXenDeveloper
changed the title
github-actions
Bot
added
💡 Feature
aXenDeveloper
marked this pull request as ready for review
August 4, 2026 21:22
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1a23614cc4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ); | ||
| } | ||
|
|
||
| if (rest.includes("//") || template.includes("..") || /\s/.test(template)) { |
There was a problem hiding this comment.
When a valid template puts {slug} between path segments, e.g. /docs/{slug}/comments, replacing the placeholder with an empty string makes rest contain //, so this check throws even though contentSearchUrl would produce a valid URL such as /docs/my-post/comments. This prevents searchable content types from using nested detail routes; check for empty segments on the actual template parts rather than on the string after deleting the placeholder.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Improving Documentation
pnpm lint:fixto fix formatting issues before opening the PR.Description
What?
Why?