improvement(search): consolidate knowledge search around live Sim Search and document-decided workspace retrieval - #8314
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
6 tasks done
Contributor
 |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 212 files
Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
Tip: instead of fixing issues one by one fix them all with cubic
Re-trigger cubic
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
waleedlatif1
force-pushed
the
chore/consolidate-knowledge-search
branch
from
September 26, 2026 01:59
e3d9a30 to
a50312d
Compare
Collaborator
Author
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@waleedlatif1 I have started the AI code review. It will take a few minutes to complete. |
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 193 files
Tip: instead of fixing issues one by one fix them all with cubic
Re-trigger cubic
…rch and document-decided workspace retrieval - Move indexed organization search under lib/sim-search/indexed, dormant behind the single isIndexedOrgSearchEnabled() gate, with a check:indexed-org-search-boundary audit keeping callers on its public entry - Decide workspace knowledge base search on the document for every principal, so workspace retrieval never waits on the projector - Remove the knowledge-async-projection, knowledge-projection-fill and knowledge-tin-keyword flags and their code paths - Trim the projector to a single periodic sweep that releases workspace marks before deciding whether a pass is owed - Remove per-source vector index builds - Scope member sync and processing recovery to the indexed search path - Remove dead code left behind by the consolidation - Add an ops runbook and scripts for recovering and maintaining a dormant search index
waleedlatif1
force-pushed
the
chore/consolidate-knowledge-search
branch
from
September 26, 2026 02:33
a50312d to
884cdd9
Compare
Collaborator
Author
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@waleedlatif1 I have started the AI code review. It will take a few minutes to complete. |
6 tasks done
waleedlatif1
added a commit
that referenced
this pull request
Sep 26, 2026
…indexes (#8333) - Since #8314, workspace knowledge search ranks on `embedding`/`document`. The only live reader of `embedding_keyword_search` and `embedding_keyword_tin` is dormant indexed search, and only for bases that are search indexes. Yet every chunk write still upserted a keyword row, ran a Tin `DELETE` on insert, and every document insert fanned an ACL/connector sync out to the projections - Script migration `0025_scope_keyword_projections`, all `CREATE OR REPLACE` in one transaction with the 0024 lock-timeout retry: - `sync_embedding_keyword_search` takes the shared membership lock and writes keyword rows only for `is_search_index` bases. On an update that moves a chunk out of a search index, it deletes the chunk's row - A new flip trigger backfills a base's keyword rows when it becomes a search index and deletes them when it stops being one, under the exclusive membership lock. It is separate from the Tin flip trigger, which only exists where Tin is installed. Its upsert skips unchanged rows - The Tin chunk trigger skips its `DELETE` on insert, since a new chunk can't have a Tin row - `projection_source_acl_sync` becomes `AFTER UPDATE OF connector_id, acl ... WHEN` the value actually changed. A new document has no chunks yet (FK), so the insert arm never matched anything - The projector's keyword page writes only for search-index bases, keyed on the base flag - Fresh installs, `db:push`, and late Tin adoption end with the same trigger definitions: 0019 installs the final guarded, key-share-locked Tin triggers, and 0025 re-runs after 0016. A `db:push` re-run of the legacy 0016 backfill may refill keyword rows for non-search bases; they're unread and left in place like the existing ones - Both adoption backfills read their chunks `FOR KEY SHARE`, so a chunk delete racing an adoption is waited out instead of failing it - Rollback floor: v0.9.1 and earlier ranked workspace keyword search through `embedding_keyword_search`. Once 0025 has run, don't roll back below v0.9.2, and self-hosters should upgrade through v0.9.2 or later - Existing keyword rows for non-search bases are left in place (unread) and removed separately
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
lib/sim-search/indexed/(andapp/**/indexed/for its UI) behind one gate,isIndexedOrgSearchEnabled(), and every indexed use case refuses on its own while dormantembedding_keyword_search, per-source walks) now only runs for search-index bases while indexed search is on. Signed-in workspace searches run fewer statements; live source proof stays lazy; actorless legs keep their prior (unbudgeted) behaviorcandidates.ts,tag-filters.ts, andvector-leg.tsknowledge-async-projection,knowledge-projection-fill, andknowledge-tin-keywordflags and their code pathsType of Change
Testing
Type-check, lint,
check:audits, unit suites, and the knowledge/sim-search integration suites against real Postgres. New integration coverage: workspace document-decided access (incl. a gated GitHub-installation source and a positive principal), member-sync reclaim scoping, processing recovery while dormant, projection work detection and mark release in both gate states.Checklist