diff --git a/apps/docs/content/docs/search/lucid.mdx b/apps/docs/content/docs/search/lucid.mdx index 4012bff1d9b..71efecc0972 100644 --- a/apps/docs/content/docs/search/lucid.mdx +++ b/apps/docs/content/docs/search/lucid.mdx @@ -11,12 +11,18 @@ Sim uses [Lucid’s official read-only MCP server](https://help.lucid.co/hc/en-u ## Search -Search requires terms, even when filtering by date. Results are ranked for relevance and may not match the title literally. Start with a short document-title query, such as `deployment architecture`. Select `lucidchart` or `lucidspark` to narrow the product, or search both. Read a result to inspect its diagram or board structure. +Title search requires terms, even when filtering by date. Results are ranked for relevance and may not match the title literally. Start with a short document-title query, such as `deployment architecture`. Select `lucidchart` or `lucidspark` to narrow the product, or search both. Read a result to inspect its diagram or board structure. To find text inside a known document, use its UUID or Lucid URL as the native query’s `project` and enter a literal phrase such as `API Gateway`. This matches shape labels and sticky-note text, case-insensitively. It does not search notes, tags, links or comments. Title queries allow up to 400 characters; document-scoped text queries allow 200. Boolean and field operators are unsupported. Document search has no continuation and verifies at most 10 candidates. Dates use modification time; filtering and sorting the returned candidates cannot establish the newest or oldest document across the entire account. +## Browse documents + +Ask to browse your Lucid folders when you do not have a title or topic. The assistant can list the root folder or a selected child folder without guessing search terms. Each request returns one page of direct children; child folders are navigation references, not document content. Follow the continuation for more items in the same folder, or select a child folder to explore it. + +Browsing uses the same personal readonly connection. It does not recursively scan the account, follow shortcuts, or guarantee the newest or oldest document across folders. Date filters and sorting apply to the current page’s document metadata. + ## Diagram content Reads preserve Lucid’s structured pages, nodes, connections and properties, with links back to the source. Large responses can be read in successive windows of the same document version. Sim rejects incomplete or changed documents rather than treating a partial graph as complete. diff --git a/apps/docs/content/docs/search/notion.mdx b/apps/docs/content/docs/search/notion.mdx index ca185ef1a3e..cd326cf5c60 100644 --- a/apps/docs/content/docs/search/notion.mdx +++ b/apps/docs/content/docs/search/notion.mdx @@ -13,7 +13,13 @@ Use short, distinctive keywords such as `launch rollback` or a concise question Sim checks the connection's current tool access before searching. When available, it uses Notion AI search; otherwise it uses the advertised keyword-search route. Workspace plans and enabled MCP tools determine availability. Notion can return results from connected applications, but Sim's Notion provider returns only Notion pages and databases. Search Slack, Gmail, or Drive through their own providers for those sources. -The assistant can scope a query to a Notion page URL or ID when the server advertises page-scoped search. A server without that option reports the limitation. For date filters or sorting, Sim fetches up to 10 candidate pages and uses their explicit modification timestamps. They are not exhaustive date-range searches; date-only requests require search terms. Sim never substitutes the search time for a missing page date. +The assistant can scope a query to a Notion page URL or ID when the server advertises page-scoped search. A server without that option reports the limitation. When the connection’s plan and tool schema allow it, Sim sends modification-date filters and newest sorting to Notion. It then checks exact timestamps from up to 10 current page reads. Other connections use local date filtering for keyword searches; date-only searches report the missing plan capability. These bounded results do not establish exhaustive date coverage or the oldest page in the workspace. Sim never substitutes the search time for a missing page date. + +## Browse pages + +Ask for your private, shared, favorite, or recently viewed pages to browse without inventing keywords. Private and shared lists reflect the corresponding sidebar sections; they do not enumerate the whole workspace. Recently viewed pages are ranked by visits and frequency, not modification time. Follow the returned continuation to see more entries in the same list, and read a page for its contents. + +List availability is checked against the connected account’s current tools and access. Browsing keeps the same personal OAuth connection and requires no additional credentials. ## Coverage and reads @@ -21,6 +27,6 @@ Search returns a bounded, ranked selection. Pagination is used only when both th Reads fetch the exact page or database through the same member connection. Large pages can omit subtrees; Sim preserves that limitation in the returned content. Open the original page when the result says its content is incomplete. -Search is limited to a fixed read-only allowlist: tool-access inspection, search, AI search, and fetch. It cannot create or edit Notion pages. The server URL is fixed to Notion's official endpoint, and every call is validated against the current advertised tool schema. +Search is limited to a fixed read-only allowlist: tool-access inspection, search, AI search, sidebar lists, and fetch. It cannot create or edit Notion pages. The server URL is fixed to Notion's official endpoint, and every call is validated against the current advertised tool schema. See [Notion MCP setup](https://developers.notion.com/guides/mcp/get-started-with-mcp) and the [supported tools and plan behavior](https://developers.notion.com/guides/mcp/mcp-supported-tools). diff --git a/apps/sim/lib/api/contracts/mothership-assistant-tools.ts b/apps/sim/lib/api/contracts/mothership-assistant-tools.ts index 1447831aa29..60a9db7177d 100644 --- a/apps/sim/lib/api/contracts/mothership-assistant-tools.ts +++ b/apps/sim/lib/api/contracts/mothership-assistant-tools.ts @@ -4,11 +4,6 @@ import { LIVE_SEARCH_PROVIDER_IDS } from '@/lib/sim-search/live/provider-catalog export const liveSearchProviderSchema = z.enum(LIVE_SEARCH_PROVIDER_IDS) export type LiveSearchProvider = z.output -export const SEARCH_TERMS_REQUIRED = { - notion: 'Notion requires search terms. Add keywords or a concise question.', - lucid: 'Lucid requires search terms. Add document-title keywords or a literal shape-text query.', -} as const - /** * Native queries one call may send to the same provider account. Alternatives run as separate * provider searches and fuse into one ranking, so the bound keeps a call within the provider's @@ -54,6 +49,12 @@ export const nativeSearchQuerySchema = z accountId: z.string().min(1).max(200).optional(), kind: nativeSearchKindSchema.optional(), project: z.string().min(1).max(300).optional(), + browse: z + .enum(['folder', 'private', 'shared', 'favorites', 'recent']) + .optional() + .describe( + 'Queryless discovery: Lucid folder lists one folder page (omit project for root; otherwise use a returned numeric folder ID). Notion private/shared list sidebar pages, favorites lists pinned pages, recent lists recently viewed pages, not recently modified pages. These lists are not an exhaustive workspace inventory. Follow the returned cursor with the same account, browse mode, project, filters and topK.' + ), cursor: z.string().max(4000).optional(), termClauses: z.array(z.string().max(500)).max(10).optional(), modifiers: z.string().max(1000).optional(), @@ -72,11 +73,32 @@ export const nativeSearchQuerySchema = z : `${input.provider} does not support kind selection.`, }) } - if ((input.provider === 'notion' || input.provider === 'lucid') && !input.query) + if ( + input.browse && + (input.query || + input.modifiers || + input.keywordOnly || + input.termClauses?.length || + (input.browse === 'folder' ? input.provider !== 'lucid' : input.provider !== 'notion') || + (input.provider === 'notion' && input.project)) + ) + context.addIssue({ + code: 'custom', + path: ['browse'], + message: + 'Browse requires an empty query and a supported provider mode; Notion lists cannot be scoped to a page.', + }) + if (input.browse === 'folder' && input.project && !/^[1-9]\d{0,14}$/.test(input.project)) + context.addIssue({ + code: 'custom', + path: ['project'], + message: 'Lucid folder browsing requires a numeric folder ID returned by the provider.', + }) + if (input.provider === 'lucid' && !input.query && !input.browse) context.addIssue({ code: 'custom', path: ['query'], - message: SEARCH_TERMS_REQUIRED[input.provider], + message: 'Lucid requires title keywords or explicit browse: folder.', }) }) export type NativeSearchQuery = z.output @@ -113,7 +135,9 @@ export const nativeSearchQueriesSchema = z addIssue('Duplicate native query.') else if ( hasSearchKinds(query.provider) && - earlier.some((previous) => !previous.kind || !query.kind) + earlier.some( + (previous) => !previous.browse && !query.browse && (!previous.kind || !query.kind) + ) ) addIssue( 'A GitHub, GitLab, HubSpot, Lucid, Google Meet, or Zoom query without a kind already searches its default kinds; give each query on this account a kind.' @@ -134,6 +158,10 @@ export const liveSearchAccountStatusSchema = z.object({ status: z.enum(['ok', 'partial', 'reconnect', 'rate_limited', 'unavailable', 'timeout']), message: z.string().optional(), nextCursor: z.string().optional(), + folders: z + .array(z.object({ id: z.string(), name: z.string() })) + .max(10) + .optional(), retryAfterSeconds: z.number().optional(), }) export type LiveSearchAccountStatus = z.output @@ -198,7 +226,7 @@ export const searchWorkspaceInputSchema = workspaceSearchFiltersSchema nativeQueries: nativeSearchQueriesSchema .optional() .describe( - `Live search only: queries in a provider's own language (Drive q, Gmail operators, JQL, CQL, GitHub qualifiers, Slack RTS, plain Linear/Fireflies/HubSpot/Lucid/Zoom terms, bounded local Google Meet text matching, Granola natural-language questions, Notion keywords or AI questions when available). Blank queries require a date bound or sortBy newest/oldest; Notion and Lucid always require search terms. Up to ${MAX_NATIVE_QUERIES_PER_ACCOUNT} per account run separately and merge; one GitHub, GitLab, or HubSpot query without a kind searches GitHub issues (plus code when the query has no date bound or boolean operators, as its status message says), GitLab issues, merge requests, and code, or every HubSpot CRM kind; other collections, and multiple queries on one account, each need a kind, which may repeat. HubSpot kinds are contacts, companies, deals, and tickets; Lucid kinds are lucidchart and lucidspark. Google Meet kinds are transcript and smart_notes (note metadata and Docs link only); it searches bounded recent conference artifacts with 30-day retention. Zoom kind is meeting and searches past occurrences; read for transcripts and separately labeled summaries. Use Drive for saved Meet note bodies and older transcripts; Drive dates mean file modification time. HubSpot, Lucid, Zoom and Meet reject ownership filters. Lucid searches titles with no search continuation; project can scope a literal shape-text query to one known document UUID or Lucid URL. Read for structured diagram evidence. Dates and sorting cover only retrieved candidates, not globally newest/oldest matches. Write queries from the returned live guidance and account IDs; each account status names the queryIndex its cursor belongs to. Omit for simple cross-provider terms.` + `Live search only: queries in a provider's own language (Drive q, Gmail operators, JQL, CQL, GitHub qualifiers, Slack RTS, plain Linear/Fireflies/HubSpot/Lucid/Zoom terms, bounded local Google Meet text matching, Granola natural-language questions, Notion keywords or AI questions when available). Blank queries require a date bound, sortBy newest/oldest, or explicit browse mode. Lucid browse folder lists root or a numeric folder project; Notion browse private/shared/favorites/recent lists sidebar pages. Recent means viewed, not modified. Up to ${MAX_NATIVE_QUERIES_PER_ACCOUNT} per account run separately and merge; one GitHub, GitLab, or HubSpot query without a kind searches GitHub issues (plus code when the query has no date bound or boolean operators, as its status message says), GitLab issues, merge requests, and code, or every HubSpot CRM kind; other collections, and multiple content queries on one account, each need a kind, which may repeat; explicit browse queries may select distinct folders or sidebar sections without a kind. HubSpot kinds are contacts, companies, deals, and tickets; Lucid kinds are lucidchart and lucidspark. Google Meet kinds are transcript and smart_notes (note metadata and Docs link only); it searches bounded recent conference artifacts with 30-day retention. Zoom kind is meeting and searches past occurrences; read for transcripts and separately labeled summaries. Use Drive for saved Meet note bodies and older transcripts; Drive dates mean file modification time. HubSpot, Lucid, Zoom and Meet reject ownership filters. Lucid title search has no continuation; folder browsing is paginated and returns child folders in account coverage; project can scope a literal shape-text query to one known document UUID or Lucid URL. Read for structured diagram evidence. Dates and sorting cover only retrieved candidates, not globally newest/oldest matches. Write queries from the returned live guidance and account IDs; each account status names the queryIndex its cursor belongs to. Omit for simple cross-provider terms.` ), query: z .string() @@ -206,7 +234,7 @@ export const searchWorkspaceInputSchema = workspaceSearchFiltersSchema .max(2000) .default('') .describe( - 'Search terms, without dates already supplied as filters. May be empty for a live listing with a date bound or sortBy newest or oldest where supported; Notion requires search terms.' + 'Search terms, without dates already supplied as filters. May be empty for a live listing with a date bound or sortBy newest or oldest where supported, or use an explicit native browse mode. Notion date-only search depends on plan capabilities.' ), topK: z .number() @@ -227,7 +255,11 @@ export const searchWorkspaceInputSchema = workspaceSearchFiltersSchema input.sortBy === 'newest' || input.sortBy === 'oldest' ) - if (!input.query && !input.nativeQueries?.some((query) => query.query) && !bounded) + if ( + !input.query && + !input.nativeQueries?.some((query) => query.query || query.browse) && + !bounded + ) context.addIssue({ code: 'custom', path: ['query'], @@ -243,7 +275,7 @@ export const searchWorkspaceInputSchema = workspaceSearchFiltersSchema path: ['endDate'], message: 'endDate must be after startDate.', }) - if (input.nativeQueries?.some((query) => !query.query) && !bounded) + if (input.nativeQueries?.some((query) => !query.query && !query.browse) && !bounded) context.addIssue({ code: 'custom', path: ['nativeQueries'], diff --git a/apps/sim/lib/mothership/generated/sim-assistant-tools.generated.ts b/apps/sim/lib/mothership/generated/sim-assistant-tools.generated.ts index 2c7108b798c..f09e343936b 100644 --- a/apps/sim/lib/mothership/generated/sim-assistant-tools.generated.ts +++ b/apps/sim/lib/mothership/generated/sim-assistant-tools.generated.ts @@ -24,11 +24,6 @@ export const liveSearchProviderSchema = z.enum([ ]) export type LiveSearchProvider = z.output -export const SEARCH_TERMS_REQUIRED = { - notion: 'Notion requires search terms. Add keywords or a concise question.', - lucid: 'Lucid requires search terms. Add document-title keywords or a literal shape-text query.', -} as const - /** * Native queries one call may send to the same provider account. Alternatives run as separate * provider searches and fuse into one ranking, so the bound keeps a call within the provider's @@ -74,6 +69,12 @@ export const nativeSearchQuerySchema = z accountId: z.string().min(1).max(200).optional(), kind: nativeSearchKindSchema.optional(), project: z.string().min(1).max(300).optional(), + browse: z + .enum(['folder', 'private', 'shared', 'favorites', 'recent']) + .optional() + .describe( + 'Queryless discovery: Lucid folder lists one folder page (omit project for root; otherwise use a returned numeric folder ID). Notion private/shared list sidebar pages, favorites lists pinned pages, recent lists recently viewed pages, not recently modified pages. These lists are not an exhaustive workspace inventory. Follow the returned cursor with the same account, browse mode, project, filters and topK.' + ), cursor: z.string().max(4000).optional(), termClauses: z.array(z.string().max(500)).max(10).optional(), modifiers: z.string().max(1000).optional(), @@ -92,11 +93,32 @@ export const nativeSearchQuerySchema = z : `${input.provider} does not support kind selection.`, }) } - if ((input.provider === 'notion' || input.provider === 'lucid') && !input.query) + if ( + input.browse && + (input.query || + input.modifiers || + input.keywordOnly || + input.termClauses?.length || + (input.browse === 'folder' ? input.provider !== 'lucid' : input.provider !== 'notion') || + (input.provider === 'notion' && input.project)) + ) + context.addIssue({ + code: 'custom', + path: ['browse'], + message: + 'Browse requires an empty query and a supported provider mode; Notion lists cannot be scoped to a page.', + }) + if (input.browse === 'folder' && input.project && !/^[1-9]\d{0,14}$/.test(input.project)) + context.addIssue({ + code: 'custom', + path: ['project'], + message: 'Lucid folder browsing requires a numeric folder ID returned by the provider.', + }) + if (input.provider === 'lucid' && !input.query && !input.browse) context.addIssue({ code: 'custom', path: ['query'], - message: SEARCH_TERMS_REQUIRED[input.provider], + message: 'Lucid requires title keywords or explicit browse: folder.', }) }) export type NativeSearchQuery = z.output @@ -133,7 +155,9 @@ export const nativeSearchQueriesSchema = z addIssue('Duplicate native query.') else if ( hasSearchKinds(query.provider) && - earlier.some((previous) => !previous.kind || !query.kind) + earlier.some( + (previous) => !previous.browse && !query.browse && (!previous.kind || !query.kind) + ) ) addIssue( 'A GitHub, GitLab, HubSpot, Lucid, Google Meet, or Zoom query without a kind already searches its default kinds; give each query on this account a kind.' @@ -154,6 +178,10 @@ export const liveSearchAccountStatusSchema = z.object({ status: z.enum(['ok', 'partial', 'reconnect', 'rate_limited', 'unavailable', 'timeout']), message: z.string().optional(), nextCursor: z.string().optional(), + folders: z + .array(z.object({ id: z.string(), name: z.string() })) + .max(10) + .optional(), retryAfterSeconds: z.number().optional(), }) export type LiveSearchAccountStatus = z.output @@ -218,7 +246,7 @@ export const searchWorkspaceInputSchema = workspaceSearchFiltersSchema nativeQueries: nativeSearchQueriesSchema .optional() .describe( - `Live search only: queries in a provider's own language (Drive q, Gmail operators, JQL, CQL, GitHub qualifiers, Slack RTS, plain Linear/Fireflies/HubSpot/Lucid/Zoom terms, bounded local Google Meet text matching, Granola natural-language questions, Notion keywords or AI questions when available). Blank queries require a date bound or sortBy newest/oldest; Notion and Lucid always require search terms. Up to ${MAX_NATIVE_QUERIES_PER_ACCOUNT} per account run separately and merge; one GitHub, GitLab, or HubSpot query without a kind searches GitHub issues (plus code when the query has no date bound or boolean operators, as its status message says), GitLab issues, merge requests, and code, or every HubSpot CRM kind; other collections, and multiple queries on one account, each need a kind, which may repeat. HubSpot kinds are contacts, companies, deals, and tickets; Lucid kinds are lucidchart and lucidspark. Google Meet kinds are transcript and smart_notes (note metadata and Docs link only); it searches bounded recent conference artifacts with 30-day retention. Zoom kind is meeting and searches past occurrences; read for transcripts and separately labeled summaries. Use Drive for saved Meet note bodies and older transcripts; Drive dates mean file modification time. HubSpot, Lucid, Zoom and Meet reject ownership filters. Lucid searches titles with no search continuation; project can scope a literal shape-text query to one known document UUID or Lucid URL. Read for structured diagram evidence. Dates and sorting cover only retrieved candidates, not globally newest/oldest matches. Write queries from the returned live guidance and account IDs; each account status names the queryIndex its cursor belongs to. Omit for simple cross-provider terms.` + `Live search only: queries in a provider's own language (Drive q, Gmail operators, JQL, CQL, GitHub qualifiers, Slack RTS, plain Linear/Fireflies/HubSpot/Lucid/Zoom terms, bounded local Google Meet text matching, Granola natural-language questions, Notion keywords or AI questions when available). Blank queries require a date bound, sortBy newest/oldest, or explicit browse mode. Lucid browse folder lists root or a numeric folder project; Notion browse private/shared/favorites/recent lists sidebar pages. Recent means viewed, not modified. Up to ${MAX_NATIVE_QUERIES_PER_ACCOUNT} per account run separately and merge; one GitHub, GitLab, or HubSpot query without a kind searches GitHub issues (plus code when the query has no date bound or boolean operators, as its status message says), GitLab issues, merge requests, and code, or every HubSpot CRM kind; other collections, and multiple content queries on one account, each need a kind, which may repeat; explicit browse queries may select distinct folders or sidebar sections without a kind. HubSpot kinds are contacts, companies, deals, and tickets; Lucid kinds are lucidchart and lucidspark. Google Meet kinds are transcript and smart_notes (note metadata and Docs link only); it searches bounded recent conference artifacts with 30-day retention. Zoom kind is meeting and searches past occurrences; read for transcripts and separately labeled summaries. Use Drive for saved Meet note bodies and older transcripts; Drive dates mean file modification time. HubSpot, Lucid, Zoom and Meet reject ownership filters. Lucid title search has no continuation; folder browsing is paginated and returns child folders in account coverage; project can scope a literal shape-text query to one known document UUID or Lucid URL. Read for structured diagram evidence. Dates and sorting cover only retrieved candidates, not globally newest/oldest matches. Write queries from the returned live guidance and account IDs; each account status names the queryIndex its cursor belongs to. Omit for simple cross-provider terms.` ), query: z .string() @@ -226,7 +254,7 @@ export const searchWorkspaceInputSchema = workspaceSearchFiltersSchema .max(2000) .default('') .describe( - 'Search terms, without dates already supplied as filters. May be empty for a live listing with a date bound or sortBy newest or oldest where supported; Notion requires search terms.' + 'Search terms, without dates already supplied as filters. May be empty for a live listing with a date bound or sortBy newest or oldest where supported, or use an explicit native browse mode. Notion date-only search depends on plan capabilities.' ), topK: z .number() @@ -247,7 +275,11 @@ export const searchWorkspaceInputSchema = workspaceSearchFiltersSchema input.sortBy === 'newest' || input.sortBy === 'oldest' ) - if (!input.query && !input.nativeQueries?.some((query) => query.query) && !bounded) + if ( + !input.query && + !input.nativeQueries?.some((query) => query.query || query.browse) && + !bounded + ) context.addIssue({ code: 'custom', path: ['query'], @@ -263,7 +295,7 @@ export const searchWorkspaceInputSchema = workspaceSearchFiltersSchema path: ['endDate'], message: 'endDate must be after startDate.', }) - if (input.nativeQueries?.some((query) => !query.query) && !bounded) + if (input.nativeQueries?.some((query) => !query.query && !query.browse) && !bounded) context.addIssue({ code: 'custom', path: ['nativeQueries'], diff --git a/apps/sim/lib/mothership/generated/tool-catalog-v1.ts b/apps/sim/lib/mothership/generated/tool-catalog-v1.ts index bf94e2d5695..6a0cc58e12e 100644 --- a/apps/sim/lib/mothership/generated/tool-catalog-v1.ts +++ b/apps/sim/lib/mothership/generated/tool-catalog-v1.ts @@ -6097,7 +6097,7 @@ export const SearchWorkspace: ToolCatalogEntry = { }, nativeQueries: { description: - "Live search only: queries in a provider's own language (Drive q, Gmail operators, JQL, CQL, GitHub qualifiers, Slack RTS, plain Linear/Fireflies/HubSpot/Lucid/Zoom terms, bounded local Google Meet text matching, Granola natural-language questions, Notion keywords or AI questions when available). Blank queries require a date bound or sortBy newest/oldest; Notion and Lucid always require search terms. Up to 4 per account run separately and merge; one GitHub, GitLab, or HubSpot query without a kind searches GitHub issues (plus code when the query has no date bound or boolean operators, as its status message says), GitLab issues, merge requests, and code, or every HubSpot CRM kind; other collections, and multiple queries on one account, each need a kind, which may repeat. HubSpot kinds are contacts, companies, deals, and tickets; Lucid kinds are lucidchart and lucidspark. Google Meet kinds are transcript and smart_notes (note metadata and Docs link only); it searches bounded recent conference artifacts with 30-day retention. Zoom kind is meeting and searches past occurrences; read for transcripts and separately labeled summaries. Use Drive for saved Meet note bodies and older transcripts; Drive dates mean file modification time. HubSpot, Lucid, Zoom and Meet reject ownership filters. Lucid searches titles with no search continuation; project can scope a literal shape-text query to one known document UUID or Lucid URL. Read for structured diagram evidence. Dates and sorting cover only retrieved candidates, not globally newest/oldest matches. Write queries from the returned live guidance and account IDs; each account status names the queryIndex its cursor belongs to. Omit for simple cross-provider terms.", + "Live search only: queries in a provider's own language (Drive q, Gmail operators, JQL, CQL, GitHub qualifiers, Slack RTS, plain Linear/Fireflies/HubSpot/Lucid/Zoom terms, bounded local Google Meet text matching, Granola natural-language questions, Notion keywords or AI questions when available). Blank queries require a date bound, sortBy newest/oldest, or explicit browse mode. Lucid browse folder lists root or a numeric folder project; Notion browse private/shared/favorites/recent lists sidebar pages. Recent means viewed, not modified. Up to 4 per account run separately and merge; one GitHub, GitLab, or HubSpot query without a kind searches GitHub issues (plus code when the query has no date bound or boolean operators, as its status message says), GitLab issues, merge requests, and code, or every HubSpot CRM kind; other collections, and multiple content queries on one account, each need a kind, which may repeat; explicit browse queries may select distinct folders or sidebar sections without a kind. HubSpot kinds are contacts, companies, deals, and tickets; Lucid kinds are lucidchart and lucidspark. Google Meet kinds are transcript and smart_notes (note metadata and Docs link only); it searches bounded recent conference artifacts with 30-day retention. Zoom kind is meeting and searches past occurrences; read for transcripts and separately labeled summaries. Use Drive for saved Meet note bodies and older transcripts; Drive dates mean file modification time. HubSpot, Lucid, Zoom and Meet reject ownership filters. Lucid title search has no continuation; folder browsing is paginated and returns child folders in account coverage; project can scope a literal shape-text query to one known document UUID or Lucid URL. Read for structured diagram evidence. Dates and sorting cover only retrieved candidates, not globally newest/oldest matches. Write queries from the returned live guidance and account IDs; each account status names the queryIndex its cursor belongs to. Omit for simple cross-provider terms.", minItems: 1, maxItems: 9, type: 'array', @@ -6149,6 +6149,12 @@ export const SearchWorkspace: ToolCatalogEntry = { ], }, project: { type: 'string', minLength: 1, maxLength: 300 }, + browse: { + description: + 'Queryless discovery: Lucid folder lists one folder page (omit project for root; otherwise use a returned numeric folder ID). Notion private/shared list sidebar pages, favorites lists pinned pages, recent lists recently viewed pages, not recently modified pages. These lists are not an exhaustive workspace inventory. Follow the returned cursor with the same account, browse mode, project, filters and topK.', + type: 'string', + enum: ['folder', 'private', 'shared', 'favorites', 'recent'], + }, cursor: { type: 'string', maxLength: 4000 }, termClauses: { maxItems: 10, type: 'array', items: { type: 'string', maxLength: 500 } }, modifiers: { type: 'string', maxLength: 1000 }, @@ -6161,7 +6167,7 @@ export const SearchWorkspace: ToolCatalogEntry = { query: { default: '', description: - 'Search terms, without dates already supplied as filters. May be empty for a live listing with a date bound or sortBy newest or oldest where supported; Notion requires search terms.', + 'Search terms, without dates already supplied as filters. May be empty for a live listing with a date bound or sortBy newest or oldest where supported, or use an explicit native browse mode. Notion date-only search depends on plan capabilities.', type: 'string', maxLength: 2000, }, diff --git a/apps/sim/lib/mothership/generated/tool-schemas-v1.ts b/apps/sim/lib/mothership/generated/tool-schemas-v1.ts index 07020b2a423..369aee5b4cf 100644 --- a/apps/sim/lib/mothership/generated/tool-schemas-v1.ts +++ b/apps/sim/lib/mothership/generated/tool-schemas-v1.ts @@ -6043,7 +6043,7 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { }, nativeQueries: { description: - "Live search only: queries in a provider's own language (Drive q, Gmail operators, JQL, CQL, GitHub qualifiers, Slack RTS, plain Linear/Fireflies/HubSpot/Lucid/Zoom terms, bounded local Google Meet text matching, Granola natural-language questions, Notion keywords or AI questions when available). Blank queries require a date bound or sortBy newest/oldest; Notion and Lucid always require search terms. Up to 4 per account run separately and merge; one GitHub, GitLab, or HubSpot query without a kind searches GitHub issues (plus code when the query has no date bound or boolean operators, as its status message says), GitLab issues, merge requests, and code, or every HubSpot CRM kind; other collections, and multiple queries on one account, each need a kind, which may repeat. HubSpot kinds are contacts, companies, deals, and tickets; Lucid kinds are lucidchart and lucidspark. Google Meet kinds are transcript and smart_notes (note metadata and Docs link only); it searches bounded recent conference artifacts with 30-day retention. Zoom kind is meeting and searches past occurrences; read for transcripts and separately labeled summaries. Use Drive for saved Meet note bodies and older transcripts; Drive dates mean file modification time. HubSpot, Lucid, Zoom and Meet reject ownership filters. Lucid searches titles with no search continuation; project can scope a literal shape-text query to one known document UUID or Lucid URL. Read for structured diagram evidence. Dates and sorting cover only retrieved candidates, not globally newest/oldest matches. Write queries from the returned live guidance and account IDs; each account status names the queryIndex its cursor belongs to. Omit for simple cross-provider terms.", + "Live search only: queries in a provider's own language (Drive q, Gmail operators, JQL, CQL, GitHub qualifiers, Slack RTS, plain Linear/Fireflies/HubSpot/Lucid/Zoom terms, bounded local Google Meet text matching, Granola natural-language questions, Notion keywords or AI questions when available). Blank queries require a date bound, sortBy newest/oldest, or explicit browse mode. Lucid browse folder lists root or a numeric folder project; Notion browse private/shared/favorites/recent lists sidebar pages. Recent means viewed, not modified. Up to 4 per account run separately and merge; one GitHub, GitLab, or HubSpot query without a kind searches GitHub issues (plus code when the query has no date bound or boolean operators, as its status message says), GitLab issues, merge requests, and code, or every HubSpot CRM kind; other collections, and multiple content queries on one account, each need a kind, which may repeat; explicit browse queries may select distinct folders or sidebar sections without a kind. HubSpot kinds are contacts, companies, deals, and tickets; Lucid kinds are lucidchart and lucidspark. Google Meet kinds are transcript and smart_notes (note metadata and Docs link only); it searches bounded recent conference artifacts with 30-day retention. Zoom kind is meeting and searches past occurrences; read for transcripts and separately labeled summaries. Use Drive for saved Meet note bodies and older transcripts; Drive dates mean file modification time. HubSpot, Lucid, Zoom and Meet reject ownership filters. Lucid title search has no continuation; folder browsing is paginated and returns child folders in account coverage; project can scope a literal shape-text query to one known document UUID or Lucid URL. Read for structured diagram evidence. Dates and sorting cover only retrieved candidates, not globally newest/oldest matches. Write queries from the returned live guidance and account IDs; each account status names the queryIndex its cursor belongs to. Omit for simple cross-provider terms.", minItems: 1, maxItems: 9, type: 'array', @@ -6106,6 +6106,12 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { minLength: 1, maxLength: 300, }, + browse: { + description: + 'Queryless discovery: Lucid folder lists one folder page (omit project for root; otherwise use a returned numeric folder ID). Notion private/shared list sidebar pages, favorites lists pinned pages, recent lists recently viewed pages, not recently modified pages. These lists are not an exhaustive workspace inventory. Follow the returned cursor with the same account, browse mode, project, filters and topK.', + type: 'string', + enum: ['folder', 'private', 'shared', 'favorites', 'recent'], + }, cursor: { type: 'string', maxLength: 4000, @@ -6133,7 +6139,7 @@ export const TOOL_RUNTIME_SCHEMAS: Record = { query: { default: '', description: - 'Search terms, without dates already supplied as filters. May be empty for a live listing with a date bound or sortBy newest or oldest where supported; Notion requires search terms.', + 'Search terms, without dates already supplied as filters. May be empty for a live listing with a date bound or sortBy newest or oldest where supported, or use an explicit native browse mode. Notion date-only search depends on plan capabilities.', type: 'string', maxLength: 2000, }, diff --git a/apps/sim/lib/mothership/tools/server/knowledge/workspace-search.test.ts b/apps/sim/lib/mothership/tools/server/knowledge/workspace-search.test.ts index a926b851cdc..8ae1042bd73 100644 --- a/apps/sim/lib/mothership/tools/server/knowledge/workspace-search.test.ts +++ b/apps/sim/lib/mothership/tools/server/knowledge/workspace-search.test.ts @@ -155,24 +155,6 @@ describe('Assistant retrieval tools', () => { }) }) } - it.each([{ startDate: '2026-09-01T00:00:00Z' }, { sortBy: 'newest' }, { sortBy: 'oldest' }])( - 'returns actionable validation for empty Notion native queries with %j', - async (bound) => { - const result = await searchWorkspaceServerTool.execute( - { - ...bound, - query: 'fallback terms', - nativeQueries: [{ provider: 'notion', query: ' \t ' }], - }, - { ...context, assistantSearch: undefined } - ) - expect(result).toMatchObject({ - success: false, - message: 'Notion requires search terms. Add keywords or a concise question.', - }) - expect(result).not.toHaveProperty('data') - } - ) it.each([ { provider: 'slack', kind: 'meeting' }, { provider: 'google_drive', kind: 'transcript' }, diff --git a/apps/sim/lib/sim-search/live/application.test.ts b/apps/sim/lib/sim-search/live/application.test.ts index 92573d1f542..9626a70e9e9 100644 --- a/apps/sim/lib/sim-search/live/application.test.ts +++ b/apps/sim/lib/sim-search/live/application.test.ts @@ -395,34 +395,19 @@ describe('authorized live retrieval', () => { } ) }) - describe.each(['notion', 'lucid'] as const)('%s requires terms before dispatch', (provider) => { - it.each([ - { startDate: '2026-08-01T00:00:00Z' }, - { source: ` ${provider} `, startDate: '2026-08-01T00:00:00Z' }, - { sortBy: 'newest' as const }, - { sortBy: 'oldest' as const }, - ])('rejects a provider-only listing with %j', async (bound) => { - await expect( - searchLiveKnowledge.execute({ - principal, - input: { ...input, query: ' \t ', filters: { source: provider, ...bound } }, - }) - ).rejects.toMatchObject({ code: 'validation' }) - }) - it('rejects an empty native query even with a date bound', async () => { - await expect( - searchLiveKnowledge.execute({ - principal, - input: { - ...input, - query: 'topology', - filters: { startDate: '2026-08-01T00:00:00Z' }, - nativeQueries: [{ provider, query: ' \t ' }], - }, - }) - ).rejects.toMatchObject({ - issues: expect.arrayContaining([expect.objectContaining({ path: [0, 'query'] })]), + it('rejects an empty Lucid native query without an explicit browse mode', async () => { + await expect( + searchLiveKnowledge.execute({ + principal, + input: { + ...input, + query: 'topology', + filters: { startDate: '2026-08-01T00:00:00Z' }, + nativeQueries: [{ provider: 'lucid', query: ' \t ' }], + }, }) + ).rejects.toMatchObject({ + issues: expect.arrayContaining([expect.objectContaining({ path: [0, 'query'] })]), }) }) it.each([undefined, 'google_drive'])( diff --git a/apps/sim/lib/sim-search/live/application.ts b/apps/sim/lib/sim-search/live/application.ts index 86d3239bc79..3eb7f8e96b6 100644 --- a/apps/sim/lib/sim-search/live/application.ts +++ b/apps/sim/lib/sim-search/live/application.ts @@ -13,7 +13,6 @@ import { liveSearchProviderSchema, type NativeSearchQuery, nativeSearchQueriesSchema, - SEARCH_TERMS_REQUIRED, workspaceSearchFiltersSchema, } from '@/lib/api/contracts/mothership-assistant-tools' import { canonicalJson, fingerprint, instantScopePart } from '@/lib/api/cursor-binding' @@ -73,14 +72,20 @@ const signedContinuationSchema = boundContinuationSchema.extend({ signature: z.string().regex(/^[a-f0-9]{64}$/), }) -function continuationSignature(provider: 'hubspot' | 'zoom', value: BoundContinuation): string { +type BoundProvider = 'hubspot' | 'zoom' | 'lucid' | 'notion' + +function hasBoundContinuation(provider: string): provider is BoundProvider { + return ['hubspot', 'zoom', 'lucid', 'notion'].includes(provider) +} + +function continuationSignature(provider: BoundProvider, value: BoundContinuation): string { return hmacSha256Hex( `live-search-continuation:${provider}:${canonicalJson(value)}`, env.BETTER_AUTH_SECRET ) } -function readBoundContinuation(provider: 'hubspot' | 'zoom', value: string): BoundContinuation { +function readBoundContinuation(provider: BoundProvider, value: string): BoundContinuation { try { const prefix = `${provider}:` if (!value.startsWith(prefix) || value.length > (provider === 'hubspot' ? 512 : 4000)) @@ -96,12 +101,12 @@ function readBoundContinuation(provider: 'hubspot' | 'zoom', value: string): Bou } catch { throw new NativeSearchError( 'unavailable', - `Invalid ${provider === 'zoom' ? 'Zoom' : 'HubSpot'} cursor. Restart this search without a cursor.` + `Invalid ${provider} cursor. Restart this search without a cursor.` ) } } -function writeBoundContinuation(provider: 'hubspot' | 'zoom', value: BoundContinuation): string { +function writeBoundContinuation(provider: BoundProvider, value: BoundContinuation): string { const payload = boundContinuationSchema.parse(value) const signed = { ...payload, signature: continuationSignature(provider, payload) } const cursor = `${provider}:${Buffer.from(JSON.stringify(signed)).toString('base64url')}` @@ -358,7 +363,8 @@ export const searchLiveKnowledge = defineAuthorizedKnowledgeUseCase({ if ( (!input.query.trim() && !hasDateBounds(input.filters) && - !queries?.some((query) => query.query)) || + !queries?.some((query) => query.query || query.browse)) || + (!queries && input.filters?.source === 'lucid' && !input.query.trim()) || input.query.length > 2000 || !Number.isInteger(input.topK) || input.topK < 1 || @@ -366,19 +372,13 @@ export const searchLiveKnowledge = defineAuthorizedKnowledgeUseCase({ ) throw new OrchestrationError('validation', 'Invalid live search query or result limit') const filters = input.filters - if ( - !queries && - (filters?.source === 'notion' || filters?.source === 'lucid') && - !input.query.trim() - ) - throw new OrchestrationError('validation', SEARCH_TERMS_REQUIRED[filters.source]) if ( filters?.startDate && filters.endDate && Date.parse(filters.startDate) >= Date.parse(filters.endDate) ) throw new OrchestrationError('validation', 'endDate must be after startDate') - if (queries?.some((query) => !query.query) && !hasDateBounds(filters)) + if (queries?.some((query) => !query.query && !query.browse) && !hasDateBounds(filters)) throw new OrchestrationError('validation', 'Empty native queries require a date bound') const searchSignal = input.signal ? AbortSignal.any([input.signal, AbortSignal.timeout(20_000)]) @@ -421,7 +421,7 @@ export const searchLiveKnowledge = defineAuthorizedKnowledgeUseCase({ let queryNative = native let continuationScope: string | undefined let listingEndDate: string | undefined - if (account.provider === 'hubspot' || account.provider === 'zoom') { + if (hasBoundContinuation(account.provider)) { continuationScope = fingerprint( canonicalJson({ provider: account.provider, @@ -430,6 +430,9 @@ export const searchLiveKnowledge = defineAuthorizedKnowledgeUseCase({ account: account.id, query: (native?.query ?? input.query).trim(), kind: native?.kind, + ...(['lucid', 'notion'].includes(account.provider) + ? { browse: native?.browse, project: native?.project, topK: input.topK } + : {}), sort: requestedFilters?.sortBy ?? 'relevance', startDate: instantScopePart(requestedFilters?.startDate), endDate: instantScopePart(requestedFilters?.endDate), @@ -540,10 +543,16 @@ export const searchLiveKnowledge = defineAuthorizedKnowledgeUseCase({ ? 'No readable matches on this page. Continue with nextCursor for more.' : undefined, ]), + ...(page.folders + ? { + folders: page.folders.map((folder) => ({ + ...folder, + name: safeContent(folder.name, input.resultSecretRegistry), + })), + } + : {}), nextCursor: - page.nextCursor && - continuationScope && - (account.provider === 'hubspot' || account.provider === 'zoom') + page.nextCursor && continuationScope && hasBoundContinuation(account.provider) ? writeBoundContinuation(account.provider, { v: 2, scope: continuationScope, diff --git a/apps/sim/lib/sim-search/live/lucid-mcp.ts b/apps/sim/lib/sim-search/live/lucid-mcp.ts index 23dcdf357bd..34ed17a7324 100644 --- a/apps/sim/lib/sim-search/live/lucid-mcp.ts +++ b/apps/sim/lib/sim-search/live/lucid-mcp.ts @@ -1,4 +1,5 @@ import { isRecordLike } from '@sim/utils/object' +import { truncate } from '@sim/utils/string' import { mapWithConcurrency } from '@/lib/core/utils/concurrency' import { dateSortDirection, @@ -80,6 +81,7 @@ export async function searchLucidMcp( input: NativeSearchInput ): Promise { const query = nativeText(input) + if (input.native?.browse) return browseFolder(client, input) if (!query.trim()) invalid('Lucid requires search terms, including for date-filtered searches.') const project = input.native?.project if ([...query].length > (project ? 200 : 400)) @@ -202,6 +204,100 @@ export async function searchLucidMcp( } } +/** A single folder page stays within the metadata budget and never recursively expands folders. */ +async function browseFolder( + client: ManagedSearchMcpClient, + input: NativeSearchInput +): Promise { + const native = input.native + if ( + native?.browse !== 'folder' || + nativeText(input) || + native.modifiers || + native.termClauses?.length || + native.keywordOnly + ) + invalid('Lucid folder browsing requires an empty query without search operators.') + if (native.project && !/^[1-9]\d{0,14}$/.test(native.project)) + invalid('Lucid folder browsing requires a numeric folder ID.') + if (client.hasTool?.('lucid_list_folder_contents') !== true) + invalid('Lucid folder browsing is unavailable on this connection.') + if (native.kind && !PRODUCTS.some((product) => product === native.kind)) + invalid('Lucid supports lucidchart and lucidspark document kinds.') + const limit = Math.max(1, Math.min(MAX_CANDIDATES, input.limit)) + const result = object( + await client.call('lucid_list_folder_contents', { + page_size: limit, + ...(native.project ? { folder_id: Number(native.project) } : {}), + ...(native.cursor ? { page_token: native.cursor } : {}), + }) + ) + if (!Array.isArray(result.items) || result.items.length > limit) + invalid('Lucid returned an invalid or oversized folder page; restart with a smaller page.') + if ( + result.nextPageToken !== undefined && + (typeof result.nextPageToken !== 'string' || + !result.nextPageToken || + result.nextPageToken.length > 2048 || + result.nextPageToken === native.cursor) + ) + invalid('Lucid returned an invalid folder continuation.') + const folders: NonNullable = [] + const candidates: { id: string; kind: string }[] = [] + const seen = new Set() + let dropped = false + for (const item of result.items) { + const row = object(item) + if (row.isShortcut !== false) { + dropped = true + continue + } + if (row.type === 'folder') { + const id = + typeof row.id === 'number' && Number.isSafeInteger(row.id) ? String(row.id) : string(row.id) + if (!/^[1-9]\d{0,14}$/.test(id) || typeof row.name !== 'string') { + dropped = true + continue + } + if (!seen.has(`folder:${id}`)) folders.push({ id, name: truncate(row.name, 200) }) + seen.add(`folder:${id}`) + continue + } + const id = string(row.id) + const kind = string(row.product) + if ( + row.type !== 'document' || + !UUID.test(id) || + !PRODUCTS.some((product) => product === kind) + ) { + dropped = true + continue + } + if (native.kind && native.kind !== kind) continue + if (!seen.has(id)) candidates.push({ id, kind }) + seen.add(id) + } + const documents = await mapWithConcurrency(candidates, 3, async (candidate) => { + const document = await metadata(client, candidate.id) + if (!document || document.kind !== candidate.kind) { + dropped = true + return undefined + } + return document + }) + const nextCursor = string(result.nextPageToken) || undefined + return { + documents: documents.filter((document) => document !== undefined), + folders, + nextCursor, + hasMore: Boolean(nextCursor), + partial: dropped || hasDateBounds(input.filters) || Boolean(dateSortDirection(input.filters)), + message: + 'Lucid lists one folder’s direct children, not the entire account. Child folders are navigation references: use browse folder with their ID as project. Continue the same folder with nextCursor, including when no documents matched this page. Dates and sorting apply only to this page’s current document metadata; folder names and document titles are not diagram evidence.' + + (dropped ? ' Shortcuts, unsupported items or unreadable metadata were excluded.' : ''), + } +} + function manifest(value: unknown, id: string, kind: string | undefined) { const row = object(value) const url = resource(string(row.edit_url)) diff --git a/apps/sim/lib/sim-search/live/managed-mcp-config.ts b/apps/sim/lib/sim-search/live/managed-mcp-config.ts index 6606a39581b..d5aa3b32d1a 100644 --- a/apps/sim/lib/sim-search/live/managed-mcp-config.ts +++ b/apps/sim/lib/sim-search/live/managed-mcp-config.ts @@ -4,8 +4,23 @@ export const MANAGED_SEARCH_MCP_READ_TOOLS = { fireflies: ['fireflies_get_transcripts', 'fireflies_get_transcript', 'fireflies_get_summary'], granola: ['query_granola_meetings', 'list_meetings', 'get_meetings', 'get_meeting_transcript'], hubspot: ['get_user_details', 'search_crm_objects', 'get_crm_objects'], - lucid: ['search', 'fetch', 'lucid_search_document', 'lucid_get_document_metadata'], - notion: ['notion-get-tool-access', 'notion-search', 'notion-ai-search', 'notion-fetch'], + lucid: [ + 'search', + 'fetch', + 'lucid_search_document', + 'lucid_get_document_metadata', + 'lucid_list_folder_contents', + ], + notion: [ + 'notion-get-tool-access', + 'notion-search', + 'notion-ai-search', + 'notion-fetch', + 'notion-list-private-pages', + 'notion-list-shared-pages', + 'notion-list-favorite-pages', + 'notion-list-recent-pages', + ], zoom: ['search_meetings', 'get_meeting_assets'], } as const diff --git a/apps/sim/lib/sim-search/live/managed-mcp.integration.ts b/apps/sim/lib/sim-search/live/managed-mcp.integration.ts index 3beb8aa1e8f..43a6df18f81 100644 --- a/apps/sim/lib/sim-search/live/managed-mcp.integration.ts +++ b/apps/sim/lib/sim-search/live/managed-mcp.integration.ts @@ -36,19 +36,28 @@ const RESOURCE = 'https://mcp.lucid.app/mcp/readonly' const DOCUMENT = '00000000-0000-4000-8000-000000000001' const SECOND_DOCUMENT = '00000000-0000-4000-8000-000000000002' const TITLE = 'Synthetic topology' -const editUrl = `https://lucid.app/lucidchart/${DOCUMENT}/edit` const actors = [0, 1].map(() => ({ userId: generateId(), organizationId: generateId(), credentialId: `mcp-cg-${generateId()}`, token: generateId(), + provider: 'lucid' as 'lucid' | 'notion', })) actors.push({ userId: generateId(), organizationId: actors[0].organizationId, credentialId: `mcp-cg-${generateId()}`, token: generateId(), + provider: 'lucid', }) +actors.push({ + userId: generateId(), + organizationId: generateId(), + credentialId: `mcp-cg-${generateId()}`, + token: generateId(), + provider: 'notion', +}) +const NOTION_RESOURCE = 'https://mcp.notion.com/mcp' const sessions = new Map() const serverIds = new Map() const protocols: Server[] = [] @@ -98,20 +107,32 @@ const providerServer = createServer(async (request, response) => { if (failDiscovery && params?.cursor) throw new Error('Synthetic discovery failure') return { ...(failDiscovery ? { nextCursor: 'second-page' } : {}), - tools: ['search', 'fetch', 'lucid_get_document_metadata', 'write_document'].map( - (name) => ({ - name, - inputSchema: { - type: 'object' as const, - properties: { - query: { type: 'string' }, - product: { type: 'array', items: { type: 'string' } }, - last_modified_after: { type: 'string' }, - }, - additionalProperties: name !== 'search', + tools: [ + 'search', + 'fetch', + 'lucid_get_document_metadata', + 'lucid_list_folder_contents', + 'notion-get-tool-access', + 'notion-search', + 'notion-list-private-pages', + 'notion-list-shared-pages', + 'notion-fetch', + 'write_document', + ].map((name) => ({ + name, + inputSchema: { + type: 'object' as const, + properties: { + query: { type: 'string' }, + cursor: { type: 'string' }, + limit: { type: 'number' }, + sort: { type: 'string' }, + product: { type: 'array', items: { type: 'string' } }, + last_modified_after: { type: 'string' }, }, - }) - ), + additionalProperties: name !== 'search', + }, + })), } }) protocol.setRequestHandler(CallToolRequestSchema, async ({ params }) => { @@ -119,11 +140,68 @@ const providerServer = createServer(async (request, response) => { await onTool?.(params.name, params.arguments ?? {}) const documentId = params.arguments?.query === 'second topology' || - params.arguments?.document_id === SECOND_DOCUMENT + params.arguments?.document_id === SECOND_DOCUMENT || + params.arguments?.id === SECOND_DOCUMENT ? SECOND_DOCUMENT : DOCUMENT const documentTitle = documentId === SECOND_DOCUMENT ? 'Second topology' : TITLE const documentUrl = `https://lucid.app/lucidchart/${documentId}/edit` + if (params.name === 'notion-get-tool-access') + return payload({ + current_tool_access: { + search: { status: 'available' }, + list_private_pages: { status: 'available' }, + list_shared_pages: { status: 'available' }, + }, + }) + if (params.name === 'notion-search' || params.name.startsWith('notion-list-')) + return payload({ + results: [ + { + type: 'page', + title: TITLE, + url: `https://www.notion.so/${params.arguments?.cursor ? SECOND_DOCUMENT : DOCUMENT}`, + }, + ], + ...(params.arguments?.cursor ? {} : { nextCursor: 'offset:1' }), + }) + if (params.name === 'notion-fetch') + return payload({ + id: params.arguments?.id, + text: 'Private page evidence', + page_last_edited_at: '2026-09-01T12:00:00Z', + }) + if (params.name === 'lucid_list_folder_contents' && params.arguments?.folder_id === 42) + return payload({ + items: [ + { + id: SECOND_DOCUMENT, + type: 'document', + product: 'lucidchart', + name: 'Second topology', + isShortcut: false, + }, + ], + }) + if (params.name === 'lucid_list_folder_contents') + return payload( + params.arguments?.page_token + ? { + items: [ + { + id: DOCUMENT, + type: 'document', + product: 'lucidchart', + name: TITLE, + isShortcut: false, + }, + ], + } + : { + items: [{ id: 42, type: 'folder', name: 'Architecture', isShortcut: false }], + nextPageToken: 'page-two', + } + ) if (params.name === 'search') return payload({ results: [{ id: documentId, title: documentTitle, url: documentUrl }] }) if (params.name === 'lucid_get_document_metadata') @@ -137,9 +215,9 @@ const providerServer = createServer(async (request, response) => { lastModified: '2026-09-01T12:00:00Z', }) const manifest = { - document_id: DOCUMENT, - title: TITLE, - edit_url: editUrl, + document_id: documentId, + title: documentTitle, + edit_url: documentUrl, metadata: { page_count: 1, page_region_counts: [1] }, } if (params.arguments?.metadata_only) return payload(manifest) @@ -158,7 +236,10 @@ const providerServer = createServer(async (request, response) => { requestedChunks: [ { chunkIndex: 0, - data: { nodes: [{ label: `Private diagram ${actor}` }], edges: [] }, + data: { + nodes: [{ label: `Private diagram ${actor}: ${documentTitle}` }], + edges: [], + }, }, ], }, @@ -192,12 +273,13 @@ beforeAll(async () => { origin = `http://127.0.0.1:${address.port}/mcp` const guarded = pinnedFetch.createGuardedMcpFetch vi.spyOn(pinnedFetch, 'createGuardedMcpFetch').mockImplementation((url) => { - if (url !== RESOURCE) throw new Error('Unexpected MCP fixture origin') + if (!url || ![RESOURCE, NOTION_RESOURCE].includes(url)) + throw new Error('Unexpected MCP fixture origin') const transport = guarded(origin) openTransports.add(transport) return { fetch: (input, init) => { - if (new URL(input instanceof Request ? input.url : input).href !== RESOURCE) + if (new URL(input instanceof Request ? input.url : input).href !== url) throw new Error('Unexpected MCP fixture destination') return transport.fetch(origin, init) }, @@ -238,7 +320,10 @@ beforeAll(async () => { organizationId: actor.organizationId, credentialGroupId: group.id, userId: actor.userId, - validated: { input: { connectorId: 'lucid' }, url: RESOURCE }, + validated: { + input: { connectorId: actor.provider }, + url: actor.provider === 'notion' ? NOTION_RESOURCE : RESOURCE, + }, }, tx ) @@ -278,7 +363,11 @@ beforeAll(async () => { }) await db .insert(organizationSearchIntegration) - .values({ organizationId: actor.organizationId, connectorType: 'lucid', approved: true }) + .values({ + organizationId: actor.organizationId, + connectorType: actor.provider, + approved: true, + }) .onConflictDoNothing() } }) @@ -412,6 +501,8 @@ describe('managed Search operation sessions', () => { { queryIndex: 1, status: 'ok' }, ]) expect(events.slice(offset).filter((event) => event.method === 'initialize')).toHaveLength(1) + for (const result of found.results) + expect(JSON.stringify(await read(result.documentId))).toContain(result.documentName) expect(openTransports.size).toBe(0) } finally { clearTimeout(timer) @@ -419,6 +510,209 @@ describe('managed Search operation sessions', () => { onTool = undefined } }) + it('binds folder continuations to account, folder, page size and filters', async () => { + const browse = ( + index = 0, + options: { + cursor?: string + project?: string + topK?: number + query?: string + startDate?: string + sortBy?: 'newest' + } = {} + ) => { + const actor = actors[index] + return searchLiveKnowledge.execute({ + principal: createSessionPrincipal({ userId: actor.userId, sessionId: generateId() }), + input: { + organizationId: actor.organizationId, + query: '', + topK: options.topK ?? 10, + filters: { + source: 'lucid', + ...(options.startDate ? { startDate: options.startDate } : {}), + ...(options.sortBy ? { sortBy: options.sortBy } : {}), + }, + nativeQueries: [ + { + provider: 'lucid', + query: options.query ?? '', + ...(options.query ? {} : { browse: 'folder' as const }), + accountId: actor.credentialId, + ...(options.cursor ? { cursor: options.cursor } : {}), + ...(options.project ? { project: options.project } : {}), + }, + ], + }, + }) + } + const first = await browse() + expect(first.live?.accounts[0].folders).toEqual([{ id: '42', name: 'Architecture' }]) + const cursor = first.live?.accounts[0].nextCursor + expect(cursor).toBeTruthy() + expect((await browse(0, { cursor })).results).toHaveLength(1) + const actor = actors[0] + const parallel = await searchLiveKnowledge.execute({ + principal: createSessionPrincipal({ userId: actor.userId, sessionId: generateId() }), + input: { + organizationId: actor.organizationId, + query: '', + topK: 10, + filters: { source: 'lucid' }, + nativeQueries: [ + { provider: 'lucid', query: '', browse: 'folder', accountId: actor.credentialId, cursor }, + { + provider: 'lucid', + query: '', + browse: 'folder', + accountId: actor.credentialId, + project: '42', + }, + ], + }, + }) + expect(parallel.results.map((result) => result.documentName).sort()).toEqual([ + 'Second topology', + TITLE, + ]) + + for (const options of [{ project: '42' }, { topK: 5 }, { startDate: '2026-09-01T00:00:00Z' }]) { + expect((await browse(0, options)).live?.accounts[0]?.status).not.toBe('unavailable') + const result = await browse(0, { ...options, cursor }) + expect(result.results).toHaveLength(0) + expect(result.live?.accounts[0]).toMatchObject({ status: 'unavailable' }) + } + expect((await browse(2, { cursor })).live?.accounts[0]).toMatchObject({ status: 'unavailable' }) + expect((await browse(0, { cursor: `${cursor}tampered` })).live?.accounts[0]).toMatchObject({ + status: 'unavailable', + }) + const sorted = await browse(0, { sortBy: 'newest' }) + expect( + (await browse(0, { sortBy: 'newest', cursor: sorted.live?.accounts[0].nextCursor })).results + ).toHaveLength(1) + expect(openTransports.size).toBe(0) + }) + it.each([ + { startDate: '2026-09-01T00:00:00Z' }, + { sortBy: 'newest' }, + { sortBy: 'oldest' }, + ] as const)( + 'rejects plain empty Lucid searches with %j before opening the provider', + async (filters) => { + const actor = actors[0] + const before = events.length + await expect( + searchLiveKnowledge.execute({ + principal: createSessionPrincipal({ userId: actor.userId, sessionId: generateId() }), + input: { + organizationId: actor.organizationId, + query: '', + topK: 10, + filters: { source: 'lucid', ...filters }, + }, + }) + ).rejects.toMatchObject({ code: 'validation' }) + expect(events.length).toBe(before) + } + ) + it('combines Lucid folder browsing and typed title search while preserving each query outcome', async () => { + const actor = actors[0] + const found = await searchLiveKnowledge.execute({ + principal: createSessionPrincipal({ userId: actor.userId, sessionId: generateId() }), + input: { + organizationId: actor.organizationId, + query: '', + topK: 10, + filters: { source: 'lucid' }, + nativeQueries: [ + { provider: 'lucid', query: '', browse: 'folder', accountId: actor.credentialId }, + { + provider: 'lucid', + query: 'topology', + kind: 'lucidchart', + accountId: actor.credentialId, + }, + ], + }, + }) + expect(found.results).toHaveLength(1) + expect(found.live?.accounts).toEqual([ + expect.objectContaining({ + queryIndex: 0, + folders: [{ id: '42', name: 'Architecture' }], + nextCursor: expect.any(String), + }), + expect.objectContaining({ queryIndex: 1, status: 'ok' }), + ]) + expect(openTransports.size).toBe(0) + }) + it.each(['newest', 'oldest'] as const)( + 'continues a plain Notion topical search sorted %s', + async (sortBy) => { + const actor = actors[3] + const principal = createSessionPrincipal({ userId: actor.userId, sessionId: generateId() }) + const input = { + organizationId: actor.organizationId, + query: 'topology', + topK: 10, + filters: { source: 'notion' as const, sortBy }, + } + const first = await searchLiveKnowledge.execute({ principal, input }) + expect(first.results, JSON.stringify(first.live)).toHaveLength(1) + const cursor = first.live?.accounts[0].nextCursor + expect(cursor).toBeTruthy() + const next = await searchLiveKnowledge.execute({ + principal, + input: { + ...input, + nativeQueries: [ + { provider: 'notion', query: input.query, accountId: actor.credentialId, cursor }, + ], + }, + }) + expect(next.results, JSON.stringify(next.live)).toHaveLength(1) + expect(next.results[0].documentId).not.toBe(first.results[0].documentId) + expect(openTransports.size).toBe(0) + } + ) + it('binds Notion continuation to its sidebar mode and preserves independent page reads', async () => { + const actor = actors[3] + const browse = (mode: 'private' | 'shared', cursor?: string) => + searchLiveKnowledge.execute({ + principal: createSessionPrincipal({ userId: actor.userId, sessionId: generateId() }), + input: { + organizationId: actor.organizationId, + query: '', + topK: 10, + filters: { source: 'notion' }, + nativeQueries: [ + { + provider: 'notion', + query: '', + browse: mode, + accountId: actor.credentialId, + ...(cursor ? { cursor } : {}), + }, + ], + }, + }) + const first = await browse('private') + expect(first.results, JSON.stringify(first.live)).toHaveLength(1) + const cursor = first.live?.accounts[0].nextCursor + expect(cursor).toBeTruthy() + const next = await browse('private', cursor) + expect(next.results).toHaveLength(1) + expect(next.results[0].documentId).not.toBe(first.results[0].documentId) + expect(JSON.stringify(await read(next.results[0].documentId, 3))).toContain( + 'Private page evidence' + ) + expect((await browse('shared')).results).toHaveLength(1) + const changed = await browse('shared', cursor) + expect(changed.results).toHaveLength(0) + expect(changed.live?.accounts[0].status).toBe('unavailable') + expect(openTransports.size).toBe(0) + }) it('isolates simultaneous users and rejects another organization’s signed document reference', async () => { const results = await Promise.all([search(0), search(1)]) const documents = await Promise.all( diff --git a/apps/sim/lib/sim-search/live/notion-mcp.ts b/apps/sim/lib/sim-search/live/notion-mcp.ts index c9ab482b082..b9d095267bf 100644 --- a/apps/sim/lib/sim-search/live/notion-mcp.ts +++ b/apps/sim/lib/sim-search/live/notion-mcp.ts @@ -1,4 +1,9 @@ -import { dateSortDirection, hasDateBounds, nativeText } from '@/lib/sim-search/live/dates' +import { + dateSortDirection, + hasDateBounds, + nativeDateBounds, + nativeText, +} from '@/lib/sim-search/live/dates' import { array, NativeSearchError, object, string } from '@/lib/sim-search/live/http' import type { ManagedSearchMcpClient } from '@/lib/sim-search/live/managed-mcp' import type { NativeDocument, NativePage, NativeSearchInput } from '@/lib/sim-search/live/types' @@ -59,17 +64,49 @@ function searchDocument(row: Record): NativeDocument | undefine } } +const BROWSE_TOOLS = { + private: 'notion-list-private-pages', + shared: 'notion-list-shared-pages', + favorites: 'notion-list-favorite-pages', + recent: 'notion-list-recent-pages', +} as const + /** Tool visibility and workspace-plan access are separate; the current access map owns routing. */ -async function searchTool(client: ManagedSearchMcpClient): Promise { +async function searchTool( + client: ManagedSearchMcpClient, + browse?: NativeSearchInput['native'] +): Promise<{ tool: string; restrictions: Record }> { const access = object(object(await client.call('notion-get-tool-access', {})).current_tool_access) - const ai = string(object(access.ai_search).status) - const keyword = string(object(access.search).status) + if (browse?.browse) { + if (!(browse.browse in BROWSE_TOOLS)) + throw new NativeSearchError('unavailable', 'Unsupported Notion browse mode.') + const name = BROWSE_TOOLS[browse.browse as keyof typeof BROWSE_TOOLS] + const entry = object(access[name.replace('notion-', '').replaceAll('-', '_')]) + if ( + client.hasTool?.(name) !== true || + !['available', 'available_with_limit'].includes(string(entry.status)) + ) + throw new NativeSearchError( + 'unavailable', + 'Notion browsing is unavailable for this connection.' + ) + return { tool: name, restrictions: object(entry.restricted_parameters) } + } + const ai = object(access.ai_search) + const keyword = object(access.search) const exposed = (name: string) => client.hasTool?.(name) !== false - if (ai === 'available' && exposed('notion-ai-search')) return 'notion-ai-search' - if (['available', 'available_with_limit'].includes(keyword) && exposed('notion-search')) - return 'notion-search' - if (['upgrade_required', 'plan_required'].includes(ai) && exposed('notion-ai-search')) - return 'notion-ai-search' + if (ai.status === 'available' && exposed('notion-ai-search')) + return { tool: 'notion-ai-search', restrictions: object(ai.restricted_parameters) } + if ( + ['available', 'available_with_limit'].includes(string(keyword.status)) && + exposed('notion-search') + ) + return { tool: 'notion-search', restrictions: object(keyword.restricted_parameters) } + if ( + ['upgrade_required', 'plan_required'].includes(string(ai.status)) && + exposed('notion-ai-search') + ) + return { tool: 'notion-ai-search', restrictions: object(ai.restricted_parameters) } throw new NativeSearchError( 'unavailable', 'Notion search is unavailable for this connection. Check the enabled tools and workspace plan.' @@ -81,24 +118,63 @@ export async function searchNotionMcp( input: NativeSearchInput ): Promise { const query = nativeText(input) - if (!query) + const browsing = Boolean(input.native?.browse) + if ( + browsing && + (query || + input.native?.project || + input.native?.modifiers || + input.native?.termClauses?.length || + input.native?.keywordOnly) + ) throw new NativeSearchError( 'unavailable', - 'Notion requires search terms. Use short keywords or a concise question, then narrow by dates.' + 'Notion sidebar browsing requires an empty query without page scope or search operators.' ) - const tool = await searchTool(client) + const { tool, restrictions } = await searchTool(client, input.native) + const supported = (path: string) => + client.hasArgument?.(tool, path) === true && + !Object.keys(restrictions).some((key) => path === key || path.startsWith(`${key}.`)) const localFilters = hasDateBounds(input.filters) || Boolean(dateSortDirection(input.filters)) - const args: Record = { query, query_type: 'internal' } - if (input.native?.project) { - const scope = notionResource(input.native.project) - if (!scope || !client.hasArgument?.(tool, 'page_url')) + const args: Record = browsing ? {} : { query, query_type: 'internal' } + let providerDates = false + if (!browsing) { + if (input.native?.project) { + const scope = notionResource(input.native.project) + if (!scope || !supported('page_url')) + throw new NativeSearchError( + 'unavailable', + 'Notion page scope requires a Notion page URL or ID and a connection advertising page_url. Search by page title if scoped search is unavailable.' + ) + args.page_url = scope.url + } + const bounds = nativeDateBounds(input) + const start = + bounds.start && supported('filters.last_edited_date_range.start_date') + ? bounds.start + : undefined + const end = + bounds.end && supported('filters.last_edited_date_range.end_date') ? bounds.end : undefined + if (start || end) { + // Date-only provider bounds have no timezone; widen candidate days before exact local checks. + const day = (value: string, offset: number) => + new Date(Date.parse(value) + offset * 86_400_000).toISOString().slice(0, 10) + args.filters = { + last_edited_date_range: { + ...(start ? { start_date: day(start, -1) } : {}), + ...(end ? { end_date: day(end, 2) } : {}), + }, + } + providerDates = true + } + if (input.filters?.sortBy === 'newest' && supported('sort')) args.sort = 'last_edited' + if (!query && !providerDates && !args.sort) throw new NativeSearchError( 'unavailable', - 'Notion page scope requires a Notion page URL or ID and a connection advertising page_url. Search by page title if scoped search is unavailable.' + 'Notion date-only search requires supported modification filters or newest sorting on the workspace plan. Use an explicit private, shared, favorites or recent browse mode, or supply search terms.' ) - args.page_url = scope.url } - const limit = Math.min(input.limit, localFilters ? 10 : 50) + const limit = Math.max(1, Math.min(input.limit, localFilters ? 10 : 50)) if (client.hasArgument?.(tool, 'page_size')) args.page_size = limit else if (client.hasArgument?.(tool, 'limit')) args.limit = limit const cursorKey = client.hasArgument?.(tool, 'cursor') @@ -152,7 +228,17 @@ export async function searchNotionMcp( } documents = hydrated } - const next = string(result.next_cursor ?? result.nextCursor) + const continuation = result.next_cursor ?? result.nextCursor + if ( + continuation !== undefined && + continuation !== null && + (typeof continuation !== 'string' || + !continuation || + continuation.length > 2048 || + continuation === input.native?.cursor) + ) + throw new NativeSearchError('unavailable', 'Notion returned an invalid search continuation.') + const next = string(continuation) const nextCursor = cursorKey && next && !clipped ? next : undefined const notices = array(result.notices).length > 0 const aiSearch = @@ -174,7 +260,9 @@ export async function searchNotionMcp( cappedWithoutCoverage || (hasMore && !nextCursor), message: - 'Notion searches page content with the connected member’s current access. Only Notion pages and databases are returned; connected-app results are excluded. Read important matches before relying on them.' + + (browsing + ? `Notion lists ${input.native?.browse} sidebar pages and databases, not an exhaustive workspace inventory. Recent means viewed/frequency, not last modified. List entries are navigation metadata; read pages for content evidence.` + : 'Notion searches page content with the connected member’s current access. Only Notion pages and databases are returned; connected-app results are excluded. Read important matches before relying on them.') + (tool === 'notion-search' ? ' This connection uses keyword search; use short, specific title or content terms.' : '') + @@ -184,8 +272,11 @@ export async function searchNotionMcp( (notices ? ' Notion returned plan or search notices; requested coverage may be limited.' : '') + + (providerDates + ? ' Provider modification-day filters narrow candidates; exact timestamps are checked from current page metadata.' + : '') + (localFilters - ? ' Dates and sorting use freshly fetched page modification timestamps for at most 10 candidates; this does not exhaustively search a date range.' + ? ' Dates and sorting use freshly fetched page modification timestamps for at most 10 candidates; this does not exhaustively search a date range or establish global oldest/newest results.' : '') + (hasMore && !nextCursor ? ' More results may exist, but no safe continuation is available. Narrow the query.' diff --git a/apps/sim/lib/sim-search/live/provider-discovery.integration.ts b/apps/sim/lib/sim-search/live/provider-discovery.integration.ts new file mode 100644 index 00000000000..3e9fdac5e6a --- /dev/null +++ b/apps/sim/lib/sim-search/live/provider-discovery.integration.ts @@ -0,0 +1,361 @@ +import { createServer } from 'node:http' +import type { AddressInfo } from 'node:net' +import { Server } from '@modelcontextprotocol/sdk/server/index.js' +import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js' +import { CallToolRequestSchema, ListToolsRequestSchema } from '@modelcontextprotocol/sdk/types.js' +import { generateId } from '@sim/utils/id' +import { toRecord } from '@sim/utils/object' +import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest' +import { nativeSearchQuerySchema } from '@/lib/api/contracts/mothership-assistant-tools' +import { McpClient } from '@/lib/mcp/client' +import { compileMcpToolSchema } from '@/lib/mcp/tool-schema' +import type { McpToolSchemaProperty } from '@/lib/mcp/types' +import { searchLucidMcp } from '@/lib/sim-search/live/lucid-mcp' +import type { ManagedSearchMcpClient } from '@/lib/sim-search/live/managed-mcp' +import { managedMcpPayload } from '@/lib/sim-search/live/managed-mcp-payload' +import { searchNotionMcp } from '@/lib/sim-search/live/notion-mcp' + +const ID = '00000000-0000-4000-8000-000000000001' +const ID2 = '00000000-0000-4000-8000-000000000002' +const stamp = '2026-09-20T12:00:00Z' +const folder = { id: 42, type: 'folder', name: 'Architecture', isShortcut: false } +const lucidDocument = { + id: ID, + type: 'document', + name: 'Topology', + product: 'lucidchart', + isShortcut: false, +} +const notionDocument = { type: 'page', title: 'Decisions', url: `https://www.notion.so/${ID}` } +const field = { type: 'string' } +const tool = ( + name: string, + properties: Record, + required: string[] = [] +) => ({ + name, + inputSchema: { type: 'object' as const, properties, required, additionalProperties: false }, +}) +const schemas = [ + tool('lucid_list_folder_contents', { + folder_id: { type: 'integer' }, + page_size: { type: 'integer', minimum: 1, maximum: 200 }, + page_token: field, + }), + tool('lucid_get_document_metadata', { document_id: field }, ['document_id']), + tool('notion-get-tool-access', {}), + ...['private', 'shared', 'favorite', 'recent'].map((list) => + tool(`notion-list-${list}-pages`, { limit: { type: 'number' }, cursor: field }) + ), + ...['notion-search', 'notion-ai-search'].map((name) => + tool( + name, + { + query: field, + query_type: { enum: ['internal'] }, + page_size: { type: 'integer', maximum: 50 }, + page_url: field, + sort: { enum: ['relevance', 'last_edited', 'created'] }, + filters: { + type: 'object', + properties: { + last_edited_date_range: { + type: 'object', + properties: { + start_date: { type: 'string', pattern: '^\\d{4}-\\d{2}-\\d{2}$' }, + end_date: { type: 'string', pattern: '^\\d{4}-\\d{2}-\\d{2}$' }, + }, + }, + }, + additionalProperties: false, + }, + }, + ['query'] + ) + ), + tool('notion-fetch', { id: field }, ['id']), +] +let restricted = false +let providerOffset = 0 +let ai = false +let mode = '' +let calls = 0 +const requests: { name: string; args: Record }[] = [] +const protocol = new Server( + { name: 'discovery-fixture', version: '1' }, + { capabilities: { tools: {} } } +) +const payload = (data: unknown) => ({ + content: [{ type: 'text' as const, text: JSON.stringify(data) }], +}) +protocol.setRequestHandler(ListToolsRequestSchema, async () => ({ tools: schemas })) +protocol.setRequestHandler(CallToolRequestSchema, async ({ params }) => { + calls++ + if (calls > 12) throw Error('Exceeded per-query provider budget') + const args = params.arguments ?? {} + requests.push({ name: params.name, args }) + const schema = schemas.find((value) => value.name === params.name) + if (!schema || !compileMcpToolSchema(schema.inputSchema)(args)) + throw Error('Provider rejected arguments') + if (params.name === 'lucid_list_folder_contents') { + if (mode === 'oversize') + return payload({ + items: Array.from({ length: 11 }, () => lucidDocument), + nextPageToken: 'after-omitted', + }) + if (mode === 'invalid-cursor') return payload({ items: [], nextPageToken: 123 }) + if (mode === 'unsafe') + return payload({ + items: [ + { ...lucidDocument, isShortcut: true }, + { id: 'https://evil.invalid', type: 'folder', name: 'Unsafe', isShortcut: false }, + ], + }) + if (args.folder_id === 42) return payload({ items: [{ ...lucidDocument, id: ID2 }] }) + return payload( + args.page_token ? { items: [lucidDocument] } : { items: [folder], nextPageToken: 'page-two' } + ) + } + if (params.name === 'lucid_get_document_metadata') + return payload({ + documentId: args.document_id, + title: 'Topology', + product: 'lucidchart', + viewUrl: `https://lucid.app/lucidchart/${args.document_id}/view`, + version: 7, + pageCount: 1, + lastModified: stamp, + }) + if (params.name === 'notion-get-tool-access') + return payload({ + current_tool_access: { + [ai ? 'ai_search' : 'search']: { + status: 'available', + restricted_parameters: restricted + ? { + [mode === 'restricted-child' + ? 'filters.last_edited_date_range.start_date' + : 'filters.last_edited_date_range']: 'Plan restriction', + sort: 'Plan restriction', + } + : {}, + }, + ...Object.fromEntries( + ['private', 'shared', 'favorite', 'recent'].map((list) => [ + `list_${list}_pages`, + { status: 'available' }, + ]) + ), + }, + }) + if (params.name.startsWith('notion-list-')) + return payload( + args.cursor + ? { results: [{ ...notionDocument, url: `https://www.notion.so/${ID2}` }] } + : { results: [notionDocument], nextCursor: 'offset:1' } + ) + if (params.name === 'notion-fetch') + return payload({ + id: args.id, + text: 'Authoritative page content', + page_last_edited_at: + mode === 'date-boundary' + ? args.id === ID + ? '2026-09-19T11:00:00Z' + : '2026-09-20T10:59:59Z' + : stamp, + }) + if (params.name === 'notion-search' || params.name === 'notion-ai-search') { + const dates = toRecord(toRecord(args.filters).last_edited_date_range) + if (mode === 'restricted-child' && dates.start_date) throw Error('Restricted field was sent') + if (!args.query && !args.sort && !Object.keys(dates).length) + throw Error('Empty search has no constraint') + if (mode === 'date-boundary') { + const candidates = [ + { ...notionDocument, edited: '2026-09-19T11:00:00Z' }, + { ...notionDocument, url: `https://www.notion.so/${ID2}`, edited: '2026-09-20T10:59:59Z' }, + ] + return payload({ + results: candidates.filter((row) => { + const date = new Date(Date.parse(row.edited) + providerOffset * 3_600_000) + .toISOString() + .slice(0, 10) + return ( + (typeof dates.start_date !== 'string' || date >= dates.start_date) && + (typeof dates.end_date !== 'string' || date < dates.end_date) + ) + }), + }) + } + return payload({ + type: ai ? 'ai_search' : 'workspace_search', + results: [notionDocument], + ...(mode === 'notice' ? { notices: [{ message: 'Filter was ignored' }] } : {}), + }) + } + throw Error('Unexpected provider operation') +}) +const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: generateId }) +const server = createServer((req, res) => { + void transport.handleRequest(req, res).catch(() => res.end()) +}) +let client: McpClient +let available = new Set(schemas.map((value) => value.name)) +const reader: ManagedSearchMcpClient = { + hasTool: (name) => available.has(name), + hasArgument(name, path) { + let schema: Record = toRecord( + schemas.find((value) => value.name === name)?.inputSchema + ) + for (const key of path.split('.')) { + schema = toRecord(toRecord(schema.properties)[key]) + if (!Object.keys(schema).length) return false + } + return true + }, + async call(name, args) { + return managedMcpPayload( + await client.callTool( + { name, arguments: args }, + { signal: AbortSignal.timeout(5000), timeoutMs: 5000 } + ), + name.startsWith('notion') ? 'Notion' : 'Lucid' + ) + }, +} +const browse = (provider: string, mode: string, extra = {}) => ({ + query: '', + limit: 10, + scopes: [], + native: nativeSearchQuerySchema.parse({ provider, query: '', browse: mode, ...extra }), +}) +beforeAll(async () => { + await protocol.connect(transport) + await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)) + client = new McpClient({ + config: { + id: 'discovery-fixture', + name: 'Discovery', + transport: 'streamable-http', + url: `http://127.0.0.1:${(server.address() as AddressInfo).port}/mcp`, + authType: 'none', + }, + resolvedIP: '127.0.0.1', + securityPolicy: { requireConsent: false, auditLevel: 'none' }, + }) + await client.connect() + await client.listTools() +}) +beforeEach(() => { + calls = 0 + requests.length = 0 + mode = '' + restricted = false + ai = false + available = new Set(schemas.map((value) => value.name)) +}) +afterAll(async () => { + await client?.disconnect() + await protocol.close() + await transport.close() + await new Promise((resolve) => { + server.close(() => resolve()) + server.closeAllConnections() + }) +}) + +describe('Provider discovery over real MCP HTTP', () => { + it('lists Lucid root folders, continues empty document pages, and reads a selected child folder', async () => { + const first = await searchLucidMcp(reader, browse('lucid', 'folder')) + expect(first).toMatchObject({ + documents: [], + folders: [{ id: '42', name: 'Architecture' }], + nextCursor: 'page-two', + }) + calls = 0 + const next = await searchLucidMcp( + reader, + browse('lucid', 'folder', { cursor: first.nextCursor }) + ) + expect(next.documents.map((item) => item.id)).toEqual([ID]) + calls = 0 + const child = await searchLucidMcp(reader, browse('lucid', 'folder', { project: '42' })) + expect(child.documents.map((item) => item.id)).toEqual([ID2]) + }) + for (const failure of ['oversize', 'invalid-cursor']) + it(`rejects ${failure} folder pages instead of issuing a lossy continuation`, async () => { + mode = failure + await expect(searchLucidMcp(reader, browse('lucid', 'folder'))).rejects.toThrow() + }) + it('excludes shortcuts and malformed folder identities with incomplete coverage', async () => { + mode = 'unsafe' + const page = await searchLucidMcp(reader, browse('lucid', 'folder')) + expect(page).toMatchObject({ documents: [], partial: true }) + expect(page.folders ?? []).toEqual([]) + }) + for (const list of ['private', 'shared', 'favorites', 'recent']) + it(`browses Notion ${list} with cursor without guessing keywords`, async () => { + const first = await searchNotionMcp(reader, browse('notion', list)) + expect(first.documents.map((item) => item.id)).toEqual([ID]) + expect(first.nextCursor).toBe('offset:1') + calls = 0 + const next = await searchNotionMcp( + reader, + browse('notion', list, { cursor: first.nextCursor }) + ) + expect(next.documents.map((item) => item.id)).toEqual([ID2]) + expect(requests.some((call) => call.name === 'notion-search')).toBe(false) + }) + it('refuses a browse tool missing from the connection', async () => { + available.delete('notion-list-private-pages') + await expect(searchNotionMcp(reader, browse('notion', 'private'))).rejects.toThrow( + /unavailable|advertise/i + ) + }) + it.each([-12, 14])( + 'retains boundary candidates with provider day offset %s and returns current timestamps', + async (offset) => { + providerOffset = offset + mode = 'date-boundary' + const page = await searchNotionMcp(reader, { + query: '', + limit: 10, + scopes: [], + filters: { + startDate: '2026-09-20T01:00:00+14:00', + endDate: '2026-09-21T01:00:00+14:00', + sortBy: 'newest', + }, + }) + expect(page.documents.map((document) => [document.id, document.modifiedAt])).toEqual([ + [ID, '2026-09-19T11:00:00Z'], + [ID2, '2026-09-20T10:59:59Z'], + ]) + expect(requests.find((call) => call.name === 'notion-search')?.args.filters).toBeTruthy() + expect(requests.find((call) => call.name === 'notion-search')?.args.sort).toBe('last_edited') + } + ) + it.each(['restricted-parent', 'restricted-child'])( + 'refuses date-only search when the plan restricts %s', + async (restriction) => { + mode = restriction + restricted = true + await expect( + searchNotionMcp(reader, { + query: '', + limit: 10, + scopes: [], + filters: { startDate: stamp, sortBy: 'newest' }, + }) + ).rejects.toThrow(/plan|supported|available/i) + } + ) + it('preserves topical AI routing and reports dropped provider constraints', async () => { + ai = true + mode = 'notice' + const page = await searchNotionMcp(reader, { query: 'launch decisions', limit: 10, scopes: [] }) + expect(page.documents[0]?.id).toBe(ID) + expect(page.partial).toBe(true) + expect(requests.some((call) => call.name === 'notion-ai-search')).toBe(true) + }) +}) diff --git a/apps/sim/lib/sim-search/live/providers.ts b/apps/sim/lib/sim-search/live/providers.ts index 177e8613ee5..1f74acef8f5 100644 --- a/apps/sim/lib/sim-search/live/providers.ts +++ b/apps/sim/lib/sim-search/live/providers.ts @@ -64,7 +64,7 @@ export const LIVE_SEARCH_PROVIDERS = { transport: 'managed_mcp', guide: { syntax: - 'Nonempty document-title keywords, at most 400 characters. Results are relevance-ranked, not guaranteed literal title matches. The provider returns at most 200 relevance-ranked candidates; Sim verifies metadata for at most 10. Search has no continuation and cannot enumerate the account. For a browse-all request without a title or topic, ask for one instead of guessing keywords.', + 'Nonempty document-title keywords, at most 400 characters. Results are relevance-ranked, not guaranteed literal title matches. The provider returns at most 200 relevance-ranked candidates; Sim verifies metadata for at most 10. Title search has no continuation. To discover available documents without guessing keywords, use native query {provider: lucid, query: empty string, browse: folder}; omit project for the root folder, or pass a returned numeric folder ID. Each page lists direct children, with child folders in account coverage; follow the cursor with the same account, mode, project, filters and topK. Do not claim recursive or whole-account completeness.', scope: 'kind lucidchart or lucidspark selects a product; omit to search both. To search shape text within a known document, set project to its UUID or Lucid URL and use one literal substring of at most 200 characters. Dates use modification time; sorting and end dates apply only to retrieved candidates, not the entire account.', example: 'deployment architecture', @@ -264,9 +264,9 @@ export const LIVE_SEARCH_PROVIDERS = { transport: 'managed_mcp', guide: { syntax: - 'Natural-language or plain keyword content search through Notion MCP. Search terms are required even with dates or sorting; this search cannot enumerate the account. For a browse-all request without a title or topic, ask for one instead of guessing keywords. Availability depends on the connected account and plan; results are restricted to Notion pages, excluding connected apps.', + 'Natural-language or plain keyword content search through Notion MCP. For navigation without a topic, use a queryless native browse mode: private or shared for sidebar pages, favorites for pinned pages, recent for recently viewed pages. These bounded, paginated lists are not an exhaustive workspace inventory; recent is not last modified. Follow the account cursor with the same browse mode, account, filters and topK. Availability depends on the connected account and plan; results are restricted to Notion pages, excluding connected apps.', scope: - 'project optionally takes a known Notion page URL when the advertised tool supports page scoping. Dates use explicit last-edited timestamps; results without those timestamps cannot satisfy date filters. Read a result for page content.', + 'project optionally takes a known Notion page URL when the advertised tool supports page scoping. Modification filters and newest sorting are pushed to the provider only if the advertised schema and plan support them. Date-only search requires those capabilities; use a sidebar browse mode or terms otherwise. Exact date checks use freshly fetched last-edited timestamps for at most 10 candidates; oldest is local ordering, not global oldest discovery. Read a result for page content.', example: 'deployment rollback checklist', avoid: 'Treating REST title search as full-content search, unsupported boolean qualifiers, claiming exhaustive results, or assuming advanced filters were applied when the provider reports they were dropped.', diff --git a/apps/sim/lib/sim-search/live/types.ts b/apps/sim/lib/sim-search/live/types.ts index 54afa4a6704..7f582aabfa5 100644 --- a/apps/sim/lib/sim-search/live/types.ts +++ b/apps/sim/lib/sim-search/live/types.ts @@ -61,6 +61,8 @@ export interface NativeDocument { export interface NativePage { documents: NativeDocument[] + /** Provider folder references are navigation hints, not readable document identities. */ + folders?: { id: string; name: string }[] /** Continues this exact query and account; implies more results exist. */ nextCursor?: string /** More matches exist beyond this page but cannot be continued through it. */ diff --git a/scripts/check-unused-exports.baseline.json b/scripts/check-unused-exports.baseline.json index f1bdccd4cae..7525802641b 100644 --- a/scripts/check-unused-exports.baseline.json +++ b/scripts/check-unused-exports.baseline.json @@ -1770,7 +1770,6 @@ "apps/sim/lib/api/contracts/memory.ts#memoryWorkspaceQuerySchema", "apps/sim/lib/api/contracts/mothership-assistant-tools.ts#liveSearchAccountStatusSchema", "apps/sim/lib/api/contracts/mothership-assistant-tools.ts#liveSearchCoverageSchema", - "apps/sim/lib/api/contracts/mothership-assistant-tools.ts#nativeSearchQuerySchema", "apps/sim/lib/api/contracts/mothership-assistant-tools.ts#workspaceKnowledgeSearchResultSchema", "apps/sim/lib/api/contracts/mothership-chat-images.ts#InlineChatImageParams", "apps/sim/lib/api/contracts/mothership-chat-images.ts#InlineChatImageQuery",