|
| 1 | +--- |
| 2 | +description: How stored content (workspace files, run files, knowledge, table rows) reaches Chat's model, and the per-path policy. Read before changing a use case, route or Chat surface that returns such content. |
| 3 | +paths: |
| 4 | + - "apps/sim/lib/core/application/**" |
| 5 | + - "apps/sim/lib/mothership/content-delivery/**" |
| 6 | + - "apps/sim/lib/workspace-files/application/**" |
| 7 | + - "apps/sim/lib/knowledge/application/**" |
| 8 | + - "apps/sim/lib/table/application/**" |
| 9 | + - "apps/sim/lib/workflows/application/**" |
| 10 | + - "apps/sim/lib/mothership/chat/application/**" |
| 11 | + - "apps/sim/lib/**/application/operations.ts" |
| 12 | + - "apps/sim/lib/mothership/agent-cli/**" |
| 13 | + - "apps/sim/lib/mothership/tools/**" |
| 14 | + - "apps/sim/app/api/v2/**" |
| 15 | +--- |
| 16 | + |
| 17 | +# Sim Content Delivery to Chat |
| 18 | + |
| 19 | +Content derived from a provenance-bearing store reaches Chat's model only after **each item** it |
| 20 | +came from is admitted by the gate of the surface it leaves through. The stores are workspace files |
| 21 | +(and their versions), run files, knowledge documents and chunks, and table rows. "Derived" covers |
| 22 | +bytes, text, excerpts, chunks, cells, previews, base64, archives, rendered or compiled output, |
| 23 | +error text that quotes content, and public links. Counts, ordering, and match locations are ordinary |
| 24 | +metadata; this boundary protects supported literal secret values, not arbitrary inference. Intentional |
| 25 | +human uploads remain ordinary input and are not scanned. |
| 26 | + |
| 27 | +Only Copilot principals (`delegated` / `organization_delegated` with `serviceId: 'copilot'`) are |
| 28 | +classified. Every other caller is admitted without a provenance read, so the UI, API keys, MCP and |
| 29 | +the workflow executor see byte-identical results. |
| 30 | + |
| 31 | +## The pieces |
| 32 | + |
| 33 | +- `lib/core/application/content-delivery.ts`: the primitive. |
| 34 | + - `admitContent(principal, sources)` returns per-item verdicts. `requireAdmittedContent` is the |
| 35 | + single-item form and throws `ContentRefusedError` (403). |
| 36 | + - `admitWrittenContent` is for a committed write that returns content. A refusal or gate failure |
| 37 | + does not undo the write: audit and effects run, then the operation answers with the refusal. |
| 38 | + Settled tool results use the same deferral, but do not claim that a mutation was saved. |
| 39 | + - A source is `{ kind, id, provenance, view }`. `view` is `'complete'` (text whose every entry |
| 40 | + may appear), `{ derived: value }` (only entries occurring in `value`), `'opaque'` (bytes no |
| 41 | + redaction reaches: binary, base64, archives, images, public links), or `'excerpt'` (text already |
| 42 | + cut before admission, where a secret fragment cannot be redacted). Both gates withhold an |
| 43 | + excerpt with known secret entries; raw opaque bytes may still enter the workbench with history. |
| 44 | + - `redactBeforeCutting` projects complete text before selecting a line window. The session gate |
| 45 | + creates the existing redactor from the operation's bounded evidence only when a cut needs it; |
| 46 | + ordinary byte reads do not decrypt a second time or reload history. |
| 47 | +- Gates, installed by the surface: the **registry gate** (Chat tools, CLI, mounts) imports entries |
| 48 | + into the registry egress projects with; the **session gate** (run_code sandbox proxy) records |
| 49 | + them into the workbench history. Neither admits `unknown`, with one exception: the CLI's direct |
| 50 | + single-file download (`files get`) admits it under `unknown: 'taint'`, which latches the call's |
| 51 | + registry so the whole output is withheld and the workbench copy stays unknown. It needs a |
| 52 | + registry to latch; without one it refuses. One refused item never latches the shared registry. |
| 53 | +- `observeContentOutsideModel` reads for Chat without classifying, for content that never reaches |
| 54 | + the model (the user's own edit preview; a derived file that carries its sources' provenance). |
| 55 | + Name why at each use. |
| 56 | + |
| 57 | +## Rules |
| 58 | + |
| 59 | +1. **Every operation that admits Copilot declares `contentDelivery`**: `'sourced'` when its result |
| 60 | + can carry content from a store, `'none'` otherwise. Definition throws |
| 61 | + without it, and `copilot-route-inventory.test.ts` lists every Chat-reachable route without one. |
| 62 | +2. **A `sourced` use case accounts for what it returns**, before it loads content: call |
| 63 | + `admitContent` (with `[]` when this call returns none). Otherwise Chat gets |
| 64 | + `ContentDeliveryUnaccountedError`, and the transports turn the response into a 403. |
| 65 | +3. **Classify before loading bytes**, and exclude refused items per item. Withhold a whole result |
| 66 | + only when its items cannot be separated (one file, one render, one compile). |
| 67 | +4. **A new `sourced` route needs a case in `lib/mothership/__integration__/content-delivery.integration.ts`.** |
| 68 | + `copilot-content-routes.json` lists them; the suite fails for a listed route it does not drive. |
| 69 | + Refresh the list with `vitest run lib/api/server/routes/copilot-route-inventory.test.ts --update=all`. |
| 70 | +5. **Do not reach a store's bytes around a use case for Chat.** A helper that reads content for |
| 71 | + Chat goes through the gated use case, or names `observeContentOutsideModel` and why. |
| 72 | + |
| 73 | +## Policy |
| 74 | + |
| 75 | +U = `unknown`, E = exact with secret entries, R/C = unrecorded or clean. Each item is decided on its |
| 76 | +own unless noted. |
| 77 | + |
| 78 | +| Path | U | E | R / C | |
| 79 | +|---|---|---|---| |
| 80 | +| File download (CLI `files get`) | Delivered once; invocation withheld from the model; workbench copy stays unknown | Redacted | Delivered | |
| 81 | +| File download (run_code proxy) | Refused before bytes enter workbench history | Recorded in workbench history | Delivered | |
| 82 | +| Requested mounts | Refused before code executes, listed in `excludedInputs`; later calls remain usable | Recorded in workbench history | Delivered | |
| 83 | +| File text, version text, `files read`, grep | 403 | Redacted (grep redacts before clipping) | Delivered | |
| 84 | +| Bulk download zip | Left out, reported in the archive manifest | Left out (opaque), reported in the archive manifest | Included | |
| 85 | +| `files search` | Match location kept; clipped preview withheld | Same; read/grep provides safely redacted text | Returned | |
| 86 | +| File edit (`PATCH` content) | 403 before reading | Diagnostics redacted | Unchanged | |
| 87 | +| Chat public share activation, and seeing an existing link (any surface, run_code included) | 403 | 403 (a link leaves every gate) | Shares | |
| 88 | +| Rendered view, streamed image, inline attachments | Refused | Refused (opaque) | Delivered | |
| 89 | +| Doc compile | Actual file read refused | Evidence imported before broker bytes/errors return | Compiles, including dynamic file references | |
| 90 | +| Run file download | 403 | Redacted | Delivered | |
| 91 | +| Run read / execute / resume `base64` | `base64` removed, descriptor kept | Same | Inlined (sync runs only) | |
| 92 | +| KB chunks, search, documents, tags | Item dropped (single read: 403) | Redacted | Delivered | |
| 93 | +| KB export | Stream fails before refused source bytes; no successful archive receipt | Refused (opaque) | Streams originals/chunks with admission at each source | |
| 94 | +| Table export download | Existing unsupported Chat operation | | | |
| 95 | +| Table rows, query, read, `table_selection` | Row dropped (single read: 403) | Redacted | Delivered | |
| 96 | +| Row search | Locations returned without cell values | Same | Returned | |
| 97 | +| Table import from a file | Rows stored unknown | Per-cell entries; a row whose coercion rewrote a secret, or whose raw cells cannot be checked, is stored unknown | Exact-empty, as before | |
| 98 | +| `tools execute` | Settled result withheld, with no automatic retry | Named masking through the existing registry | Returned, subject to existing tool authorization | |
| 99 | + |
| 100 | +Search pagination and aggregate totals retain their database meaning even when an item or preview is |
| 101 | +withheld. Do not scan unrelated table rows or whole document collections to conceal counts. |
| 102 | +A refused requested mount stops code before it runs, avoiding an older file at the same sandbox path. |
| 103 | +Registry admission commits only when the complete item fits the existing capacity; a refused item |
| 104 | +must not consume capacity or permanently disable the parent registry. |
0 commit comments