Merge staging into Oracle Fusion foundation

Retain the Oracle origin and credential safeguards while respecting staging test cleanup. Adopt shared test mocks and keep unused internal types private for current audit compatibility.

Author: Bill Leoutsakos

.agents/skills/add-block-preview/SKILL.md

@@ -53,7 +53,7 @@ To pull an already-GA block from discovery surfaces on hosted (incident, depreca
 - **Clone-not-remove:** gated blocks stay in `getAllBlocks()` output as clones with `hideFromToolbar: true` — `.find`-by-type consumers rely on this. Never filter them out.
 - **Keys are registry block types.** Never `custom_block_*` (parse drops them — custom blocks have their own enabled/disabled lifecycle).
 - **The shared hidden-predicate is `isHiddenUnder`** (`apps/sim/blocks/visibility/context.ts`). Never restate the preview/disabled rule inline at a new consumer.
-- **Process-global caches stay ungated.** `getStaticComponentFiles` (VFS) and `getExposedIntegrationTools` build the ungated universe; per-viewer filtering happens at stamp/consumer time. Never move gating into a shared builder.
+- **Process-global caches stay ungated.** Shared builders such as `getExposedIntegrationTools` (`apps/sim/lib/integrations/tool-catalog.ts`) build the ungated universe; per-viewer filtering happens at consumer time via `isHiddenUnder`. Never move gating into a shared builder.
 - Gating is **surface hiding, not secrecy** — the full config ships in the client JS bundle. Anything truly secret cannot be a registered block.
 
 ## Tests

.agents/skills/add-block/SKILL.md

@@ -35,7 +35,6 @@ export const {ServiceName}Block: BlockConfig = {
   docsLink: 'https://docs.sim.ai/integrations/{service}',
   category: 'tools',                    // 'tools' | 'blocks' | 'triggers'
   integrationType: IntegrationType.X,   // Primary category (see IntegrationType enum)
-  tags: ['oauth', 'api'],              // Cross-cutting tags (see IntegrationTag type)
   bgColor: '#HEXCOLOR',                 // Brand color
   icon: {ServiceName}Icon,
 
@@ -63,7 +62,7 @@ export const {ServiceName}Block: BlockConfig = {
   },
 
   inputs: {
-    // Optional: define expected inputs from other blocks
+    // Required: the params the block accepts, keyed by tool param / canonical id
   },
 
   outputs: {
@@ -74,7 +73,7 @@ export const {ServiceName}Block: BlockConfig = {
 
 ## SubBlock Types Reference
 
-**Critical:** Every subblock `id` must be unique within the block. Duplicate IDs cause conflicts even with different conditions.
+**Critical:** Give every subblock a unique `id`: duplicates collide silently (the last definition wins). `blocks.test.ts` fails a duplicate within one condition unless the copies are a basic/advanced mode-swap pair, one basic plus trigger-mode copies, or all carry `canonicalParamId`. The only sanctioned cross-condition reuse is the hosted-key `apiKey` pair (`add-hosted-key` skill), where both fields deliberately share one value.
 
 ### Text Inputs
 ```typescript
@@ -129,6 +128,7 @@ export const {ServiceName}Block: BlockConfig = {
   id: 'credential',
   title: 'Account',
   type: 'oauth-input',
+  canonicalParamId: 'oauthCredential',
   serviceId: '{service}',  // Must match OAuth provider service key
   requiredScopes: getScopesForService('{service}'),  // Import from @/lib/oauth/utils
   placeholder: 'Select account',
@@ -310,7 +310,7 @@ When several fields are mutually exclusive alternatives, mark them all `required
 other paths ever get a chance to supply the value.
 
 **Constraints (block-wide):**
-- `canonicalParamId` must not equal any subblock `id` in the block.
+- `canonicalParamId` may equal only the `id` of a member of its own group, as `channel` does in the canonicalParamId Pattern below; it must never equal any other subblock's `id`. (`blocks.test.ts` enforces the case of a subblock with no `canonicalParamId`.)
 - One canonical id links exactly one basic/advanced pair for one logical parameter. Groups are keyed by canonical id across every subblock and hold one `basicId`, so two operations that each need a pair need two canonical ids.
 - All members of a group share the same `required` status.
 
@@ -370,8 +370,6 @@ Declare the **canonical** id with `type: 'json'` — the subblock ids never reac
 ```typescript
 inputs: {
   file: { type: 'json', description: 'File to upload (UserFile or reference)' },
-  // Legacy field for backwards compatibility
-  fileContent: { type: 'string', description: 'Legacy: base64 encoded content' },
 }
 ```
 
@@ -500,6 +498,7 @@ Controls which UI view shows the field.
 - `'advanced'` - Only in advanced view
 - `'both'` - Both views (default if not specified)
 - `'trigger'` - Only in trigger configuration
+- `'trigger-advanced'` - The advanced side of a trigger field (a canonical pair member, or a standalone field under the block-level advanced toggle)
 
 ### canonicalParamId Pattern
 
@@ -616,7 +615,7 @@ tools: {
 - `items` property - This is only for tool outputs with array types
 
 Block outputs only support:
-- `type` - The data type ('string', 'number', 'boolean', 'json', 'array')
+- `type` - The data type ('string', 'number', 'boolean', 'json', 'array', 'file', 'file[]', 'any')
 - `description` - Human readable description
 - `condition` - Optional visibility condition
 - `hiddenFromDisplay` - Optional flag to hide from the output display
@@ -679,7 +678,7 @@ export const ServiceV2Block: BlockConfig = {
     access: ServiceBlock.tools?.access?.map(id => `${id}_v2`) || [],
     config: {
       tool: createVersionedToolSelector({
-        baseToolSelector: (params) => (ServiceBlock.tools?.config as any)?.tool(params),
+        baseToolSelector: (params) => ServiceBlock.tools.config?.tool(params) ?? 'service_default',
         suffix: '_v2',
         fallbackToolId: 'service_default_v2',
       }),
@@ -697,7 +696,7 @@ export const ServiceV2Block: BlockConfig = {
 Register the block in `apps/sim/blocks/registry-maps.ts` — add the import and an entry to each map alphabetically:
 
 ```typescript
-import { ServiceBlock, ServiceBlockMeta } from '@/blocks/blocks/service'
+import { ServiceBlock, ServiceBlockMeta } from '@/blocks/blocks/{service}'
 
 export const BLOCK_REGISTRY: Record<string, BlockConfig> = {
   // ... existing blocks ...
@@ -726,11 +725,23 @@ export const ServiceBlock: BlockConfig = {
   docsLink: 'https://docs.sim.ai/integrations/service',
   category: 'tools',
   integrationType: IntegrationType.DeveloperTools,
-  tags: ['oauth', 'api'],
   bgColor: '#FF6B6B',
   icon: ServiceIcon,
   authMode: AuthMode.OAuth,
 
+  // Sentence rules: apps/sim/blocks/AGENTS.md → "Canvas sentences"
+  canvasPresentation: {
+    defaultTitle: 'Create Resource',
+    sentences: {
+      byOperation: {
+        create: [{ text: 'Create resource', field: 'name', core: true }],
+        read: [{ text: 'Read resource', field: 'resourceId', core: true }],
+        update: [{ text: 'Update resource', field: 'resourceId', core: true }],
+        delete: [{ text: 'Delete resource', field: 'resourceId', core: true }],
+      },
+    },
+  },
+
   subBlocks: [
     {
       id: 'operation',
@@ -748,6 +759,7 @@ export const ServiceBlock: BlockConfig = {
       id: 'credential',
       title: 'Service Account',
       type: 'oauth-input',
+      canonicalParamId: 'oauthCredential',
       serviceId: 'service',
       requiredScopes: getScopesForService('service'),
       placeholder: 'Select account',
@@ -778,6 +790,13 @@ export const ServiceBlock: BlockConfig = {
     },
   },
 
+  inputs: {
+    operation: { type: 'string', description: 'Operation to perform' },
+    oauthCredential: { type: 'string', description: 'Service access token' },
+    resourceId: { type: 'string', description: 'Resource ID' },
+    name: { type: 'string', description: 'Resource name' },
+  },
+
   outputs: {
     id: { type: 'string', description: 'Resource ID' },
     name: { type: 'string', description: 'Resource name' },
@@ -902,16 +921,16 @@ Every block declares a one-line prose summary that replaces its card's field row
 
 ```
 Slack                                    ← header (already names the block)
-Posts ⟨Ship it 🚀⟩ to ⟨#eng⟩             ← the sentence; ⟨…⟩ are live value chips
+Post ⟨Ship it 🚀⟩ to ⟨#eng⟩              ← the sentence; ⟨…⟩ are live value chips
 ```
 
 Write one `byOperation` entry per operation dropdown option (or a single `default`
 when the block has no operation dropdown).
 
-**The full authoring contract — voice, structure, and the two mistakes that break
+**The full authoring contract — voice, structure, and the four mistakes that break
 cards silently — is `apps/sim/blocks/AGENTS.md` → "Canvas sentences". Read it
-before writing any.** The two failures worth repeating here, because both are
-invisible at runtime:
+before writing any.** Two of those four are worth repeating here, because both
+are invisible at runtime:
 
 1. A clause naming only one member of a `canonicalParamId` pair drops the sentence
    for every advanced-mode user. List all members:
@@ -927,35 +946,27 @@ bun run apps/sim/scripts/check-canvas-sentences.ts --block={service}
 
 ## Generated artifacts
 
+When adding or changing `sunset.replacedBy`, run `bun run generate:block-successors` and commit
+`apps/sim/lib/permission-groups/block-successors.generated.ts`. Authorization uses this generated
+map to resolve legacy and current block IDs consistently without importing the executable registry.
+Verify it with `bun run check:block-successors`.
+
 Adding a block on its own needs no **tool metadata** regeneration — a block references existing
 tool IDs through `tools.access` and does not change any tool's shape.
 
 But if the same change also adds, edits **or removes** a tool, run `bun run tool-metadata:generate` and commit the result, or CI fails on stale artifacts. That matters here because a block's `outputs` are authored to match its tools' outputs, and the UI reads those from the generated metadata, not the executable registry — an unregenerated tool change makes the block's outputs disagree with what the panel renders. See `.agents/skills/tool-registry-boundary/SKILL.md`.
 
-A visible integration block does require the generated integration catalog and docs to be refreshed.
-After adding or changing one, run:
-
-```bash
-bun run scripts/generate-docs.ts
-bun run deployment-config:generate
-bun run integration-catalog:check
-bun run deployment-config:check
-bun run docs:check
-```
-
-The catalog check independently derives deployment metadata from the executable block registry and
-compares it with the committed `packages/deployment-config/src/integrations.json`. The deployment
-config check verifies the generated service-account facts against the canonical OAuth registry and
-catalog. `docs:check` re-renders every generated docs artifact in memory and fails on any committed
-file that differs — it runs in CI via `check:audits`, so commit the full generator output. If the
-generator also trues up pages an earlier PR left stale, commit that catch-up too; reverting it as
-"unrelated drift" makes `docs:check` fail. Review the generated diff and keep only intentional
-changes.
+A visible integration block does require the generated integration catalog and docs to be refreshed:
+`bun run tool-metadata:generate` (only when a tool changed), `bun run scripts/generate-docs.ts`,
+`bun run deployment-config:generate`, then `bun run check:audits`. Also run
+`bun run apps/sim/scripts/check-block-registry.ts origin/staging` (CI runs it outside `check:audits`). Commit the
+full generator output. For what each check verifies, see the `validate-integration` skill →
+Regenerate Derived Artifacts.
 
 ## Checklist Before Finishing
 
 - [ ] `integrationType` is set to the correct `IntegrationType` enum value
-- [ ] `tags` array includes all applicable `IntegrationTag` values
+- [ ] `{Service}BlockMeta.tags` lists every applicable `IntegrationTag` (tags live on the meta, not the block)
 - [ ] All subBlocks have `id`, `title` (except switch), and `type`
 - [ ] Conditions use correct syntax (field, value, not, and)
 - [ ] DependsOn set for fields that need other values
@@ -969,6 +980,7 @@ changes.
 - [ ] Tools.config.tool returns correct tool ID (snake_case)
 - [ ] Outputs match tool outputs
 - [ ] Block + meta registered in registry-maps.ts (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`)
+- [ ] If `sunset.replacedBy` changed: regenerated and committed the block successor map; `bun run check:block-successors` passes
 - [ ] If any tool was added, changed or removed alongside the block: ran `bun run tool-metadata:generate` and committed the artifacts
 - [ ] Ran `bun run scripts/generate-docs.ts`, reviewed the generated diff, and committed the integration catalog changes
 - [ ] `bun run integration-catalog:check` passes
@@ -990,7 +1002,7 @@ Validate the block against every tool in `tools.access`:
 2. **For each tool, verify the block has correct:**
    - SubBlock inputs that cover all required tool params (with correct `condition` to show for that operation)
    - SubBlock input types that match the tool param types (e.g., dropdown for enums, short-input for strings)
-   - `tools.config.params` correctly maps subBlock IDs to tool param names (if they differ)
+   - Each subBlock (or its `canonicalParamId`) is named exactly after the tool param it fills. A required `user-only` param that is only renamed in `tools.config.params` fails `bun run apps/sim/scripts/check-block-registry.ts origin/staging`; remap only optional or `user-or-llm` params
    - Type coercions in `tools.config.params` for any params that need conversion (Number(), Boolean(), JSON.parse())
 3. **Verify block outputs** cover the key fields returned by all tools
 4. **Verify conditions** — each subBlock should only show for the operations that actually use it

.agents/skills/add-column-type/SKILL.md

@@ -12,7 +12,7 @@ A `case 'yourtype':` outside `column-types/` fails **silently** when missed (a w
 
 ## Hard Rule: the compiler tells you what to do
 
-Do **not** hunt for places to edit. Add your type to the `ColumnType` union first and let `tsc` produce the list:
+Do **not** hunt for places to edit. Append your type's id to the `COLUMN_TYPES` array in `column-types/types.ts` first (`ColumnType` derives from it) and let `tsc` produce the list:
 
 ```bash
 cd apps/sim && bun run type-check
@@ -86,7 +86,7 @@ export function Type{Pascal}(props: SVGProps<SVGSVGElement>) {
 
 ## Step 3: Write the type file
 
-`apps/sim/lib/table/column-types/{name}.ts`. Copy the closest existing type and change what differs. Every field is required by the interface, so the compiler enumerates them for you — read the TSDoc in `types.ts` rather than guessing.
+`apps/sim/lib/table/column-types/{name}.ts`. Copy the closest existing type and change what differs. Required fields are compiler-enforced; optional hooks (`isCompatibleWith`, `salvage`, `valueForEquality`, `filterOperatorsFor`, …) default sensibly — read the TSDoc in `types.ts` before overriding.
 
 The three that are easy to get wrong:
 
@@ -132,29 +132,29 @@ Registering the *type* is compiler-enforced. Registering its *metadata* is not,
 | `column-types/types.ts` `TYPE_SPECIFIC_COLUMN_KEYS` | it is never stripped on conversion, and poisons the target type |
 | `lib/api/contracts/tables.ts` — the schema slot in all three column schemas, plus `refineColumnOptions` | zod strips it at the boundary; silently never saved |
 | `columns/service.ts` `addTableColumn` param type | callers cannot pass it |
-| A metadata-only update path (`updateColumnCurrency` is the model) + a branch in both column routes + the copilot tool | changing it on an existing column is a silent 200 no-op |
+| A metadata-only update in `lib/table/columns/service.ts` (`updateColumnCurrency` is the model) + a branch in `performUpdateTableColumn` in `lib/table/orchestration/columns.ts` | changing it on an existing column is a silent 200 no-op |
 | `column-config-sidebar.tsx` | no UI to set it |
 | `table-grid.tsx` delete-column undo + `use-table-undo.ts` restore | undo silently resets it to the default |
 
 `normalizeColumn`, `buildConvertedColumn`, and the undo snapshot read `TYPE_SPECIFIC_COLUMN_KEYS` generically, so those three are already zero-edit.
 
-**Known gap:** the metadata-only update path is ~6 near-identical copies (service + 2 routes + copilot). A `metadataUpdate` descriptor on `ColumnTypeServerDefinition` would collapse them; until that exists, copy `currency`'s.
+Copy `currency`'s service function and orchestration branch.
 
 ## Checklist Before Finishing
 
-- [ ] Added to the `ColumnType` union in `column-types/types.ts`
-- [ ] `column-types/{id}.ts` created, every interface field filled in
+- [ ] Id appended to `COLUMN_TYPES` in `column-types/types.ts`
+- [ ] `column-types/{id}.ts` created, every required field filled in
 - [ ] Registered in **both** `registry.ts` and `registry.server.ts`
 - [ ] Icon added, centered on the family's optical center, exported alphabetically
 - [ ] `migrateCellsTo` / `migrateCellsFrom` added if the stored bytes change
 - [ ] New metadata keys added to `TYPE_SPECIFIC_COLUMN_KEYS` + `FOREIGN_METADATA_VERB`
-- [ ] Unit tests for `coerce` / `isCompatibleWith` round-trips, verified to fail without the code
+- [ ] Unit tests for `coerce` / `isCompatibleWith` round-trips only if they pass the `test-audit` authoring gate, verified to fail without the code
 - [ ] Docs row added to `apps/docs/content/docs/tables/index.mdx`
 
 ## Final Validation (Required)
 
 1. **`cd apps/sim && bun run type-check`** — must be clean. If any file *outside* `column-types/` errors, that file has a hardcoded type list; fix it to read the registry.
 2. **Grep for leaks** — `grep -rnE "(===|!==) '{id}'|case '{id}':" apps/sim --include='*.ts' --include='*.tsx' | grep -v column-types/`. (All three forms: a plain `!==` and a `case` are how half of `currency`'s real branches are written.) Hits are expected; judge each. A hit is fine when it mounts a specific React component or encodes a genuinely one-off behavior (`json`'s mono textarea, `date`'s timezone-aware parsing). A hit is a **leak** when it restates something the registry could answer — an icon, a label, a colour, an operator set, a cast, a coercion. Leaks get a registry field, not a new branch.
-3. **Run the suite** — `bunx vitest run lib/table 'app/workspace/[workspaceId]/tables' lib/api app/api/table app/api/v1 lib/copilot/tools/server/table`. Existing tests must pass **unchanged**; needing to edit one means you changed behavior for the other types.
-4. **`bun run lint:check`, `bun run check:api-validation`, `bun run check:client-boundary`** from the repo root.
+3. **Run the suite** — `bun run --cwd apps/sim test lib/table 'app/workspace/[workspaceId]/tables' lib/api app/api/table app/api/v1`. Existing tests must pass **unchanged**; needing to edit one means you changed behavior for the other types.
+4. **`bun run lint`, `bun run check:api-validation:strict`, `bun run check:client-boundary`** from the repo root.
 5. **Exercise it in the running app** on a table with one column of every type: create, edit inline / in the expanded popover / in the row modal, paste from a spreadsheet, filter, sort, convert to and from other types, export CSV, undo a column delete.