From 383522bcb305f5ea267e0f59badbd6110d3bd5cf Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Tue, 29 Sep 2026 13:18:46 -0700 Subject: [PATCH] docs(skills): clarify workflow validation and execution diagnostics --- skills/build-workflow/SKILL.md | 12 ++++++++++++ skills/deploy-workflow/SKILL.md | 3 +++ skills/run-tool/SKILL.md | 15 +++++++++++++-- skills/run-workflow/SKILL.md | 29 ++++++++++++++++++----------- 4 files changed, 46 insertions(+), 13 deletions(-) diff --git a/skills/build-workflow/SKILL.md b/skills/build-workflow/SKILL.md index f9ba477..45f0dd4 100644 --- a/skills/build-workflow/SKILL.md +++ b/skills/build-workflow/SKILL.md @@ -24,6 +24,9 @@ are regenerated; do not recreate that flow with a sequence of graph edits. asked for a new workflow. - Read an existing draft with `sim --output json workflows state get ` before editing it. Preserve blocks, edges, variables, and deployment state outside the requested change. +- State reads preserve configuration for editing and can contain secrets. Keep that state private; + `workflows export` is a sanitized portable representation that clears credentials and, by default, + workspace resource bindings. Use state reads for in-place edits. - If the workflow is locked or read-only, stop instead of attempting an alternate mutation path. ## Design before encoding @@ -139,10 +142,15 @@ are regenerated; do not recreate that flow with a sequence of graph edits. `sim --output json blocks get `. - Use the returned block id, operation ids, input ids, modes, conditions, credential fields, and outputs exactly. Never invent them from a display name or underlying tool id. +- Preserve an existing block's saved type/version when editing; an unversioned catalog lookup can + resolve a newer definition. - Inspect `tools list` or `tools get` only when the block response points to a tool and its parameter or output contract is needed. - Resolve credentials and resource identifiers before writing them into a graph. Do not embed raw secrets in an operations file. +- In Function code, use the supported `{{KEY}}` secret references (JavaScript: + `const key = {{KEY}};`) within the configured secret scope. Do not assume workspace secrets appear + in `process.env`, or widen selected-secret access to work around a missing reference. - Discover trigger behavior from the catalog. A service trigger may be an integration block with trigger mode enabled, while a built-in trigger may have its own block type; never substitute a trigger configuration id for a block id. @@ -232,6 +240,10 @@ prefix; `params.name` does. ## Apply atomically, then verify +Use `--dry-run --atomic` to validate a proposed batch when needed. Validation executes no blocks +and does not suppress writes during a later run. Dry-run `previewBlockIds` are provisional; later +requests must use the committed response's `mintedBlockIds`. + Pass the batch as inline JSON on the first attempt and apply it once with atomic behavior. Do not create a staging file preemptively: diff --git a/skills/deploy-workflow/SKILL.md b/skills/deploy-workflow/SKILL.md index af17211..b547644 100644 --- a/skills/deploy-workflow/SKILL.md +++ b/skills/deploy-workflow/SKILL.md @@ -13,6 +13,9 @@ post-deployment state. - Read the draft and current deployment before changing anything. - Confirm the draft has no required-field lint issues or unresolved credentials and has completed an appropriate manual run. +- Check prior edit responses for skipped operations and dropped inputs, then reread the saved draft + to verify the intended blocks, connections, and enabled states. Deployment does not establish + that earlier edits succeeded or that temporary test changes were restored. - If a live deployment already exists, explain whether this publishes a newer draft or changes its access configuration. - Never create, rotate, or reveal an API key unless the user separately asked for key management. diff --git a/skills/run-tool/SKILL.md b/skills/run-tool/SKILL.md index 1687b49..71c9959 100644 --- a/skills/run-tool/SKILL.md +++ b/skills/run-tool/SKILL.md @@ -15,6 +15,9 @@ Never put a live credential in the command. `sim --output json tools get `. Never guess a tool id or a parameter name. - An unversioned name resolves to the newest version visible in the workspace, and the response echoes the id that answered. Use that id in the call. +- Use a Function block for code requiring workflow execution context. An executor-delegation + error from `function_execute` is not a missing user credential; do not invent internal context + fields or change authentication to bypass it. ## Bind auth from the declaration, not from habit @@ -40,6 +43,10 @@ Then bind the credential by the tool's own shape: supplies its own and bills the workspace. - Otherwise the tool takes its own `user-only` key parameter; pass a `{{VAR_NAME}}` reference. +Your Sim login governs platform access; the selected provider credential determines the external +identity. A different OAuth connection can act as a different bot or user. Do not substitute one +to bypass an ownership error or a mismatch between advertised and accepted authentication modes. + `--input` accepts exactly what `tools get` publishes as yours to send. An undeclared key, a `hidden` one, or a credential under any name is a `400` that names the offending field — read it rather than guessing at a spelling. @@ -53,11 +60,15 @@ sim --output json tools execute --credential-id --input '{"...": " `--input` also accepts `@path` or `@-`, which is how a payload too large or too awkward to quote reaches the command. -A tool that ran and refused exits non-zero with `status: "failed"` and the reason in `error.message`: -the call reached the service and the service declined. Report that reason. A `403` carrying +A tool result with `status: "failed"` exits non-zero and reports its reason in `error.message`. +Read that reason to distinguish execution setup failure from a provider refusal; the status alone +does not prove the service received the request. A `403` carrying `error.details.code` `INTEGRATION_NOT_ALLOWED` is a workspace policy decision, not a fixable argument — say so and stop. +Tool results follow the provider's pagination contract. When more pages remain, use the published +cursor before reporting a total; the CLI's resource-list pagination does not paginate tool results. + ## Invariants - Never print a resolved secret, an OAuth token, or a profile credential. You hold a credential id diff --git a/skills/run-workflow/SKILL.md b/skills/run-workflow/SKILL.md index b8f8c14..d96eb01 100644 --- a/skills/run-workflow/SKILL.md +++ b/skills/run-workflow/SKILL.md @@ -44,19 +44,18 @@ sim --output json workflows run \ Do not guess a source run or synthesize upstream outputs. Confirm that the source run belongs to the workflow and contains the state the selected block needs. -## Runs longer than about a minute +## Long runs and uncertain responses -A manual run holds its HTTP connection open for the entire execution, and `--manual` refuses -`--async`. On hosted deployments the fronting load balancer drops idle connections after roughly a -minute (observed; re-verify on the current deployment), and a dropped connection cancels the run. -The failure surfaces as a transport error such as `Could not reach : fetch failed`, not as -a timeout, so it reads like network flakiness. It is not: retrying the same synchronous run fails -the same way, and a sequence of such attempts corrupts the evidence - a deterministic workflow -starts looking nondeterministic because most of its recorded attempts are transport casualties. +Current CLI and server versions use heartbeat responses for ordinary synchronous runs, including +`--manual`. A long-running draft does not require deployment. Use `--follow` when live progress +helps; it does not make execution asynchronous. For an intended deployed run, use `--async` and +`workflows runs wait --workflow --wait-timeout 3600`, or another explicit bound. -When a workflow can plausibly exceed a minute, deploy it and run `--async`, then wait with -`workflows runs wait` or poll `workflows runs get` with a stopping bound. Reserve synchronous -manual runs for graphs that finish quickly. +After a timeout or connection loss, inspect the known run before starting another execution; +external actions may already have occurred. Ordinary runs without `--follow` accept `--run-id` +to choose the identifier before sending. It is not an idempotency key: a claimed ID conflicts +rather than replaying the result, and a missing run record does not prove nothing executed. +Do not automatically retry with a fresh or omitted ID. ## Keep output focused @@ -65,6 +64,9 @@ manual runs for graphs that finish quickly. when the user needs those diagnostics. - Use `--async` only for deployed runs that should return immediately. Then wait with `workflows runs wait` or inspect with `workflows runs get`; do not poll without a stopping bound. +- Check `sim logs get --help` before using optional diagnostic flags. When it lists + `--no-include-workflow-state`, use that flag to omit the saved graph from a log read. The trace + still loads; use selected-output reads when only particular results matter. ## Diagnose failures @@ -75,6 +77,11 @@ manual runs for graphs that finish quickly. 4. Correct the graph with the build skill. Do not hide a deterministic failure behind retries or a different execution mode. +Request acceptance, workflow completion, and successful tool results are different outcomes. +A workflow can complete after handling a failed tool call. Inspect nested tool errors and the +actual output before concluding that research or delivery succeeded; an absent trace is not proof +of success. + Four properties of runs and run records that mislead diagnosis when unknown: - A run record has a lifecycle. `logs get` returns NOT_FOUND for a run that is still in flight and