Skip to content

fix(workflows): stop safeStringify node budget from silently truncating artifacts - #596

Merged
tt-a1i merged 7 commits into
openpi-dev:mainfrom
onychen:fix/workflow-artifact-node-budget
Sep 22, 2026
Merged

tt-a1i merged 7 commits into
openpi-dev:mainfrom
onychen:fix/workflow-artifact-node-budget

Conversation

@onychen

@onychen onychen commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Problem

Fixes #558. The four safeStringify callers in extensions/workflows/artifacts.ts that write workflow.json, transcripts.json, and result.json pass only maxBytes and inherit serialization.ts's DEFAULT_MAX_NODES = 20_000. That node budget is a global counter over the whole value tree, so it is reached long before the 1–2 MiB byte budget. When it is, toSerializable inserts a "[truncated: node limit]" marker and breaks — producing a structurally valid file that reads back as a clean completed run while trailing data is silently gone. No reader consumes the marker.

Reachable on default config: 12 agents each with ~202 transcript entries (the upstream ceiling TRANSCRIPT_MAX_ENTRIES = 200) writes only 11 of 12 transcripts, at 28% of the 2 MiB byte cap; the 12th agent hydrates as an empty transcript in /workflows with no notice. Every other safeStringify call in the repo already passes an explicit maxNodes (model.ts:604, progress-projection.ts:65, sandbox.ts:164,378), and persistWorkflowAgentResult uses fail-closed encodeCompleteJson — these four sites were simply missed. journal.ts:281-289 already documents this exact hazard for a sibling artifact.

Value

Terminal workflow artifacts stop losing data silently — the failure mode AGENTS.md's "runtime owns … exact terminal evidence" contract is meant to prevent. Within the valid configured range the artifacts are written in full; genuine overflow is reported, not hidden.

Approach

Make the byte budget the binding limit instead of the incidental 20k-node default. A value dense enough to reach N nodes always serializes to more than N bytes, so pinning maxNodes to each artifact's byte budget guarantees the honest, reported byte cap is what binds.

  • transcripts.json — new boundedTranscriptsArtifact helper keeps whole agents in index order until the next would exceed the 2 MiB budget (binary-searched for exactness), and reports the dropped tail via a new transcriptsOmitted: { agents, entries } field rather than letting the serializer silently drop it. Empty-transcript hydration was the most user-hostile symptom, so partial retention + a reported count is warranted here.
  • workflow.json (terminal + final manifest) and result.json — pass an explicit maxNodes equal to the byte budget. The manifest is bounded-shape metadata that never truncates within the valid agent range (delivery/status/transcriptArtifact preserved); result.json holds arbitrary script data, so genuine overflow falls back to the existing honest {truncated, reason, preview} stub → read-back downgrades to uncertain. This is the issue's second suggested direction.
  • Read/displaytranscriptsOmitted is mirrored on WorkflowDetails (like logsDropped), validated on untrusted read-back in normalizePersistedWorkflowDetails, and surfaced in buildWorkflowReport and completion alerts.

Out of scope (per the issue): serialization.ts defaults are unchanged, and retention.ts's in-memory settled projection keeps its own omitted accounting — the canonical on-disk workflow.json carries transcriptsOmitted, so a re-read restores it.

Validation

  • bunx biome format / biome lint / tsc --noEmit — clean on all changed files. Config-, docs-, and discipline-contract checks pass (internal budget, not a package-owned user choice, so no setup/README updates required).
  • New tests/extensions/workflows/artifact-node-budget.test.ts (5 tests, all pass): the reported 12-agent bug; partial-retention + reported count on 2 MiB overflow; the boundedTranscriptsArtifact unit contract; 1024-agent manifest surviving without losing delivery/status/transcriptArtifact; honest result.json overflow.
  • All 347 workflow tests pass (2 platform-skipped), including artifacts, retention, dashboard, narrator, and completion-projection.
  • Pre-existing, unrelated failures in the full suite (web/ui deps not installed → Vitest + check:web; plan-mode ANSI rendering; an installed pi-coding-agent 0.84.1 vs 0.85.1 version-drift assertion) were verified to reproduce on clean main with this change stashed.

Impact

  • User-visible: /workflows now shows an "N of M agent transcript(s) omitted" notice instead of silently rendering empty transcripts; large runs no longer read back as clean completed with missing data.
  • Model-visible context/tools: none.
  • Runtime/lifecycle: the manifest no longer drops the delivery receipt at high agent counts, so a pending background completion is redelivered on restart as intended.
  • Persisted config/data: adds an optional transcriptsOmitted field to persisted workflow.json; older records without it read back unchanged.
  • Compatibility/risk: low. Behavior changes only at the previously-silent truncation boundary; normal-range workflow output format is unchanged.

🤖 Generated with Claude Code

…ng artifacts

The four safeStringify callers that write workflow.json, transcripts.json,
and result.json passed only maxBytes and inherited serialization.ts's
DEFAULT_MAX_NODES = 20_000. That global node counter is reached long before
the byte budget, so toSerializable inserted a "[truncated: node limit]"
marker and broke — producing a structurally valid file that reads back as a
clean `completed` run while trailing data was silently gone. No reader
consumes the marker. Reachable on default config: 12 agents each with ~202
transcript entries wrote only 11 of 12 transcripts at 28% of the 2 MiB cap;
the 12th hydrated as an empty transcript in /workflows with no notice.

Pin maxNodes to each artifact's byte budget so the honest, reported byte cap
is the only binding limit (a value dense enough to reach N nodes always
serializes to more than N bytes). For transcripts.json, additionally assemble
the file with a new boundedTranscriptsArtifact helper that keeps whole agents
in index order until the byte budget is reached and reports the dropped tail
via a new transcriptsOmitted field, surfaced in the dashboard report and
completion alerts (mirroring logsDropped). The manifest and result.json fall
back to the existing honest byte-cap stub on genuine overflow.

Fixes openpi-dev#558.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the area:workflows Workflow engine, capability, skills, or tests label Sep 20, 2026

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

节点预算修复和磁盘回读路径本身是对的,但新增的省略证据还没有穿过 settled/completion 内存投影。请补齐这个降级路径及回归测试后再合并。

Comment thread extensions/workflows/completion-projection.ts
…on projections

Addresses PR review: the omission evidence stopped at the disk read-back and
never reached the in-memory projections. The settled projection
(projectWorkflowDetails) rebuilds WorkflowDetails from a field whitelist that
dropped transcriptsOmitted, and completionEnvelope() builds its envelope from
that projection — so a restored or evicted run lost the notice, and the
completion alert never fired.

- retention.ts: pass transcriptsOmitted through makeProjection's candidate
  (like logsDropped), and add it to the byte-pressure drop list so the bound
  still converges under an extreme budget.
- completion-projection.ts: surface it in the expanded operator report next to
  the transcripts artifact (the collapsed alert was already added).
- tests: assert it survives projectWorkflowDetails and appears in both the
  alerts and expanded evidence of buildWorkflowCompletionDisplay.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@onychen

onychen commented Sep 20, 2026

Copy link
Copy Markdown
Contributor Author

@tt-a1i 已补齐降级路径,见 b63a3a7:

  • retention.ts:transcriptsOmitted 现在穿过 makeProjection 的 candidate(与 logsDropped 一致),并加入字节压力下的可丢弃白名单,保证极小预算仍能收敛。这也让 completionEnvelope() 经由 projectWorkflowDetails 构造的信封保留该证据,重启恢复/驱逐后不再丢失。
  • completion-projection.ts:在展开的 operator report 里紧挨 transcripts artifact 补了一行(折叠态 alert 上一版已加)。
  • 回归测试:断言该字段能穿过 projectWorkflowDetails,并同时出现在 buildWorkflowCompletionDisplay 的 alerts 与 expanded 证据中。

全部 349 个 workflow 测试通过(2 个平台跳过),format/lint/typecheck 干净。

@onychen

onychen commented Sep 20, 2026

Copy link
Copy Markdown
Contributor Author

@tt-a1i 刚看到 Background terminals (Windows) check 失败了,排查了一下:

  • 失败的是 tests/extensions/ai-providers/antigravity.test.ts:1183streamAntigravity bounds the full SSE lifetime after the first event,报的是 Timed out waiting for the first SSE event(预期匹配 /next SSE event/)。
  • 这个用例用真实定时器把 timeoutMs 压到 5ms 去竞速一个 mock 的 ReadableStream(同文件里的模式),本身就对调度抖动很敏感;Windows runner 在这次跑动里显然被挤到超过 5ms 才推进到第一个事件。
  • 这个文件和 extensions/ai-providers/** 都不在本 PR 的 diff 里(git diff main --stat 只涉及 extensions/workflows/* 和新增测试),本地 checkout 对照 upstream/main 复现同样的用例逻辑,和这次改动无关,是环境性的偶发超时,不是本 PR 引入的回归。
  • 我没有仓库的 admin 权限,gh run rerun --failed 提示需要仓库管理权限,麻烦您帮忙重跑一下这个失败的 job?其余 5 个 check(Node 22/24/26、Web E2E、Apply area labels)都是绿的。

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

审查提交:b63a3a7629b6a6fedfd035729ddeb86ee42d442e

本轮未发现新的阻断问题。以下结论仅覆盖所列验证范围,不是合并批准。

验证范围

15 artifact-node-budget/retention/completion tests pass. Latest head copies transcriptsOmitted into settled projection and covers completion display; prior retention finding addressed.

限制与后续

No full-suite run. Existing safeStringify depth/string truncation is unchanged and not claimed fixed.

已核对的 CI 失败

Run 35496500076:Windows:Antigravity SSE lifetime 测试期望 next SSE event,却在 first SSE event 超时;失败位于本 PR 未修改的 provider 用例,尚未证明是偶发还是底层回归。

…ail view

The omission evidence only reached saved report.md (buildWorkflowReport),
the completion alert, and disk. The interactive detail page (renderDetail)
never read the field, so after transcripts.json overflows its byte budget a
user opening a dropped agent still saw an empty transcript labeled "this run
predates transcript capture" — the exact misdirection issue openpi-dev#558 targets.

- renderDetail now shows a run-level omission notice, reserving its row so the
  exact-height layout is preserved.
- The transcript view's emptyText is run-aware: a dropped agent reads as
  "omitted ... to stay within its byte budget" instead of "predates capture".
- Regression test drives WorkflowDashboard.render() through both surfaces.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@onychen

onychen commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

补充一个自查时发现的相关问题,已一并修复(commit 6e17db9)。这个不在之前的 review 里,是我复查这次改动的 UI 影响面时找到的。

问题transcriptsOmitted 之前只到达了三个地方——磁盘 workflow.json、completion alert、以及用户按 s 保存的 report.mdbuildWorkflowReport)。而 /workflows 交互式详情页走的是 renderDetail,它根本没读这个字段。所以 transcripts.json 超过 2 MiB 后,用户进详情页、打开被丢弃的 agent,仍然看到空 transcript,而且 transcript 视图的默认文案是 "this run predates transcript capture"——正是 #558 想消除的误导,只不过之前变成了"磁盘/alert 有证据、主界面没有证据、而且主界面还在说错话"。

修复extensions/workflows/dashboard.ts):

  1. renderDetail 现在在 run 级别显示省略提示(N of M agent transcript(s) omitted from transcripts.json (byte budget); full data on disk.),并为这一行预留高度,保持原有的精确行高布局不变。
  2. transcript 视图的 emptyText 改为 run-aware:当 details.transcriptsOmitted 存在时,被打开的空 transcript 显示 "omitted ... to stay within its byte budget",不再是错误的 "predates transcript capture"。

回归测试tests/extensions/workflows/dashboard.test.ts):新增 WorkflowDashboard.render() 用例,直接驱动详情页与 transcript 两个界面——断言详情页出现省略提示、行高仍精确、transcript 空态文案改为字节预算措辞且不含 "predates"。

验证:dashboard 套件 40/40 通过;全部 workflow 测试 0 失败(execute.e2e 的 2 个差值是既有的平台 skip);format / lint 干净;typecheck 无新增错误(TuiMouseEvent 那条在干净 main 上就存在,是 pi-tui 版本漂移,与本改动无关)。

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

复审提交 e5373b988cb26f5c1f628d8106f376c17cc6da7b

未发现新的阻断问题。新增投影将 transcriptsOmitted 带到运行详情,详情页给出 run 级别的省略数量,进入空 transcript 时也不再错误显示为“旧运行尚未捕获”;高度预算同步扣除提示行,测试覆盖精确布局和文案。当前所有远端检查通过。

验证边界:分支目前落后于 main,合并前仍需更新并确认最终 head;本结论不扩大到既有 safeStringify 深度或字符串截断语义。

@tt-a1i
tt-a1i merged commit cc831a7 into openpi-dev:main Sep 22, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:workflows Workflow engine, capability, skills, or tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

workflows: transcripts.json / workflow.json 复用 safeStringify 默认 maxNodes=20_000,达到上限后静默丢弃 agent transcript 与终态字段

2 participants