Skip to content

fix(runtime): accept sequence observations and share bound execution - #5475

Open
testikun wants to merge 5 commits into
apache:mainfrom
testikun:codex/cu-observation-lease
Open

testikun wants to merge 5 commits into
apache:mainfrom
testikun:codex/cu-observation-lease

Conversation

@testikun

@testikun testikun commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #5474. After element_sequence returned a fresh observation, its next bound action was rejected as reobserve_required: the frame was registered but the session was not activated. Accept all six fresh-observation paths with a pre-capture lease check and synchronous frame/session activation. This also declines captures invalidated while in flight.

A separate second commit shares the existing bound-action execution protocol between a single semantic action and each sequence step: recheck the action lease after presentation, preserve partial delivery as outcome_unknown, and apply the same frame retirement/confirmation rules. Sequence still owns target lookup, progress, capture policy and response formatting; ordinary type/key is unchanged. Host/fake-backend timing checks do not establish real OS input delivery. Refs #4909; independent of open #5458.

Verification

  • On unmodified main, the new closing-observation regression failed with actual: reobserve_required; after the fix it passes. The presentation-stop and partial-delivery regressions also failed before the second commit and pass after it.
  • 148 tests across eight runtime CU test files and the computer-use cross-layer contract passed after rebuilding the affected workspaces.
  • npm run lint, npm run format:check, npm run build, npm run typecheck, npm run check:asf-headers, and both prescribed knip workspace checks passed.
  • Local npm test was stopped at user request to avoid a large suite; before interruption, several concurrent runtime-host tests reported readiness/terminal timeouts. The first remote test run exposed a new-Turn observe lease regression in this PR; it failed the cross-layer contract, was fixed in the first commit, and the focused cross-layer test is green locally. The corrected head passed remote build, typecheck, and affected standard workspace tests. The later Runtime Host test step failed on runtime-resource-process with a 60-second PTY control-close timeout (not in the changed code). This account lacks Actions rerun permission, so a maintainer needs to rerun that job. The full suite is not claimed green pending a privileged rerun of the Runtime Host step. No real Desktop/model/OS CU reproduction was run.
  • Ablation: removing the presentation-time lease recheck dispatched a stopped second step; removing observation lease validation published an invalidated closing frame and accepted a stopped wait result. Both checks were restored.

AI use

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Codex (OpenAI) authored the scoped runtime changes, regression tests, and this description; the contributor reviews and owns submission. Both commits carry Generated-by trailers.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actions github-actions Bot added the effort/L Under 1000 readable lines label Sep 18, 2026
@testikun

Copy link
Copy Markdown
Contributor Author

Status for maintainers: this PR is mergeable and has no review threads. The hosted Runtime Host step failed only in the unchanged runtime-resource-process PTY integrity test after its 60s close wait; the affected Computer Use build and focused tests passed. I tried gh run rerun 35312795350 --failed, but GitHub requires repository admin rights for this account. Please rerun that failed check and review the PR; no unrelated CI-only commit has been added.

Generated-by: OpenAI Codex
Comment thread packages/runtime/src/computer-use-tools.ts
@testikun

Copy link
Copy Markdown
Contributor Author

Synced current apache/main into head 74fdb22 on September 20, 2026. The merge was conflict-free and the effective PR diff remains limited to the two Computer Use runtime files. GitHub now reports MERGEABLE; fresh CI is running. Local full build was blocked by an unrelated current-main storage typing failure in read_usage, outside this PR diff. Please review the updated head.

@me2seeks

Copy link
Copy Markdown
Contributor

Review:推荐合并(Approve with nits)

我按对抗式复核逐调用点比对了 packages/runtime/src/computer-use-tools.ts 的改动,未发现高危正确性回归,重构在各调用点行为等价。这是一个质量较高的修复,推荐合并。

复核确认的点

  • blocked_url / user_stopped 区分没有丢:base 里 if (activated.status !== 'active') 那段其实是死代码——freshObservationSucceeded() 只在 !canObserve() 时不迁移,而前一行 validateObservationLease(...).ok 已要求 canObserve(),所以 activated.status 恒为 'active'。新代码用 state.beforeAction() 兜底,blockReason() 仍能把 blocked_url / user_stopped 正确映射出来。
  • observingRecord 提前计算是必要的:新回合 sessionObservationreobserveRequired() 推进 generation;base 里租约在 bump 之前取、且 registerObservation 不校验租约,属于「碰巧能跑」。重排后配合 acceptObservation 的租约校验才是自洽的。
  • finish() 不会双调也不会泄漏runWithPresentationfinished 幂等标志,executeBoundAction 每条分支恰好调一次。
  • 测试覆盖充分:新增 9 个回归用例(closing observation 可被下一步使用、in-flight 失效捕获被拒、部分投递保留为 outcome_unknown、异常释放 presentation 等),PR 说明里还做了 ablation 验证。

两个低危 nit(不阻塞合并)

  1. observingRecord! 非空断言偏脆弱computer-use-tools.ts:2313observingRecord!,其赋值在 :1667,中间隔了约 150 行控制流。当前确实安全(该行在 if (input.action === 'observe') 块内,且 sessionObservation 不会返回 undefined),但断言依赖远距离不变量。建议改成局部 const record = observingRecord! 加注释,或把 record 作为参数传下去。

  2. 缺少真实 OS 输入投递验证:PR 自述「host/fake-backend 时序检查不能证明真实 OS 输入投递」,且「未跑真实 Desktop/model/OS CU 复现」。这是已知证据缺口而非缺陷,合入前值得在真实环境跑一次序列操作。

关于 CI

本地 npm test 被中断、远端 Runtime Host 步骤在未改动的 runtime-resource-process PTY 用例上超时——这些都不在本次改动范围内,最新 head 的 build / typecheck / 受影响 workspace 测试已通过。建议由有权限的维护者 rerun 一次 Runtime Host 步骤后合并。

@me2seeks me2seeks left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

推荐合并。逐调用点复核未发现高危正确性回归,重构行为等价;仅两个低危 nit(observingRecord! 远距离非空断言、缺真实 OS 输入验证),不阻塞。详见上方 review 评论。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/L Under 1000 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(runtime): sequence closing observation cannot be used for the next action

2 participants