diff --git a/docs/architecture/rfcs/goal-instance-identity-and-orphan-recovery-v0.md b/docs/architecture/rfcs/goal-instance-identity-and-orphan-recovery-v0.md index 23b84bf36e..c68ae6dccd 100644 --- a/docs/architecture/rfcs/goal-instance-identity-and-orphan-recovery-v0.md +++ b/docs/architecture/rfcs/goal-instance-identity-and-orphan-recovery-v0.md @@ -2,10 +2,10 @@ - **RFC status:** Accepted - **Supersedes / closes:** none -- **Delivery maturity:** Identity/recovery proposal; codec prerequisite shipped in #4917 +- **Delivery maturity:** Partial source-session lifetime and binding-owner implementation; quota settlement qualified in #5389; activation, orphan recovery and product acceptance remain held - **Authors / owners:** LoopX contributors - **Created:** 2026-09-23 -- **Last normative revision:** 2026-09-23 +- **Last normative revision:** 2026-10-02 - **Implementation baseline:** `23edcb19c70394480e3a9ebe8a960f5a320c5342` - **Related contracts:** [Issue #4801](https://github.com/loopx-project/loopx/issues/4801), [orphan fence slice #4808](https://github.com/loopx-project/loopx/pull/4808) - **Language mirror:** [Chinese semantic mirror](goal-instance-identity-and-orphan-recovery-v0.zh-CN.md) @@ -718,13 +718,50 @@ already prevents execution. | M4: orphan recovery | Reuse fence/diagnosis/path/backup owners; journaled file-state resolution with exact legacy cleanup | Preview, destructive/retry/rollback negatives and original-entry readback. Archive/delete can ship earlier without identity creation; native-provider adoption stays blocked. | | M5: migration and product acceptance | M2–M4 plus multi-Goal quiescence/reconnection; packaged frontend and qualified Lark | 2–3 workers, dependency artifact, interrupted A, recreated B and late A return; unrelated Goal progresses; B accepted independently. No duplicate protected effect. | -M0 is a prerequisite checkpoint, not the next unstarted task. The next slice -owns M1 compatibility/consumer characterization and the M2 local lifecycle seam; -do not ship a field-only milestone as the ABA outcome. M2 and M3 describe +M0 is shipped; retain the implemented source-session lifetime transaction and +qualified binding owners. Continue from the remaining owner/compatibility gaps +in the binding inventory and the M4 recovery path tracked by #5206; do not +restart the codec or count fields as the ABA outcome. M2 and M3 describe implementation order, not permission to activate a partially fenced system. R2/R3 consume the completed local slice; R6 service adoption and D1–D3 provider promotion retain their own acceptance. No new paid cohort or soak is authorized. +### Product integration through existing roadmap journeys + +Use [#5206](https://github.com/loopx-project/loopx/issues/5206) for lifetime and +recovery delivery and the existing R1–R3/G1 owners under +[#4574](https://github.com/loopx-project/loopx/issues/4574) for product adoption. +The [golden-query lifecycle variant](../../product/use-cases/steward/golden-queries.md#goal-lifetime-creation-collaboration-and-recovery) +is the common scenario specification. These are planned integration exits, not +claims that the source profile is activated or that G1/M5 has passed. + +| Priority / existing journey | Lifetime contribution and companion owner | Decisive observable result | +| --- | --- | --- | +| P0 · GQ01/02, R1/R2: create a Goal and create/reuse an Agent | Source lifecycle publishes the exact GoalRef; registry/onboarding and session owners reconcile creation retries and bind authorized work | Lost responses and repeated clicks create no duplicate Goal, Agent or executor; an old same-alias attachment cannot attach itself to the new Goal | +| P0 · GQ05/11/12, R2/R3/G1: dependent artifacts and independent review | Collaboration carries instance identity through request, adoption, result and original-conversation return; Todo/lease and quota retain their own acceptance/effect checks | Two real cycles use exact artifact versions; late A work cannot complete, debit or enter B's accepted synthesis; B and unrelated work still progress | +| P0 · GQ08/09, R2/R3: stop, correct and resume | Session/execution generation and claim/lease fence stale execution within one Goal; this RFC fences a retired Goal lifetime | Ordinary correction/reconnect/resume retains the GoalRef; explicit retirement/recreation changes it; both paths preserve the owed result without duplicate effects | +| P1 · GQ15, S7/R2/R3: mixed team within an agreed budget | #5389 isolates quota spend/replay/repair/void/readback by instance; existing quota and scheduler owners enforce team allocation | Replay does not double-charge or charge a successor; the team allowance is not copied to each worker. Instance accounting alone does not qualify shared-budget allocation | +| P1 · R5/M4/M5: recover orphaned work at create/connect entry | Existing diagnosis and lifecycle resolution drive preview, backup, apply/resume and explicit reconnection | The packaged App and CLI show the selected disposition and recovery result; no guessed candidate or silent adoption of old authority | +| P2 · GQ16, R6/G3: local/cloud work and reconnect | Authenticated service, remote binding, revocation and lease owners consume exact identity after their own qualification | A returning remote executor cannot commit or debit a successor; local-file evidence does not qualify a remote provider | + +Keep Goal lifetime, registered Agent identity, host session/execution generation +and work request/attempt identity distinct. Reusing an Agent grants no new Goal +authority; changing its model, reconnecting or opening a Turn does not mint a +Goal instance. Intent corrections and artifact revisions retain their existing +R4/work-graph owners. Identity matching neither transfers a lease nor accepts an +artifact. No new Agent factory, scheduler or parallel task ledger is needed. + +First qualify creation/reuse, two collaboration cycles and same-lifetime +interruption on an already supported profile. In an isolated qualification +environment, add the M5 retirement/recreation variant only after the selected +source profile's M2/M3 owner, old-writer and effect-drain gates are satisfied; +include M4 if orphan resolution is exercised. Component fixtures may run before +that gate but cannot certify the live journey. Use the packaged App with +independent CLI/source readback; Lark retains separate transport/audience +qualification. Record entry/retry, receiver adoption, artifact acceptance, +instance-scoped settlement and original-route return separately. #5389 qualifies +only `quota_settlement`; missing companion owners remain open in #5206. + ## 12. Open decisions and holds 1. **Supported package/profile matrix:** release and host owners pin all supported @@ -840,6 +877,33 @@ promotion retain their own acceptance. No new paid cohort or soak is authorized. drain, unsupported/warm binary coverage, or any other M3 row. `execution_authority: false` and the overall activation hold remain. +### 2026-09-30: M3 quota settlement owner candidate + +- **Baseline:** `3ec049e13`. +- **Proposed:** Bind source-profile quota spend, replay, receipt repair, void, + settlement readback, and rolling-window accounting to the caller-captured + exact GoalRef. Python hands the ordered run-index and Goal-lifecycle lock + witnesses to the TypeScript accounting owner. TypeScript validates both + witnesses and reuses `decideFirstPartyHostRuntime(require_current)`. + Single-phase accounting adopts the witnesses through artifact commit; + multi-phase monitor accounting borrows them while Python retains the + enclosing lock scope across preflight, provider writeback, and commit. +- **Evidence:** TypeScript and Python integration tests publish same-alias Goal + B after Goal A capture and prove that stale A writes nothing. They also cover + B-only spend and void, cross-instance replay and prepared-receipt repair + rejection, exact settlement readback, per-instance rolling-window accounting, + mixed alias/exact fallback history, and exact auxiliary monitor preflight, + exception, commit, and replay under one admission. +- **Compatibility:** Non-source spend, replay, void, and readback requests omit + GoalRef and source admission. Their persisted records, receipts, response + payloads, and lock behavior retain the legacy shape. +- **Remaining hold:** This qualifies only the `quota_settlement` inventory row. + The source-profile provider journey remains behind its existing activation + gate; the native monitor evidence does not claim that route is activated. + Unsupported and warm binaries, downstream external-effect drain, and every + other unqualified M3 owner remain blocked. `execution_authority: false` and + the overall activation hold remain unchanged. + ## Appendix B: Decision log | Date | Decision | Owner / approval | Alternatives | Normative sections changed | diff --git a/docs/architecture/rfcs/goal-instance-identity-and-orphan-recovery-v0.zh-CN.md b/docs/architecture/rfcs/goal-instance-identity-and-orphan-recovery-v0.zh-CN.md index 8ccb2d1c92..b345c7afd6 100644 --- a/docs/architecture/rfcs/goal-instance-identity-and-orphan-recovery-v0.zh-CN.md +++ b/docs/architecture/rfcs/goal-instance-identity-and-orphan-recovery-v0.zh-CN.md @@ -2,10 +2,10 @@ - **RFC 状态：** 已接受 - **替代 / 关闭：** 无 -- **交付成熟度：** Identity/recovery 提案；codec 前置已由 #4917 交付 +- **交付成熟度：** source-session 生命周期与 binding owner 部分实现；#5389 资格化 quota settlement；activation、孤儿恢复与产品验收仍受限 - **作者／Owner：** LoopX contributors - **创建日期：** 2026-09-23 -- **最后规范修订：** 2026-09-23 +- **最后规范修订：** 2026-10-02 - **实现基线：** `23edcb19c70394480e3a9ebe8a960f5a320c5342` - **相关契约：** [Issue #4801](https://github.com/loopx-project/loopx/issues/4801)、[orphan fence 切片 #4808](https://github.com/loopx-project/loopx/pull/4808) - **语言镜像：** [英文语义镜像](goal-instance-identity-and-orphan-recovery-v0.md) @@ -652,11 +652,40 @@ retry 必须幂等。External manual cleanup 只作提示，因为 logical revoc | M4：orphan recovery | 复用 fence/diagnosis/path/backup owner；journaled file-state resolution 与精确 legacy cleanup | Preview、destructive/retry/rollback negative、原入口 readback；archive/delete 可提前交付但不创建身份，native-provider adoption 仍阻塞。 | | M5：迁移与产品验收 | M2–M4、多 Goal quiescence/reconnection、packaged frontend、已资格化 Lark | 2–3 worker、一条产物依赖、中断 A、重建 B、迟到 A return；无关 Goal 继续，B 独立验收；无重复 protected effect。 | -M0 是前置 checkpoint，不是下一个尚未开始的任务。下一切片承担 M1 compatibility/ -consumer 刻画及 M2 本地 lifecycle seam，不能把加字段当作 ABA outcome。M2/M3 +M0 已交付；保留已实现的 source-session lifetime transaction 和已资格化 binding +owner。继续处理 binding inventory 的剩余 owner/compatibility 缺口以及 #5206 +跟踪的 M4 恢复路径；不重建 codec，也不把加字段当作 ABA outcome。M2/M3 描述实现顺序，不授权激活部分有 fence 的系统。R2/R3 消费完成的本地切片；R6 service adoption、D1–D3 provider promotion 保留各自验收。不授权付费 cohort/soak。 +### 通过既有 roadmap 旅程接入产品 + +生命周期与恢复交付继续归 [#5206](https://github.com/loopx-project/loopx/issues/5206)， +产品采用归 [#4574](https://github.com/loopx-project/loopx/issues/4574) 下既有 R1–R3/G1 +owner。[golden-query 生命周期变体](../../product/use-cases/steward/golden-queries.md#goal-lifetime-creation-collaboration-and-recovery) +是共用场景规范。以下是计划中的集成出口，不表示 source profile 已激活或 G1/M5 已通过。 + +| 优先级 / 既有旅程 | 生命周期贡献与配套 owner | 决定性的可观察结果 | +| --- | --- | --- | +| P0 · GQ01/02、R1/R2：创建 Goal，创建或复用 Agent | Source lifecycle 发布精确 GoalRef；registry/onboarding 与 session owner 核对创建重试并绑定获授权工作 | 响应丢失、重复点击不产生重复 Goal、Agent 或 executor；同名旧 attachment 不能自行接入新 Goal | +| P0 · GQ05/11/12、R2/R3/G1：依赖产物与独立复核 | Collaboration 在请求、采用、结果与原会话返回中携带实例身份；Todo/lease 与 quota 保留各自验收和效果检查 | 两轮真实协作使用精确产物版本；迟到 A 工作不能完成 B、扣 B 的额度或进入 B 已验收汇总；B 和无关工作继续推进 | +| P0 · GQ08/09、R2/R3：停止、纠偏、恢复 | Session/执行代次和 claim/lease 隔离同一 Goal 内过期执行；本 RFC 隔离已退役 Goal 的生命周期 | 普通纠偏、重连、恢复保留 GoalRef；明确退役后重建才改变它；两条路径都保留结果返回义务且不重复执行效果 | +| P1 · GQ15、S7/R2/R3：既定预算内的混合小队 | #5389 按实例隔离 quota spend/replay/repair/void/readback；团队分配仍由既有 quota/scheduler owner 强制执行 | 重放不重复扣账，也不扣继任实例；团队额度不会复制给每个 worker。实例记账本身不资格化共享预算分配 | +| P1 · R5/M4/M5：从创建/接入入口恢复孤儿工作 | 既有诊断与生命周期 resolution 驱动 preview、backup、apply/resume 和明确重连 | Packaged App 与 CLI 显示选定处置和恢复结果；不猜测 candidate，不静默继承旧 authority | +| P2 · GQ16、R6/G3：本地/云端协作与重连 | 认证服务、远端 binding、撤销和 lease owner 在独立资格化后消费精确身份 | 返回的远端 executor 不能向继任实例提交或扣账；本地文件证据不资格化远端 provider | + +区分 Goal 生命周期、注册 Agent 身份、host session/执行代次和工作请求/尝试身份。 +复用 Agent 不授予新 Goal 权限；换模型、重连或新开 Turn 不生成 Goal 实例。 +意图纠偏和产物修订继续归既有 R4/work-graph owner。身份匹配不转移 lease，也不验收 +产物。不新增 Agent factory、scheduler 或平行任务账本。 + +先在已支持 profile 上验收创建/复用、两轮协作和同生命周期中断。隔离资格化环境中的 +M5 退役/重建变体，须等待所选 source profile 的 M2/M3 owner、旧 writer 排除与 +effect-drain 门槛全部满足；涉及孤儿 resolution 时还需 M4。组件夹具可提前运行， +但不能证明真实旅程完成。以 packaged App 和独立 CLI/source 回读验收；Lark 保留 +独立 transport/受众资格。分别记录入口/重试、接收方采用、产物验收、实例内结算和 +原路返回。#5389 仅资格化 `quota_settlement`；缺失配套 owner 继续在 #5206 跟踪。 + ## 12. 未决事项与 hold 1. **受支持 package/profile matrix：** release/host owner 在 M2 activation 前固定 @@ -761,6 +790,30 @@ service adoption、D1–D3 provider promotion 保留各自验收。不授权付 binary 或其他 M3 行已完成。`execution_authority: false` 和总 activation hold 保持不变。 +### 2026-09-30：M3 quota settlement owner 候选 + +- **基线：** `3ec049e13`。 +- **候选实现：** Source profile 的 quota spend、replay、receipt repair、void、 + settlement readback 与 rolling-window accounting 均绑定调用方预先捕获的精确 + GoalRef。Python 按顺序把 run-index 与 Goal lifecycle lock witness 交接给 + TypeScript accounting owner。TypeScript 校验两个 witness，并复用 + `decideFirstPartyHostRuntime(require_current)`。单阶段 accounting 会接管 + witness 直至 artifact commit；多阶段 monitor accounting 只借用 witness， + 由 Python 在 preflight、provider writeback 与 commit 的完整外层范围内持锁。 +- **证据：** TypeScript 与 Python 集成测试在 Goal A 捕获后发布同名 Goal B， + 证明迟到的 A 不产生任何写入。测试还覆盖 B 独立 spend/void、跨实例 replay + 和 prepared receipt repair 拒绝、精确 settlement readback，以及按实例隔离的 + rolling-window accounting、alias/exact 混合 fallback 历史，以及在同一 + admission 下的 exact auxiliary monitor preflight、异常、commit 与 replay。 +- **兼容性：** 非 source 的 spend、replay、void 与 readback 请求不携带 GoalRef + 或 source admission；其持久化 record、receipt、响应 payload 和锁行为保持 + legacy 形态。 +- **剩余 hold：** 本切片只资格化 `quota_settlement` inventory 行。Source + profile provider journey 仍受既有 activation gate 限制，native monitor + 证据不表示该路径已启用。不支持及常驻 + binary、downstream external-effect drain 和其他未资格化 M3 owner 继续受阻。 + `execution_authority: false` 和总 activation hold 保持不变。 + ## 附录 B：决策日志 | 日期 | 决策 | Owner／批准 | 替代方案 | 变更的规范章节 | diff --git a/docs/architecture/rfcs/loopx-overall-roadmap-v0.md b/docs/architecture/rfcs/loopx-overall-roadmap-v0.md index 25b1c7f766..7d832bacbf 100644 --- a/docs/architecture/rfcs/loopx-overall-roadmap-v0.md +++ b/docs/architecture/rfcs/loopx-overall-roadmap-v0.md @@ -449,6 +449,29 @@ Keep existing R/G/M/A identifiers and canonical Todos; do not create a parallel roadmap, scheduler or achievement ledger. Release claims require observed results, not this plan or merged prerequisite PRs. +### Goal lifetime across creation and small-team delivery + +R1–R3/G1 adopt the [Goal-instance RFC's product integration path](goal-instance-identity-and-orphan-recovery-v0.md#product-integration-through-existing-roadmap-journeys) +through the existing [golden-query lifecycle variant](../../product/use-cases/steward/golden-queries.md#goal-lifetime-creation-collaboration-and-recovery). +Prioritize GQ01/02 creation and Agent reuse, GQ05/11/12 dependent delivery and +review, then GQ08/09 correction/stop/resume as one small-team journey. Distinguish +ordinary continuation within one Goal from explicit retirement and same-alias +recreation: registered Agent, host session/execution generation and work attempt +remain separate identities. Old work must not settle a successor's quota, +complete its work or enter its accepted synthesis; current and unrelated work +must still progress and return to the initiating conversation. + +R5 lifetime/recovery delivery stays with #5206; R1–R3 retain entry, receiver +adoption, independent acceptance and return. #5389 qualifies the quota owner, +not the whole journey. Run the ordinary pilot on a supported profile; qualify +the isolated recreation variant only after the selected profile's M2/M3 gates, +plus M4 where recovery is used. Activation and `execution_authority: false` +holds remain. P1 adds GQ15 budget allocation and orphan recovery at create/connect; +P2 adds GQ16 authenticated cross-host recovery. Packaged App and independent CLI +readback come first, with Lark separately qualified. These refine existing +acceptance, without adding a roadmap milestone or treating a prerequisite merge +as product completion. + ### Collaboration and Handoff Between LoopX Agents Participants are long-running LoopX Agents with their own goals, commitments, frontiers and execution bindings, not merely temporary subtasks inside the steward process. Manager→worker and worker→worker share one collaboration contract. Workers can request help, provide results, challenge dependencies and propose replanning without asking the steward to relay every message. The steward owns overall progress and synthesis, not a serial transit point for every message or commit. @@ -708,12 +731,16 @@ L3 checkpoint: standalone acquisition/takeover, atomic claim admission and maint - **Exit:** affected real CLI/backend, immutable baseline versus candidate comparison, negative/mutation coverage, three-arm rehearsal and applicable D2 soak of at least ten days. D3 retains explicit cutover approval. This audit runs no new soak and promotes no provider. - **Rollback:** reviewed fenced export/import and schema-aware downgrade; replacing a binary cannot restore old write authority. -The [Goal instance/recovery proposal](goal-instance-identity-and-orphan-recovery-v0.md) -adds a bounded R5 dependency for R2/R3 retirement and late-result safety. Its M0 -codec is shipped; lifetime admission, commit fencing and recovery are not. Reuse -TS transaction and existing provider owners, qualify the local path first, and -keep R6 service identity and D1–D3 promotion separate. This checkpoint does not -activate identity or require every R1–R4 change to wait for the full lifecycle. +The [Goal instance/recovery RFC](goal-instance-identity-and-orphan-recovery-v0.md) +adds a bounded R5 dependency for R2/R3 retirement and late-result safety. The +codec and source-session lifetime transaction exist; attached Chat, handoff and +Turn-journal fences have individual qualification, and #5389 adds quota +settlement. First-party host enforcement is partial; remaining inventory owners, +old-writer/effect-drain qualification, M4 recovery and M5 product acceptance stay +open in #5206. Reuse those TS/provider owners and the creation/collaboration +journey above. R6 service identity and D1–D3 promotion remain separate. This +checkpoint does not activate identity or make every R1–R4 change wait for the +full lifecycle. ### R6: Local/Cloud Convergence diff --git a/docs/architecture/rfcs/loopx-overall-roadmap-v0.zh-CN.md b/docs/architecture/rfcs/loopx-overall-roadmap-v0.zh-CN.md index ac79d75100..b5f4e8bb7e 100644 --- a/docs/architecture/rfcs/loopx-overall-roadmap-v0.zh-CN.md +++ b/docs/architecture/rfcs/loopx-overall-roadmap-v0.zh-CN.md @@ -359,6 +359,22 @@ P0 首批是负责人路由和真实 2–3-worker 协调：两轮并行汇合、 验收集在运行前冻结成对基线/候选任务、结果与注意力指标、反例和逐入口证据；所有真实运行初始都未验收。沿用 R/G/M/A 编号和 canonical Todo，不另建路线图、调度器或成绩账本。发布主张依据实际结果，不能以规划或前置 PR 合并代替。 +### 创建与小团队交付中的 Goal 生命周期 + +R1–R3/G1 通过既有 [golden-query 生命周期变体](../../product/use-cases/steward/golden-queries.md#goal-lifetime-creation-collaboration-and-recovery) +接入 [Goal-instance RFC 的产品集成路径](goal-instance-identity-and-orphan-recovery-v0.zh-CN.md#通过既有-roadmap-旅程接入产品)。 +优先将 GQ01/02 的创建与 Agent 复用、GQ05/11/12 的依赖交付与复核、GQ08/09 的 +纠偏/停止/恢复串成一条小团队旅程。区分同一 Goal 的正常继续和明确退役后的同名重建； +注册 Agent、host session/执行代次、工作尝试仍是独立身份。旧工作不能结算继任者的 +额度、完成其工作或进入其已验收汇总；当前工作和无关工作仍须推进并返回发起会话。 + +R5 生命周期/恢复交付继续归 #5206；R1–R3 保留入口、接收方采用、独立验收和返回。 +#5389 资格化 quota owner，不代表整条旅程完成。普通试点使用已支持 profile；隔离的 +重建变体须满足所选 profile 的 M2/M3 门槛，涉及恢复时还需 M4。保留 activation 与 +`execution_authority: false` hold。P1 增加 GQ15 预算分配及创建/接入时的孤儿恢复； +P2 增加 GQ16 的认证跨主机恢复。先做 packaged App 与独立 CLI 回读，Lark 单独资格化。 +这些细化既有验收，不新增 roadmap 里程碑，也不把前置 PR 合并记为产品完成。 + ## 6. 核心交付路径：R1–R7 执行卡 | 卡 | 优先级 / 可验收结果 | 硬前置 | 可同时推进但无需等待 | @@ -515,11 +531,14 @@ L3 检查点：独立领取/接管、原子 claim 准入与维护共用 typed le - **退出：** 相关真实 CLI/backend、不可变 baseline 与候选对照、负例/mutation、三臂演练及适用 D2 至少十日 soak；D3 切换保留明确批准。此次审计没有执行新的 soak，也未晋升 provider。 - **回滚：** 按已审阅的 fenced export/import 和 schema-aware downgrade，不能靠替换二进制恢复旧写权威。 -[Goal instance/recovery 提案](goal-instance-identity-and-orphan-recovery-v0.zh-CN.md) -为 R2/R3 retirement 和迟到结果安全提供有界 R5 依赖。M0 codec 已交付，lifetime -admission、commit fence 和 recovery 尚未交付。复用 TS transaction 与既有 provider -owner，先资格化本地路径；R6 service identity、D1–D3 promotion 独立验收。本检查点 -不激活 identity，也不要求所有 R1–R4 改动等待完整 lifecycle。 +[Goal instance/recovery RFC](goal-instance-identity-and-orphan-recovery-v0.zh-CN.md) +为 R2/R3 retirement 和迟到结果安全提供有界 R5 依赖。Codec 和 source-session lifetime +transaction 已存在；attached Chat、handoff、Turn journal 的 fence 已分别资格化， +#5389 补充 quota settlement。第一方 host enforcement 仍为部分实现；其余 inventory +owner、旧 writer/effect-drain 资格、M4 恢复、M5 产品验收继续在 #5206 保持开放。 +复用这些 TS/provider owner 和前述创建/协作旅程；R6 service identity 与 D1–D3 +promotion 独立验收。本检查点不激活 identity，也不要求所有 R1–R4 改动等待完整 +lifecycle。 ### R6：本地与云端汇合 diff --git a/docs/product/use-cases/steward/golden-queries.md b/docs/product/use-cases/steward/golden-queries.md index ba5bb1dcd3..e74afd993d 100644 --- a/docs/product/use-cases/steward/golden-queries.md +++ b/docs/product/use-cases/steward/golden-queries.md @@ -451,6 +451,48 @@ capacity/fairness, per-cohort cost/latency budgets and a problem with enough independent work to justify the cohort. These two are roadmap targets, not ready-to-run scale fixtures or a reason to spawn idle workers now. +### Goal lifetime: creation, collaboration and recovery + +This planned variant composes GQ01/02, GQ05/11/12 and GQ08/09 for R1–R3/G1 and +[Goal-instance RFC M5](../../../architecture/rfcs/goal-instance-identity-and-orphan-recovery-v0.md#product-integration-through-existing-roadmap-journeys). +Reuse the existing public research question or one bounded engineering delivery; +the user asks for an outcome, not instance IDs or lifecycle protocol fields. +Keep implementation tracking in #5206 and the existing R1–R3 owners under #4574. + +1. **Create/connect.** Retain the original request, create its Goal and reuse a + qualified responsible Agent; create additional workers only within the + existing authorization. Lose the creation response and retry/reload. Verify + one committed operation and no duplicate Goal, Agent, session or driver. +2. **Collaborate.** Use 2–3 real workers for two artifact/adoption/review cycles, + with one owner correction and an independent reviewer. Observe the producer, + exact consumed version, receiver decision, acceptance and synthesized return. +3. **Resume the same lifetime.** Interrupt one worker while another progresses. + Resume through the qualified binding and current claim/lease. GoalRef stays + unchanged; a stale execution generation cannot commit twice. A correction, + model change or reconnect does not create a new Goal. +4. **Retire and recreate.** In a separate disposable variant, pause old Goal A + before a result/settlement returns, retire it through its lifecycle owner, + then create B with the same alias and a different instance. Preserve A's + attachments and deliver the late result and a duplicate callback. A remains + inspectable as history; its work cannot debit B, complete B's Todo or satisfy + B's join. Historical accepted A effects remain A's; rejection is not erasure. +5. **Recover and return.** B independently completes valid work and returns its + accepted result to its initiating conversation; an unrelated Goal progresses. + Check source state, settlement and original-route return independently of + the UI. Show an actionable stale-binding/reconnection outcome without asking + the user to relay results. If orphan state is introduced, exercise M4's + preview, backup, explicit disposition, interrupted apply/resume and readback. + +The same-lifetime pilot uses an already supported profile. The live recreation +variant requires the selected source profile's full M2/M3 qualification; +orphan recovery additionally requires M4. Isolated component tests can run +earlier but do not authorize activation or certify this real-model journey. +#5389 covers the quota-settlement owner only. Keep App, CLI, Lark and provider +results separate, with passed/failed/untested evidence; existing release-only +paid-evaluation and frozen-budget rules apply. GQ15 later checks that a team +allowance is not multiplied across members; GQ16 separately qualifies remote +identity, revocation, network recovery and old-executor fencing. + ## Required variants and independent observations | Boundary | Fixture or intervention | What the observer must establish | diff --git a/loopx/capabilities/multi_subagent/cli.py b/loopx/capabilities/multi_subagent/cli.py index a9fcf0f2eb..96cf8d52e8 100644 --- a/loopx/capabilities/multi_subagent/cli.py +++ b/loopx/capabilities/multi_subagent/cli.py @@ -2,7 +2,13 @@ from __future__ import annotations +import argparse + from ...agent_registry import load_goal_from_registry, registered_agent_ids_for_goal +from ...control_plane.goals.first_party_host_admission import ( + capture_first_party_host_goal_ref, +) +from ...control_plane.goals.source_session_registry_state import exact_goal_ref from ...orchestration import compact_orchestration_policy from .native_child_receipts import load_native_child_activity, record_native_child @@ -16,6 +22,7 @@ def register_native_child_commands(subparsers, add_format): parser.add_argument("--goal-id", required=True) parser.add_argument("--agent-id", required=True) parser.add_argument("--turn-instance-id", required=True) + parser.add_argument("--goal-instance-id", help=argparse.SUPPRESS) parser.add_argument("--operation-id", help="Stable identity for one host-native child operation.") parser.add_argument("--stage", choices=("decision", "result", "review")) parser.add_argument("--operation", choices=("spawn", "followup", "skip")) @@ -31,6 +38,17 @@ def handle_native_child_command(args, registry_path, runtime_root, print_payload if args.command != "native-child": return None try: + goal_instance_id = str( + getattr(args, "goal_instance_id", None) or "" + ).strip() + goal_ref = ( + exact_goal_ref(args.goal_id, goal_instance_id) + if goal_instance_id + else capture_first_party_host_goal_ref( + registry_path=registry_path, + goal_id=args.goal_id, + ) + ) goal = load_goal_from_registry(registry_path, args.goal_id) if goal is None or args.agent_id not in registered_agent_ids_for_goal(goal): raise ValueError("coordinator is not registered for this Goal") @@ -50,6 +68,7 @@ def handle_native_child_command(args, registry_path, runtime_root, print_payload payload = {"ok": True, "native_child_activity": load_native_child_activity( runtime_root, goal_id=args.goal_id, agent_id=args.agent_id, turn_instance_id=args.turn_instance_id, configured_limit=configured_limit, + registry_path=registry_path, goal_ref=goal_ref, )} else: if not args.operation_id or not args.stage or not args.outcome: @@ -62,6 +81,7 @@ def handle_native_child_command(args, registry_path, runtime_root, print_payload entrypoint_id=args.entrypoint_id, reason_code=args.reason_code, evidence_ref=args.evidence_ref, validation_ref=args.validation_ref, execute=args.execute, + registry_path=registry_path, goal_ref=goal_ref, ) except (OSError, ValueError, KeyError, RuntimeError) as exc: payload = {"ok": False, "error": str(exc)} diff --git a/loopx/capabilities/multi_subagent/native_child_receipts.py b/loopx/capabilities/multi_subagent/native_child_receipts.py index ead59dcd0b..1044b845b5 100644 --- a/loopx/capabilities/multi_subagent/native_child_receipts.py +++ b/loopx/capabilities/multi_subagent/native_child_receipts.py @@ -13,6 +13,7 @@ from pathlib import Path from typing import Any +from ...control_plane.quota.accounting_admission import quota_accounting_admission from ...control_plane.quota.settlement import read_heartbeat_settlement from ...control_plane.runtime.public_safety import validate_public_safe_value from ...rollout_event_log import ( @@ -64,20 +65,33 @@ def _details(event: Mapping[str, Any]) -> dict[str, Any]: def _events_for_turn( events: Sequence[Mapping[str, Any]], *, goal_id: str, agent_id: str, turn_instance_id: str, + goal_ref: Mapping[str, Any] | None = None, ) -> list[dict[str, Any]]: return [dict(event) for event in events if event.get("event_kind") in EVENT_KINDS.values() and event.get("goal_id") == goal_id and event.get("agent_id") == agent_id - and event.get("run_id") == turn_instance_id] + and event.get("run_id") == turn_instance_id + and ( + event.get("goal_ref") == dict(goal_ref) + if goal_ref is not None + else "goal_ref" not in event + )] def native_child_activity( events: Sequence[Mapping[str, Any]], *, goal_id: str, agent_id: str, turn_instance_id: str, configured_limit: int, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: """One read model for CLI, agent-context and product projections.""" - rows = _events_for_turn(events, goal_id=goal_id, agent_id=agent_id, turn_instance_id=turn_instance_id) + rows = _events_for_turn( + events, + goal_id=goal_id, + agent_id=agent_id, + turn_instance_id=turn_instance_id, + goal_ref=goal_ref, + ) operations: dict[str, dict[str, Any]] = {} for event in rows: details = _details(event) @@ -134,32 +148,80 @@ def native_child_activity( def load_native_child_activity( runtime_root: Path, *, goal_id: str, agent_id: str, turn_instance_id: str, configured_limit: int, + goal_ref: Mapping[str, Any] | None = None, + registry_path: Path | None = None, ) -> dict[str, Any]: - source = iter_rollout_events(rollout_event_log_path(runtime_root, goal_id)) - events = [event for event in source - if event.get("event_kind") in EVENT_KINDS.values() - and event.get("agent_id") == agent_id - and event.get("run_id") == turn_instance_id] - return native_child_activity( - events, goal_id=goal_id, agent_id=agent_id, - turn_instance_id=turn_instance_id, configured_limit=configured_limit, - ) + with quota_accounting_admission( + runtime_root=runtime_root, + registry_path=registry_path, + goal_id=goal_id, + goal_ref=goal_ref, + operation="native-child-read", + lock_legacy_index=False, + ) as source_admission: + if source_admission is not None: + readback = read_heartbeat_settlement( + runtime_root, + goal_id=goal_id, + agent_id=agent_id, + todo_id=None, + turn_instance_id=turn_instance_id, + resolve_original_binding=True, + registry_path=registry_path, + goal_ref=goal_ref, + source_admission=source_admission, + borrow_source_admission=True, + ) + if readback is None or readback.identity.value is None: + raise ValueError( + "native child read requires an admitted, settlement-bound Turn guard" + ) + source = iter_rollout_events( + rollout_event_log_path(runtime_root, goal_id) + ) + events = [ + event + for event in source + if event.get("event_kind") in EVENT_KINDS.values() + and event.get("agent_id") == agent_id + and event.get("run_id") == turn_instance_id + and ( + event.get("goal_ref") == dict(goal_ref) + if goal_ref is not None + else "goal_ref" not in event + ) + ] + return native_child_activity( + events, + goal_id=goal_id, + agent_id=agent_id, + turn_instance_id=turn_instance_id, + configured_limit=configured_limit, + goal_ref=goal_ref, + ) def latest_native_child_activity( events: Sequence[Mapping[str, Any]], *, goal_id: str, configured_limit: int, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any] | None: """Expose only the latest reported Turn in existing Goal status surfaces.""" observations = [event for event in events if event.get("goal_id") == goal_id and event.get("event_kind") in EVENT_KINDS.values() - and event.get("agent_id") and event.get("run_id")] + and event.get("agent_id") and event.get("run_id") + and ( + event.get("goal_ref") == dict(goal_ref) + if goal_ref is not None + else "goal_ref" not in event + )] if not observations: return None latest = max(observations, key=lambda event: str(event.get("recorded_at") or "")) return native_child_activity( events, goal_id=goal_id, agent_id=str(latest["agent_id"]), turn_instance_id=str(latest["run_id"]), configured_limit=configured_limit, + goal_ref=goal_ref, ) @@ -212,13 +274,16 @@ def _normalized_fields( raise ValueError("stage must be decision, result or review") -def record_native_child( +def _record_native_child( *, runtime_root: Path, goal_id: str, agent_id: str, turn_instance_id: str, operation_id: str, configured_limit: int, stage: str, outcome: str, operation: str | None = None, entrypoint_id: str | None = None, reason_code: str | None = None, evidence_ref: str | None = None, validation_ref: str | None = None, execute: bool = False, + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, + source_admission: Mapping[str, Any] | None = None, ) -> dict[str, Any]: """Preview or append a typed report; never launch a child or spend quota.""" goal_id = _id(goal_id, field="goal_id") @@ -233,7 +298,13 @@ def record_native_child( ) log_path = rollout_event_log_path(runtime_root, goal_id) events = load_rollout_events(log_path) - prior = _events_for_turn(events, goal_id=goal_id, agent_id=agent_id, turn_instance_id=turn_instance_id) + prior = _events_for_turn( + events, + goal_id=goal_id, + agent_id=agent_id, + turn_instance_id=turn_instance_id, + goal_ref=goal_ref, + ) existing = next((event for event in prior if event.get("case_id") == operation_id and event.get("event_kind") == EVENT_KINDS[stage]), None) @@ -244,6 +315,9 @@ def report_admission() -> Mapping[str, Any]: readback = read_heartbeat_settlement( runtime_root, goal_id=goal_id, agent_id=agent_id, todo_id=None, turn_instance_id=turn_instance_id, resolve_original_binding=True, + registry_path=registry_path, goal_ref=goal_ref, + source_admission=source_admission, + borrow_source_admission=source_admission is not None, ) if readback is None or readback.identity.value is None or readback.identity.failure is not None: reason = (readback.identity.failure.reason @@ -261,7 +335,8 @@ def report_admission() -> Mapping[str, Any]: def validate_transition(observed: Sequence[Mapping[str, Any]]) -> None: admission = report_admission() current = _events_for_turn(observed, goal_id=goal_id, agent_id=agent_id, - turn_instance_id=turn_instance_id) + turn_instance_id=turn_instance_id, + goal_ref=goal_ref) decisions = {str(item.get("case_id")): item for item in current if item.get("event_kind") == EVENT_KINDS["decision"]} if stage == "decision": @@ -293,12 +368,20 @@ def validate_transition(observed: Sequence[Mapping[str, Any]]) -> None: goal_id=goal_id, event_kind=EVENT_KINDS[stage], agent_id=agent_id, run_id=turn_instance_id, case_id=operation_id, status=fields["outcome"], details=fields, recorded_at=(existing or {}).get("recorded_at"), + goal_ref=goal_ref, ) appended = False if execute: stored, appended = append_rollout_event_once( log_path, event, - identity_fields=("goal_id", "event_kind", "agent_id", "run_id", "case_id"), + identity_fields=( + "goal_id", + "event_kind", + "agent_id", + "run_id", + "case_id", + *(("goal_ref",) if goal_ref is not None else ()), + ), precondition=lambda: validate_transition(load_rollout_events(log_path)), ) if _details(stored) != fields: @@ -318,5 +401,46 @@ def validate_transition(observed: Sequence[Mapping[str, Any]]) -> None: events if execute or existing else [*events, event], goal_id=goal_id, agent_id=agent_id, turn_instance_id=turn_instance_id, configured_limit=configured_limit, + goal_ref=goal_ref, ), } + + +def record_native_child( + *, runtime_root: Path, goal_id: str, agent_id: str, + turn_instance_id: str, operation_id: str, configured_limit: int, + stage: str, outcome: str, operation: str | None = None, + entrypoint_id: str | None = None, reason_code: str | None = None, + evidence_ref: str | None = None, validation_ref: str | None = None, + execute: bool = False, registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, +) -> dict[str, Any]: + """Preview or append one report under its exact quota owner.""" + + with quota_accounting_admission( + runtime_root=runtime_root, + registry_path=registry_path, + goal_id=goal_id, + goal_ref=goal_ref, + operation="native-child-report", + lock_legacy_index=False, + ) as source_admission: + return _record_native_child( + runtime_root=runtime_root, + goal_id=goal_id, + agent_id=agent_id, + turn_instance_id=turn_instance_id, + operation_id=operation_id, + configured_limit=configured_limit, + stage=stage, + outcome=outcome, + operation=operation, + entrypoint_id=entrypoint_id, + reason_code=reason_code, + evidence_ref=evidence_ref, + validation_ref=validation_ref, + execute=execute, + registry_path=registry_path, + goal_ref=goal_ref, + source_admission=source_admission, + ) diff --git a/loopx/cli_commands/agent_context.py b/loopx/cli_commands/agent_context.py index c93a394874..ae4ad5af99 100644 --- a/loopx/cli_commands/agent_context.py +++ b/loopx/cli_commands/agent_context.py @@ -1,8 +1,14 @@ """Read-only lifecycle context for hosts whose native tools bypass LoopX Turn.""" +import argparse + from ..agent_registry import load_goal_from_registry, registered_agent_ids_for_goal from ..capabilities.multi_subagent.native_child_receipts import load_native_child_activity from ..control_plane.agent_context import project_goal_agent_context +from ..control_plane.goals.first_party_host_admission import ( + capture_first_party_host_goal_ref, +) +from ..control_plane.goals.source_session_registry_state import exact_goal_ref from ..orchestration import compact_orchestration_policy @@ -37,9 +43,29 @@ def register_agent_context(subparsers, add_format): "--turn-instance-id", help="Read durable native child activity for this exact admitted Turn.", ) + parser.add_argument("--goal-instance-id", help=argparse.SUPPRESS) def handle_agent_context(args, registry_path, runtime_root, print_payload, output_format): + goal_instance_id = str( + getattr(args, "goal_instance_id", None) or "" + ).strip() + try: + goal_ref = ( + exact_goal_ref(args.goal_id, goal_instance_id) + if goal_instance_id + else capture_first_party_host_goal_ref( + registry_path=registry_path, + goal_id=args.goal_id, + ) + ) + except (OSError, ValueError, RuntimeError) as exc: + print_payload( + {"ok": False, "error": str(exc)}, + output_format(args), + render_agent_context, + ) + return 1 goal = load_goal_from_registry(registry_path, args.goal_id) if goal is None or args.agent_id not in registered_agent_ids_for_goal(goal): print_payload( @@ -118,6 +144,8 @@ def handle_agent_context(args, registry_path, runtime_root, print_payload, outpu runtime_root, goal_id=args.goal_id, agent_id=args.agent_id, turn_instance_id=args.turn_instance_id, configured_limit=int(orchestration["max_children"]), + registry_path=registry_path, + goal_ref=goal_ref, ) observations["native_child_activity"] = native_activity context = project_goal_agent_context( diff --git a/loopx/cli_commands/project_lifecycle_refresh_state.py b/loopx/cli_commands/project_lifecycle_refresh_state.py index 1e702c233c..65ea45b5bb 100644 --- a/loopx/cli_commands/project_lifecycle_refresh_state.py +++ b/loopx/cli_commands/project_lifecycle_refresh_state.py @@ -24,6 +24,10 @@ from ..control_plane.goals.goal_vision_policy import ( GOAL_VISION_ADVANCEMENT_POLICY_CHOICES, ) +from ..control_plane.goals.first_party_host_admission import ( + capture_first_party_host_goal_ref, +) +from ..control_plane.goals.source_session_registry_state import exact_goal_ref from ..control_plane.quota.settlement import ( attach_settlement_progress, read_heartbeat_settlement, @@ -86,6 +90,7 @@ def register_refresh_state_command( binding = context_parser.add_mutually_exclusive_group(required=True) binding.add_argument("--todo-id") binding.add_argument("--replan-obligation-id") + context_parser.add_argument("--goal-instance-id", help=argparse.SUPPRESS) context_parser.add_argument("--project") context_parser.add_argument("--state-file") context_parser.add_argument("--dependency-todo-id", action="append", default=[], @@ -178,6 +183,7 @@ def register_refresh_state_command( "value on retries." ), ) + refresh_state_parser.add_argument("--goal-instance-id", help=argparse.SUPPRESS) refresh_state_parser.add_argument("--completion-todo-id", help=argparse.SUPPRESS) refresh_state_parser.add_argument("--completion-turn-key", help=argparse.SUPPRESS) refresh_state_parser.add_argument( @@ -391,6 +397,17 @@ def handle_refresh_state_command( if args.command == "checkpoint-context": from ..control_plane.goals.checkpoint_context_io import read_checkpoint_context, render_checkpoint_context try: + goal_instance_id = str( + getattr(args, "goal_instance_id", None) or "" + ).strip() + goal_ref = ( + exact_goal_ref(args.goal_id, goal_instance_id) + if goal_instance_id + else capture_first_party_host_goal_ref( + registry_path=registry_path, + goal_id=args.goal_id, + ) + ) payload = read_checkpoint_context( registry_path=registry_path, runtime_root_override=args.runtime_root, goal_id=args.goal_id, agent_id=args.agent_id, todo_id=args.todo_id, @@ -398,6 +415,7 @@ def handle_refresh_state_command( project=Path(args.project).expanduser() if args.project else None, state_file=Path(args.state_file).expanduser() if args.state_file else None, dependency_todo_ids=args.dependency_todo_id, + goal_ref=goal_ref, ) except Exception as exc: payload = {"ok": False, "error": str(exc), @@ -411,6 +429,7 @@ def handle_refresh_state_command( agent_vision_packet: dict[str, object] | None = None progress_observation: dict[str, object] | None = None merge_agent_vision_patch = False + goal_ref: dict[str, str] | None = None try: inline_vision_packet = inline_agent_vision_packet(args) if args.agent_vision_json and inline_vision_packet: @@ -446,6 +465,20 @@ def handle_refresh_state_command( if not isinstance(loaded_usage, dict): raise ValueError("--usage-json must be a JSON object") usage_measurement = loaded_usage + goal_instance_id = str( + getattr(args, "goal_instance_id", None) or "" + ).strip() + if goal_instance_id and not getattr(args, "turn_instance_id", None): + raise ValueError("--goal-instance-id requires --turn-instance-id") + if getattr(args, "turn_instance_id", None): + goal_ref = ( + exact_goal_ref(args.goal_id, goal_instance_id) + if goal_instance_id + else capture_first_party_host_goal_ref( + registry_path=registry_path, + goal_id=args.goal_id, + ) + ) except Exception as exc: payload = { "ok": False, @@ -507,6 +540,7 @@ def handle_refresh_state_command( ), dry_run=bool(args.dry_run), sync_global=not bool(args.no_global_sync), + goal_ref=goal_ref, ) except Exception as exc: payload = { @@ -583,6 +617,7 @@ def handle_refresh_state_command( registry_path=registry_path, runtime_root_arg=args.runtime_root, event_kind="refresh_state", + goal_ref=goal_ref, agent_id=args.agent_id, todo_id=getattr(args, "todo_id", None), run_id=getattr(args, "turn_instance_id", None), @@ -628,6 +663,7 @@ def handle_refresh_state_command( else [] ), "run_id", + *(["goal_ref"] if goal_ref is not None else []), ] if getattr(args, "turn_instance_id", None) else None @@ -650,6 +686,8 @@ def handle_refresh_state_command( replan_obligation_id=getattr( args, "replan_obligation_id", None ), + registry_path=registry_path, + goal_ref=goal_ref, ) if settlement_readback is None: raise RuntimeError( @@ -658,6 +696,7 @@ def handle_refresh_state_command( settlement_result = settlement_readback.delivery attach_settlement_progress( payload, settlement_readback, registry_path=registry_path, runtime_root=runtime_root, + goal_ref=goal_ref, ) payload["settlement_result"] = settlement_result_payload( settlement_result diff --git a/loopx/cli_commands/quota.py b/loopx/cli_commands/quota.py index d37eaccf2a..72f9a3ab3d 100644 --- a/loopx/cli_commands/quota.py +++ b/loopx/cli_commands/quota.py @@ -18,6 +18,10 @@ ) from ..control_plane.effect_runtime import EffectRuntimeRejected from ..control_plane.capability_hooks import InteractionProjectionHookRegistration +from ..control_plane.goals.first_party_host_admission import ( + capture_first_party_host_goal_ref, +) +from ..control_plane.goals.source_session_registry_state import exact_goal_ref from ..control_plane.quota.cli_projection import ( compact_quota_monitor_poll_cli_payload, compact_quota_plan_cli_payload, @@ -107,6 +111,35 @@ RolloutEventAppender = Callable[..., dict[str, object]] +def _quota_goal_ref( + args: argparse.Namespace, + *, + registry_path: Path, +) -> dict[str, str] | None: + goal_id = str(getattr(args, "goal_id", None) or "").strip() + if not goal_id: + return None + goal_instance_id = str( + getattr(args, "goal_instance_id", None) or "" + ).strip() + if goal_instance_id: + return exact_goal_ref(goal_id, goal_instance_id) + if args.quota_command not in { + "should-run", + "monitor-poll", + "scheduler-ack", + "scheduler-ack-current", + "scheduler-fail-current", + "spend-slot", + "void-slot", + }: + return None + return capture_first_party_host_goal_ref( + registry_path=registry_path, + goal_id=goal_id, + ) + + def _effective_spend_turn_instance_id( payload: Mapping[str, object], *, @@ -162,6 +195,7 @@ def _record_automatic_heartbeat_stall( interaction_projection_hooks: tuple[InteractionProjectionHookRegistration, ...], action_selection: RequestedQuotaActionSelection, cache_metadata: object, + goal_ref: Mapping[str, object] | None, ) -> tuple[dict[str, object], dict[str, object], object, str]: """Commit and reproject the automatic no-spend heartbeat observation.""" @@ -173,6 +207,7 @@ def _record_automatic_heartbeat_stall( goal_id=args.goal_id, agent_id=args.agent_id, turn_instance_id=turn_id, + goal_ref=goal_ref, ) if ( payload.get("effective_action") @@ -189,6 +224,7 @@ def _record_automatic_heartbeat_stall( agent_id=args.agent_id, available_capabilities=args.available_capabilities, turn_instance_id=turn_id, + goal_ref=goal_ref, scheduler_execution_context=context.scheduler_context, operator_inbox_urgency_projector=context.operator_inbox_urgency_projector, bounded_research_frontier_projector=project_live_explore_composition_frontier, @@ -232,6 +268,7 @@ def _record_automatic_heartbeat_stall( ), turn_instance_id=turn_id, interaction_projection_hooks=interaction_projection_hooks, + goal_ref=goal_ref, ) rebuilt["heartbeat_stall_writeback"] = { "turn_instance_id": turn_id, @@ -300,6 +337,35 @@ def _dispatch_quota_turn_start_hooks( return dispatch, local_private_state_mutated +def _prepare_quota_command_execution( + args: argparse.Namespace, + *, + registry_path: Path, + runtime_root_arg: str | None, +) -> tuple[ + dict[str, str] | None, + dict[str, object] | None, + QuotaCommandContext, +]: + goal_ref = _quota_goal_ref(args, registry_path=registry_path) + turn_start_hook_dispatch, turn_start_mutated = _dispatch_quota_turn_start_hooks( + args, + registry_path=registry_path, + runtime_root_arg=runtime_root_arg, + ) + context = prepare_quota_command_context( + args, + registry_path=registry_path, + runtime_root_arg=runtime_root_arg, + status_collector=collect_status, + operator_inbox_urgency_projector_factory=( + build_lark_operator_inbox_urgency_projector + ), + force_projection_refresh=turn_start_mutated, + ) + return goal_ref, turn_start_hook_dispatch, context + + def _attach_turn_start_hook_dispatch( payload: dict[str, object], dispatch: Mapping[str, object] | None, @@ -368,21 +434,14 @@ def handle_quota_command( heartbeat_stall_observation = "not_evaluated" detail_sections: frozenset[str] = frozenset() context: QuotaCommandContext | None = None + goal_ref: dict[str, str] | None = None try: - turn_start_hook_dispatch, turn_start_mutated = _dispatch_quota_turn_start_hooks( - args, - registry_path=registry_path, - runtime_root_arg=runtime_root_arg, - ) - context = prepare_quota_command_context( - args, - registry_path=registry_path, - runtime_root_arg=runtime_root_arg, - status_collector=collect_status, - operator_inbox_urgency_projector_factory=( - build_lark_operator_inbox_urgency_projector - ), - force_projection_refresh=turn_start_mutated, + goal_ref, turn_start_hook_dispatch, context = ( + _prepare_quota_command_execution( + args, + registry_path=registry_path, + runtime_root_arg=runtime_root_arg, + ) ) ( heartbeat_turn_id, @@ -421,6 +480,7 @@ def handle_quota_command( args, runtime_root=runtime_root, turn_instance_id=heartbeat_turn_id, + goal_ref=goal_ref, ) heartbeat_receipt_existing = action_selection.receipt payload = build_live_quota_should_run_decision( @@ -455,6 +515,7 @@ def handle_quota_command( turn_instance_id=heartbeat_turn_id, interaction_projection_hooks=interaction_projection_hooks, turn_start_hook_dispatch=turn_start_hook_dispatch, + goal_ref=goal_ref, ) _attach_turn_start_hook_dispatch(payload, turn_start_hook_dispatch) action_selection_preflight = ( @@ -464,6 +525,7 @@ def handle_quota_command( registry_path=registry_path, context=context, selection=action_selection, + goal_ref=goal_ref, ) ) action_selection_preflight_failed = action_selection_preflight.rejected @@ -491,6 +553,7 @@ def handle_quota_command( runtime_root=runtime_root, turn_instance_id=heartbeat_turn_id, existing=heartbeat_receipt_existing, + goal_ref=goal_ref, ) else: ( @@ -508,6 +571,7 @@ def handle_quota_command( interaction_projection_hooks=interaction_projection_hooks, action_selection=action_selection, cache_metadata=cache_metadata, + goal_ref=goal_ref, ) heartbeat_receipt_ready = True elif args.quota_command == "monitor-poll": @@ -519,6 +583,7 @@ def handle_quota_command( turn_instance_id=heartbeat_turn_id, scheduler_execution_context=scheduler_context, operator_inbox_urgency_projector=operator_inbox_urgency_projector, + goal_ref=goal_ref, monitor_poll_recorder=record_quota_monitor_poll, status_reloader=lambda: collect_status( registry_path=registry_path, @@ -542,6 +607,7 @@ def handle_quota_command( turn_instance_id=heartbeat_turn_id, scheduler_context=scheduler_context, operator_inbox_urgency_projector=operator_inbox_urgency_projector, + goal_ref=goal_ref, ) elif args.quota_command == "spend-slot": payload = spend_quota_slot( @@ -557,6 +623,8 @@ def handle_quota_command( todo_id=args.todo_id, turn_instance_id=heartbeat_turn_id, replan_obligation_id=args.replan_obligation_id, + registry_path=registry_path, + goal_ref=goal_ref, ) elif args.quota_command == "void-slot": payload = void_quota_slot( @@ -568,6 +636,8 @@ def handle_quota_command( reason_summary=args.reason_summary, agent_id=args.agent_id, operator_inbox_urgency_projector=operator_inbox_urgency_projector, + registry_path=registry_path, + goal_ref=goal_ref, ) else: payload = build_quota_plan(status_payload, mode=args.quota_command) @@ -677,6 +747,7 @@ def handle_quota_command( registry_path=registry_path, runtime_root_arg=runtime_root_arg, event_kind="quota_should_run", + goal_ref=goal_ref, agent_id=args.agent_id, run_id=heartbeat_turn_id, status=str( @@ -690,13 +761,20 @@ def handle_quota_command( ), details=rollout_details, allow_failed=True, - idempotency_fields=["goal_id", "event_kind", "agent_id", "run_id"], + idempotency_fields=[ + "goal_id", + "event_kind", + "agent_id", + "run_id", + *(["goal_ref"] if goal_ref is not None else []), + ], ) receipt = find_heartbeat_receipt( runtime_root, goal_id=args.goal_id, agent_id=args.agent_id, turn_instance_id=heartbeat_turn_id, + goal_ref=goal_ref, ) if receipt: rollout_event_value = payload.get("rollout_event") @@ -736,6 +814,7 @@ def handle_quota_command( registry_path=registry_path, runtime_root_arg=runtime_root_arg, event_kind=QUOTA_EVENT_KINDS[args.quota_command], + goal_ref=goal_ref, agent_id=args.agent_id, todo_id=rollout_todo_id, run_id=( @@ -766,6 +845,7 @@ def handle_quota_command( todo_id=rollout_todo_id, turn_instance_id=spend_turn_instance_id, replan_obligation_id=rollout_replan_obligation_id, + goal_ref=goal_ref, ) attach_reward_memory_ingest_after_spend( payload, @@ -777,6 +857,7 @@ def handle_quota_command( todo_id=rollout_todo_id, turn_instance_id=spend_turn_instance_id, replan_obligation_id=rollout_replan_obligation_id, + goal_ref=goal_ref, ) attach_reward_memory_recall_after_should_run( payload, diff --git a/loopx/cli_commands/quota_action_selection.py b/loopx/cli_commands/quota_action_selection.py index f0e5690ea1..749737e23a 100644 --- a/loopx/cli_commands/quota_action_selection.py +++ b/loopx/cli_commands/quota_action_selection.py @@ -82,6 +82,7 @@ def load_requested_quota_action_selection( *, runtime_root: Path, turn_instance_id: str | None, + goal_ref: Mapping[str, object] | None = None, ) -> RequestedQuotaActionSelection: requested_todo_id = _requested_quota_action_todo_id(args) if not turn_instance_id: @@ -98,6 +99,7 @@ def load_requested_quota_action_selection( goal_id=args.goal_id, agent_id=args.agent_id, turn_instance_id=turn_instance_id, + goal_ref=goal_ref, ) if not existing: return RequestedQuotaActionSelection( @@ -234,6 +236,7 @@ def reconcile_requested_quota_action_selection( registry_path: Path, context: QuotaCommandContext, selection: RequestedQuotaActionSelection, + goal_ref: Mapping[str, object] | None = None, ) -> ActionSelectionPreflightResult: recovery = _requested_quota_action_selection_preflight( payload, @@ -265,6 +268,7 @@ def reconcile_requested_quota_action_selection( scheduler_args=render_scheduler_execution_args( scheduler_execution_context=context.scheduler_context ), + goal_ref=goal_ref, ) receipt, receipt_status, receipt_appended = _retain_deferred_action_selection( payload, @@ -272,6 +276,7 @@ def reconcile_requested_quota_action_selection( runtime_root=context.runtime_root, turn_instance_id=context.heartbeat_turn_id, selection=selection, + goal_ref=goal_ref, ) return ActionSelectionPreflightResult( rejected=True, @@ -304,6 +309,7 @@ def _retain_deferred_action_selection( runtime_root: Path, turn_instance_id: str | None, selection: RequestedQuotaActionSelection, + goal_ref: Mapping[str, object] | None, ) -> tuple[dict[str, object] | None, str, bool]: """Append a deferred explicit choice without granting settlement authority.""" @@ -323,5 +329,6 @@ def _retain_deferred_action_selection( turn_instance_id=turn_instance_id, todo_id=selection.requested_todo_id, reason=str(qualification.get("reason") or "current_delivery_gate"), + goal_ref=goal_ref, ) return retained, "selection_retained" if appended else "replayed", appended diff --git a/loopx/cli_commands/quota_monitor_poll.py b/loopx/cli_commands/quota_monitor_poll.py index 838776bd5c..95bd8639de 100644 --- a/loopx/cli_commands/quota_monitor_poll.py +++ b/loopx/cli_commands/quota_monitor_poll.py @@ -22,6 +22,7 @@ def _receipt_bound_monitor_todo_id( *, runtime_root: Path, turn_instance_id: str | None, + goal_ref: Mapping[str, object] | None, ) -> str | None: if not turn_instance_id: return None @@ -30,6 +31,7 @@ def _receipt_bound_monitor_todo_id( goal_id=args.goal_id, agent_id=args.agent_id, turn_instance_id=turn_instance_id, + goal_ref=goal_ref, ) if receipt is None: raise HeartbeatReceiptIdentityConflictError( @@ -57,6 +59,7 @@ def record_quota_monitor_poll_for_cli( operator_inbox_urgency_projector: Callable[..., dict[str, object]], status_reloader: Callable[[], dict[str, object]], monitor_poll_recorder: Callable[..., dict[str, object]], + goal_ref: Mapping[str, object] | None = None, ) -> dict[str, object]: """Execute the monitor-poll CLI request with receipt-bound settlement identity.""" return monitor_poll_recorder( @@ -87,10 +90,12 @@ def record_quota_monitor_poll_for_cli( task_lease_expected_version=getattr(args, "task_lease_expected_version", None), use_current_task_lease=bool(getattr(args, "use_current_task_lease", False)), turn_instance_id=turn_instance_id, + goal_ref=goal_ref, receipt_bound_todo_id=_receipt_bound_monitor_todo_id( args, runtime_root=runtime_root, turn_instance_id=turn_instance_id, + goal_ref=goal_ref, ), scheduler_execution_context=scheduler_execution_context, operator_inbox_urgency_projector=operator_inbox_urgency_projector, diff --git a/loopx/cli_commands/quota_registration.py b/loopx/cli_commands/quota_registration.py index 5a51aa2064..d3c3790575 100644 --- a/loopx/cli_commands/quota_registration.py +++ b/loopx/cli_commands/quota_registration.py @@ -188,6 +188,7 @@ def register_quota_command( "refresh-state, spend, and retries." ), ) + quota_parser.add_argument("--goal-instance-id", help=argparse.SUPPRESS) quota_parser.add_argument( "--scheduler-host-facts-chunk", dest="scheduler_host_facts_chunks", diff --git a/loopx/cli_commands/quota_reward_memory.py b/loopx/cli_commands/quota_reward_memory.py index 61179d8f00..c448a7de5c 100644 --- a/loopx/cli_commands/quota_reward_memory.py +++ b/loopx/cli_commands/quota_reward_memory.py @@ -76,6 +76,7 @@ def attach_reward_memory_ingest_after_spend( todo_id: str | None, turn_instance_id: str, replan_obligation_id: str | None, + goal_ref: Mapping[str, Any] | None = None, ) -> None: if not execute or payload.get("ok") is not True: return @@ -86,6 +87,8 @@ def attach_reward_memory_ingest_after_spend( todo_id=todo_id, turn_instance_id=turn_instance_id, replan_obligation_id=replan_obligation_id, + registry_path=registry_path, + goal_ref=goal_ref, ) identity = readback.identity.value if readback else None writeback_event = readback.writeback_event if readback else None diff --git a/loopx/cli_commands/quota_scheduler_followup.py b/loopx/cli_commands/quota_scheduler_followup.py index 0e9fbef4ea..fe39744c3d 100644 --- a/loopx/cli_commands/quota_scheduler_followup.py +++ b/loopx/cli_commands/quota_scheduler_followup.py @@ -42,6 +42,7 @@ def _build_scheduler_followup_decision( | SchedulerExecutionContextResolution | None, operator_inbox_urgency_projector: Callable[..., dict[str, object]], + goal_ref: Mapping[str, object] | None = None, ) -> dict[str, object]: """Rebuild a scheduler follow-up from the originating receipt-bound Turn.""" @@ -74,6 +75,7 @@ def _build_scheduler_followup_decision( interaction_projection_hooks=( repository_delivery_interaction_hook(repo_path=Path.cwd()), ), + goal_ref=goal_ref, ) @@ -88,6 +90,7 @@ def build_scheduler_followup_payload( | SchedulerExecutionContextResolution | None, operator_inbox_urgency_projector: Callable[..., dict[str, object]], + goal_ref: Mapping[str, object] | None = None, ) -> dict[str, object]: """Execute one scheduler ACK/failure command against its live decision.""" @@ -97,6 +100,7 @@ def build_scheduler_followup_payload( goal_id=args.goal_id, agent_id=args.agent_id, turn_instance_id=turn_instance_id, + goal_ref=goal_ref, ) if turn_instance_id else None @@ -167,6 +171,7 @@ def build_scheduler_followup_payload( ), scheduler_context=scheduler_context, operator_inbox_urgency_projector=operator_inbox_urgency_projector, + goal_ref=goal_ref, ) receipt_todo_id = ( heartbeat_receipt_settlement_todo_id(heartbeat_receipt) diff --git a/loopx/cli_commands/todo.py b/loopx/cli_commands/todo.py index af217661f2..bb7ef902da 100644 --- a/loopx/cli_commands/todo.py +++ b/loopx/cli_commands/todo.py @@ -39,6 +39,10 @@ build_task_planning_packet, render_task_planning_packet, ) +from ..control_plane.goals.first_party_host_admission import ( + capture_first_party_host_goal_ref, +) +from ..control_plane.goals.source_session_registry_state import exact_goal_ref from ..todos import ( add_goal_todo, archive_completed_todos, @@ -87,13 +91,16 @@ def _read_todo_turn_settlement( - args: argparse.Namespace, *, runtime_root: Path, + args: argparse.Namespace, *, registry_path: Path, runtime_root: Path, + goal_ref: dict[str, str] | None, ) -> QuotaSettlementReadback: """Transport the original lifecycle tuple to the TS identity owner.""" readback = read_heartbeat_settlement( runtime_root, goal_id=args.goal_id, agent_id=args.agent_id, todo_id=args.todo_id, turn_instance_id=args.turn_instance_id, + registry_path=registry_path, + goal_ref=goal_ref, ) if readback is None: raise RuntimeError("exact settlement readback unexpectedly returned not-found") @@ -124,6 +131,7 @@ def _completion_settlement_error( def _completion_settlement_plan( identity: SettlementIdentity, *, args: argparse.Namespace, registry_path: Path, runtime_root: Path, + goal_ref: dict[str, str] | None, ) -> dict[str, object]: """Render the native plan with the original route and supplied lease facts.""" actor_args = "" @@ -145,6 +153,7 @@ def _completion_settlement_plan( goal_id=identity.goal_id, agent_id=identity.agent_id, todo_id=identity.todo_id, turn_instance_id=identity.turn_instance_id, command_prefix=prefix, scoped_cli_args="", lifecycle_actor_args=actor_args, writeback_path_args=path_args, + goal_ref=goal_ref, ).as_dict() @@ -274,6 +283,7 @@ def handle_todo_command( renderer = _render_todo_receipt elif args.todo_command == "result-read": renderer = itemgetter("text") + goal_ref: dict[str, str] | None = None try: if args.todo_command is None: raise ValueError( @@ -283,6 +293,18 @@ def handle_todo_command( ) validate_shared_todo_options(args) validate_capability_gap_options(args) + if getattr(args, "turn_instance_id", None): + goal_instance_id = str( + getattr(args, "goal_instance_id", None) or "" + ).strip() + goal_ref = ( + exact_goal_ref(args.goal_id, goal_instance_id) + if goal_instance_id + else capture_first_party_host_goal_ref( + registry_path=registry_path, + goal_id=args.goal_id, + ) + ) if args.todo_command == "plan": validate_todo_plan_options(args) payload = build_task_planning_packet( @@ -512,7 +534,10 @@ def handle_todo_command( if getattr(args, "turn_instance_id", None): runtime_root = resolve_runtime_root(load_registry(registry_path), runtime_root_arg) settlement_readback = _read_todo_turn_settlement( - args, runtime_root=runtime_root, + args, + registry_path=registry_path, + runtime_root=runtime_root, + goal_ref=goal_ref, ) settlement_result = settlement_readback.identity assert settlement_result.value is not None @@ -558,7 +583,11 @@ def handle_todo_command( settlement_result ), "settlement_plan": _completion_settlement_plan( - identity, args=args, registry_path=registry_path, runtime_root=runtime_root, + identity, + args=args, + registry_path=registry_path, + runtime_root=runtime_root, + goal_ref=goal_ref, ), "error": completion_error, } @@ -631,7 +660,12 @@ def handle_todo_command( validate_todo_supersede_options(args) supersede_readback = ( _read_todo_turn_settlement( - args, runtime_root=resolve_runtime_root(load_registry(registry_path), runtime_root_arg), + args, + registry_path=registry_path, + runtime_root=resolve_runtime_root( + load_registry(registry_path), runtime_root_arg + ), + goal_ref=goal_ref, ) if args.turn_instance_id else None ) payload = supersede_goal_todo( @@ -676,6 +710,8 @@ def handle_todo_command( ) else: raise ValueError("unsupported todo command") + if goal_ref is not None: + payload["goal_ref"] = dict(goal_ref) except Exception as exc: from ..usage_ping import capture_failure capture_failure(exc) @@ -704,6 +740,8 @@ def handle_todo_command( agent_id=args.agent_id, todo_id=args.todo_id, turn_instance_id=getattr(args, "turn_instance_id", None), + registry_path=registry_path, + goal_ref=goal_ref, ) if settlement_readback is None: raise RuntimeError("exact settlement readback unexpectedly returned not-found") diff --git a/loopx/cli_commands/todo_argument_validation.py b/loopx/cli_commands/todo_argument_validation.py index ae274b5e73..e20124a839 100644 --- a/loopx/cli_commands/todo_argument_validation.py +++ b/loopx/cli_commands/todo_argument_validation.py @@ -16,6 +16,7 @@ ("--update-operation-id", "update_operation_id"), ("--update-expected-provider-revision", "update_expected_provider_revision"), ("--turn-instance-id", "turn_instance_id"), + ("--goal-instance-id", "goal_instance_id"), ("--completion-identity-key", "completion_identity_key"), ("--replan-obligation-id", "replan_obligation_id"), ("--status", "status"), @@ -541,6 +542,14 @@ def validate_shared_todo_options(args: argparse.Namespace) -> None: raise ValueError( "--turn-instance-id is supported only by todo complete/supersede settlement" ) + if getattr(args, "goal_instance_id", None) and ( + args.todo_command not in {"complete", "supersede"} + or not getattr(args, "turn_instance_id", None) + ): + raise ValueError( + "--goal-instance-id is supported only by turn-scoped todo " + "complete/supersede settlement" + ) if getattr(args, "update_operation_id", None) is not None and args.todo_command != "update": raise ValueError("--update-operation-id is supported only by todo update") if getattr(args, "update_expected_provider_revision", None) is not None and args.todo_command != "update": diff --git a/loopx/cli_commands/todo_event.py b/loopx/cli_commands/todo_event.py index ea5edcc170..558e147270 100644 --- a/loopx/cli_commands/todo_event.py +++ b/loopx/cli_commands/todo_event.py @@ -1,7 +1,7 @@ from __future__ import annotations import argparse -from collections.abc import Callable +from collections.abc import Callable, Mapping from pathlib import Path from ..control_plane.coordination.local_authority import ( @@ -87,11 +87,17 @@ def append_todo_rollout_event( or (payload.get("idempotent_replay") and not turn_instance_id) ): return + goal_ref = ( + payload.get("goal_ref") + if isinstance(payload.get("goal_ref"), Mapping) + else None + ) append_cli_rollout_event( payload, registry_path=registry_path, runtime_root_arg=runtime_root_arg, event_kind=TODO_EVENT_KINDS.get(args.todo_command, "todo_update"), + goal_ref=goal_ref, agent_id=args.agent_id or args.claimed_by, todo_id=args.todo_id or str(payload.get("todo_id") or "").strip() or None, run_id=turn_instance_id, @@ -138,6 +144,7 @@ def append_todo_rollout_event( "agent_id", "todo_id", "run_id", + *(["goal_ref"] if goal_ref is not None else []), *(["status"] if terminal_closeout else []), ] if turn_instance_id diff --git a/loopx/cli_commands/todo_registration.py b/loopx/cli_commands/todo_registration.py index 0b1bbb3df0..1a23b7db37 100644 --- a/loopx/cli_commands/todo_registration.py +++ b/loopx/cli_commands/todo_registration.py @@ -85,6 +85,7 @@ def register_todo_command( "turn-scoped quota guard and reuse it on retries." ), ) + todo_parser.add_argument("--goal-instance-id", help=argparse.SUPPRESS) todo_parser.add_argument( "--completion-identity-key", help=( diff --git a/loopx/cli_commands/turn.py b/loopx/cli_commands/turn.py index e71de571ef..7ce1473462 100644 --- a/loopx/cli_commands/turn.py +++ b/loopx/cli_commands/turn.py @@ -147,6 +147,7 @@ def handle_turn_command( runtime_root=runtime_root, runtime_root_arg=runtime_root_arg, turn_start_hook_dispatch=turn_start_hook_dispatch, + goal_ref=goal_ref, ) operator_inbox_urgency_projector = decision_owner.operator_inbox_urgency_projector scheduler_context = decision_owner.scheduler_execution_context @@ -272,6 +273,12 @@ def handle_turn_command( strict_goal_admission = goal_admission if goal_admission.enabled else None if strict_goal_admission is not None: strict_goal_admission.require_current() + resumed_goal_ref = payload.get("goal_ref") + goal_ref = ( + dict(resumed_goal_ref) + if isinstance(resumed_goal_ref, Mapping) + else None + ) if payload.get("route", {}).get("kind") == "capability_action_required": # The normal host transaction forbids Core mutations. A # capability may prepare artifacts and require authored input; @@ -289,6 +296,7 @@ def handle_turn_command( registry_path=registry_path, runtime_root=runtime_root, runtime_root_arg=runtime_root_arg, + goal_ref=goal_ref, goal_admission=strict_goal_admission, operator_environ=operator_environ, operator_inbox_urgency_projector=operator_inbox_urgency_projector, diff --git a/loopx/cli_commands/turn_decision.py b/loopx/cli_commands/turn_decision.py index 8db3b1f10b..f51a819922 100644 --- a/loopx/cli_commands/turn_decision.py +++ b/loopx/cli_commands/turn_decision.py @@ -87,6 +87,7 @@ def _build_turn_decision( scheduler_execution_context: Mapping[str, Any], operator_inbox_urgency_projector: Callable[..., dict[str, Any]], turn_start_hook_dispatch: Mapping[str, Any] | None = None, + goal_ref: Mapping[str, object] | None = None, ) -> Callable[..., dict[str, Any]]: """Return the ``build_turn_decision`` every Turn owner resolves through. @@ -117,6 +118,7 @@ def build_turn_decision( ), requested_action_todo_id=requested_action_todo_id, turn_start_hook_dispatch=dict(turn_start_hook_dispatch or {}), + goal_ref=goal_ref, interaction_projection_hooks=( periodic_report_pending_intent_interaction_hook( registry_path=registry_path, @@ -194,6 +196,7 @@ def build_fresh_turn_decision_owner( runtime_root: Path, runtime_root_arg: str | None, turn_start_hook_dispatch: Mapping[str, Any] | None = None, + goal_ref: Mapping[str, object] | None = None, ) -> FreshTurnDecisionOwner: """Read the live status and derive the shared decision inputs from it. @@ -230,6 +233,7 @@ def build_fresh_turn_decision_owner( scheduler_execution_context=scheduler_execution_context, operator_inbox_urgency_projector=operator_inbox_urgency_projector, turn_start_hook_dispatch=turn_start_hook_dispatch, + goal_ref=goal_ref, ), requested_todo_id=getattr(args, "todo_id", None), ) diff --git a/loopx/cli_commands/turn_run_once.py b/loopx/cli_commands/turn_run_once.py index c253edf8ab..355c3c6ecd 100644 --- a/loopx/cli_commands/turn_run_once.py +++ b/loopx/cli_commands/turn_run_once.py @@ -70,6 +70,7 @@ def execute_turn_run_once( registry_path: Path, runtime_root: Path, runtime_root_arg: str | None, + goal_ref: Mapping[str, object] | None, goal_admission: FirstPartyHostGoalAdmission | None, operator_environ: Mapping[str, str], operator_inbox_urgency_projector: Callable[..., dict[str, Any]], @@ -196,6 +197,7 @@ def append_settlement_event( registry_path=registry_path, runtime_root_arg=runtime_root_arg, event_kind=event_kind, + goal_ref=goal_ref, agent_id=settlement_identity.agent_id, todo_id=settlement_identity.todo_id, run_id=settlement_identity.turn_instance_id, @@ -211,6 +213,7 @@ def append_settlement_event( "agent_id", "todo_id", "run_id", + *(("goal_ref",) if goal_ref is not None else ()), *(("status",) if event_kind == "todo_complete" else ()), ], ) @@ -296,6 +299,7 @@ def writeback( completion_turn_key=completion_turn_key, dry_run=False, sync_global=not bool(args.no_global_sync), + goal_ref=goal_ref, ) if refresh.get("ok") and ( refresh.get("appended") @@ -477,6 +481,8 @@ def spend(*, effect_ref: str) -> dict[str, object]: ), operator_inbox_urgency_projector=operator_inbox_urgency_projector, effect_ref=effect_ref, + registry_path=registry_path, + goal_ref=goal_ref, ) if spent.get("ok") and ( spent.get("appended") @@ -490,6 +496,8 @@ def spend(*, effect_ref: str) -> dict[str, object]: todo_id=settlement_identity.todo_id, turn_instance_id=settlement_identity.turn_instance_id, replan_obligation_id=settlement_identity.replan_obligation_id, + registry_path=registry_path, + goal_ref=goal_ref, ) if readback is None: raise RuntimeError(EXACT_SETTLEMENT_READBACK_NOT_FOUND) @@ -580,6 +588,8 @@ def writeback_resolver(effect_ref: str) -> dict[str, object]: todo_id=settlement_identity.todo_id, turn_instance_id=settlement_identity.turn_instance_id, replan_obligation_id=settlement_identity.replan_obligation_id, + registry_path=registry_path, + goal_ref=goal_ref, ) if readback is None: raise RuntimeError(EXACT_SETTLEMENT_READBACK_NOT_FOUND) @@ -622,6 +632,8 @@ def spend_resolver(effect_ref: str) -> dict[str, object]: todo_id=settlement_identity.todo_id, turn_instance_id=settlement_identity.turn_instance_id, replan_obligation_id=settlement_identity.replan_obligation_id, + registry_path=registry_path, + goal_ref=goal_ref, ) if readback is None: raise RuntimeError(EXACT_SETTLEMENT_READBACK_NOT_FOUND) @@ -668,6 +680,8 @@ def terminal_closeout_resolver(effect_ref: str) -> dict[str, object]: todo_id=settlement_identity.todo_id, turn_instance_id=settlement_identity.turn_instance_id, replan_obligation_id=settlement_identity.replan_obligation_id, + registry_path=registry_path, + goal_ref=goal_ref, ) if readback is None: raise RuntimeError(EXACT_SETTLEMENT_READBACK_NOT_FOUND) @@ -726,6 +740,7 @@ def scheduler(_spend_payload: dict[str, object]) -> dict[str, object]: bounded_research_frontier_projector=( project_live_explore_composition_frontier ), + goal_ref=goal_ref, ) hint = ( latest.get("scheduler_hint") @@ -829,6 +844,7 @@ def on_managed_start_admitted() -> None: settlement_identity, semantic_replan_guard_scoped=replan_guard_scoped, semantic_replan_obligation_id=replan_obligation_id, + goal_ref=goal_ref, ) managed_cadence = managed_cadence_start( diff --git a/loopx/cli_rollout.py b/loopx/cli_rollout.py index b6714b5c7a..54cbdbd822 100644 --- a/loopx/cli_rollout.py +++ b/loopx/cli_rollout.py @@ -1,5 +1,6 @@ from __future__ import annotations +from collections.abc import Mapping from pathlib import Path from .history import load_registry @@ -18,6 +19,7 @@ def append_cli_rollout_event( registry_path: Path, runtime_root_arg: str | None, event_kind: str, + goal_ref: Mapping[str, object] | None = None, agent_id: str | None = None, todo_id: str | None = None, case_id: str | None = None, @@ -52,6 +54,7 @@ def append_cli_rollout_event( event = build_rollout_event( goal_id=goal_id, event_kind=event_kind, + goal_ref=goal_ref, agent_id=agent_id or str(payload.get("agent_id") or "").strip() or None, todo_id=todo_id or str(payload.get("todo_id") or "").strip() or None, case_id=case_id, diff --git a/loopx/control_plane/effect_program.ts b/loopx/control_plane/effect_program.ts index 0b5c97730d..ae2280c4a1 100644 --- a/loopx/control_plane/effect_program.ts +++ b/loopx/control_plane/effect_program.ts @@ -186,6 +186,7 @@ export interface SettlementStep { export interface SettlementPlan { identity: SettlementIdentity; steps: readonly SettlementStep[]; + goal_ref?: JsonObject; } export type SettlementBindGate = diff --git a/loopx/control_plane/goals/checkpoint_commit.ts b/loopx/control_plane/goals/checkpoint_commit.ts index 42292cc782..b70fbf4fb7 100644 --- a/loopx/control_plane/goals/checkpoint_commit.ts +++ b/loopx/control_plane/goals/checkpoint_commit.ts @@ -13,7 +13,17 @@ import {requireLocalAuthorityRuntimeRoot} from "../coordination/local_authority_ import {canonicalAuthoritySha256} from "../coordination/authority_store_codec.ts"; import {legacyCoordinationTodoLockPath} from "../coordination/legacy_writer_lock_paths.ts"; import {shadowMaintenanceLockPath, requireShadowPrimaryWriteAllowed} from "../coordination/shadow_management.ts"; -import {readQuotaSettlement, QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA} from "../quota/settlement_readback.ts"; +import { + QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA, + readAdmittedQuotaSettlementFromSnapshot, + readQuotaSettlementFromSnapshot, + readQuotaSettlementSnapshot, +} from "../quota/settlement_readback.ts"; +import { + parseQuotaAccountingOwner, + quotaAccountingOwnerLocks, + requireCurrentQuotaAccountingOwner, +} from "../quota/source_admission.ts"; import {evaluateCheckpointReadContext} from "./checkpoint_read_context.ts"; import {withCheckpointAuthority} from "./checkpoint_authority.ts"; import {goalPathSegment} from "../rollout_receipt_log.ts"; @@ -112,16 +122,29 @@ export async function commitCheckpoint(value: unknown): Promise { const statePath = resolve(requireNonEmptyString(request.state_file, "state_file")); const targets = [indexPath, shadowMaintenanceLockPath(root, identity.goal_id), legacyCoordinationTodoLockPath(root, identity.goal_id), statePath].map(path => resolve(path)); + const owner = parseQuotaAccountingOwner({ + goalRefValue: request.goal_ref, + sourceAdmissionValue: request.source_admission, + runtimeRoot: root, + goalId: identity.goal_id, + }); const retry = requireJsonObject(request.refresh_retry, "refresh retry"); const receiptPath = join(root, "goals", identity.goal_id, "checkpoint-contexts", `${createHash("sha256").update(identity.effect_id).digest("hex")}.json`); async function admission(): Promise { indexBytes(indexPath); - return await readQuotaSettlement({schema_version: QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA, + const value = {schema_version: QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA, runtime_root: root, goal_id: identity.goal_id, agent_id: identity.agent_id, todo_id: identity.todo_id, replan_obligation_id: identity.replan_obligation_id, turn_instance_id: identity.turn_instance_id, - infer_turn_instance_id: false, allow_unbound_binding: false, refresh_retry: retry}); + infer_turn_instance_id: false, allow_unbound_binding: false, refresh_retry: retry, + ...(owner.kind === "exact_source" + ? {goal_ref: request.goal_ref, source_admission: request.source_admission} + : {})}; + const snapshot = await readQuotaSettlementSnapshot(root, identity.goal_id); + return owner.kind === "exact_source" + ? readAdmittedQuotaSettlementFromSnapshot(value, snapshot) + : readQuotaSettlementFromSnapshot(value, snapshot); } function replay(readback: JsonObject): JsonObject | null { const recovery = requireJsonObject(readback.refresh_recovery, "refresh recovery"); @@ -134,33 +157,85 @@ export async function commitCheckpoint(value: unknown): Promise { return committedArtifacts(prior, runsDir); } - const claims: {target: string; token: string; claim: FileMutationLockClaim}[] = []; + const claims: { + target: string; + token: string; + claim: FileMutationLockClaim; + borrowed: boolean; + }[] = []; let adopted = false; + let sourceClaimed = false; try { if (!Array.isArray(request.locks) || request.locks.length !== targets.length) { throw new EffectRuntimeRequestError("checkpoint requires the complete internal lock handoff"); } - for (const [i, value] of request.locks.entries()) { + const checkpointWitnesses = request.locks.map((value, i) => { const witness = requireJsonObject(value, "lock witness"); const token = requireNonEmptyString(witness.token, "lock token"); if (resolve(String(witness.target)) !== targets[i]) throw new EffectRuntimeRequestError("checkpoint lock target mismatch"); - const claim = await claimFileMutationLock(targets[i], token); + return { + target: targets[i], + pid: witness.pid, + token, + }; + }); + const sourceLocks = quotaAccountingOwnerLocks(owner); + if ( + owner.kind === "exact_source" + && ( + sourceLocks[0].target !== checkpointWitnesses[0].target + || sourceLocks[0].pid !== checkpointWitnesses[0].pid + || sourceLocks[0].token !== checkpointWitnesses[0].token + ) + ) { + throw new EffectRuntimeRequestError( + "checkpoint index handoff does not match quota source admission", + ); + } + const handoff = owner.kind === "exact_source" + ? [ + ...sourceLocks.map((witness) => ({...witness, borrowed: true})), + ...checkpointWitnesses.slice(1).map((witness) => ({ + ...witness, + borrowed: false, + })), + ] + : checkpointWitnesses.map((witness) => ({...witness, borrowed: false})); + for (const [i, witness] of handoff.entries()) { + const claim = await claimFileMutationLock(witness.target, witness.token); if (!claim) break; - claims.push({target: targets[i], token, claim}); - const owner = await mutationLockOwner(targets[i]); - if (owner?.token !== token || owner.pid !== witness.pid) break; - if (i === targets.length - 1) adopted = true; + claims.push({ + target: witness.target, + token: witness.token, + claim, + borrowed: witness.borrowed, + }); + const current = await mutationLockOwner(witness.target); + if (current?.token !== witness.token || current.pid !== witness.pid) break; + if (owner.kind === "exact_source" && i === sourceLocks.length - 1) { + sourceClaimed = true; + } + if (i === handoff.length - 1) adopted = true; } if (!adopted) { // A runtime retry can arrive after a successful save released the locks. // It may only read an exact committed result, never reuse the old handoff. - for (const entry of claims.splice(0).reverse()) await releaseFileMutationLockClaim(entry.claim); + if (sourceClaimed) { + requireCurrentQuotaAccountingOwner(owner); + const result = replay(await admission()); + if (result) return result; + unknown("checkpoint lock handoff expired"); + } + for (const entry of claims.splice(0).reverse()) { + await releaseFileMutationLockClaim(entry.claim); + } return await withFileMutationLock(indexPath, async () => { const result = replay(await admission()); if (result) return result; unknown("checkpoint lock handoff expired"); }); } + requireCurrentQuotaAccountingOwner(owner); const readback = await admission(); const repeated = replay(readback); if (repeated) return repeated; @@ -210,7 +285,9 @@ export async function commitCheckpoint(value: unknown): Promise { // The effect owns the end of the handed-off critical section. Releasing the // markers here also handles a caller that timed out but is still alive. for (const entry of claims.reverse()) { - if (adopted) await releaseFileMutationLock(entry.target, entry.token, entry.claim, true); + if (adopted && !entry.borrowed) { + await releaseFileMutationLock(entry.target, entry.token, entry.claim, true); + } else await releaseFileMutationLockClaim(entry.claim); } } diff --git a/loopx/control_plane/goals/checkpoint_context_io.py b/loopx/control_plane/goals/checkpoint_context_io.py index 96dc33c83a..8eed8f9cd5 100644 --- a/loopx/control_plane/goals/checkpoint_context_io.py +++ b/loopx/control_plane/goals/checkpoint_context_io.py @@ -1,6 +1,7 @@ """Source/receipt I/O only; checkpoint_read_context.ts owns decision semantics.""" from __future__ import annotations +from collections.abc import Mapping from contextlib import ExitStack, contextmanager import hashlib import json @@ -8,7 +9,7 @@ from typing import Any, Iterator from uuid import uuid4 -from ...file_lock import exclusive_cross_runtime_file_lock, exclusive_run_index_lock, cross_runtime_lock_witness +from ...file_lock import cross_runtime_lock_witness, exclusive_cross_runtime_file_lock from ...history import load_index, load_registry from ...paths import resolve_runtime_root from ...registry import atomic_write_json @@ -16,6 +17,7 @@ from ..coordination.legacy_writer_fence import legacy_coordination_todo_lock_path from ..coordination.shadow_management import shadow_maintenance_lock_target, require_shadow_primary_write_allowed from ..effect_runtime import effect_runtime_result, EffectRuntimeRejected +from ..quota.accounting_admission import quota_accounting_admission from ..quota.settlement import SettlementIdentity, read_heartbeat_settlement from ..todos.active_state_todo_parser import parse_todo_source from ..todos.machine_region import find_todo_source_regions @@ -89,6 +91,7 @@ def _source_guard(root: Path, goal_id: str, state_file: Path) -> Iterator[None]: def _local_source_facts( root: Path, registry_path: Path, state_file: Path, identity: SettlementIdentity, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: text = state_file.read_text(encoding="utf-8") metadata, body = split_state_frontmatter(text) @@ -99,6 +102,15 @@ def _local_source_facts( active, archived, _ = parse_todo_source(text) todos = [*active["user"], *active["agent"], *archived] runs, _ = load_index(root / "goals" / identity.goal_id / "runs" / "index.jsonl") + runs = [ + run + for run in runs + if ( + run.get("goal_ref") == dict(goal_ref) + if goal_ref is not None + else "goal_ref" not in run + ) + ] newest = [run for _, run in sorted(enumerate(runs), key=lambda pair: (str(pair[1].get("generated_at") or ""), pair[0]), reverse=True)] return { @@ -109,12 +121,24 @@ def _local_source_facts( } -def _source_facts(root: Path, registry_path: Path, state_file: Path, identity: SettlementIdentity) -> dict[str, Any]: +def _source_facts( + root: Path, + registry_path: Path, + state_file: Path, + identity: SettlementIdentity, + goal_ref: Mapping[str, Any] | None = None, +) -> dict[str, Any]: # The typed owner derives Todo and complete acceptance from one head and # fails closed after cutover. Local parsed Markdown cannot override it. return _checkpoint_effect("goal.checkpoint_read_context.source", { "runtime_root": str(root.resolve()), "goal_id": identity.goal_id, - "facts": _local_source_facts(root, registry_path, state_file, identity), + "facts": _local_source_facts( + root, + registry_path, + state_file, + identity, + goal_ref, + ), }) @@ -123,6 +147,7 @@ def read_checkpoint_context( agent_id: str, todo_id: str | None, turn_instance_id: str, replan_obligation_id: str | None = None, project: Path | None = None, state_file: Path | None = None, dependency_todo_ids: list[str] | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: # Local import avoids a cycle with refresh-state's persistence adapter. from ...state_refresh import resolve_goal_state, registered_agents_for_goal @@ -134,17 +159,28 @@ def read_checkpoint_context( project_override=project, state_file_override=state_file) if agent_id not in registered_agents_for_goal(goal): raise ValueError("checkpoint-context requires a registered Agent") - with exclusive_run_index_lock(root / "goals" / goal_id / "runs" / "index.jsonl", operation="checkpoint-context"): + with quota_accounting_admission( + runtime_root=root, + registry_path=registry_path, + goal_id=goal_id, + goal_ref=goal_ref, + operation="checkpoint-context", + handoff_legacy_index=True, + ) as source_admission: require_complete_checkpoint_index(root / "goals" / goal_id / "runs" / "index.jsonl") readback = read_heartbeat_settlement(root, goal_id=goal_id, agent_id=agent_id, - todo_id=todo_id, turn_instance_id=turn_instance_id, replan_obligation_id=replan_obligation_id) + todo_id=todo_id, turn_instance_id=turn_instance_id, + replan_obligation_id=replan_obligation_id, + registry_path=registry_path, goal_ref=goal_ref, + source_admission=source_admission, + borrow_source_admission=source_admission is not None) if readback is None or readback.identity.value is None or readback.writeback_run is None: raise ValueError("checkpoint-context requires the original committed Turn writeback") identity = readback.identity.value with _source_guard(root, goal_id, path): result = _evaluate(phase="read", identity=identity.as_dict(), prior=readback.writeback_run, read_context_id=uuid4().hex, dependency_todo_ids=dependency_todo_ids or [], - facts=_source_facts(root, registry_path, path, identity)) + facts=_source_facts(root, registry_path, path, identity, goal_ref)) receipt = result.pop("receipt") atomic_write_json(_receipt_path(root, identity), receipt) return {**result, "read_context_id": receipt["read_context_id"], "settlement_identity": identity.as_dict(), @@ -158,6 +194,7 @@ def read_checkpoint_context( def checkpoint_commit_guard( *, runtime_root: Path, registry_path: Path, state_file: Path, identity: SettlementIdentity, read_context_id: str | None, + goal_ref: Mapping[str, Any] | None = None, ) -> Iterator[dict[str, Any]]: """Capture/preview under source locks. The native save repeats the check under the real provider fence; this preliminary check is not the commit.""" @@ -167,13 +204,21 @@ def checkpoint_commit_guard( except FileNotFoundError: receipt = None result = _evaluate(phase="check", identity=identity.as_dict(), read_context_id=read_context_id, - receipt=receipt, facts=_source_facts(runtime_root, registry_path, state_file, identity)) + receipt=receipt, facts=_source_facts( + runtime_root, + registry_path, + state_file, + identity, + goal_ref, + )) yield result def commit_checkpoint_run( *, runtime_root: Path, registry_path: Path, state_file: Path, identity: SettlementIdentity, refresh_retry: dict[str, Any], record: dict[str, Any], index_record: dict[str, Any], markdown: str, + goal_ref: Mapping[str, Any] | None = None, + source_admission: Mapping[str, Any] | None = None, ) -> dict[str, Any]: """Handoff the held locks and parsed bytes to one native save operation.""" root = runtime_root.resolve() @@ -185,8 +230,20 @@ def commit_checkpoint_run( "locks": [cross_runtime_lock_witness(target) for target in targets], "state_sha256": hashlib.sha256(state_file.read_bytes()).hexdigest(), "index_sha256": hashlib.sha256(index.read_bytes()).hexdigest(), - "facts": _local_source_facts(root, registry_path, state_file, identity), + "facts": _local_source_facts( + root, + registry_path, + state_file, + identity, + goal_ref, + ), "refresh_retry": refresh_retry, "record": record, "index_record": index_record, "markdown": markdown, + **({"goal_ref": dict(goal_ref)} if goal_ref is not None else {}), + **( + {"source_admission": dict(source_admission)} + if source_admission is not None + else {} + ), }) if not isinstance(result, dict) or not isinstance(result.get("ok"), bool): raise RuntimeError("invalid typed checkpoint commit result") diff --git a/loopx/control_plane/goals/first_party_host_admission.py b/loopx/control_plane/goals/first_party_host_admission.py index ae315c30b4..dc05f89433 100644 --- a/loopx/control_plane/goals/first_party_host_admission.py +++ b/loopx/control_plane/goals/first_party_host_admission.py @@ -33,7 +33,7 @@ def __init__(self, code: str) -> None: self.code = code -def _source_authority(registry_path: Path, goal_id: str) -> dict[str, Any]: +def source_goal_authority(registry_path: Path, goal_id: str) -> dict[str, Any]: if not registry_path.is_file(): return {"kind": "unavailable", "reason": "registry_missing"} try: @@ -79,7 +79,7 @@ def capture_first_party_host_goal_ref( guard_path(requested_registry, goal_id), operation="first_party_host_goal_capture", ): - authority = _source_authority(requested_registry, goal_id) + authority = source_goal_authority(requested_registry, goal_id) if authority.get("kind") != "present": raise FirstPartyHostRuntimeRejected( "goal_not_registered" @@ -149,7 +149,7 @@ def _decision( ), "operation": operation, "planned_goal_ref": self.planned_goal_ref, - "authority": _source_authority( + "authority": source_goal_authority( self.registry_path, self.goal_id, ), @@ -220,7 +220,7 @@ def source_journal_admission( "profile_id": SOURCE_SESSION_PROFILE_ID, "registry_path": str(self.registry_path), "planned_goal_ref": self.planned_goal_ref, - "authority": _source_authority( + "authority": source_goal_authority( self.registry_path, self.goal_id, ), diff --git a/loopx/control_plane/goals/goal_ref_validation.py b/loopx/control_plane/goals/goal_ref_validation.py new file mode 100644 index 0000000000..72848a8f11 --- /dev/null +++ b/loopx/control_plane/goals/goal_ref_validation.py @@ -0,0 +1,22 @@ +from __future__ import annotations + +import re + + +GOAL_ID = re.compile(r"^[A-Za-z0-9._:-]{1,200}$") +GOAL_INSTANCE_ID = re.compile(r"^ginst_[0-9a-f]{32}$") + + +def require_goal_id(goal_id: str) -> None: + if not GOAL_ID.fullmatch(goal_id): + raise ValueError("source-session goal_id must be 1-200 safe characters") + + +def exact_goal_ref(goal_id: str, goal_instance_id: str) -> dict[str, str]: + require_goal_id(goal_id) + if not GOAL_INSTANCE_ID.fullmatch(goal_instance_id): + raise ValueError("goal_instance_id must be a Goal instance identifier") + return { + "goal_id": goal_id, + "goal_instance_id": goal_instance_id, + } diff --git a/loopx/control_plane/goals/source_session_registry_state.py b/loopx/control_plane/goals/source_session_registry_state.py index 0b9a8fa370..2d6dc1bd84 100644 --- a/loopx/control_plane/goals/source_session_registry_state.py +++ b/loopx/control_plane/goals/source_session_registry_state.py @@ -3,16 +3,16 @@ import hashlib import json from pathlib import Path -import re from typing import Any from ...registry import atomic_write_json from ..projects.registry_codec import SOURCE_SESSION_PROFILE_ID from ..todos.active_state_editing import fsync_state_directory - - -GOAL_ID = re.compile(r"^[A-Za-z0-9._:-]{1,200}$") -GOAL_INSTANCE_ID = re.compile(r"^ginst_[0-9a-f]{32}$") +from .goal_ref_validation import ( + GOAL_INSTANCE_ID, + exact_goal_ref, + require_goal_id as require_goal_id, +) def canonical_digest(value: object) -> str: @@ -43,21 +43,6 @@ def write_journal(path: Path, payload: dict[str, Any]) -> None: fsync_state_directory(path) -def require_goal_id(goal_id: str) -> None: - if not GOAL_ID.fullmatch(goal_id): - raise ValueError("source-session goal_id must be 1-200 safe characters") - - -def exact_goal_ref(goal_id: str, goal_instance_id: str) -> dict[str, str]: - require_goal_id(goal_id) - if not GOAL_INSTANCE_ID.fullmatch(goal_instance_id): - raise ValueError("goal_instance_id must be a Goal instance identifier") - return { - "goal_id": goal_id, - "goal_instance_id": goal_instance_id, - } - - def required_list( registry: dict[str, Any], field: str, diff --git a/loopx/control_plane/host_adapter_settlement.py b/loopx/control_plane/host_adapter_settlement.py index ac450d5f57..df57273faf 100644 --- a/loopx/control_plane/host_adapter_settlement.py +++ b/loopx/control_plane/host_adapter_settlement.py @@ -62,6 +62,7 @@ class HostTodoSettlementRequest: vision_path: str | None = None vision_unchanged_reason: str | None = None checkpoint_read_context_id: str | None = None + goal_ref: Mapping[str, str] | None = None class HostCliRunner(Protocol): @@ -102,6 +103,8 @@ def _request_payload( } if provider_outcomes is not None: payload["provider_outcomes"] = provider_outcomes + if request.goal_ref is not None: + payload["goal_ref"] = dict(request.goal_ref) if request.vision_path or request.vision_unchanged_reason or phase in {"vision_refresh", "vision_context"}: payload.update( schema_version="loopx_host_todo_completion_transaction_v1", diff --git a/loopx/control_plane/quota/accounting_admission.py b/loopx/control_plane/quota/accounting_admission.py new file mode 100644 index 0000000000..3ea97a9eea --- /dev/null +++ b/loopx/control_plane/quota/accounting_admission.py @@ -0,0 +1,113 @@ +from __future__ import annotations + +from collections.abc import Iterator, Mapping +from contextlib import contextmanager +from pathlib import Path +from typing import Any + +from ...file_lock import ( + cross_runtime_lock_witness, + exclusive_cross_runtime_file_lock, + exclusive_file_lock, + exclusive_run_index_lock, +) +from ..projects.registry_codec import SOURCE_SESSION_PROFILE_ID + + +QUOTA_SOURCE_ADMISSION_SCHEMA = "loopx_quota_source_admission_v0" + + +def _planned_goal_ref( + value: Mapping[str, Any] | None, + *, + goal_id: str, +) -> dict[str, str] | None: + if value is None: + return None + from ..goals.source_session_registry_state import exact_goal_ref + + planned = exact_goal_ref( + str(value.get("goal_id") or ""), + str(value.get("goal_instance_id") or ""), + ) + if planned["goal_id"] != goal_id: + raise ValueError("quota GoalRef does not match goal_id") + return planned + + +def _uses_source_profile( + registry_path: Path | None, + goal_id: str, +) -> bool: + if registry_path is None or not registry_path.is_file(): + return False + from ..goals.first_party_host_admission import source_goal_authority + + return source_goal_authority(registry_path, goal_id).get("kind") in { + "present", + "absent", + } + + +@contextmanager +def quota_accounting_admission( + *, + runtime_root: Path, + registry_path: Path | None, + goal_id: str, + goal_ref: Mapping[str, Any] | None, + operation: str, + lock_legacy_index: bool = True, + handoff_legacy_index: bool = False, +) -> Iterator[dict[str, Any] | None]: + """Hold quota's index/source locks until the TypeScript owner adopts them.""" + + root = runtime_root.expanduser().resolve() + registry = registry_path.expanduser().resolve() if registry_path else None + planned = _planned_goal_ref(goal_ref, goal_id=goal_id) + source_profile = _uses_source_profile(registry, goal_id) + if not source_profile: + if planned is not None: + raise ValueError("quota GoalRef requires a source-session registry") + if not lock_legacy_index: + yield None + return + index_path = root / "goals" / goal_id / "runs" / "index.jsonl" + lock = ( + exclusive_run_index_lock + if handoff_legacy_index + else exclusive_file_lock + ) + with lock(index_path, operation=operation): + yield None + return + if planned is None or registry is None: + raise ValueError("source-session quota commit requires an exact GoalRef") + + from ..goals.first_party_host_admission import source_goal_authority + from ..goals.source_session_registry_state import guard_path + + index_path = root / "goals" / goal_id / "runs" / "index.jsonl" + source_target = guard_path(registry, goal_id) + with exclusive_run_index_lock(index_path, operation=operation): + with exclusive_cross_runtime_file_lock( + source_target, + operation=operation, + ): + yield { + "schema_version": QUOTA_SOURCE_ADMISSION_SCHEMA, + "profile_id": SOURCE_SESSION_PROFILE_ID, + "registry_path": str(registry), + "planned_goal_ref": planned, + "authority": source_goal_authority(registry, goal_id), + "locks": [ + { + "role": "run_index", + **cross_runtime_lock_witness(index_path), + }, + { + "role": "source_guard", + **cross_runtime_lock_witness(source_target), + }, + ], + } diff --git a/loopx/control_plane/quota/accounting_artifact_transaction.ts b/loopx/control_plane/quota/accounting_artifact_transaction.ts index 70a0fde025..576deb4323 100644 --- a/loopx/control_plane/quota/accounting_artifact_transaction.ts +++ b/loopx/control_plane/quota/accounting_artifact_transaction.ts @@ -10,6 +10,7 @@ import { atomicWriteText, withFileMutationLock, } from "../effect_runtime_io.ts"; +import { parseExactGoalRef } from "../goals/goal_instance_identity.ts"; import { jsonObject, optionalNonEmptyString as optionalString, @@ -65,12 +66,17 @@ export interface QuotaAccountingArtifactReceipt extends JsonObject { index_record: JsonObject; markdown: string; payload: JsonObject; + goal_ref?: JsonObject; } export type QuotaAccountingEffectResolution = | { kind: "absent" } | { kind: "matched"; record: JsonObject } - | { kind: "conflict"; reason: string }; + | { + kind: "conflict"; + reason: string; + reasonCode: "effect_id_conflict" | "goal_instance_conflict"; + }; export interface QuotaAccountingArtifactPrepareContext { jsonPath: string; @@ -101,6 +107,8 @@ export interface QuotaAccountingArtifactCommitRequest { effectId: string; requestDigest: string; expectedIndexDigest: string | null; + goalRef?: JsonObject; + indexLockHeld?: boolean; prepare: ( context: QuotaAccountingArtifactPrepareContext, ) => @@ -117,7 +125,10 @@ export type QuotaAccountingArtifactCommitOutcome = | { status: "conflict"; reason: string; - reasonCode: "effect_id_conflict" | "index_digest_conflict"; + reasonCode: + | "effect_id_conflict" + | "goal_instance_conflict" + | "index_digest_conflict"; indexDigest: string | null; } | { @@ -156,6 +167,42 @@ function sha256Bytes(value: Uint8Array): string { return `sha256:${createHash("sha256").update(value).digest("hex")}`; } +function exactGoalRef(value: unknown, label: string): JsonObject | null { + if (value === undefined) return null; + const parsed = parseExactGoalRef(value); + if (parsed.kind === "invalid") { + throw new EffectRuntimeRequestError( + `${label} must be an exact GoalRef`, + "malformed_transaction_receipt", + ); + } + return { + goal_id: parsed.value.goalId.value, + goal_instance_id: parsed.value.goalInstanceId.value, + }; +} + +function sameGoalRef(left: JsonObject | null, right: JsonObject | null): boolean { + return left?.goal_id === right?.goal_id + && left?.goal_instance_id === right?.goal_instance_id; +} + +function ownerConflict( + existing: JsonObject | null, + requested: JsonObject | null, + label: string, +): string | null { + if (existing === null && requested === null) return null; + if (sameGoalRef(existing, requested)) return null; + if (existing === null) { + return `${label} belongs to a legacy Goal alias and cannot authorize an exact Goal instance`; + } + if (requested === null) { + return `${label} belongs to an exact Goal instance and cannot be consumed without GoalRef admission`; + } + return `${label} belongs to a different Goal instance`; +} + function runStem(generatedAt: string): string { const stem = generatedAt.replace(/[^0-9A-Za-z-]+/g, "-").replace(/^-+|-+$/g, ""); if (!stem) { @@ -384,6 +431,7 @@ function resolveEffectIdentity( contract: QuotaAccountingArtifactContract, record: JsonObject, expectedEffectId: string, + expectedGoalRef: JsonObject | null, ): QuotaAccountingEffectResolution { const rawMetadata = record[contract.metadataField]; const recordEffect = effectIdentityValue(record.effect_ref); @@ -395,6 +443,7 @@ function resolveEffectIdentity( return { kind: "conflict", reason: `${contract.label} index row has malformed effect metadata`, + reasonCode: "effect_id_conflict", }; } const metadataEffect = effectIdentityValue(metadata?.effect_id); @@ -409,6 +458,7 @@ function resolveEffectIdentity( return { kind: "conflict", reason: `${contract.label} index row has malformed effect identity`, + reasonCode: "effect_id_conflict", }; } if ( @@ -419,6 +469,19 @@ function resolveEffectIdentity( return { kind: "conflict", reason: `${contract.label} index row has conflicting effect identities`, + reasonCode: "effect_id_conflict", + }; + } + const goalConflict = ownerConflict( + exactGoalRef(record.goal_ref, `${contract.label} index row goal_ref`), + expectedGoalRef, + `${contract.label} effect identity`, + ); + if (goalConflict) { + return { + kind: "conflict", + reason: goalConflict, + reasonCode: "goal_instance_conflict", }; } return { kind: "matched", record }; @@ -428,11 +491,17 @@ export function resolveQuotaAccountingEffect( kind: QuotaAccountingArtifactKind, records: readonly JsonObject[], effectId: string, + goalRef: JsonObject | null = null, ): QuotaAccountingEffectResolution { const contract = contractFor(kind); for (const record of [...records].reverse()) { if (record.classification !== contract.classification) continue; - const resolution = resolveEffectIdentity(contract, record, effectId); + const resolution = resolveEffectIdentity( + contract, + record, + effectId, + goalRef, + ); if (resolution.kind !== "absent") return resolution; } return { kind: "absent" }; @@ -443,6 +512,8 @@ export async function lookupQuotaAccountingReplay( indexPath: string, effectId: string, readOnly: boolean, + goalRef: JsonObject | null = null, + indexLockHeld = false, ): Promise<{ resolution: QuotaAccountingEffectResolution; indexDigest: string | null; @@ -454,11 +525,14 @@ export async function lookupQuotaAccountingReplay( kind, parseQuotaAccountingIndex(content), effectId, + goalRef, ), indexDigest: await quotaAccountingIndexDigest(indexPath), }; }; - return readOnly ? await lookup() : await withFileMutationLock(indexPath, lookup); + return readOnly || indexLockHeld + ? await lookup() + : await withFileMutationLock(indexPath, lookup); } function receiptObject( @@ -481,6 +555,10 @@ function receiptObject( receipt.expected_index_bytes, "receipt.expected_index_bytes", ); + const goalRef = exactGoalRef( + receipt.goal_ref, + "transaction receipt goal_ref", + ); if (expectedIndexBytes < 0) { throw new EffectRuntimeRequestError( "receipt.expected_index_bytes cannot be negative", @@ -503,6 +581,7 @@ function receiptObject( index_record: requiredObject(receipt.index_record, "receipt.index_record"), markdown: requiredString(receipt.markdown, "receipt.markdown"), payload: requiredObject(receipt.payload, "receipt.payload"), + ...(goalRef === null ? {} : { goal_ref: goalRef }), }; } @@ -573,6 +652,28 @@ function validateReceiptPaths( receipt.index_record[contract.metadataField], `receipt.index_record.${contract.metadataField}`, ); + const receiptGoalRef = receipt.goal_ref ?? null; + const quotaEvent = requiredObject( + receipt.record.quota_event, + "receipt.record.quota_event", + ); + for (const [label, value] of [ + ["record", receipt.record.goal_ref], + ["quota event", quotaEvent.goal_ref], + ["index record", receipt.index_record.goal_ref], + ["payload", receipt.payload.goal_ref], + ] as const) { + const projected = exactGoalRef( + value, + `receipt ${label} goal_ref`, + ); + if (!sameGoalRef(projected, receiptGoalRef)) { + throw new EffectRuntimeRequestError( + `${contract.label} transaction receipt ${label} GoalRef does not match its owner`, + "malformed_transaction_receipt", + ); + } + } for (const [label, projection, expected] of [ ["record classification", receipt.record.classification, contract.classification], ["index classification", receipt.index_record.classification, contract.classification], @@ -740,11 +841,12 @@ async function ensureReceiptArtifacts( kind, index.records, receipt.effect_id, + receipt.goal_ref ?? null, ); if (matchResolution.kind === "conflict") { throw new EffectRuntimeRequestError( matchResolution.reason, - "effect_id_conflict", + matchResolution.reasonCode, ); } const match = matchResolution.kind === "matched" @@ -794,11 +896,28 @@ export async function commitQuotaAccountingArtifactTransaction( ): Promise { const contract = contractFor(request.kind); const indexPath = join(request.runsDir, "index.jsonl"); - return await withFileMutationLock(indexPath, async () => { + const requestedGoalRef = exactGoalRef( + request.goalRef, + `${contract.label} request goal_ref`, + ); + const commit = async (): Promise => { await rejectSymlinkPath(indexPath, `${contract.label} run index`); const receiptPath = transactionPath(contract, request.runsDir, request.effectId); const existingReceipt = await readReceipt(contract, receiptPath, request.runsDir); if (existingReceipt) { + const conflict = ownerConflict( + existingReceipt.goal_ref ?? null, + requestedGoalRef, + `${contract.label} transaction`, + ); + if (conflict) { + return { + status: "conflict", + reason: conflict, + reasonCode: "goal_instance_conflict", + indexDigest: await quotaAccountingIndexDigest(indexPath), + }; + } if ( existingReceipt.effect_id !== request.effectId || existingReceipt.request_digest !== request.requestDigest @@ -849,12 +968,13 @@ export async function commitQuotaAccountingArtifactTransaction( request.kind, currentRecords, request.effectId, + requestedGoalRef, ); if (duplicateResolution.kind === "conflict") { return { status: "conflict", reason: duplicateResolution.reason, - reasonCode: "effect_id_conflict", + reasonCode: duplicateResolution.reasonCode, indexDigest: currentDigest, }; } @@ -902,6 +1022,7 @@ export async function commitQuotaAccountingArtifactTransaction( index_record: preparation.indexRecord, markdown: preparation.markdown, payload: preparation.payload, + ...(requestedGoalRef === null ? {} : { goal_ref: requestedGoalRef }), } satisfies QuotaAccountingArtifactReceipt; validateReceiptPaths(contract, request.runsDir, prepared); await atomicWriteJson(receiptPath, prepared); @@ -916,5 +1037,8 @@ export async function commitQuotaAccountingArtifactTransaction( receipt: committedReceipt, indexDigest: await quotaAccountingIndexDigest(indexPath), }; - }); + }; + return request.indexLockHeld + ? await commit() + : await withFileMutationLock(indexPath, commit); } diff --git a/loopx/control_plane/quota/effect_program.py b/loopx/control_plane/quota/effect_program.py index 316acc3bcb..df1a79fa29 100644 --- a/loopx/control_plane/quota/effect_program.py +++ b/loopx/control_plane/quota/effect_program.py @@ -26,6 +26,7 @@ receipt_bound_terminal_phase, settlement_result_payload, ) +from ..goals.goal_ref_validation import exact_goal_ref __all__ = [ "SETTLEMENT_IDENTITY_SCHEMA_VERSION", @@ -76,6 +77,23 @@ def _settlement_actor_args(arguments: str, agent_id: str) -> str: return f"{arguments} --agent-id {shlex.quote(agent_id)}" +def _settlement_goal_ref( + goal_ref: Mapping[str, object] | None, + *, + goal_id: str, +) -> dict[str, str] | None: + if goal_ref is None: + return None + + normalized = exact_goal_ref( + str(goal_ref.get("goal_id") or ""), + str(goal_ref.get("goal_instance_id") or ""), + ) + if normalized["goal_id"] != goal_id: + raise ValueError("settlement GoalRef does not match goal_id") + return normalized + + def build_codex_app_settlement_plan( *, goal_id: str, @@ -89,6 +107,7 @@ def build_codex_app_settlement_plan( writeback_path_args: str = "", delivery_boundary: str | None = None, quota_spend_source: str = "heartbeat", + goal_ref: Mapping[str, object] | None = None, ) -> SettlementPlan: return build_turn_scoped_cli_settlement_plan( goal_id=goal_id, @@ -102,6 +121,7 @@ def build_codex_app_settlement_plan( writeback_path_args=writeback_path_args, delivery_boundary=delivery_boundary, quota_spend_source=quota_spend_source, + goal_ref=goal_ref, ) @@ -118,6 +138,7 @@ def build_turn_scoped_cli_settlement_plan( writeback_path_args: str = "", delivery_boundary: str | None = None, quota_spend_source: str = "heartbeat", + goal_ref: Mapping[str, object] | None = None, ) -> SettlementPlan: if bool(todo_id) == bool(replan_obligation_id): raise ValueError( @@ -138,6 +159,7 @@ def build_turn_scoped_cli_settlement_plan( ) scoped_cli_args = _settlement_actor_args(scoped_cli_args, identity.agent_id) lifecycle_actor_args = _settlement_actor_args(lifecycle_actor_args, identity.agent_id) + normalized_goal_ref = _settlement_goal_ref(goal_ref, goal_id=identity.goal_id) quoted_turn = _quoted_turn_ref(turn_instance_id) binding_arg = ( f" --todo-id {shlex.quote(todo_id)}" @@ -145,6 +167,11 @@ def build_turn_scoped_cli_settlement_plan( else f" --replan-obligation-id {shlex.quote(str(replan_obligation_id))}" ) turn_arg = f" --turn-instance-id {quoted_turn}" + goal_ref_arg = ( + f" --goal-instance-id {shlex.quote(normalized_goal_ref['goal_instance_id'])}" + if normalized_goal_ref is not None + else "" + ) boundary_arg = ( " --delivery-boundary in_flight_continuation" if delivery_boundary == "in_flight_continuation" @@ -153,22 +180,23 @@ def build_turn_scoped_cli_settlement_plan( cli_prefix = command_prefix.strip() or "loopx" ordinary_completion = ( f"{cli_prefix} todo complete --goal-id {shlex.quote(goal_id)}{binding_arg}" - f"{lifecycle_actor_args}{turn_arg} --evidence ''" + f"{lifecycle_actor_args}{turn_arg}{goal_ref_arg} --evidence ''" ) terminal_closeout = ordinary_completion + " --no-follow-up" writeback = ( f"{cli_prefix} refresh-state --goal-id {shlex.quote(goal_id)} " "--classification --delivery-batch-scale " - f"--delivery-outcome {boundary_arg}{binding_arg}{turn_arg}" + f"--delivery-outcome {boundary_arg}{binding_arg}{turn_arg}{goal_ref_arg}" f"{scoped_cli_args}{writeback_path_args}" ) spend = ( f"{cli_prefix} quota spend-slot --goal-id {shlex.quote(goal_id)} --slots 1 " - f"--source {quota_spend_source} --execute{binding_arg}{turn_arg}" + f"--source {quota_spend_source} --execute{binding_arg}{turn_arg}{goal_ref_arg}" f"{scoped_cli_args}" ) payload = effect_runtime_result("settlement.turn_scoped_cli_plan", { "identity": identity.as_dict(), "delivery_boundary": delivery_boundary, + **({"goal_ref": normalized_goal_ref} if normalized_goal_ref is not None else {}), "command_templates": { "todo_completion": ordinary_completion, "durable_writeback": writeback, "quota_spend": spend, "terminal_closeout": terminal_closeout, @@ -219,4 +247,20 @@ def settlement_binding_args(plan: Mapping[str, Any] | None) -> str: if todo_id else f" --replan-obligation-id {shlex.quote(replan_obligation_id)}" ) - return binding_arg + (f" --turn-instance-id {_quoted_turn_ref(turn_instance_id)}") + goal_ref = plan.get("goal_ref") + goal_ref_arg = "" + if isinstance(goal_ref, Mapping): + normalized_goal_ref = _settlement_goal_ref( + goal_ref, + goal_id=str(identity.get("goal_id") or ""), + ) + assert normalized_goal_ref is not None + goal_ref_arg = ( + f" --goal-instance-id " + f"{shlex.quote(normalized_goal_ref['goal_instance_id'])}" + ) + return ( + binding_arg + + f" --turn-instance-id {_quoted_turn_ref(turn_instance_id)}" + + goal_ref_arg + ) diff --git a/loopx/control_plane/quota/heartbeat_receipt.py b/loopx/control_plane/quota/heartbeat_receipt.py index f02fec2e14..bd8fad289d 100644 --- a/loopx/control_plane/quota/heartbeat_receipt.py +++ b/loopx/control_plane/quota/heartbeat_receipt.py @@ -27,7 +27,20 @@ def _heartbeat_receipt_events( goal_id: str, agent_id: str, turn_instance_id: str, + goal_ref: Mapping[str, object] | None, ) -> list[dict[str, object]]: + from ..goals.source_session_registry_state import exact_goal_ref + + expected_goal_ref = ( + exact_goal_ref( + str(goal_ref.get("goal_id") or ""), + str(goal_ref.get("goal_instance_id") or ""), + ) + if goal_ref is not None + else None + ) + if expected_goal_ref is not None and expected_goal_ref["goal_id"] != goal_id: + raise ValueError("heartbeat receipt GoalRef does not match goal_id") return [ event for event in events @@ -35,6 +48,11 @@ def _heartbeat_receipt_events( and str(event.get("goal_id") or "") == goal_id and str(event.get("agent_id") or "") == agent_id and str(event.get("run_id") or "") == turn_instance_id + and ( + event.get("goal_ref") == expected_goal_ref + if expected_goal_ref is not None + else "goal_ref" not in event + ) ] @@ -154,6 +172,7 @@ def find_heartbeat_receipt( goal_id: str, agent_id: str, turn_instance_id: str, + goal_ref: Mapping[str, object] | None = None, ) -> dict[str, object] | None: events = load_rollout_events(rollout_event_log_path(runtime_root, goal_id)) return _effective_heartbeat_receipt( @@ -162,6 +181,7 @@ def find_heartbeat_receipt( goal_id=goal_id, agent_id=agent_id, turn_instance_id=turn_instance_id, + goal_ref=goal_ref, ) ) @@ -172,6 +192,7 @@ def ensure_turn_heartbeat_settlement_receipt( *, semantic_replan_guard_scoped: bool, semantic_replan_obligation_id: str | None, + goal_ref: Mapping[str, object] | None = None, ) -> dict[str, object]: """Idempotently bind a Turn-created quota guard to its settlement identity. @@ -199,6 +220,7 @@ def ensure_turn_heartbeat_settlement_receipt( goal_id=identity.goal_id, agent_id=identity.agent_id, turn_instance_id=identity.turn_instance_id, + goal_ref=goal_ref, ) effective = _effective_heartbeat_receipt(matching) expected = ( @@ -261,6 +283,7 @@ def ensure_turn_heartbeat_settlement_receipt( receipt = build_rollout_event( goal_id=identity.goal_id, event_kind="quota_should_run", + goal_ref=goal_ref, agent_id=identity.agent_id, todo_id=identity.todo_id, run_id=identity.turn_instance_id, @@ -282,6 +305,7 @@ def retain_pending_heartbeat_action_selection( turn_instance_id: str, todo_id: str, reason: str, + goal_ref: Mapping[str, object] | None = None, ) -> tuple[dict[str, object], bool]: """Append an identity-less revision retaining one explicit Todo choice. @@ -306,6 +330,7 @@ def retain_pending_heartbeat_action_selection( goal_id=goal_id, agent_id=agent_id, turn_instance_id=turn_instance_id, + goal_ref=goal_ref, ) effective = _effective_heartbeat_receipt(matching) if effective is None: @@ -341,6 +366,7 @@ def retain_pending_heartbeat_action_selection( retained = build_rollout_event( goal_id=goal_id, event_kind="quota_should_run", + goal_ref=goal_ref, agent_id=agent_id, run_id=turn_instance_id, status="action_selection_deferred", @@ -368,6 +394,7 @@ def upgrade_identityless_heartbeat_receipt( status: str, summary: str, details: Mapping[str, object], + goal_ref: Mapping[str, object] | None = None, ) -> tuple[dict[str, object], bool]: """Append one settlement-bound receipt after an identity-less same-turn guard. @@ -414,6 +441,7 @@ def upgrade_identityless_heartbeat_receipt( goal_id=goal_id, agent_id=agent_id, turn_instance_id=turn_instance_id, + goal_ref=goal_ref, ) effective = _effective_heartbeat_receipt(matching) if effective is None: @@ -467,6 +495,7 @@ def upgrade_identityless_heartbeat_receipt( corrected = build_rollout_event( goal_id=goal_id, event_kind="quota_should_run", + goal_ref=goal_ref, agent_id=agent_id, todo_id=normalized_todo_id or None, run_id=turn_instance_id, diff --git a/loopx/control_plane/quota/live_decision.py b/loopx/control_plane/quota/live_decision.py index 802bebc7fb..a44eb76586 100644 --- a/loopx/control_plane/quota/live_decision.py +++ b/loopx/control_plane/quota/live_decision.py @@ -492,6 +492,7 @@ def build_live_quota_should_run_decision( interaction_projection_hooks: Sequence[InteractionProjectionHookRegistration] | None = None, turn_start_hook_dispatch: Mapping[str, Any] | None = None, + goal_ref: Mapping[str, object] | None = None, ) -> dict[str, Any]: """Build one live CLI decision while keeping host observation injectable.""" resolved_context = resolve_scheduler_execution_context(scheduler_execution_context) @@ -539,6 +540,8 @@ def build_live_quota_should_run_decision( todo_id=receipt_bound_todo_id, turn_instance_id=turn_instance_id, replan_obligation_id=receipt_bound_replan_obligation_id, + registry_path=registry_path, + goal_ref=goal_ref, ) receipt_bound_monitor_phase = ( settlement_readback.monitor_phase if settlement_readback else None @@ -610,6 +613,7 @@ def build_live_quota_should_run_decision( receipt_bound_replan_guard_scoped=receipt_bound_replan_guard_scoped, turn_instance_id=turn_instance_id, runtime_root=runtime_root, + goal_ref=goal_ref, ) _apply_retained_action_selection_reentry( payload, @@ -663,7 +667,8 @@ def build_live_quota_should_run_decision( ) if original_replan_settlement and settlement_readback is not None: attach_settlement_progress(payload, settlement_readback, - registry_path=registry_path, runtime_root=runtime_root) + registry_path=registry_path, runtime_root=runtime_root, + goal_ref=goal_ref) if receipt_bound_replay_phase is not ReceiptBoundReplayPhase.SETTLED and not original_replan_settlement: apply_unsettled_host_turn_recovery_if_required( payload, @@ -674,6 +679,7 @@ def build_live_quota_should_run_decision( current_turn_instance_id=turn_instance_id, available_capabilities=available_capabilities, scheduler_execution_context=resolved_context, + goal_ref=goal_ref, ) if ( receipt_bound_todo_id and agent_id and turn_instance_id diff --git a/loopx/control_plane/quota/monitor_poll.py b/loopx/control_plane/quota/monitor_poll.py index 1dcc907306..2ee4a3a585 100644 --- a/loopx/control_plane/quota/monitor_poll.py +++ b/loopx/control_plane/quota/monitor_poll.py @@ -8,7 +8,6 @@ from pathlib import Path from typing import Any -from ...file_lock import LockAcquisitionPolicy, exclusive_file_lock from ...turn_identity import normalize_turn_instance_id from ..effect_runtime import EffectRuntimeRejected, effect_runtime_result from ..runtime.time import now_local_iso @@ -27,6 +26,7 @@ ) from ..todos.todo_semantics import todo_item_task_class from .decision_summary import compact_quota_decision, quota_decision_agent_id +from .accounting_admission import quota_accounting_admission from .spend_sources import DEFAULT_SLOT_SPEND_SOURCE QUOTA_MONITOR_POLL_CLASSIFICATION = "quota_monitor_poll" @@ -296,6 +296,8 @@ def _request( observation: dict[str, Any], provider_receipt: Mapping[str, Any] | None = None, status_reload_warning: Mapping[str, Any] | None = None, + goal_ref: Mapping[str, Any] | None = None, + source_admission: Mapping[str, Any] | None = None, ) -> dict[str, Any]: return { "schema_version": ("loopx_quota_monitor_poll_commit_request_v1" if observation.get("lease_proof") is not None @@ -319,6 +321,12 @@ def _request( if status_reload_warning is not None else None ), + **({"goal_ref": dict(goal_ref)} if goal_ref is not None else {}), + **( + {"source_admission": dict(source_admission)} + if source_admission is not None + else {} + ), } @@ -397,6 +405,7 @@ def _find_monitor_poll_turn( turn_instance_id: str, todo_id: str | None = None, target_key: str | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any] | None: normalized_todo_id = normalize_todo_id(todo_id) if todo_id else None normalized_target_key = str(target_key or "").strip() or None @@ -416,6 +425,11 @@ def _find_monitor_poll_turn( and str(row.get("goal_id") or "") == goal_id and str(row.get("agent_id") or "") == agent_id and str(row.get("turn_instance_id") or "") == turn_instance_id + and ( + row.get("goal_ref") == dict(goal_ref) + if goal_ref is not None + else "goal_ref" not in row + ) and ( normalized_todo_id is None or normalize_todo_id(row.get("todo_id")) == normalized_todo_id @@ -438,6 +452,7 @@ def find_quota_monitor_poll_turn( turn_instance_id: str, todo_id: str | None = None, target_key: str | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any] | None: """Return the latest matching monitor observation for one heartbeat turn.""" @@ -451,6 +466,7 @@ def find_quota_monitor_poll_turn( turn_instance_id=normalized_turn_id, todo_id=todo_id, target_key=target_key, + goal_ref=goal_ref, ) @@ -471,6 +487,7 @@ def _monitor_poll_effect_id( turn_instance_id: str | None, todo_id: str | None, target_key: str | None, + goal_ref: Mapping[str, Any] | None, ) -> str: if not turn_instance_id: return f"quota-monitor-poll:{goal_id}:{uuid.uuid4().hex}" @@ -485,6 +502,7 @@ def _monitor_poll_effect_id( turn_instance_id=turn_instance_id, todo_id=todo_id, target_key=None if todo_id else target_key, + goal_ref=goal_ref, ) existing_effect_id = _persisted_monitor_effect_id(existing) if existing_effect_id: @@ -717,6 +735,7 @@ def record_quota_monitor_poll_for_decision( turn_instance_id: str | None = None, _index_lock_held: bool = False, status_reloader: Callable[[], dict[str, Any]] | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: del render_markdown normalized_turn_id = normalize_turn_instance_id(turn_instance_id) @@ -736,6 +755,7 @@ def record_quota_monitor_poll_for_decision( turn_instance_id=normalized_turn_id, todo_id=safe_todo_id, target_key=safe_target_key, + goal_ref=goal_ref, ) if use_current_task_lease: from .monitor_poll_lease_transport import current_monitor_lease_proof @@ -829,7 +849,9 @@ def failure( ) return payload - def transact() -> tuple[dict[str, Any], dict[str, Any]]: + def transact( + source_admission: Mapping[str, Any] | None, + ) -> tuple[dict[str, Any], dict[str, Any]]: common = { "effect_id": effect_id, "runtime_root": runtime_root, @@ -841,6 +863,8 @@ def transact() -> tuple[dict[str, Any], dict[str, Any]]: "turn_instance_id": normalized_turn_id, "decision": decision, "observation": observation, + "goal_ref": goal_ref, + "source_admission": source_admission, } after_status = deepcopy(status_payload) provider_needed = bool(safe_todo_id or safe_target_key) @@ -887,16 +911,22 @@ def transact() -> tuple[dict[str, Any], dict[str, Any]]: return native, after_status try: - if execute and not _index_lock_held: - with exclusive_file_lock( - index_path, - policy=LockAcquisitionPolicy.MONITOR, - agent_id=decision_agent_id or agent_id, - operation="quota_monitor_poll_index", - ): - native, after_status = transact() + if _index_lock_held: + if goal_ref is not None: + raise ValueError( + "exact GoalRef monitor poll requires quota source admission" + ) + native, after_status = transact(None) else: - native, after_status = transact() + with quota_accounting_admission( + runtime_root=runtime_root, + registry_path=registry_path, + goal_id=goal_id, + goal_ref=goal_ref, + operation="quota_monitor_poll_index", + lock_legacy_index=execute, + ) as source_admission: + native, after_status = transact(source_admission) except ValueError as exc: payload = failure( str(exc), diff --git a/loopx/control_plane/quota/monitor_poll_commit.ts b/loopx/control_plane/quota/monitor_poll_commit.ts index 6ba82cbfff..f906f690aa 100644 --- a/loopx/control_plane/quota/monitor_poll_commit.ts +++ b/loopx/control_plane/quota/monitor_poll_commit.ts @@ -8,7 +8,17 @@ import { basename, dirname, join, resolve } from "node:path"; import { monitorSuccessorIntent, monitorSuccessorRoute } from "../scheduler/monitor_successor.ts"; import { normalizeTodoCapabilities } from "../todos/work_requirements.ts"; import { parseProjectionDelivery } from "../todos/projection_delivery.ts"; -import { readQuotaSettlement, QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA } from "./settlement_readback.ts"; +import { + QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA, + readQuotaSettlementForAdmittedOwnerFromSnapshot, + readQuotaSettlementSnapshot, +} from "./settlement_readback.ts"; +import { + parseQuotaAccountingOwner, + quotaGoalRef, + withBorrowedQuotaAccountingOwner, + type QuotaAccountingOwner, +} from "./source_admission.ts"; import type { JsonObject } from "../effect_program.ts"; import { EffectRuntimeRequestError } from "../effect_runtime_errors.ts"; @@ -131,6 +141,7 @@ interface MonitorRequest { observation: MonitorObservation; provider_receipt: JsonObject | null; status_reload_warning: JsonObject | null; + owner: QuotaAccountingOwner; } interface MonitorProviderPlan extends JsonObject { @@ -467,6 +478,12 @@ function requestObject(value: unknown): MonitorRequest { observation, provider_receipt: jsonObject(request.provider_receipt), status_reload_warning: jsonObject(request.status_reload_warning), + owner: parseQuotaAccountingOwner({ + goalRefValue: request.goal_ref, + sourceAdmissionValue: request.source_admission, + runtimeRoot: runtimeRoot ?? "", + goalId, + }), }; } @@ -548,13 +565,17 @@ interface Admission { async function readAuxiliarySettlement(request: MonitorRequest): Promise { if (!request.runtime_root || !request.turn_instance_id || !request.observation.actor_agent_id || !request.observation.settlement_todo_id) return null; - return await readQuotaSettlement({ + const snapshot = await readQuotaSettlementSnapshot( + request.runtime_root, + request.goal_id, + ); + return readQuotaSettlementForAdmittedOwnerFromSnapshot({ schema_version: QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA, runtime_root: request.runtime_root, goal_id: request.goal_id, agent_id: request.observation.actor_agent_id, todo_id: request.observation.settlement_todo_id, turn_instance_id: request.turn_instance_id, replan_obligation_id: null, infer_turn_instance_id: false, allow_unbound_binding: false, - }); + }, request.owner, snapshot); } async function auxiliaryMonitorAllowed( @@ -845,6 +866,11 @@ function buildRecord(request: MonitorRequest, allowed: Admission): JsonObject { monitor_target: target, monitor_event: event, }; + const goalRef = quotaGoalRef(request.owner); + if (goalRef !== null) { + record.goal_ref = goalRef; + event.goal_ref = goalRef; + } if (request.decision.agent_id) { record.agent_id = request.decision.agent_id; event.agent_id = request.decision.agent_id; @@ -875,6 +901,7 @@ function requestDigest(request: MonitorRequest): string { // projected decision during a retry, while the logical observation remains // the same effect. Mutable phase/CAS/provider fields are fenced separately. const observation: JsonObject = { ...request.observation }; + const goalRef = quotaGoalRef(request.owner); if (!observation.settlement_todo_id) delete observation.settlement_todo_id; return sha256(pythonJson({ schema_version: request.schema_version, @@ -883,6 +910,7 @@ function requestDigest(request: MonitorRequest): string { goal_id: request.goal_id, source: request.source, turn_instance_id: request.turn_instance_id, + ...(goalRef === null ? {} : { goal_ref: goalRef }), observation, })); } @@ -1455,6 +1483,8 @@ function indexRecordFor( request_digest: fingerprint, }, }; + const goalRef = quotaGoalRef(request.owner); + if (goalRef !== null) indexRecord.goal_ref = goalRef; for (const [field, value] of Object.entries({ agent_id: record.agent_id, turn_instance_id: record.turn_instance_id, @@ -1581,6 +1611,8 @@ async function payloadFor( : `${request.execute ? "appended" : "dry-run preview"} monitor poll event: ` + `${request.goal_id} effective_action=${request.decision.effective_action}`, }; + const goalRef = quotaGoalRef(request.owner); + if (goalRef !== null) payload.goal_ref = goalRef; if (request.turn_instance_id) { payload.turn_instance_id = request.turn_instance_id; payload.replayed = options.replayed; @@ -2023,10 +2055,10 @@ function validateNoEffect(receipt: JsonObject | null, plan: MonitorProviderPlan) } } -export async function evaluateQuotaMonitorPollCommit( - value: unknown, +async function evaluateQuotaMonitorPollRequest( + request: MonitorRequest, + indexLockHeld: boolean, ): Promise { - const request = requestObject(value); const fingerprint = requestDigest(request); if (request.phase === "provider_rejected" && !request.execute) { throw new EffectRuntimeRequestError("provider rejection recovery requires execute"); @@ -2122,7 +2154,7 @@ export async function evaluateQuotaMonitorPollCommit( } const runsDir = join(request.runtime_root, "goals", request.goal_id, "runs"); const indexPath = join(runsDir, "index.jsonl"); - return await withFileMutationLock(indexPath, async () => { + const commit = async (): Promise => { const receiptPath = transactionPath(runsDir, request.effect_id); const existing = await readReceipt(receiptPath); if (existing) { @@ -2398,5 +2430,19 @@ export async function evaluateQuotaMonitorPollCommit( null, indexRecord, ); - }); + }; + return indexLockHeld + ? await commit() + : await withFileMutationLock(indexPath, commit); +} + +export async function evaluateQuotaMonitorPollCommit( + value: unknown, +): Promise { + const request = requestObject(value); + return await withBorrowedQuotaAccountingOwner( + request.owner, + async (indexLockHeld) => + await evaluateQuotaMonitorPollRequest(request, indexLockHeld), + ); } diff --git a/loopx/control_plane/quota/refresh_external_delivery.py b/loopx/control_plane/quota/refresh_external_delivery.py index 69cae8c31b..bf9ecadbcb 100644 --- a/loopx/control_plane/quota/refresh_external_delivery.py +++ b/loopx/control_plane/quota/refresh_external_delivery.py @@ -1,4 +1,5 @@ """Persist the typed resume decision inside the existing refresh serialization lock.""" +from collections.abc import Mapping from pathlib import Path from typing import Any @@ -9,6 +10,7 @@ def finish_external_delivery_refresh( payload: dict[str, Any], readback: QuotaSettlementReadback | None, runtime_root: Path, *, dry_run: bool, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: if readback is None: return payload @@ -29,6 +31,7 @@ def finish_external_delivery_refresh( todo_id=identity.todo_id, run_id=identity.turn_instance_id, event_kind="refresh_external_delivery", status=transition["state"], summary="Refresh external delivery preference recorded.", details=transition, + goal_ref=goal_ref, ), ) plan["transition"] = None # The planned journal effect was committed once. @@ -38,6 +41,7 @@ def finish_external_delivery_refresh( def refresh_recovery_payload( readback: QuotaSettlementReadback, *, registry_path: Path, runtime_root: Path, goal_id: str, dry_run: bool, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any] | None: recovery = readback.refresh_recovery identity = readback.identity.value @@ -50,7 +54,13 @@ def refresh_recovery_payload( # Record a requested pause before a new writeback can commit. If later # validation fails, retaining the pause is conservative and retryable. if (plan.get("transition") or {}).get("state") == "paused": - finish_external_delivery_refresh({"ok": True}, readback, runtime_root, dry_run=dry_run) + finish_external_delivery_refresh( + {"ok": True}, + readback, + runtime_root, + dry_run=dry_run, + goal_ref=goal_ref, + ) return None payload = { **(readback.writeback_run or {}), "ok": decision != "reject" and not delivery_error, @@ -62,7 +72,13 @@ def refresh_recovery_payload( "settlement_result": settlement_result_payload(readback.delivery), } if not dry_run: - attach_settlement_progress(payload, readback, registry_path=registry_path, runtime_root=runtime_root) + attach_settlement_progress( + payload, + readback, + registry_path=registry_path, + runtime_root=runtime_root, + goal_ref=goal_ref, + ) if decision == "reject": payload["error"] = ( f"{recovery['reason']}: committed writeback is unchanged; " @@ -81,4 +97,10 @@ def refresh_recovery_payload( f"--resume-external-sinks {key} instead of --suppress-external-sinks. " if key else "") + "Existing provider permissions still apply; do not repeat business mutations or spend." ) - return finish_external_delivery_refresh(payload, readback, runtime_root, dry_run=dry_run) + return finish_external_delivery_refresh( + payload, + readback, + runtime_root, + dry_run=dry_run, + goal_ref=goal_ref, + ) diff --git a/loopx/control_plane/quota/settlement.py b/loopx/control_plane/quota/settlement.py index f5317506cc..60e59a4a10 100644 --- a/loopx/control_plane/quota/settlement.py +++ b/loopx/control_plane/quota/settlement.py @@ -8,6 +8,7 @@ from ..effect_runtime import EffectRuntimeRejected, effect_runtime_result from ..settlement_driver import decode_settlement_result +from .accounting_admission import quota_accounting_admission from .effect_program import ( SETTLEMENT_IDENTITY_SCHEMA_VERSION, SETTLEMENT_PLAN_SCHEMA_VERSION, @@ -172,6 +173,7 @@ def attach_settlement_progress( *, registry_path: Path | None = None, runtime_root: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> None: """Render the TS-owned receipt progress without deriving a second settlement rule.""" progress = readback.progress @@ -197,6 +199,7 @@ def attach_settlement_progress( scoped_cli_args="", lifecycle_actor_args="", quota_spend_source=progress["quota_spend_source"], + goal_ref=goal_ref, ) payload["settlement_owed"] = { **identity.as_dict(), "schema_version": "turn_settlement_owed_v0", @@ -311,32 +314,69 @@ def read_heartbeat_settlement( allow_unbound_binding: bool = False, resolve_original_binding: bool = False, refresh_retry: dict[str, Any] | None = None, + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, + source_admission: Mapping[str, Any] | None = None, + borrow_source_admission: bool = False, ) -> QuotaSettlementReadback | None: """Read one complete heartbeat settlement through the TS domain owner.""" - try: - payload = effect_runtime_result( - "quota.settlement.read", - { - "schema_version": QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA, - "runtime_root": str(runtime_root.expanduser()), - "goal_id": goal_id, - "agent_id": agent_id, - "todo_id": todo_id, - "turn_instance_id": turn_instance_id, - "replan_obligation_id": replan_obligation_id, - "infer_turn_instance_id": infer_turn_instance_id, - "allow_unbound_binding": allow_unbound_binding, - **({"resolve_original_binding": True} if resolve_original_binding else {}), - **( - {"refresh_retry": refresh_retry} - if refresh_retry is not None - else {} - ), - }, - ) - except EffectRuntimeRejected as exc: - raise ValueError(str(exc)) from None + def read(admission: Mapping[str, Any] | None) -> Any: + try: + return effect_runtime_result( + "quota.settlement.read", + { + "schema_version": QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA, + "runtime_root": str(runtime_root.expanduser()), + "goal_id": goal_id, + "agent_id": agent_id, + "todo_id": todo_id, + "turn_instance_id": turn_instance_id, + "replan_obligation_id": replan_obligation_id, + "infer_turn_instance_id": infer_turn_instance_id, + "allow_unbound_binding": allow_unbound_binding, + **( + {"resolve_original_binding": True} + if resolve_original_binding + else {} + ), + **( + {"refresh_retry": refresh_retry} + if refresh_retry is not None + else {} + ), + **( + {"goal_ref": dict(goal_ref)} + if goal_ref is not None + else {} + ), + **( + {"source_admission": dict(admission)} + if admission is not None + else {} + ), + **( + {"borrow_source_admission": True} + if borrow_source_admission + else {} + ), + }, + ) + except EffectRuntimeRejected as exc: + raise ValueError(str(exc)) from None + + if source_admission is not None: + payload = read(source_admission) + else: + with quota_accounting_admission( + runtime_root=runtime_root, + registry_path=registry_path, + goal_id=goal_id, + goal_ref=goal_ref, + operation="quota-settlement-read", + lock_legacy_index=False, + ) as admission: + payload = read(admission) if not isinstance(payload, Mapping) or ( payload.get("schema_version") != QUOTA_SETTLEMENT_READBACK_RESULT_SCHEMA diff --git a/loopx/control_plane/quota/settlement_cli.py b/loopx/control_plane/quota/settlement_cli.py index 2096d20ca4..c0e7addfd9 100644 --- a/loopx/control_plane/quota/settlement_cli.py +++ b/loopx/control_plane/quota/settlement_cli.py @@ -38,6 +38,7 @@ def reconcile_existing_heartbeat_receipt( runtime_root: Path, turn_instance_id: str, existing: dict[str, object], + goal_ref: Mapping[str, object] | None = None, ) -> tuple[dict[str, object], str, bool, str]: """Bind an identity-less same-turn receipt without changing a bound receipt.""" @@ -130,6 +131,7 @@ def reconcile_existing_heartbeat_receipt( ), summary=f"heartbeat quota receipt upgraded for turn={turn_instance_id}", details=rollout_details, + goal_ref=goal_ref, ) if upgraded: receipt_status = "upgraded" @@ -149,6 +151,7 @@ def reconcile_existing_heartbeat_receipt_for_turn( runtime_root: Path, turn_instance_id: str, existing: dict[str, object], + goal_ref: Mapping[str, object] | None = None, ) -> tuple[dict[str, object], str, bool, str, bool]: """Reconcile an existing receipt and report whether the turn is receipt-ready.""" @@ -158,6 +161,7 @@ def reconcile_existing_heartbeat_receipt_for_turn( runtime_root=runtime_root, turn_instance_id=turn_instance_id, existing=existing, + goal_ref=goal_ref, ) return receipt, status, appended, stall_observation, True @@ -409,7 +413,11 @@ def attach_spend_settlement_result( todo_id: str | None, turn_instance_id: str, replan_obligation_id: str | None = None, + goal_ref: Mapping[str, object] | None = None, ) -> None: + registry_path = ( + Path(str(payload["registry"])) if payload.get("registry") else None + ) readback = read_heartbeat_settlement( runtime_root, goal_id=goal_id, @@ -417,12 +425,15 @@ def attach_spend_settlement_result( todo_id=todo_id, turn_instance_id=turn_instance_id, replan_obligation_id=replan_obligation_id, + registry_path=registry_path, + goal_ref=goal_ref, ) if readback is None: raise RuntimeError("exact settlement readback unexpectedly returned not-found") attach_settlement_progress( payload, readback, runtime_root=runtime_root, - registry_path=Path(str(payload["registry"])) if payload.get("registry") else None, + registry_path=registry_path, + goal_ref=goal_ref, ) identity = readback.identity.value if identity is None: diff --git a/loopx/control_plane/quota/settlement_plan.ts b/loopx/control_plane/quota/settlement_plan.ts index 37d7547ee2..ac98fab6c7 100644 --- a/loopx/control_plane/quota/settlement_plan.ts +++ b/loopx/control_plane/quota/settlement_plan.ts @@ -4,6 +4,7 @@ import { settlementIdentity, type JsonObject, type SettlementIdentityInput, type SettlementPlan, type SettlementStep, } from "../effect_program.ts"; +import {parseExactGoalRef} from "../goals/goal_instance_identity.ts"; import {requireJsonObject, requireNonEmptyString} from "../runtime_decode.ts"; export function turnScopedCliSettlementPlan(params: JsonObject): SettlementPlan { @@ -13,6 +14,18 @@ export function turnScopedCliSettlementPlan(params: JsonObject): SettlementPlan const commands = requireJsonObject(params.command_templates, "command_templates"); const writeback = requireNonEmptyString(commands.durable_writeback, "durable_writeback"); const spend = requireNonEmptyString(commands.quota_spend, "quota_spend"); + const parsedGoalRef = params.goal_ref === undefined + ? null + : parseExactGoalRef(params.goal_ref); + if ( + parsedGoalRef !== null + && ( + parsedGoalRef.kind === "invalid" + || parsedGoalRef.value.goalId.value !== identity.goal_id + ) + ) { + throw new Error("settlement plan GoalRef is invalid or does not match identity"); + } const inFlight = params.delivery_boundary === "in_flight_continuation"; const validation: SettlementStep = { kind: "validation", owner: "agent", idempotency_key_ref: "$.identity.effect_id", @@ -52,5 +65,14 @@ export function turnScopedCliSettlementPlan(params: JsonObject): SettlementPlan command_template: requireNonEmptyString(commands.terminal_closeout, "terminal_closeout"), conditional: true, }); - return {identity, steps}; + return { + identity, + steps, + ...(parsedGoalRef === null ? {} : { + goal_ref: { + goal_id: parsedGoalRef.value.goalId.value, + goal_instance_id: parsedGoalRef.value.goalInstanceId.value, + }, + }), + }; } diff --git a/loopx/control_plane/quota/settlement_readback.ts b/loopx/control_plane/quota/settlement_readback.ts index b55ce95f3a..71d7dcd0be 100644 --- a/loopx/control_plane/quota/settlement_readback.ts +++ b/loopx/control_plane/quota/settlement_readback.ts @@ -56,6 +56,13 @@ import { import { refreshExternalDelivery } from "./refresh_external_delivery.ts"; import { BLOCKED_WAIT_REQUEST_SCHEMA, prepareBlockedWait, RECEIPT_BOUND_WAIT_REQUEST_SCHEMA, projectReceiptBoundWait } from "./blocked_wait.ts"; import {nativeChildReportAdmission} from "../capabilities/native_child_admission.ts"; +import { + parseQuotaAccountingOwner, + quotaOwnerOwnsProjection, + withBorrowedQuotaAccountingOwner, + withQuotaAccountingOwner, + type QuotaAccountingOwner, +} from "./source_admission.ts"; export const QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA = "loopx_quota_settlement_readback_request_v0"; @@ -76,7 +83,9 @@ interface ReadbackRequest { infer_turn_instance_id: boolean; allow_unbound_binding: boolean; resolve_original_binding: boolean; + borrow_source_admission: boolean; refresh_retry: RefreshRetryRequest | null; + owner: QuotaAccountingOwner; } export interface QuotaSettlementReadbackSnapshot { @@ -153,7 +162,10 @@ function normalizeAgentId(value: unknown): string | null { return candidate && AGENT_ID_PATTERN.test(candidate) ? candidate : null; } -function decodeRequest(value: unknown): ReadbackRequest { +function decodeRequest( + value: unknown, + admittedOwner?: QuotaAccountingOwner, +): ReadbackRequest { const request = requireJsonObject(value, "quota settlement readback request"); if (request.schema_version !== QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA) { throw new EffectRuntimeRequestError( @@ -177,6 +189,44 @@ function decodeRequest(value: unknown): ReadbackRequest { if (request.resolve_original_binding === true && request.infer_turn_instance_id) { throw new EffectRuntimeRequestError("original binding requires an explicit Turn identity"); } + if ( + request.borrow_source_admission !== undefined + && typeof request.borrow_source_admission !== "boolean" + ) { + throw new EffectRuntimeRequestError( + "borrow_source_admission must be a boolean", + ); + } + if ( + admittedOwner !== undefined + && (request.goal_ref !== undefined || request.source_admission !== undefined) + ) { + throw new EffectRuntimeRequestError( + "admitted settlement readback must use its parsed quota owner", + "quota_source_admission_invalid", + ); + } + if ( + admittedOwner?.kind === "exact_source" + && admittedOwner.goalRef.goalId.value !== goalId + ) { + throw new EffectRuntimeRequestError( + "admitted quota owner does not match settlement goal_id", + "quota_source_admission_invalid", + ); + } + const owner = admittedOwner ?? parseQuotaAccountingOwner({ + goalRefValue: request.goal_ref, + sourceAdmissionValue: request.source_admission, + runtimeRoot, + goalId, + }); + if (request.borrow_source_admission === true && owner.kind !== "exact_source") { + throw new EffectRuntimeRequestError( + "borrow_source_admission requires exact source admission", + "quota_source_admission_invalid", + ); + } return { runtime_root: runtimeRoot, goal_id: goalId, @@ -193,7 +243,9 @@ function decodeRequest(value: unknown): ReadbackRequest { infer_turn_instance_id: request.infer_turn_instance_id, allow_unbound_binding: request.allow_unbound_binding, resolve_original_binding: request.resolve_original_binding === true, + borrow_source_admission: request.borrow_source_admission === true, refresh_retry: decodeRefreshRetry(request.refresh_retry), + owner, }; } @@ -334,6 +386,7 @@ function indexedRuns( export function committedMonitorPollFromSnapshot( snapshot: QuotaSettlementReadbackSnapshot, identity: Pick, + owner: QuotaAccountingOwner = {kind: "alias"}, ): JsonObject | null { if (snapshot.goalId !== identity.goal_id) { throw new EffectRuntimeRequestError("monitor poll snapshot scope mismatch"); @@ -342,6 +395,7 @@ export function committedMonitorPollFromSnapshot( turnKey(identity.goal_id, identity.agent_id, identity.turn_instance_id)!, ) ?? []; return [...runs].reverse().find((run) => + quotaOwnerOwnsProjection(owner, run.goal_ref) && run.classification === "quota_monitor_poll" && optionalString(run.goal_id) === identity.goal_id && optionalString(run.agent_id) === identity.agent_id && @@ -948,6 +1002,12 @@ function readQuotaSettlementFromRequest( ); } const explicitAgentId = normalizeAgentId(request.agent_id); + const ownerRuns = snapshot.runs.filter((run) => + quotaOwnerOwnsProjection(request.owner, run.goal_ref) + ); + const ownerEvents = snapshot.events.filter((event) => + quotaOwnerOwnsProjection(request.owner, event.goal_ref) + ); const explicitEvents = !request.infer_turn_instance_id && explicitAgentId !== null && request.turn_instance_id !== null ? indexedEvents( @@ -955,12 +1015,14 @@ function readQuotaSettlementFromRequest( request.goal_id, explicitAgentId, request.turn_instance_id, + ).filter((event) => + quotaOwnerOwnsProjection(request.owner, event.goal_ref) ) - : snapshot.events; + : ownerEvents; const identityResult = resolveIdentity( request, explicitEvents, - snapshot.runs, + ownerRuns, ); if (identityResult === null) { return { @@ -984,8 +1046,12 @@ function readQuotaSettlementFromRequest( identity.goal_id, identity.agent_id, identity.turn_instance_id, + ).filter((event) => + quotaOwnerOwnsProjection(request.owner, event.goal_ref) + ); + const runs = indexedRuns(snapshot, identity).filter((run) => + quotaOwnerOwnsProjection(request.owner, run.goal_ref) ); - const runs = indexedRuns(snapshot, identity); const heartbeatReceipt = effectiveHeartbeatReceipt(events, identity); if (heartbeatReceipt === null) { return failedReadback( @@ -999,7 +1065,9 @@ function readQuotaSettlementFromRequest( const writebackRun = findWriteback(runs, identity); const writebackEvent = findStepEvent(events, identity, "refresh_state"); const spendRun = findSpend( - spendCandidateRuns(snapshot, identity, runs), + spendCandidateRuns(snapshot, identity, runs).filter((run) => + quotaOwnerOwnsProjection(request.owner, run.goal_ref) + ), identity, ); const spendEvent = findStepEvent(events, identity, "quota_spend"); @@ -1026,7 +1094,11 @@ function readQuotaSettlementFromRequest( const withWriteback = settlementBindReduce(identityResult, writeback); const settled = blockedNoSpend ? withWriteback : settlementBindReduce(withWriteback, spend); const terminalSettlement = settlementBindReduce(settled, terminalCloseout); - const monitorPoll = committedMonitorPollFromSnapshot(snapshot, identity); + const monitorPoll = committedMonitorPollFromSnapshot( + snapshot, + identity, + request.owner, + ); const nestedCausality = typeof receiptDetails.delivery_workspace_causality === "object" && receiptDetails.delivery_workspace_causality !== null && !Array.isArray(receiptDetails.delivery_workspace_causality) @@ -1064,6 +1136,8 @@ function readQuotaSettlementFromRequest( workspaceCausality?.requirement, writebackRun !== null && snapshot.runs.slice( (snapshot.runPositions.get(writebackRun) ?? -1) + 1, + ).filter((run) => + quotaOwnerOwnsProjection(request.owner, run.goal_ref) ).some((run) => run.goal_id === identity.goal_id && run.agent_id === identity.agent_id && (jsonObject(run.agent_vision) !== null || jsonObject(run.vision_checkpoint)?.required === true) @@ -1113,10 +1187,34 @@ function readQuotaSettlementFromRequest( export function readQuotaSettlementFromSnapshot( value: unknown, snapshot: QuotaSettlementReadbackSnapshot, +): JsonObject { + const request = decodeRequest(value); + if (request.owner.kind !== "alias") { + throw new EffectRuntimeRequestError( + "exact source settlement readback requires live owner admission", + "quota_source_admission_invalid", + ); + } + return readQuotaSettlementFromRequest(request, snapshot); +} + +/** Read under a source admission already adopted by the enclosing transaction. */ +export function readAdmittedQuotaSettlementFromSnapshot( + value: unknown, + snapshot: QuotaSettlementReadbackSnapshot, ): JsonObject { return readQuotaSettlementFromRequest(decodeRequest(value), snapshot); } +/** Read under the parsed owner already admitted by the enclosing transaction. */ +export function readQuotaSettlementForAdmittedOwnerFromSnapshot( + value: unknown, + owner: QuotaAccountingOwner, + snapshot: QuotaSettlementReadbackSnapshot, +): JsonObject { + return readQuotaSettlementFromRequest(decodeRequest(value, owner), snapshot); +} + export async function readQuotaSettlement(value: unknown): Promise { if (jsonObject(value)?.schema_version === RECEIPT_BOUND_WAIT_REQUEST_SCHEMA) { return projectReceiptBoundWait(value); @@ -1125,8 +1223,14 @@ export async function readQuotaSettlement(value: unknown): Promise { return prepareBlockedWait(value); } const request = decodeRequest(value); - return readQuotaSettlementFromRequest( - request, - await readQuotaSettlementSnapshot(request.runtime_root, request.goal_id), + const withOwner = request.borrow_source_admission + ? withBorrowedQuotaAccountingOwner + : withQuotaAccountingOwner; + return await withOwner( + request.owner, + async () => readQuotaSettlementFromRequest( + request, + await readQuotaSettlementSnapshot(request.runtime_root, request.goal_id), + ), ); } diff --git a/loopx/control_plane/quota/should_run.py b/loopx/control_plane/quota/should_run.py index 46a87ad8e1..288f5c247f 100644 --- a/loopx/control_plane/quota/should_run.py +++ b/loopx/control_plane/quota/should_run.py @@ -147,6 +147,7 @@ def build_quota_paused_should_run_payload( codex_app_automation_id: Any = None, resolved_scheduler_context: SchedulerExecutionContextResolution, runtime_root: str | Path | None = None, + goal_ref: Mapping[str, object] | None = None, ) -> dict[str, Any]: """Project one canonical hard-pause contract with no lane contradiction. @@ -224,6 +225,8 @@ def build_quota_paused_should_run_payload( payload=payload, agent_identity=agent_identity, ) + if goal_ref is not None: + payload["goal_ref"] = dict(goal_ref) payload["automation_liveness"] = build_automation_liveness(payload) payload["interaction_contract"] = build_interaction_contract( payload, @@ -271,6 +274,7 @@ def build_quota_should_run( receipt_bound_replan_guard_scoped: bool = False, turn_instance_id: str | None = None, runtime_root: str | Path | None = None, + goal_ref: Mapping[str, object] | None = None, ) -> dict[str, Any]: safe_goal_id = str(goal_id or "").strip() resolved_scheduler_context = resolve_scheduler_execution_context( @@ -318,6 +322,7 @@ def build_quota_should_run( codex_app_automation_id=codex_app_automation_id, resolved_scheduler_context=resolved_scheduler_context, runtime_root=runtime_root, + goal_ref=goal_ref, ) prepared = _prepare_quota_should_run_item( status_payload, @@ -349,6 +354,7 @@ def build_quota_should_run( turn_instance_id=turn_instance_id, include_agent_todo_detail=include_agent_todo_detail, runtime_root=runtime_root, + goal_ref=goal_ref, ) if health_item: return { diff --git a/loopx/control_plane/quota/should_run_packet.py b/loopx/control_plane/quota/should_run_packet.py index 0aefeef31c..eb17b7977f 100644 --- a/loopx/control_plane/quota/should_run_packet.py +++ b/loopx/control_plane/quota/should_run_packet.py @@ -1472,6 +1472,7 @@ def _build_quota_should_run_payload( turn_instance_id: str | None = None, include_agent_todo_detail: bool = False, runtime_root: str | Path | None = None, + goal_ref: Mapping[str, object] | None = None, ) -> dict[str, Any]: if prepared.receipt_bound_replay_phase is ReceiptBoundReplayPhase.SETTLED: payload = _build_settled_quota_payload(prepared, route) @@ -1479,6 +1480,8 @@ def _build_quota_should_run_payload( payload = _build_active_quota_payload( prepared, route, include_agent_todo_detail=include_agent_todo_detail, ) + if goal_ref is not None: + payload["goal_ref"] = dict(goal_ref) apply_settled_monitor_precedence(payload) cadence_root = _interaction_runtime_root(runtime_root, prepared.status_payload) if cadence_root: diff --git a/loopx/control_plane/quota/slot_accounting.py b/loopx/control_plane/quota/slot_accounting.py index 387ed81304..b91971cf38 100644 --- a/loopx/control_plane/quota/slot_accounting.py +++ b/loopx/control_plane/quota/slot_accounting.py @@ -146,6 +146,8 @@ def _resolve_preview_settlement( todo_id: str | None, replan_obligation_id: str | None, turn_instance_id: str | None, + registry_path: Path | None, + goal_ref: Mapping[str, Any] | None, ) -> dict[str, Any]: if turn_instance_id and source not in TURN_SCOPED_SLOT_SPEND_SOURCES: result = SettlementResult.failed( @@ -176,6 +178,8 @@ def _resolve_preview_settlement( and not todo_id and not replan_obligation_id ), + registry_path=registry_path, + goal_ref=goal_ref, ) if readback is None: return {} @@ -382,6 +386,7 @@ def _latest_unspent_turn_settlement_run( goal_id: str, *, agent_id: str | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any] | None: """Return the latest same-agent Turn settlement that still needs accounting. @@ -395,6 +400,10 @@ def _latest_unspent_turn_settlement_run( safe_agent_id = normalize_todo_claimed_by(agent_id) for run in reversed(_load_goal_run_index_records(runtime_root, goal_id)): + if goal_ref is None and "goal_ref" in run: + continue + if goal_ref is not None and run.get("goal_ref") != dict(goal_ref): + continue run_agent_id = normalize_todo_claimed_by(run.get("agent_id")) if safe_agent_id and run_agent_id and safe_agent_id != run_agent_id: continue @@ -579,6 +588,8 @@ def build_quota_slot_preview_for_decision( replan_obligation_id: str | None = None, turn_instance_id: str | None = None, source: str = DEFAULT_SLOT_SPEND_SOURCE, + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: safe_goal_id = _validate_goal_id_path_segment(str(goal_id or "")) safe_slots = max(1, _int_number(slots, default=1)) @@ -627,6 +638,8 @@ def build_quota_slot_preview_for_decision( todo_id=normalized_todo_id, replan_obligation_id=normalized_replan_obligation_id, turn_instance_id=turn_instance_id, + registry_path=registry_path, + goal_ref=goal_ref, ) settlement_identity = settlement.get("identity") settlement_result = settlement.get("result") @@ -698,6 +711,7 @@ def build_quota_slot_preview_for_decision( Path(str(raw_runtime_root)).expanduser(), safe_goal_id, agent_id=safe_requested_agent_id, + goal_ref=goal_ref, ) if raw_runtime_root else None diff --git a/loopx/control_plane/quota/source_admission.ts b/loopx/control_plane/quota/source_admission.ts new file mode 100644 index 0000000000..47526e2a75 --- /dev/null +++ b/loopx/control_plane/quota/source_admission.ts @@ -0,0 +1,387 @@ +import { createHash } from "node:crypto"; +import { dirname, isAbsolute, join, resolve } from "node:path"; + +import type { JsonObject } from "../effect_program.ts"; +import { + EffectRuntimeConflictError, + EffectRuntimeRequestError, +} from "../effect_runtime_errors.ts"; +import { + claimFileMutationLock, + mutationLockOwner, + releaseFileMutationLock, + releaseFileMutationLockClaim, + type FileMutationLockClaim, +} from "../effect_runtime_io.ts"; +import { decideFirstPartyHostRuntime } from "../goals/first_party_host_runtime.ts"; +import { + parseExactGoalRef, + type ExactGoalRef, +} from "../goals/goal_instance_identity.ts"; +import { + requireJsonObject, + requireNonEmptyString, +} from "../runtime_decode.ts"; + +const QUOTA_SOURCE_ADMISSION_SCHEMA = "loopx_quota_source_admission_v0"; +const SOURCE_SESSION_PROFILE_ID = "source_session_v1"; + +type LockRole = "run_index" | "source_guard"; + +interface QuotaSourceLock { + role: LockRole; + target: string; + pid: number; + token: string; +} + +interface QuotaSourceAdmission { + registryPath: string; + plannedGoalRef: ExactGoalRef; + authority: unknown; + locks: readonly [QuotaSourceLock, QuotaSourceLock]; +} + +export type QuotaAccountingOwner = + | Readonly<{ kind: "alias" }> + | Readonly<{ + kind: "exact_source"; + goalRef: ExactGoalRef; + admission: QuotaSourceAdmission; + }>; + +interface ClaimedLock { + witness: QuotaSourceLock; + claim: FileMutationLockClaim; +} + +export function quotaAccountingOwnerLocks( + owner: QuotaAccountingOwner, +): readonly Readonly[] { + return owner.kind === "alias" ? [] : owner.admission.locks; +} + +function sha256(value: string): string { + return createHash("sha256").update(value, "utf8").digest("hex"); +} + +function sourceGuardTarget(registryPath: string, goalId: string): string { + return join( + dirname(registryPath), + ".loopx", + "lifecycle", + "goal-instance", + "guards", + `${sha256(goalId)}.guard`, + ); +} + +function sameGoalRef(left: ExactGoalRef, right: ExactGoalRef): boolean { + return left.goalId.value === right.goalId.value + && left.goalInstanceId.value === right.goalInstanceId.value; +} + +export function quotaGoalRef(owner: QuotaAccountingOwner): JsonObject | null { + if (owner.kind === "alias") return null; + return { + goal_id: owner.goalRef.goalId.value, + goal_instance_id: owner.goalRef.goalInstanceId.value, + }; +} + +export function requireQuotaOwnerProjection( + owner: QuotaAccountingOwner, + value: unknown, + label: string, +): void { + if (owner.kind === "alias") { + if (value === undefined) return; + throw new EffectRuntimeConflictError( + `${label} belongs to an exact Goal instance and cannot be consumed without GoalRef admission`, + "goal_instance_conflict", + ); + } + const projected = parseExactGoalRef(value); + if ( + projected.kind === "invalid" + || !sameGoalRef(projected.value, owner.goalRef) + ) { + throw new EffectRuntimeConflictError( + `${label} does not belong to the admitted Goal instance`, + "goal_instance_conflict", + ); + } +} + +export function quotaOwnerOwnsProjection( + owner: QuotaAccountingOwner, + value: unknown, +): boolean { + if (owner.kind === "alias") return value === undefined; + const projected = parseExactGoalRef(value); + return projected.kind === "parsed" + && sameGoalRef(projected.value, owner.goalRef); +} + +function quotaSourceLock( + value: unknown, + role: LockRole, + expectedTarget: string, +): QuotaSourceLock { + const witness = requireJsonObject(value, `${role} lock witness`); + if (witness.role !== role) { + throw new EffectRuntimeRequestError( + `quota source admission ${role} lock role mismatch`, + "quota_source_admission_invalid", + ); + } + const target = requireNonEmptyString( + witness.target, + `${role} lock target`, + ); + if ( + !isAbsolute(target) + || resolve(target) !== target + || target !== expectedTarget + ) { + throw new EffectRuntimeRequestError( + `quota source admission ${role} lock target mismatch`, + "quota_source_admission_invalid", + ); + } + if ( + typeof witness.pid !== "number" + || !Number.isSafeInteger(witness.pid) + || witness.pid <= 0 + ) { + throw new EffectRuntimeRequestError( + `quota source admission ${role} lock owner is invalid`, + "quota_source_admission_invalid", + ); + } + return { + role, + target, + pid: witness.pid, + token: requireNonEmptyString(witness.token, `${role} lock token`), + }; +} + +export function parseQuotaAccountingOwner( + { + goalRefValue, + sourceAdmissionValue, + runtimeRoot, + goalId, + }: { + goalRefValue: unknown; + sourceAdmissionValue: unknown; + runtimeRoot: string; + goalId: string; + }, +): QuotaAccountingOwner { + const hasGoalRef = goalRefValue !== undefined; + const hasAdmission = sourceAdmissionValue !== undefined; + if (!hasGoalRef && !hasAdmission) return { kind: "alias" }; + if (!hasGoalRef || !hasAdmission) { + throw new EffectRuntimeRequestError( + "exact quota GoalRef and source admission must be supplied together", + "quota_source_admission_invalid", + ); + } + + const goalRef = parseExactGoalRef(goalRefValue); + if ( + goalRef.kind === "invalid" + || goalRef.value.goalId.value !== goalId + ) { + throw new EffectRuntimeRequestError( + "quota GoalRef is invalid or does not match goal_id", + "quota_source_admission_invalid", + ); + } + const admission = requireJsonObject( + sourceAdmissionValue, + "quota source admission", + ); + if ( + admission.schema_version !== QUOTA_SOURCE_ADMISSION_SCHEMA + || admission.profile_id !== SOURCE_SESSION_PROFILE_ID + ) { + throw new EffectRuntimeRequestError( + "quota source admission is malformed", + "quota_source_admission_invalid", + ); + } + const registryPath = requireNonEmptyString( + admission.registry_path, + "quota source admission registry_path", + ); + if (!isAbsolute(registryPath) || resolve(registryPath) !== registryPath) { + throw new EffectRuntimeRequestError( + "quota source admission registry path must be absolute and normalized", + "quota_source_admission_invalid", + ); + } + const plannedGoalRef = parseExactGoalRef(admission.planned_goal_ref); + if ( + plannedGoalRef.kind === "invalid" + || !sameGoalRef(plannedGoalRef.value, goalRef.value) + ) { + throw new EffectRuntimeRequestError( + "quota source admission does not match the requested GoalRef", + "quota_source_admission_invalid", + ); + } + if (!Array.isArray(admission.locks) || admission.locks.length !== 2) { + throw new EffectRuntimeRequestError( + "quota source admission requires both ordered lock witnesses", + "quota_source_admission_invalid", + ); + } + const indexTarget = resolve( + join(runtimeRoot, "goals", goalId, "runs", "index.jsonl"), + ); + const guardTarget = resolve(sourceGuardTarget(registryPath, goalId)); + const runIndex = quotaSourceLock( + admission.locks[0], + "run_index", + indexTarget, + ); + const sourceGuard = quotaSourceLock( + admission.locks[1], + "source_guard", + guardTarget, + ); + return { + kind: "exact_source", + goalRef: goalRef.value, + admission: { + registryPath, + plannedGoalRef: plannedGoalRef.value, + authority: admission.authority, + locks: [runIndex, sourceGuard], + }, + }; +} + +export async function withQuotaAccountingOwner( + owner: QuotaAccountingOwner, + operation: (indexLockHeld: boolean) => Promise, +): Promise { + if (owner.kind === "alias") return await operation(false); + + const claims: ClaimedLock[] = []; + let adopted = false; + try { + for (const witness of owner.admission.locks) { + const claim = await claimFileMutationLock( + witness.target, + witness.token, + ); + if (!claim) { + throw new EffectRuntimeConflictError( + "quota source admission lock handoff expired", + "quota_source_admission_expired", + ); + } + claims.push({ witness, claim }); + const current = await mutationLockOwner(witness.target); + if ( + current?.pid !== witness.pid + || current.token !== witness.token + ) { + throw new EffectRuntimeConflictError( + "quota source admission lock owner changed", + "quota_source_admission_expired", + ); + } + } + adopted = true; + requireCurrentQuotaAccountingOwner(owner); + return await operation(true); + } finally { + for (const entry of claims.reverse()) { + if (adopted) { + await releaseFileMutationLock( + entry.witness.target, + entry.witness.token, + entry.claim, + true, + ); + } else { + await releaseFileMutationLockClaim(entry.claim); + } + } + } +} + +export function requireCurrentQuotaAccountingOwner( + owner: QuotaAccountingOwner, +): void { + if (owner.kind === "alias") return; + const decision = decideFirstPartyHostRuntime({ + profile_id: SOURCE_SESSION_PROFILE_ID, + operation: "require_current", + planned_goal_ref: quotaGoalRef(owner), + authority: owner.admission.authority, + }); + if (decision.kind === "reject") { + throw new EffectRuntimeConflictError( + `quota source admission rejected: ${decision.code}`, + decision.code, + ); + } + if (decision.kind !== "resume") { + throw new EffectRuntimeRequestError( + "quota source admission did not resume the exact GoalRef", + "quota_source_admission_invalid", + ); + } +} + +/** + * Verify a Python-owned admission without ending its surrounding transaction. + * + * The temporary claims prevent stale-owner reclamation while the callback + * runs. The caller remains responsible for releasing the underlying locks. + */ +export async function withBorrowedQuotaAccountingOwner( + owner: QuotaAccountingOwner, + operation: (indexLockHeld: boolean) => Promise, +): Promise { + if (owner.kind === "alias") return await operation(false); + + const claims: FileMutationLockClaim[] = []; + try { + for (const witness of owner.admission.locks) { + const claim = await claimFileMutationLock( + witness.target, + witness.token, + ); + if (!claim) { + throw new EffectRuntimeConflictError( + "quota source admission lock handoff expired", + "quota_source_admission_expired", + ); + } + claims.push(claim); + const current = await mutationLockOwner(witness.target); + if ( + current?.pid !== witness.pid + || current.token !== witness.token + ) { + throw new EffectRuntimeConflictError( + "quota source admission lock owner changed", + "quota_source_admission_expired", + ); + } + } + requireCurrentQuotaAccountingOwner(owner); + return await operation(true); + } finally { + for (const claim of claims.reverse()) { + await releaseFileMutationLockClaim(claim); + } + } +} diff --git a/loopx/control_plane/quota/spend_commit.py b/loopx/control_plane/quota/spend_commit.py index ee4b5a6f60..077b05185c 100644 --- a/loopx/control_plane/quota/spend_commit.py +++ b/loopx/control_plane/quota/spend_commit.py @@ -6,10 +6,15 @@ from typing import Any from ...file_lock import exclusive_file_lock -from ..effect_runtime import EffectRuntimeRejected, effect_runtime_result +from ..effect_runtime import ( + EffectRuntimeConflict, + EffectRuntimeRejected, + effect_runtime_result, +) from ..runtime.time import now_local_iso from ..todos.contract import normalize_todo_claimed_by from .decision_summary import compact_quota_decision, quota_decision_agent_id +from .accounting_admission import quota_accounting_admission from .spend_sources import DEFAULT_SLOT_SPEND_SOURCE @@ -33,6 +38,8 @@ def _quota_spend_commit_request( execute: bool, runtime_root: Path | None, expected_index_digest: str | None, + goal_ref: Mapping[str, Any] | None = None, + source_admission: Mapping[str, Any] | None = None, ) -> dict[str, Any]: # Import lazily because runtime loads history, whose quota compatibility # surface re-exports this module during package initialization. @@ -52,7 +59,7 @@ def _quota_spend_commit_request( request_preview = dict(preview) request_preview.pop("expected_index_digest", None) request_preview["after_recommended_action"] = after.get("recommended_action") - return { + request = { "schema_version": QUOTA_SPEND_COMMIT_REQUEST_SCHEMA, "runtime_root": str(runtime_root) if runtime_root is not None else None, "goal_id": goal_id, @@ -65,6 +72,11 @@ def _quota_spend_commit_request( "after": compact_quota_decision(after), "resolved_agent_id": resolved_agent_id, } + if goal_ref is not None: + request["goal_ref"] = dict(goal_ref) + if source_admission is not None: + request["source_admission"] = dict(source_admission) + return request def _quota_spend_commit_result( @@ -75,6 +87,8 @@ def _quota_spend_commit_result( execute: bool, runtime_root: Path | None, expected_index_digest: str | None, + goal_ref: Mapping[str, Any] | None = None, + source_admission: Mapping[str, Any] | None = None, ) -> Mapping[str, Any]: params = _quota_spend_commit_request( preview, @@ -83,10 +97,12 @@ def _quota_spend_commit_result( execute=execute, runtime_root=runtime_root, expected_index_digest=expected_index_digest, + goal_ref=goal_ref, + source_admission=source_admission, ) try: result = effect_runtime_result("quota.spend.commit", params) - except EffectRuntimeRejected as exc: + except (EffectRuntimeConflict, EffectRuntimeRejected) as exc: raise ValueError(str(exc)) from None if ( not isinstance(result, Mapping) @@ -105,6 +121,8 @@ def replay_quota_spend_by_effect_ref( effect_ref: str, agent_id: str | None, read_only: bool = False, + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: """Ask the native quota transaction to validate an effect replay.""" @@ -119,21 +137,31 @@ def replay_quota_spend_by_effect_ref( "replay_found": False, "reason": "effect_ref must be a non-empty string", } - try: - result = effect_runtime_result( - "quota.spend.commit", - { - "schema_version": QUOTA_SPEND_COMMIT_REQUEST_SCHEMA, - "operation": "replay", - "runtime_root": str(runtime_root.expanduser()), - "goal_id": safe_goal_id, - "effect_id": normalized_effect_ref, - "resolved_agent_id": normalize_todo_claimed_by(agent_id), - "read_only": read_only, - }, - ) - except EffectRuntimeRejected as exc: - raise ValueError(str(exc)) from None + with quota_accounting_admission( + runtime_root=runtime_root, + registry_path=registry_path, + goal_id=safe_goal_id, + goal_ref=goal_ref, + operation="quota_spend_replay", + lock_legacy_index=False, + ) as source_admission: + request: dict[str, Any] = { + "schema_version": QUOTA_SPEND_COMMIT_REQUEST_SCHEMA, + "operation": "replay", + "runtime_root": str(runtime_root.expanduser()), + "goal_id": safe_goal_id, + "effect_id": normalized_effect_ref, + "resolved_agent_id": normalize_todo_claimed_by(agent_id), + "read_only": read_only, + } + if goal_ref is not None: + request["goal_ref"] = dict(goal_ref) + if source_admission is not None: + request["source_admission"] = dict(source_admission) + try: + result = effect_runtime_result("quota.spend.commit", request) + except (EffectRuntimeConflict, EffectRuntimeRejected) as exc: + raise ValueError(str(exc)) from None if not isinstance(result, Mapping) or result.get("schema_version") != QUOTA_SPEND_COMMIT_RESULT_SCHEMA: raise RuntimeError("TypeScript quota spend replay result shape mismatch") payload = result.get("payload") @@ -173,6 +201,8 @@ def record_quota_slot_spend_from_preview( goal_id: str, execute: bool = False, source: str = DEFAULT_SLOT_SPEND_SOURCE, + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: if not preview.get("ok"): return preview @@ -197,18 +227,37 @@ def record_quota_slot_spend_from_preview( raise ValueError("quota spend preview index basis must be a string or null") if execute: index_path = runtime_root / "goals" / safe_goal_id / "runs" / "index.jsonl" - # Legacy Python run writers use this kernel lock. Hold it across the - # single TS transaction until Stage 3 moves every index writer into the - # native runtime; the TS owner also serializes native callers itself. - with exclusive_file_lock(index_path, operation="quota_spend_commit"): - result = _quota_spend_commit_result( - preview, - source=source, - generated_at=None, - execute=True, + if registry_path is None and goal_ref is None: + with exclusive_file_lock( + index_path, + operation="quota_spend_commit", + ): + result = _quota_spend_commit_result( + preview, + source=source, + generated_at=None, + execute=True, + runtime_root=runtime_root, + expected_index_digest=expected_index_digest, + ) + else: + with quota_accounting_admission( runtime_root=runtime_root, - expected_index_digest=expected_index_digest, - ) + registry_path=registry_path, + goal_id=safe_goal_id, + goal_ref=goal_ref, + operation="quota_spend_commit", + ) as source_admission: + result = _quota_spend_commit_result( + preview, + source=source, + generated_at=None, + execute=True, + runtime_root=runtime_root, + expected_index_digest=expected_index_digest, + goal_ref=goal_ref, + source_admission=source_admission, + ) else: result = _quota_spend_commit_result( preview, @@ -217,6 +266,8 @@ def record_quota_slot_spend_from_preview( execute=False, runtime_root=runtime_root, expected_index_digest=expected_index_digest, + goal_ref=None, + source_admission=None, ) payload = result.get("payload") if not isinstance(payload, Mapping): diff --git a/loopx/control_plane/quota/spend_commit.ts b/loopx/control_plane/quota/spend_commit.ts index 484e37416f..7a4f7eab4a 100644 --- a/loopx/control_plane/quota/spend_commit.ts +++ b/loopx/control_plane/quota/spend_commit.ts @@ -20,6 +20,12 @@ import { requireNonEmptyString as requiredString, requireStringLiteral, } from "../runtime_decode.ts"; +import { + parseQuotaAccountingOwner, + quotaGoalRef, + withQuotaAccountingOwner, + type QuotaAccountingOwner, +} from "./source_admission.ts"; export const QUOTA_SPEND_COMMIT_REQUEST_SCHEMA = "loopx_quota_spend_commit_request_v0"; @@ -92,6 +98,7 @@ interface QuotaSpendCommitRequest { before: CompactQuotaDecision; after: CompactQuotaDecision; resolved_agent_id: string | null; + owner: QuotaAccountingOwner; } interface QuotaSpendReplayRequest { @@ -102,6 +109,7 @@ interface QuotaSpendReplayRequest { effect_id: string; resolved_agent_id: string | null; read_only: boolean; + owner: QuotaAccountingOwner; } type SpendDisposition = @@ -224,11 +232,12 @@ function replayRequestObject(value: unknown): QuotaSpendReplayRequest { if (!isAbsolute(runtimeRoot)) { throw new EffectRuntimeRequestError("runtime_root must be absolute"); } + const goalId = safeGoalId(request.goal_id); return { schema_version: QUOTA_SPEND_COMMIT_REQUEST_SCHEMA, operation: "replay", runtime_root: runtimeRoot, - goal_id: safeGoalId(request.goal_id), + goal_id: goalId, effect_id: requiredString(request.effect_id, "effect_id").trim(), resolved_agent_id: optionalString( request.resolved_agent_id, @@ -237,6 +246,12 @@ function replayRequestObject(value: unknown): QuotaSpendReplayRequest { read_only: request.read_only === undefined ? false : requiredBoolean(request.read_only, "read_only"), + owner: parseQuotaAccountingOwner({ + goalRefValue: request.goal_ref, + sourceAdmissionValue: request.source_admission, + runtimeRoot, + goalId, + }), }; } @@ -261,6 +276,12 @@ function requestObject(value: unknown): QuotaSpendCommitRequest { `quota slot spend source must be one of: ${QUOTA_SPEND_SOURCES.join(", ")}`, ); const requestedEffectId = optionalString(request.effect_id, "effect_id")?.trim(); + const owner = parseQuotaAccountingOwner({ + goalRefValue: request.goal_ref, + sourceAdmissionValue: request.source_admission, + runtimeRoot: runtimeRoot ?? "", + goalId, + }); return { schema_version: QUOTA_SPEND_COMMIT_REQUEST_SCHEMA, effect_id: requestedEffectId || derivedEffectId( @@ -286,6 +307,7 @@ function requestObject(value: unknown): QuotaSpendCommitRequest { request.resolved_agent_id, "resolved_agent_id", )?.trim() ?? null, + owner, }; } @@ -321,6 +343,7 @@ function derivedEffectId( } function requestDigest(request: QuotaSpendCommitRequest): string { + const goalRef = quotaGoalRef(request.owner); return sha256(canonicalJson({ schema_version: request.schema_version, effect_id: request.effect_id, @@ -334,6 +357,7 @@ function requestDigest(request: QuotaSpendCommitRequest): string { before: request.before, after: request.after, resolved_agent_id: request.resolved_agent_id, + ...(goalRef === null ? {} : { goal_ref: goalRef }), })); } @@ -495,6 +519,11 @@ function buildSpendRecord( quota_event: quotaEvent, quota_spend_commit: commit, }; + const goalRef = quotaGoalRef(request.owner); + if (goalRef !== null) { + record.goal_ref = goalRef; + quotaEvent.goal_ref = goalRef; + } if (request.resolved_agent_id) { record.agent_id = request.resolved_agent_id; quotaEvent.agent_id = request.resolved_agent_id; @@ -535,11 +564,17 @@ async function evaluateQuotaSpendReplay( "runs", "index.jsonl", ); - const lookup = await lookupQuotaAccountingReplay( - "spend", - indexPath, - request.effect_id, - request.read_only, + const goalRef = quotaGoalRef(request.owner); + const lookup = await withQuotaAccountingOwner( + request.owner, + async (indexLockHeld) => await lookupQuotaAccountingReplay( + "spend", + indexPath, + request.effect_id, + request.read_only, + goalRef, + indexLockHeld, + ), ); const requestFingerprint = sha256(canonicalJson(value)); if (lookup.resolution.kind === "conflict") { @@ -563,7 +598,7 @@ async function evaluateQuotaSpendReplay( effect_ref: request.effect_id, reason: lookup.resolution.reason, }, - reason_code: "effect_id_conflict", + reason_code: lookup.resolution.reasonCode, }; } const candidate = lookup.resolution.kind === "matched" @@ -662,6 +697,8 @@ function indexRecordFor( request_digest: fingerprint, }, }; + const goalRef = quotaGoalRef(request.owner); + if (goalRef !== null) indexRecord.goal_ref = goalRef; for (const field of [ "agent_id", "effect_ref", @@ -711,6 +748,8 @@ function payloadFor( : `${request.execute ? "appended" : "dry-run preview"} quota slot spend event: ` + `${request.goal_id} ${request.before.spent_slots}->${request.after.spent_slots} slots`, }; + const goalRef = quotaGoalRef(request.owner); + if (goalRef !== null) payload.goal_ref = goalRef; if (request.execute) { payload.before = request.before; payload.after = request.after; @@ -800,40 +839,46 @@ export async function evaluateQuotaSpendCommit( ); } - const outcome = await commitQuotaAccountingArtifactTransaction({ - kind: "spend", - runsDir, - generatedAt: request.generated_at, - effectId: request.effect_id, - requestDigest: fingerprint, - expectedIndexDigest: request.expected_index_digest, - prepare: ({ jsonPath, markdownPath, indexPath: lockedIndexPath }) => { - const payload = payloadFor( - request, - record, - jsonPath, - markdownPath, - lockedIndexPath, - { appended: true, replayed: false, repaired: false }, - ); - return { - kind: "prepared", - record, - indexRecord: indexRecordFor( + const goalRef = quotaGoalRef(request.owner); + const outcome = await withQuotaAccountingOwner( + request.owner, + async (indexLockHeld) => await commitQuotaAccountingArtifactTransaction({ + kind: "spend", + runsDir, + generatedAt: request.generated_at, + effectId: request.effect_id, + requestDigest: fingerprint, + expectedIndexDigest: request.expected_index_digest, + ...(goalRef === null ? {} : { goalRef }), + indexLockHeld, + prepare: ({ jsonPath, markdownPath, indexPath: lockedIndexPath }) => { + const payload = payloadFor( request, record, jsonPath, markdownPath, - fingerprint, - ), - markdown: renderQuotaSlotMarkdown( + lockedIndexPath, + { appended: true, replayed: false, repaired: false }, + ); + return { + kind: "prepared", + record, + indexRecord: indexRecordFor( + request, + record, + jsonPath, + markdownPath, + fingerprint, + ), + markdown: renderQuotaSlotMarkdown( + payload, + QUOTA_SLOT_SPENT_CLASSIFICATION, + ), payload, - QUOTA_SLOT_SPENT_CLASSIFICATION, - ), - payload, - }; - }, - }); + }; + }, + }), + ); if (outcome.status === "conflict") { return result( diff --git a/loopx/control_plane/quota/unsettled_host_turn.py b/loopx/control_plane/quota/unsettled_host_turn.py index 7d0b5cd94a..7b02ed1ab2 100644 --- a/loopx/control_plane/quota/unsettled_host_turn.py +++ b/loopx/control_plane/quota/unsettled_host_turn.py @@ -28,6 +28,7 @@ CloseoutQueryUnavailableError, HeartbeatReceiptIdentityConflictError, ) +from .accounting_admission import quota_accounting_admission UNSETTLED_HOST_TURN_RECOVERY_SCHEMA_VERSION = "unsettled_host_turn_recovery_v0" @@ -104,6 +105,8 @@ def _prior_closeout_preflight( goal_id: str, agent_id: str, current_turn_instance_id: str | None, + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> tuple[dict[str, Any], list[str], dict[str, Any]] | None: """Ask the typed owner which prior Turn must still be closed out. @@ -113,17 +116,42 @@ def _prior_closeout_preflight( """ try: - result = effect_runtime_result( - PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_METHOD, - { - "schema_version": PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_REQUEST_SCHEMA, - "runtime_root": str(runtime_root.expanduser()), - "goal_id": goal_id, - "agent_id": agent_id, - "exclude_turn_instance_id": current_turn_instance_id, - }, - timeout=PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_TIMEOUT_SECONDS, - ) + with quota_accounting_admission( + runtime_root=runtime_root, + registry_path=registry_path, + goal_id=goal_id, + goal_ref=goal_ref, + operation="prior-host-turn-closeout-preflight", + lock_legacy_index=False, + ) as source_admission: + request_runtime_root = ( + runtime_root.expanduser().resolve() + if source_admission is not None + else runtime_root.expanduser() + ) + result = effect_runtime_result( + PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_METHOD, + { + "schema_version": ( + PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_REQUEST_SCHEMA + ), + "runtime_root": str(request_runtime_root), + "goal_id": goal_id, + "agent_id": agent_id, + "exclude_turn_instance_id": current_turn_instance_id, + **( + {"goal_ref": dict(goal_ref)} + if goal_ref is not None + else {} + ), + **( + {"source_admission": dict(source_admission)} + if source_admission is not None + else {} + ), + }, + timeout=PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_TIMEOUT_SECONDS, + ) except EffectRuntimeResponseAmbiguous as exc: # This method only reads receipts. A lost query response is not a # possibly committed mutation, and must not send the operator hunting @@ -171,6 +199,7 @@ def _unsettled_host_turn_recovery( goal_id: str, agent_id: str | None, current_turn_instance_id: str | None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any] | None: if not agent_id or not current_turn_instance_id: return None @@ -179,6 +208,8 @@ def _unsettled_host_turn_recovery( goal_id=goal_id, agent_id=agent_id, current_turn_instance_id=current_turn_instance_id, + registry_path=registry_path, + goal_ref=goal_ref, ) if preflight is None: return None @@ -244,6 +275,7 @@ def apply_unsettled_host_turn_recovery_if_required( scheduler_execution_context: ( Mapping[str, Any] | SchedulerExecutionContextResolution | None ), + goal_ref: Mapping[str, Any] | None = None, ) -> bool: """Preempt ordinary selection when the preceding host Turn lacks closeout.""" @@ -253,6 +285,7 @@ def apply_unsettled_host_turn_recovery_if_required( goal_id=goal_id, agent_id=agent_id, current_turn_instance_id=current_turn_instance_id, + goal_ref=goal_ref, ) if verdict is None: return False diff --git a/loopx/control_plane/quota/unsettled_host_turn_recovery.ts b/loopx/control_plane/quota/unsettled_host_turn_recovery.ts index c892b37277..bdeb187fd6 100644 --- a/loopx/control_plane/quota/unsettled_host_turn_recovery.ts +++ b/loopx/control_plane/quota/unsettled_host_turn_recovery.ts @@ -39,9 +39,15 @@ import { import { committedMonitorPollFromSnapshot, QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA, - readQuotaSettlementFromSnapshot, + readAdmittedQuotaSettlementFromSnapshot, readQuotaSettlementSnapshot, } from "./settlement_readback.ts"; +import { + parseQuotaAccountingOwner, + quotaOwnerOwnsProjection, + withQuotaAccountingOwner, + type QuotaAccountingOwner, +} from "./source_admission.ts"; export const PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_REQUEST_SCHEMA = "loopx_prior_host_turn_closeout_preflight_request_v0"; @@ -83,6 +89,8 @@ interface PreflightRequest { goal_id: string; agent_id: string; exclude_turn_instance_id: string | null; + owner: QuotaAccountingOwner; + owner_projection: JsonObject; } function decodePreflightRequest(value: unknown): PreflightRequest { @@ -92,13 +100,31 @@ function decodePreflightRequest(value: unknown): PreflightRequest { "Prior host Turn closeout preflight request schema mismatch", ); } + const runtimeRoot = requireNonEmptyString(request.runtime_root, "runtime_root"); + const goalId = requireNonEmptyString(request.goal_id, "goal_id"); + const owner = parseQuotaAccountingOwner({ + goalRefValue: request.goal_ref, + sourceAdmissionValue: request.source_admission, + runtimeRoot, + goalId, + }); return { - runtime_root: requireNonEmptyString(request.runtime_root, "runtime_root"), - goal_id: requireNonEmptyString(request.goal_id, "goal_id"), + runtime_root: runtimeRoot, + goal_id: goalId, agent_id: requireNonEmptyString(request.agent_id, "agent_id"), exclude_turn_instance_id: optionalHeartbeatString( request.exclude_turn_instance_id, ), + owner, + owner_projection: owner.kind === "alias" + ? {} + : { + goal_ref: requireJsonObject(request.goal_ref, "goal_ref"), + source_admission: requireJsonObject( + request.source_admission, + "source_admission", + ), + }, }; } @@ -186,6 +212,7 @@ function settlementReadbackRequest( : null, infer_turn_instance_id: false, allow_unbound_binding: false, + ...request.owner_projection, }; } @@ -209,10 +236,9 @@ function bundleFailed(readback: JsonObject, step: string): boolean { * Turn's settlement so a Turn that already settled never makes the caller read * bound facts. */ -export async function preflightPriorHostTurnCloseout( - value: unknown, +async function preflightPriorHostTurnCloseoutForOwner( + request: PreflightRequest, ): Promise { - const request = decodePreflightRequest(value); const rolloutSnapshot = await readGoalRolloutEventSnapshot( request.runtime_root, request.goal_id, @@ -221,6 +247,8 @@ export async function preflightPriorHostTurnCloseout( rolloutSnapshot, request.goal_id, request.agent_id, + )?.filter((event) => + quotaOwnerOwnsProjection(request.owner, event.goal_ref) ); const { candidates, turnsValidated } = selectCloseoutCandidates( receipts ?? [], @@ -244,7 +272,7 @@ export async function preflightPriorHostTurnCloseout( let newestSettledTurn: string | null = null; let newestAcceptedCloseout: AcceptedCloseout = "validated_writeback_and_quota_spend"; for (const selected of candidates) { - const readback = readQuotaSettlementFromSnapshot( + const readback = readAdmittedQuotaSettlementFromSnapshot( settlementReadbackRequest(request, selected), settlementSnapshot, ); @@ -273,7 +301,7 @@ export async function preflightPriorHostTurnCloseout( goal_id: request.goal_id, agent_id: request.agent_id, turn_instance_id: selected.prior_turn_instance_id, todo_id: selected.binding_id, - }) : null; + }, request.owner) : null; return { schema_version: PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_RESULT_SCHEMA, status: "candidate", @@ -295,6 +323,16 @@ export async function preflightPriorHostTurnCloseout( }; } +export async function preflightPriorHostTurnCloseout( + value: unknown, +): Promise { + const request = decodePreflightRequest(value); + return await withQuotaAccountingOwner( + request.owner, + async () => preflightPriorHostTurnCloseoutForOwner(request), + ); +} + function decodeCandidate(value: unknown): PriorHostTurnCloseoutCandidate { const raw = requireJsonObject(value, "prior host Turn recovery candidate"); const bindingKind = raw.binding_kind; diff --git a/loopx/control_plane/quota/void_commit.py b/loopx/control_plane/quota/void_commit.py index 04b946925c..652b903e76 100644 --- a/loopx/control_plane/quota/void_commit.py +++ b/loopx/control_plane/quota/void_commit.py @@ -6,8 +6,13 @@ from uuid import uuid4 from ...file_lock import exclusive_file_lock -from ..effect_runtime import EffectRuntimeRejected, effect_runtime_result +from ..effect_runtime import ( + EffectRuntimeConflict, + EffectRuntimeRejected, + effect_runtime_result, +) from ..runtime.time import now_local_iso +from .accounting_admission import quota_accounting_admission from .spend_commit import quota_spend_index_digest from .spend_sources import DEFAULT_SLOT_SPEND_SOURCE @@ -54,7 +59,7 @@ def _void_result( ) -> Mapping[str, Any]: try: result = effect_runtime_result("quota.void.commit", dict(params)) - except EffectRuntimeRejected as exc: + except (EffectRuntimeConflict, EffectRuntimeRejected) as exc: raise ValueError(str(exc)) from None if ( not isinstance(result, Mapping) @@ -132,6 +137,8 @@ def commit_quota_slot_void( effect_id: str | None = None, generated_at: str | None = None, _operation: str = "commit", + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: """Execute one TypeScript-owned quota void transaction.""" @@ -154,15 +161,29 @@ def commit_quota_slot_void( "before": dict(before), } - if execute: - # Legacy Python run writers still use the kernel lock. Hold it across - # the one native transaction until every index writer is in-process TS. - with exclusive_file_lock(index_path, operation="quota_void_commit"): + if registry_path is None and goal_ref is None: + if execute: + with exclusive_file_lock(index_path, operation="quota_void_commit"): + params["expected_index_digest"] = quota_spend_index_digest(index_path) + result = _void_result(params, expected_goal_id=safe_goal_id) + else: params["expected_index_digest"] = quota_spend_index_digest(index_path) result = _void_result(params, expected_goal_id=safe_goal_id) else: - params["expected_index_digest"] = quota_spend_index_digest(index_path) - result = _void_result(params, expected_goal_id=safe_goal_id) + with quota_accounting_admission( + runtime_root=runtime_root, + registry_path=registry_path, + goal_id=safe_goal_id, + goal_ref=goal_ref, + operation="quota_void_commit", + lock_legacy_index=execute, + ) as source_admission: + params["expected_index_digest"] = quota_spend_index_digest(index_path) + if goal_ref is not None: + params["goal_ref"] = dict(goal_ref) + if source_admission is not None: + params["source_admission"] = dict(source_admission) + result = _void_result(params, expected_goal_id=safe_goal_id) return _result_payload(result) @@ -217,6 +238,8 @@ def record_quota_slot_void_from_preview( execute: bool = False, source: str = DEFAULT_SLOT_SPEND_SOURCE, reason_summary: str | None = None, + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: del render_markdown if not preview.get("ok"): @@ -240,4 +263,6 @@ def record_quota_slot_void_from_preview( execute=execute, source=source, reason_summary=reason_summary, + registry_path=registry_path, + goal_ref=goal_ref, ) diff --git a/loopx/control_plane/quota/void_commit.ts b/loopx/control_plane/quota/void_commit.ts index 9d2448a70c..abb056decf 100644 --- a/loopx/control_plane/quota/void_commit.ts +++ b/loopx/control_plane/quota/void_commit.ts @@ -21,6 +21,13 @@ import { requireNonEmptyString as requiredString, requireStringLiteral, } from "../runtime_decode.ts"; +import { + parseQuotaAccountingOwner, + quotaGoalRef, + requireQuotaOwnerProjection, + withQuotaAccountingOwner, + type QuotaAccountingOwner, +} from "./source_admission.ts"; export const QUOTA_VOID_COMMIT_REQUEST_SCHEMA = "loopx_quota_void_commit_request_v0"; @@ -63,6 +70,7 @@ interface QuotaVoidCommitRequest { execute: boolean; expected_index_digest: string | null; before: JsonObject; + owner: QuotaAccountingOwner; } interface QuotaVoidProjectionRequest { @@ -173,12 +181,13 @@ function commitRequest(value: unknown): QuotaVoidCommitRequest { "quota void preview operation cannot execute durable effects", ); } + const goalId = safeGoalId(request.goal_id); return { schema_version: QUOTA_VOID_COMMIT_REQUEST_SCHEMA, operation, effect_id: effectId, runtime_root: runtimeRoot, - goal_id: safeGoalId(request.goal_id), + goal_id: goalId, voided_run_generated_at: typeof request.voided_run_generated_at === "string" ? request.voided_run_generated_at.trim() @@ -200,6 +209,12 @@ function commitRequest(value: unknown): QuotaVoidCommitRequest { "expected_index_digest", ), before, + owner: parseQuotaAccountingOwner({ + goalRefValue: request.goal_ref, + sourceAdmissionValue: request.source_admission, + runtimeRoot, + goalId, + }), }; } @@ -227,6 +242,7 @@ function projectionRequest(value: unknown): QuotaVoidProjectionRequest { } function requestDigest(request: QuotaVoidCommitRequest): string { + const goalRef = quotaGoalRef(request.owner); return sha256(canonicalJson({ schema_version: request.schema_version, effect_id: request.effect_id, @@ -236,6 +252,7 @@ function requestDigest(request: QuotaVoidCommitRequest): string { source: request.source, reason_summary: request.reason_summary, before: request.before, + ...(goalRef === null ? {} : { goal_ref: goalRef }), })); } @@ -406,6 +423,7 @@ async function findTargetSpend( records: readonly JsonObject[], goalId: string, generatedAt: string, + owner: QuotaAccountingOwner, ): Promise { for (const run of [...records].reverse()) { if (String(run.goal_id || goalId) !== goalId) continue; @@ -413,6 +431,12 @@ async function findTargetSpend( if (run.classification !== QUOTA_SLOT_SPENT_CLASSIFICATION) continue; const event = await readTargetEvent(runsDir, run, goalId); if (event?.event_type !== QUOTA_SLOT_SPENT_CLASSIFICATION) continue; + requireQuotaOwnerProjection(owner, run.goal_ref, "quota void target"); + requireQuotaOwnerProjection( + owner, + event.goal_ref, + "quota void target event", + ); return { run, event }; } return null; @@ -459,7 +483,7 @@ function previewFor( legacyInteger(beforeQuota.spent_slots, 0) - slots, ); after.quota = afterQuota; - return { + const preview: JsonObject = { ok: true, mode: "void-slot", dry_run: true, @@ -479,6 +503,9 @@ function previewFor( rolling_window_note: ROLLING_WINDOW_NOTE, classification: QUOTA_SLOT_VOIDED_CLASSIFICATION, }; + const goalRef = quotaGoalRef(request.owner); + if (goalRef !== null) preview.goal_ref = goalRef; + return preview; } function recordFor( @@ -488,6 +515,7 @@ function recordFor( generatedAt: string, effectId: string | null, fingerprint: string, + goalRef: JsonObject | null = null, ): JsonObject { if (preview.ok !== true) { throw new EffectRuntimeRequestError( @@ -531,6 +559,10 @@ function recordFor( request_digest: fingerprint, }; } + if (goalRef !== null) { + record.goal_ref = goalRef; + event.goal_ref = goalRef; + } return record; } @@ -550,6 +582,7 @@ function artifactsFor( request.generated_at, request.effect_id, fingerprint, + quotaGoalRef(request.owner), ); const event = requiredObject(record.quota_event, "record.quota_event"); const payload: JsonObject = { @@ -586,6 +619,11 @@ function artifactsFor( quota_void_commit: record.quota_void_commit, }; if (record.agent_id) indexRecord.agent_id = record.agent_id; + const goalRef = quotaGoalRef(request.owner); + if (goalRef !== null) { + indexRecord.goal_ref = goalRef; + payload.goal_ref = goalRef; + } return { kind: "prepared", record, @@ -609,6 +647,7 @@ async function prepareArtifacts( context.indexRecords, request.goal_id, request.voided_run_generated_at, + request.owner, ); if (!target) { const payload = missingTargetPayload( @@ -680,6 +719,7 @@ async function previewCommit( records, request.goal_id, request.voided_run_generated_at, + request.owner, ); if (!target) { const payload = missingTargetPayload( @@ -754,72 +794,77 @@ async function evaluateCommit( request.goal_id, "runs", ); - if (!request.execute) { - return await previewCommit(request, fingerprint, runsDir); - } - const outcome = await commitQuotaAccountingArtifactTransaction({ - kind: "void", - runsDir, - generatedAt: request.generated_at, - effectId: request.effect_id, - requestDigest: fingerprint, - expectedIndexDigest: request.expected_index_digest, - prepare: async (context) => - await prepareArtifacts(request, fingerprint, runsDir, context), - }); - if (outcome.status === "conflict") { - return result( - request.effect_id, - fingerprint, - "conflict", - outcome.indexDigest, - outcome.reason, - null, - { - ok: false, - mode: "void-slot", - goal_id: request.goal_id, - effect_id: request.effect_id, - appended: false, - registry_mutated: false, - }, - outcome.reasonCode, - ); - } - if (outcome.status === "not_found") { + return await withQuotaAccountingOwner(request.owner, async (indexLockHeld) => { + if (!request.execute) { + return await previewCommit(request, fingerprint, runsDir); + } + const goalRef = quotaGoalRef(request.owner); + const outcome = await commitQuotaAccountingArtifactTransaction({ + kind: "void", + runsDir, + generatedAt: request.generated_at, + effectId: request.effect_id, + requestDigest: fingerprint, + expectedIndexDigest: request.expected_index_digest, + ...(goalRef === null ? {} : { goalRef }), + indexLockHeld, + prepare: async (context) => + await prepareArtifacts(request, fingerprint, runsDir, context), + }); + if (outcome.status === "conflict") { + return result( + request.effect_id, + fingerprint, + "conflict", + outcome.indexDigest, + outcome.reason, + null, + { + ok: false, + mode: "void-slot", + goal_id: request.goal_id, + effect_id: request.effect_id, + appended: false, + registry_mutated: false, + }, + outcome.reasonCode, + ); + } + if (outcome.status === "not_found") { + return result( + request.effect_id, + fingerprint, + "not_found", + outcome.indexDigest, + outcome.reason, + null, + outcome.payload, + "target_not_found", + ); + } + const replayed = outcome.status === "replayed"; + const repaired = outcome.status === "repaired"; + const responsePayload: JsonObject = { + ...outcome.receipt.payload, + appended: outcome.status === "written" || repaired, + idempotent_replay: replayed, + transaction_repaired: repaired, + reason: replayed + ? "quota void commit replayed for the same effect identity" + : repaired + ? "quota void commit repaired its prepared durable transaction" + : outcome.receipt.payload.reason, + }; return result( request.effect_id, fingerprint, - "not_found", + outcome.status, outcome.indexDigest, - outcome.reason, - null, - outcome.payload, - "target_not_found", + String(responsePayload.reason ?? ""), + outcome.receipt.record, + responsePayload, ); - } - const replayed = outcome.status === "replayed"; - const repaired = outcome.status === "repaired"; - const responsePayload: JsonObject = { - ...outcome.receipt.payload, - appended: outcome.status === "written" || repaired, - idempotent_replay: replayed, - transaction_repaired: repaired, - reason: replayed - ? "quota void commit replayed for the same effect identity" - : repaired - ? "quota void commit repaired its prepared durable transaction" - : outcome.receipt.payload.reason, - }; - return result( - request.effect_id, - fingerprint, - outcome.status, - outcome.indexDigest, - String(responsePayload.reason ?? ""), - outcome.receipt.record, - responsePayload, - ); + }); } function evaluateProjection( diff --git a/loopx/control_plane/status/collection.py b/loopx/control_plane/status/collection.py index f0359a6c84..da2bfe5240 100644 --- a/loopx/control_plane/status/collection.py +++ b/loopx/control_plane/status/collection.py @@ -153,6 +153,8 @@ def collect_status( activation_filter is GoalActivationState.STOPPED ), events_for_goal=rollout_events.events_for_goal, + current_registry=registry, + registry_path=registry_path, ) # No later projection consumes canonical rows; release retained archive # data before assembling the rest of the display. diff --git a/loopx/control_plane/turn_driver/executor.py b/loopx/control_plane/turn_driver/executor.py index 6da35b68e3..4dc8a06956 100644 --- a/loopx/control_plane/turn_driver/executor.py +++ b/loopx/control_plane/turn_driver/executor.py @@ -997,6 +997,11 @@ def _ensure_turn_settlement_plan( transaction_plan.get("turn_instance_id") or transaction_plan.get("turn_key") ), + goal_ref=( + plan.get("goal_ref") + if isinstance(plan.get("goal_ref"), Mapping) + else None + ), ) settlement_plan = built.get("settlement_plan") if isinstance(settlement_plan, Mapping): diff --git a/loopx/control_plane/turn_driver/host_todo_completion.ts b/loopx/control_plane/turn_driver/host_todo_completion.ts index 1d17b84dd4..5dd272d20a 100644 --- a/loopx/control_plane/turn_driver/host_todo_completion.ts +++ b/loopx/control_plane/turn_driver/host_todo_completion.ts @@ -9,6 +9,7 @@ import { } from "../effect_program.ts"; import { EffectRuntimeRequestError } from "../effect_runtime_errors.ts"; import { normalizeVisionUnchangedReason } from "../goals/vision_checkpoint.ts"; +import { parseExactGoalRef } from "../goals/goal_instance_identity.ts"; import { projectMcpInteraction } from "./host_interaction.ts"; import { requireBoolean, @@ -55,6 +56,7 @@ interface HostTodoCompletionRequest { vision_path: string | null; vision_unchanged_reason: string | null; checkpoint_read_context_id: string | null; + goal_instance_id: string | null; provider_outcomes: readonly ProviderOutcome[]; } @@ -148,9 +150,24 @@ function decodeRequest( if (readContextId && phase !== "vision_refresh") { throw new EffectRuntimeRequestError("checkpoint read context belongs only to vision recovery"); } + const goalId = requireNonEmptyString(value.goal_id, "goal_id"); + const parsedGoalRef = value.goal_ref === undefined + ? null + : parseExactGoalRef(value.goal_ref); + if (parsedGoalRef?.kind === "invalid") { + throw new EffectRuntimeRequestError("host Todo completion GoalRef is malformed"); + } + if ( + parsedGoalRef?.kind === "parsed" + && parsedGoalRef.value.goalId.value !== goalId + ) { + throw new EffectRuntimeRequestError( + "host Todo completion GoalRef does not match goal_id", + ); + } const request: HostTodoCompletionRequest = { phase, - goal_id: requireNonEmptyString(value.goal_id, "goal_id"), + goal_id: goalId, agent_id: requireNonEmptyString(value.agent_id, "agent_id"), todo_id: todoId, runtime_profile: requireNonEmptyString( @@ -177,6 +194,9 @@ function decodeRequest( vision_path: visionPath, vision_unchanged_reason: unchanged, checkpoint_read_context_id: readContextId, + goal_instance_id: parsedGoalRef?.kind === "parsed" + ? parsedGoalRef.value.goalInstanceId.value + : null, provider_outcomes: [], }; if (phase === "finalize") { @@ -344,6 +364,9 @@ function writebackArgs(request: HostTodoCompletionRequest, identity: JsonObject) "--classification", "mcp_completed_turn_writeback", "--delivery-batch-scale", "single_surface", "--delivery-outcome", "outcome_progress", "--todo-id", request.todo_id, "--turn-instance-id", String(identity.turn_instance_id), + ...(request.goal_instance_id + ? ["--goal-instance-id", request.goal_instance_id] + : []), "--completion-todo-id", request.todo_id, "--completion-turn-key", String(identity.effect_id), "--no-global-sync", "--suppress-external-sinks", ...(request.vision_path ? ["--agent-vision-json", request.vision_path] : []), @@ -368,10 +391,16 @@ function providerSteps( request.todo_id, "--turn-instance-id", turnId, + ...(request.goal_instance_id + ? ["--goal-instance-id", request.goal_instance_id] + : []), ]; const lifecycleArgs = request.completion_args.filter( (arg) => arg !== "--no-follow-up", ); + if (request.goal_instance_id) { + lifecycleArgs.push("--goal-instance-id", request.goal_instance_id); + } const steps: ProviderStep[] = [ { step_kind: "guard", @@ -417,6 +446,9 @@ function providerSteps( request.todo_id, "--turn-instance-id", turnId, + ...(request.goal_instance_id + ? ["--goal-instance-id", request.goal_instance_id] + : []), ], legacy_args: null, continue_when: request.no_follow_up @@ -427,7 +459,14 @@ function providerSteps( if (request.no_follow_up) { steps.push({ step_kind: "terminal_closeout", - args: [...request.completion_args, "--turn-instance-id", turnId], + args: [ + ...request.completion_args, + "--turn-instance-id", + turnId, + ...(request.goal_instance_id + ? ["--goal-instance-id", request.goal_instance_id] + : []), + ], legacy_args: null, continue_when: null, }); @@ -981,7 +1020,10 @@ export function evaluateHostTodoCompletion(value: JsonObject): JsonObject { schema_version: HOST_TODO_COMPLETION_REDUCTION_SCHEMA_VERSION, phase, identity, args: ["checkpoint-context", "--goal-id", request.goal_id, "--agent-id", request.agent_id, "--todo-id", request.todo_id, - "--turn-instance-id", String(identity.turn_instance_id)], + "--turn-instance-id", String(identity.turn_instance_id), + ...(request.goal_instance_id + ? ["--goal-instance-id", request.goal_instance_id] + : [])], }; } if (phase === "vision_refresh") { diff --git a/loopx/control_plane/turn_driver/transaction.py b/loopx/control_plane/turn_driver/transaction.py index 2722a3940c..b96ece6d3f 100644 --- a/loopx/control_plane/turn_driver/transaction.py +++ b/loopx/control_plane/turn_driver/transaction.py @@ -199,6 +199,8 @@ def build_loopx_turn_transaction_plan( } if planned: plan["settlement_plan"] = settlement_plan.as_dict() + if goal_ref is not None: + plan["settlement_plan"]["goal_ref"] = dict(goal_ref) if normalized_instance_id is not None: plan["turn_instance_id"] = normalized_instance_id if goal_ref is not None: diff --git a/loopx/control_plane/work_items/accountable_settlement.py b/loopx/control_plane/work_items/accountable_settlement.py index 81fcfb7976..7fd8627b8a 100644 --- a/loopx/control_plane/work_items/accountable_settlement.py +++ b/loopx/control_plane/work_items/accountable_settlement.py @@ -1,5 +1,7 @@ from __future__ import annotations +from collections.abc import Mapping + from ...turn_identity import normalize_turn_instance_id from ..quota.settlement import ( SettlementPlan, @@ -25,6 +27,7 @@ def build_accountable_work_item_settlement_plan( turn_instance_id: str | None, delivery_boundary: str | None = None, command_prefix: str = "loopx", + goal_ref: Mapping[str, object] | None = None, ) -> SettlementPlan | None: if runtime_profile in APP_HEARTBEAT_SETTLEMENT_RUNTIME_PROFILES: normalized_turn_instance_id = normalize_turn_instance_id(turn_instance_id) @@ -38,6 +41,7 @@ def build_accountable_work_item_settlement_plan( lifecycle_actor_args=lifecycle_actor_args, turn_instance_id_ref=normalized_turn_instance_id, delivery_boundary=delivery_boundary, + goal_ref=goal_ref, ) if runtime_profile in VISIBLE_GOAL_SETTLEMENT_RUNTIME_PROFILES: normalized_turn_instance_id = normalize_turn_instance_id(turn_instance_id) @@ -54,6 +58,7 @@ def build_accountable_work_item_settlement_plan( turn_instance_id=normalized_turn_instance_id, delivery_boundary=delivery_boundary, quota_spend_source="visible-goal", + goal_ref=goal_ref, ) if runtime_profile is not SchedulerRuntimeProfile.GENERIC_CLI_AGENT_LOOP: return None @@ -70,4 +75,5 @@ def build_accountable_work_item_settlement_plan( lifecycle_actor_args=lifecycle_actor_args, turn_instance_id=normalized_turn_instance_id, delivery_boundary=delivery_boundary, + goal_ref=goal_ref, ) diff --git a/loopx/control_plane/work_items/action_selection_contract.py b/loopx/control_plane/work_items/action_selection_contract.py index 9f22663cad..cbcc5be008 100644 --- a/loopx/control_plane/work_items/action_selection_contract.py +++ b/loopx/control_plane/work_items/action_selection_contract.py @@ -359,6 +359,7 @@ def action_selection_recovery_command( *, registry_path: str | None = None, runtime_root: str | None = None, goal_id: str, agent_id: str | None, turn_instance_id: str | None, scheduler_args: str, available_capabilities: Any = None, + goal_ref: Mapping[str, object] | None = None, ) -> str: argv = ["loopx"] if registry_path: @@ -370,6 +371,18 @@ def action_selection_recovery_command( argv.extend(["--agent-id", agent_id]) if turn_instance_id: argv.extend(["--turn-instance-id", turn_instance_id]) + if goal_ref is not None: + from ..goals.source_session_registry_state import exact_goal_ref + + normalized_goal_ref = exact_goal_ref( + str(goal_ref.get("goal_id") or ""), + str(goal_ref.get("goal_instance_id") or ""), + ) + if normalized_goal_ref["goal_id"] != goal_id: + raise ValueError("action selection GoalRef does not match goal_id") + argv.extend( + ["--goal-instance-id", normalized_goal_ref["goal_instance_id"]] + ) for capability in runtime_capabilities_for_cli_projection(available_capabilities): argv.extend(["--available-capability", capability]) return shlex.join(argv) + scheduler_args @@ -387,6 +400,7 @@ def bind_action_selection_recovery_command( payload: dict[str, Any], *, registry_path: str, runtime_root: str, goal_id: str, agent_id: str | None, turn_instance_id: str | None, scheduler_args: str, available_capabilities: Any = None, + goal_ref: Mapping[str, object] | None = None, ) -> None: """Bind the existing recovery projection to the invoking CLI's exact argv.""" if not action_selection_needs_recovery(payload): @@ -395,6 +409,7 @@ def bind_action_selection_recovery_command( registry_path=registry_path, runtime_root=runtime_root, goal_id=goal_id, agent_id=agent_id, turn_instance_id=turn_instance_id, scheduler_args=scheduler_args, available_capabilities=available_capabilities, + goal_ref=goal_ref, ) interaction = payload["interaction_contract"] interaction["agent_channel"]["primary_action"] = command diff --git a/loopx/control_plane/work_items/interaction_contract.py b/loopx/control_plane/work_items/interaction_contract.py index 4b2b5409e2..fb7da5fe63 100644 --- a/loopx/control_plane/work_items/interaction_contract.py +++ b/loopx/control_plane/work_items/interaction_contract.py @@ -550,6 +550,26 @@ def _scoped_cli_args( return f" --agent-id {agent_id}{capability_args}" +def _goal_ref_cli_arg(payload: Mapping[str, Any]) -> str: + value = payload.get("goal_ref") + if value is None: + return "" + if not isinstance(value, Mapping): + raise ValueError("interaction GoalRef must be an object") + from ..goals.source_session_registry_state import exact_goal_ref + + goal_ref = exact_goal_ref( + str(value.get("goal_id") or ""), + str(value.get("goal_instance_id") or ""), + ) + if goal_ref["goal_id"] != str(payload.get("goal_id") or "").strip(): + raise ValueError("interaction GoalRef does not match goal_id") + return ( + " --goal-instance-id " + + shlex.quote(goal_ref["goal_instance_id"]) + ) + + def _turn_scoped_cli_settlement_context( payload: dict[str, Any], *, @@ -614,6 +634,11 @@ def _turn_scoped_cli_settlement_context( == "in_flight_continuation" else None ), + goal_ref=( + payload.get("goal_ref") + if isinstance(payload.get("goal_ref"), Mapping) + else None + ), ) return ( plan.as_dict() if plan is not None else None, @@ -678,7 +703,7 @@ def _selection_recovery_command( agent_id=identity.get("agent_id"), runtime_root=runtime_root, turn_instance_id=turn_instance_id, available_capabilities=available_capabilities, scheduler_args=render_scheduler_execution_args(scheduler_execution_context=scheduler_execution_context), - ) + ) + _goal_ref_cli_arg(payload) def _render_replan_successor_closeout_guard( @@ -751,10 +776,11 @@ def interaction_next_cli_actions( runtime_root=runtime_root, ) if selection_command_template: - return [selection_command_template] + return [selection_command_template + _goal_ref_cli_arg(payload)] + goal_ref_arg = _goal_ref_cli_arg(payload) typed_quota_guard = ( f"{command_prefix} --format json quota should-run --goal-id {goal_id}" - f"{scoped_cli_args}{scheduler_args}" + f"{scoped_cli_args}{scheduler_args}{goal_ref_arg}" if scheduler_args else "rerun the typed quota_guard from the current host packet" ) @@ -764,7 +790,7 @@ def interaction_next_cli_actions( return [turn_reentry_action] typed_monitor_poll = ( f"{command_prefix} quota monitor-poll --goal-id {goal_id}{scoped_cli_args}" - f"{scheduler_args} --execute" + f"{scheduler_args}{goal_ref_arg} --execute" if scheduler_args else "use the current host packet's typed monitor command" ) @@ -1431,6 +1457,7 @@ def _build_interaction_cli_channel( f"{shlex.quote(safe_turn_instance_id)} --todo-id " f"{shlex.quote(selected_monitor_id)}{target_args} --use-current-task-lease --result-hash " f'"${{{AUXILIARY_MONITOR_RESULT_HASH_ENV}:?}}"' + f"{_goal_ref_cli_arg(payload)}" ) auxiliary_projection.update( { diff --git a/loopx/goal_mode_mcp.py b/loopx/goal_mode_mcp.py index 1bec5811bb..3a201afc09 100644 --- a/loopx/goal_mode_mcp.py +++ b/loopx/goal_mode_mcp.py @@ -5,7 +5,7 @@ import shutil import subprocess import sys -from collections.abc import Callable +from collections.abc import Callable, Mapping from dataclasses import dataclass from typing import Annotated, Any @@ -62,6 +62,19 @@ def context(self) -> tuple[str | None, str | None]: def bound_agent_id(self) -> str | None: return self.state().get("agent_id") + def goal_ref(self) -> dict[str, str] | None: + value = self.state().get("goal_ref") + if not isinstance(value, Mapping): + return None + goal_id = value.get("goal_id") + goal_instance_id = value.get("goal_instance_id") + if not isinstance(goal_id, str) or not isinstance(goal_instance_id, str): + return None + return { + "goal_id": goal_id, + "goal_instance_id": goal_instance_id, + } + def command_prefix(self) -> list[str]: executable = shutil.which("loopx") return [executable] if executable else [sys.executable, "-m", "loopx.cli"] @@ -252,6 +265,7 @@ def complete_task( execution_mode=self.config.execution_mode, completion_args=tuple(args), no_follow_up=no_follow_up, + goal_ref=self.goal_ref(), ) with host_vision_request(request, agent_vision, vision_unchanged_reason) as authored: return settle_host_todo_completion(authored, run_cli=self.run_cli) @@ -274,6 +288,7 @@ def review_task_vision( scheduler_owner=self.config.scheduler_owner, execution_mode=self.config.execution_mode, completion_args=(), checkpoint_read_context_id=read_context_id or None, + goal_ref=self.goal_ref(), ) with host_vision_request(request, agent_vision, vision_unchanged_reason) as authored: return refresh_host_todo_vision(authored, run_cli=self.run_cli) diff --git a/loopx/quota.py b/loopx/quota.py index eb90263190..54c2cb23af 100644 --- a/loopx/quota.py +++ b/loopx/quota.py @@ -374,6 +374,25 @@ def goal_quota_with_spend_ledger( continue if str(run.get("goal_id") or goal_id) != goal_id: continue + goal_instance_id = ( + str(goal.get("goal_instance_id") or "").strip() + if goal is not None + else "" + ) + run_goal_ref = ( + run.get("goal_ref") + if isinstance(run.get("goal_ref"), Mapping) + else None + ) + if goal_instance_id: + if ( + run_goal_ref is None + or run_goal_ref.get("goal_id") != goal_id + or run_goal_ref.get("goal_instance_id") != goal_instance_id + ): + continue + elif run_goal_ref is not None: + continue generated_at = _parse_timestamp(run.get("generated_at")) if ( generated_at is None @@ -894,6 +913,7 @@ def build_quota_should_run( receipt_bound_replan_guard_scoped: bool = False, turn_instance_id: str | None = None, runtime_root: str | Path | None = None, + goal_ref: Mapping[str, object] | None = None, ) -> dict[str, Any]: from .control_plane.quota.should_run import ( build_quota_should_run as _build_quota_should_run, @@ -919,6 +939,7 @@ def build_quota_should_run( receipt_bound_replan_guard_scoped=receipt_bound_replan_guard_scoped, turn_instance_id=turn_instance_id, runtime_root=runtime_root, + goal_ref=goal_ref, ) @@ -962,6 +983,8 @@ def build_quota_slot_preview( turn_instance_id: str | None = None, effect_ref: str | None = None, source: str = DEFAULT_SLOT_SPEND_SOURCE, + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: safe_goal_id = _validate_goal_id_path_segment(str(goal_id or "")) basis_available, expected_index_digest = _quota_spend_index_basis( @@ -997,6 +1020,8 @@ def build_quota_slot_preview( replan_obligation_id=replan_obligation_id, turn_instance_id=turn_instance_id, source=source, + registry_path=registry_path, + goal_ref=goal_ref, ) if preview.get("ok") and basis_available: preview["expected_index_digest"] = expected_index_digest @@ -1114,6 +1139,7 @@ def record_quota_monitor_poll( Callable[..., Mapping[str, Any] | None] | None ) = None, status_reloader: Callable[[], dict[str, Any]] | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: safe_goal_id = _validate_goal_id_path_segment(str(goal_id or "")) normalized_requested_todo_id = normalize_todo_id(todo_id) if todo_id else None @@ -1161,6 +1187,7 @@ def should_run(current_status: dict[str, Any]) -> dict[str, Any]: scheduler_execution_context=scheduler_execution_context, operator_inbox_urgency_projector=operator_inbox_urgency_projector, receipt_bound_todo_id=normalized_receipt_todo_id, + goal_ref=goal_ref, ) before = should_run(status_payload) @@ -1221,6 +1248,7 @@ def should_run(current_status: dict[str, Any]) -> dict[str, Any]: use_current_task_lease=use_current_task_lease, turn_instance_id=turn_instance_id, status_reloader=status_reloader, + goal_ref=goal_ref, ) continuation = result.get("turn_continuation") or {} if ( @@ -1244,6 +1272,8 @@ def should_run(current_status: dict[str, Any]) -> dict[str, Any]: agent_id=agent_id, todo_id=normalized_receipt_todo_id, turn_instance_id=turn_instance_id, + registry_path=registry_path, + goal_ref=goal_ref, ) if readback is None or readback.identity.value is None: raise RuntimeError( @@ -1251,6 +1281,7 @@ def should_run(current_status: dict[str, Any]) -> dict[str, Any]: ) attach_settlement_progress( result, readback, registry_path=registry_path, runtime_root=runtime_root, + goal_ref=goal_ref, ) identity = readback.identity.value prefix = "loopx" @@ -1273,6 +1304,7 @@ def should_run(current_status: dict[str, Any]) -> dict[str, Any]: scoped_cli_args=scoped_args, lifecycle_actor_args="", quota_spend_source=readback.progress["quota_spend_source"], + goal_ref=goal_ref, ) result["settlement_resume"] = { "schema_version": "auxiliary_monitor_settlement_resume_v0", @@ -1337,6 +1369,8 @@ def void_quota_slot( reason_summary: str | None = None, agent_id: str | None = None, operator_inbox_urgency_projector: Callable[..., dict[str, Any]] | None = None, + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: safe_goal_id = _normalize_quota_void_goal_id(goal_id) before = build_quota_should_run( @@ -1353,6 +1387,8 @@ def void_quota_slot( execute=execute, source=source, reason_summary=reason_summary, + registry_path=registry_path, + goal_ref=goal_ref, ) @@ -1374,6 +1410,8 @@ def spend_quota_slot( replan_obligation_id: str | None = None, turn_instance_id: str | None = None, effect_ref: str | None = None, + registry_path: Path | None = None, + goal_ref: Mapping[str, Any] | None = None, ) -> dict[str, Any]: safe_goal_id = _validate_goal_id_path_segment(str(goal_id or "")) normalized_effect_ref = str(effect_ref or "").strip() @@ -1404,6 +1442,8 @@ def spend_quota_slot( effect_ref=normalized_effect_ref, agent_id=agent_id, read_only=not execute, + registry_path=registry_path, + goal_ref=goal_ref, ) if replay.get("replay_found"): if not replay.get("ok"): @@ -1428,6 +1468,11 @@ def spend_quota_slot( "agent_id": replay.get("agent_id"), "effect_ref": normalized_effect_ref, "reason": "quota spend replayed for the same provider effect", + **( + {"goal_ref": dict(goal_ref)} + if goal_ref is not None + else {} + ), } if turn_instance_id and source not in TURN_SCOPED_SLOT_SPEND_SOURCES: return { @@ -1463,6 +1508,8 @@ def spend_quota_slot( turn_instance_id=None, infer_turn_instance_id=True, allow_unbound_binding=recover_unbound_visible_goal, + registry_path=registry_path, + goal_ref=goal_ref, ) inferred_result = ( settlement_readback.identity @@ -1499,6 +1546,8 @@ def spend_quota_slot( todo_id=todo_id, turn_instance_id=turn_instance_id, replan_obligation_id=replan_obligation_id, + registry_path=registry_path, + goal_ref=goal_ref, ) if settlement_readback is None: raise RuntimeError("exact settlement readback unexpectedly returned not-found") @@ -1534,6 +1583,11 @@ def spend_quota_slot( "settlement_identity": identity.as_dict(), "settlement_result": settlement_result_payload(spent_result), "reason": "quota spend receipt replayed for the same settlement identity", + **( + {"goal_ref": dict(goal_ref)} + if goal_ref is not None + else {} + ), } prior_spend_run = settlement_readback.spend_run if prior_spend_run is not None: @@ -1551,6 +1605,11 @@ def spend_quota_slot( "settlement_identity": identity.as_dict(), "settlement_result": settlement_result_payload(spent_result), "reason": "quota spend run exists; repair its missing settlement receipt", + **( + {"goal_ref": dict(goal_ref)} + if goal_ref is not None + else {} + ), } preview = build_quota_slot_preview( status_payload, @@ -1566,6 +1625,8 @@ def spend_quota_slot( turn_instance_id=turn_instance_id, effect_ref=normalized_effect_ref or None, source=source, + registry_path=registry_path, + goal_ref=goal_ref, ) if not preview.get("ok"): return preview @@ -1576,4 +1637,6 @@ def spend_quota_slot( goal_id=safe_goal_id, execute=execute, source=source, + registry_path=registry_path, + goal_ref=goal_ref, ) diff --git a/loopx/rollout_event_log.py b/loopx/rollout_event_log.py index a0e4a6c33c..de2f9be78d 100644 --- a/loopx/rollout_event_log.py +++ b/loopx/rollout_event_log.py @@ -173,6 +173,24 @@ def _normalized_event_kind(event_kind: str) -> str: return text +def _safe_goal_ref( + value: Mapping[str, Any] | None, + *, + goal_id: str, +) -> dict[str, str] | None: + if value is None: + return None + from .control_plane.goals.source_session_registry_state import exact_goal_ref + + goal_ref = exact_goal_ref( + str(value.get("goal_id") or ""), + str(value.get("goal_instance_id") or ""), + ) + if goal_ref["goal_id"] != goal_id: + raise ValueError("rollout event goal_ref does not match goal_id") + return goal_ref + + def _event_id(payload: Mapping[str, Any]) -> str: """Identify one event occurrence, including its observation timestamp.""" @@ -249,6 +267,7 @@ def build_rollout_event( *, goal_id: str, event_kind: str, + goal_ref: Mapping[str, Any] | None = None, agent_id: str | None = None, todo_id: str | None = None, case_id: str | None = None, @@ -323,6 +342,9 @@ def build_rollout_event( "absolute_paths_recorded": False, }, } + safe_goal_ref = _safe_goal_ref(goal_ref, goal_id=safe_goal_id) + if safe_goal_ref is not None: + payload["goal_ref"] = safe_goal_ref optional_scalars = { "agent_id": agent_id, "todo_id": todo_id, @@ -454,7 +476,11 @@ def append_rollout_event_once( fields = tuple(str(field).strip() for field in identity_fields if str(field).strip()) if not fields: raise ValueError("rollout idempotency identity_fields are required") - missing = [field for field in fields if payload.get(field) in {None, ""}] + missing = [ + field + for field in fields + if payload.get(field) is None or payload.get(field) == "" + ] if missing: raise ValueError( "rollout idempotency fields must be populated: " + ", ".join(missing) diff --git a/loopx/semantics/goal_instance_binding_inventory_v1.json b/loopx/semantics/goal_instance_binding_inventory_v1.json index 9a3a42ad93..878bd8d4e7 100644 --- a/loopx/semantics/goal_instance_binding_inventory_v1.json +++ b/loopx/semantics/goal_instance_binding_inventory_v1.json @@ -180,19 +180,22 @@ "locator": "/goals//runs/index.jsonl#quota_event", "revision_signal": "run_generated_at and effect_id", "content_digest_signal": "quota accounting artifact digest", - "observed_reference": "goal_id", + "observed_reference": "goal_id + goal_instance_id for source_session_v1; goal_id otherwise", "producer_sites": [ - "loopx/control_plane/quota/spend_commit.py::record_quota_slot_spend_from_preview" + "loopx/control_plane/quota/accounting_admission.py::quota_accounting_admission", + "loopx/control_plane/quota/spend_commit.py::record_quota_slot_spend_from_preview", + "loopx/control_plane/quota/void_commit.py::commit_quota_slot_void" ], "consumer_sites": [ - "loopx/control_plane/quota/settlement_readback.ts::readQuotaSettlementFromSnapshot", + "loopx/control_plane/quota/settlement_readback.ts::readQuotaSettlement", + "loopx/control_plane/quota/slot_accounting.py::_latest_unspent_turn_settlement_run", "loopx/control_plane/quota/slot_accounting.py::net_quota_slot_spend" ], "effect_boundary": "loopx/control_plane/quota/accounting_artifact_transaction.ts::commitQuotaAccountingArtifactTransaction", "authority_role": "quota_accounting_ledger", - "current_identity_strength": "goal_alias_only", + "current_identity_strength": "exact_goal_ref_enforced", "cleanup_support": "typed_quota_void", - "m1_disposition": "alias_only_inventory", + "m1_disposition": "m3_qualified", "target_milestone": "M3" }, { diff --git a/loopx/semantics/project_registry_io_manifest_v1.json b/loopx/semantics/project_registry_io_manifest_v1.json index 81b7d018fb..5c1da1b7ae 100644 --- a/loopx/semantics/project_registry_io_manifest_v1.json +++ b/loopx/semantics/project_registry_io_manifest_v1.json @@ -751,7 +751,7 @@ }, { "site": "loopx/cli_commands/project_lifecycle_refresh_state.py::.handle_refresh_state_command::codec_read:load_registry#1", - "line": 562, + "line": 596, "column": 17, "kind": "codec_read", "api": "load_registry", @@ -759,7 +759,7 @@ }, { "site": "loopx/cli_commands/project_lifecycle_refresh_state.py::.handle_refresh_state_command::codec_read:load_registry#2", - "line": 641, + "line": 677, "column": 17, "kind": "codec_read", "api": "load_registry", @@ -775,7 +775,7 @@ }, { "site": "loopx/cli_commands/quota.py::._dispatch_quota_turn_start_hooks::codec_read:load_registry#1", - "line": 272, + "line": 309, "column": 37, "kind": "codec_read", "api": "load_registry", @@ -879,7 +879,7 @@ }, { "site": "loopx/cli_commands/todo.py::._validated_replan_successor_obligation::codec_read:load_registry#1", - "line": 178, + "line": 187, "column": 16, "kind": "codec_read", "api": "load_registry", @@ -887,7 +887,7 @@ }, { "site": "loopx/cli_commands/todo.py::.handle_todo_command::codec_read:load_registry#1", - "line": 309, + "line": 331, "column": 49, "kind": "codec_read", "api": "load_registry", @@ -895,7 +895,7 @@ }, { "site": "loopx/cli_commands/todo.py::.handle_todo_command::codec_read:load_registry#2", - "line": 325, + "line": 347, "column": 24, "kind": "codec_read", "api": "load_registry", @@ -903,7 +903,7 @@ }, { "site": "loopx/cli_commands/todo.py::.handle_todo_command::codec_read:load_registry#3", - "line": 333, + "line": 355, "column": 24, "kind": "codec_read", "api": "load_registry", @@ -911,7 +911,7 @@ }, { "site": "loopx/cli_commands/todo.py::.handle_todo_command::codec_read:load_registry#4", - "line": 513, + "line": 535, "column": 53, "kind": "codec_read", "api": "load_registry", @@ -919,15 +919,15 @@ }, { "site": "loopx/cli_commands/todo.py::.handle_todo_command::codec_read:load_registry#5", - "line": 634, - "column": 61, + "line": 666, + "column": 25, "kind": "codec_read", "api": "load_registry", "classification": "codec_api" }, { "site": "loopx/cli_commands/todo.py::.handle_todo_command::codec_read:load_registry#6", - "line": 698, + "line": 734, "column": 13, "kind": "codec_read", "api": "load_registry", @@ -935,7 +935,7 @@ }, { "site": "loopx/cli_commands/todo.py::.handle_todo_command::codec_read:load_registry#7", - "line": 740, + "line": 778, "column": 38, "kind": "codec_read", "api": "load_registry", @@ -951,7 +951,7 @@ }, { "site": "loopx/cli_rollout.py::.append_cli_rollout_event::codec_read:load_registry#1", - "line": 50, + "line": 52, "column": 24, "kind": "codec_read", "api": "load_registry", @@ -1135,7 +1135,7 @@ }, { "site": "loopx/control_plane/goals/checkpoint_context_io.py::.read_checkpoint_context::codec_read:load_registry#1", - "line": 131, + "line": 156, "column": 16, "kind": "codec_read", "api": "load_registry", @@ -1318,17 +1318,17 @@ "classification": "codec_api" }, { - "site": "loopx/control_plane/goals/first_party_host_admission.py::._source_authority::codec_read:load_project_registry#1", - "line": 40, - "column": 20, + "site": "loopx/control_plane/goals/first_party_host_admission.py::.capture_first_party_host_goal_ref::codec_read:load_project_registry#1", + "line": 75, + "column": 16, "kind": "codec_read", "api": "load_project_registry", "classification": "codec_api" }, { - "site": "loopx/control_plane/goals/first_party_host_admission.py::.capture_first_party_host_goal_ref::codec_read:load_project_registry#1", - "line": 75, - "column": 16, + "site": "loopx/control_plane/goals/first_party_host_admission.py::.source_goal_authority::codec_read:load_project_registry#1", + "line": 40, + "column": 20, "kind": "codec_read", "api": "load_project_registry", "classification": "codec_api" @@ -2071,7 +2071,7 @@ }, { "site": "loopx/state_refresh.py::.refresh_state_run::codec_read:load_registry#1", - "line": 900, + "line": 901, "column": 16, "kind": "codec_read", "api": "load_registry", diff --git a/loopx/state_refresh.py b/loopx/state_refresh.py index 02d7f10deb..924e4cba3f 100644 --- a/loopx/state_refresh.py +++ b/loopx/state_refresh.py @@ -3,7 +3,7 @@ import hashlib import json import re -from contextlib import ExitStack, nullcontext +from contextlib import ExitStack from pathlib import Path from typing import Any @@ -30,6 +30,7 @@ finish_external_delivery_refresh, refresh_recovery_payload, ) from .control_plane.quota.blocked_retry import require_blocked_retry_wait +from .control_plane.quota.accounting_admission import quota_accounting_admission from .control_plane.coordination.local_authority import local_authority_is_promoted from .control_plane.todos.active_state_todo_parser import parse_active_state_todos from .control_plane.quota.settlement import ( @@ -100,7 +101,6 @@ from .control_plane.goals.checkpoint_context_io import ( checkpoint_commit_guard, commit_checkpoint_run, require_complete_checkpoint_index, inspect_checkpoint_replay, ) -from .file_lock import exclusive_run_index_lock from .registry import registry_goals, resolve_state_file from .runtime import validate_goal_id_path_segment from .state_projection import ( @@ -825,6 +825,7 @@ def refresh_state_run( dry_run: bool, sync_global: bool = True, external_delivery: dict[str, Any] | None = None, + goal_ref: dict[str, str] | None = None, ) -> dict[str, Any]: from .control_plane.todos.provider_projection import recover_refresh_todo_projection @@ -901,9 +902,15 @@ def refresh_state_run( runtime_root = resolve_runtime_root(registry, runtime_root_override, registry_path=registry_path) # State-dependent admission through the final append remains serialized. # Only pure input validation runs before this transitional persistence lock. - with (nullcontext() if dry_run else exclusive_run_index_lock( - runtime_root / "goals" / safe_goal_id / "runs" / "index.jsonl", operation="refresh-state" - )): + with quota_accounting_admission( + runtime_root=runtime_root, + registry_path=registry_path, + goal_id=safe_goal_id, + goal_ref=goal_ref, + operation="refresh-state", + lock_legacy_index=not dry_run, + handoff_legacy_index=not dry_run, + ) as source_admission: settlement_identity = None settlement_result = None delivery_workspace_causality = None @@ -926,6 +933,10 @@ def refresh_state_run( todo_id=todo_id, turn_instance_id=turn_instance_id, replan_obligation_id=normalized_replan_obligation_id, + registry_path=registry_path, + goal_ref=goal_ref, + source_admission=source_admission, + borrow_source_admission=source_admission is not None, refresh_retry=(refresh_retry_request := { "checkpoint_read_context_id": checkpoint_read_context_id, "external_delivery": external_delivery, @@ -968,7 +979,7 @@ def refresh_state_run( inspect_checkpoint_replay(runtime_root, safe_goal_id, prior_writeback_run) recovery_payload = refresh_recovery_payload( settlement_readback, registry_path=registry_path, runtime_root=runtime_root, - goal_id=safe_goal_id, dry_run=dry_run, + goal_id=safe_goal_id, dry_run=dry_run, goal_ref=goal_ref, ) if recovery_payload is not None: return recover_refresh_todo_projection( @@ -1406,6 +1417,9 @@ def refresh_state_run( dry_run=dry_run, autonomous_replan_recorded_requested=bool(autonomous_replan_recorded), ) + if goal_ref is not None: + for projection in (record, index_record, payload): + projection["goal_ref"] = dict(goal_ref) # GH-C95 producer boundary: attach the typed run_usage_v0 row before the # durable record and index rows are written, so malformed or negative usage # fails the whole refresh instead of entering run history. The booking lock @@ -1418,6 +1432,7 @@ def refresh_state_run( runtime_root=runtime_root, registry_path=registry_path, state_file=resolved_state_file, identity=settlement_identity, read_context_id=checkpoint_read_context_id, + goal_ref=goal_ref, )) for projection in (record, index_record, payload): projection["vision_checkpoint"] = { @@ -1496,7 +1511,8 @@ def refresh_state_run( saved = commit_checkpoint_run(runtime_root=runtime_root, registry_path=registry_path, state_file=resolved_state_file, identity=settlement_identity, refresh_retry=refresh_retry_request, record=record, index_record=index_record, - markdown=render_state_refresh_markdown(payload) + "\n") + markdown=render_state_refresh_markdown(payload) + "\n", + goal_ref=goal_ref, source_admission=source_admission) for projection in (record, index_record, payload): projection["vision_checkpoint"]["read_context"] = saved["context"] for projection in (index_record, payload): @@ -1598,13 +1614,23 @@ def refresh_state_run( runtime_root, goal_id=safe_goal_id, agent_id=settlement_identity.agent_id, todo_id=settlement_identity.todo_id, turn_instance_id=settlement_identity.turn_instance_id, replan_obligation_id=settlement_identity.replan_obligation_id, + registry_path=registry_path, goal_ref=goal_ref, + source_admission=source_admission, + borrow_source_admission=source_admission is not None, ) if committed_readback is None: raise RuntimeError("committed refresh settlement readback missing") - attach_settlement_progress(payload, committed_readback, registry_path=registry_path, runtime_root=runtime_root) + attach_settlement_progress( + payload, committed_readback, registry_path=registry_path, + runtime_root=runtime_root, goal_ref=goal_ref, + ) return recover_refresh_todo_projection( finish_external_delivery_refresh( - payload, settlement_readback, runtime_root, dry_run=dry_run, + payload, + settlement_readback, + runtime_root, + dry_run=dry_run, + goal_ref=goal_ref, ), registry_path=registry_path, runtime_root=runtime_root, goal_id=safe_goal_id, project=resolved_project, state_file=resolved_state_file, diff --git a/loopx/status.py b/loopx/status.py index 9e293a2afe..ea6912b0d4 100644 --- a/loopx/status.py +++ b/loopx/status.py @@ -1090,6 +1090,8 @@ def build_attention_queue( include_stopped_goal_context: bool = False, events_for_goal: EventsForGoal | None = None, todo_snapshot: _CanonicalTodoSnapshot | None = None, + current_registry: dict[str, Any] | None = None, + registry_path: Path | None = None, ) -> dict[str, Any]: def request_active_state_todo_fields( goal: dict[str, Any], @@ -1189,9 +1191,25 @@ def request_active_state_todo_fields( and int(orchestration.get("max_children") or 0) > 0 ): continue + goal_ref = None + if (current_registry or {}).get("profile_id") == "source_session_v1": + matches = [ + goal + for goal in (current_registry or {}).get("goals") or [] + if isinstance(goal, dict) + and goal.get("id") == goal_id + and goal.get("status") == "active" + and isinstance(goal.get("goal_instance_id"), str) + ] + if len(matches) == 1: + goal_ref = { + "goal_id": goal_id, + "goal_instance_id": matches[0]["goal_instance_id"], + } native_activity = latest_native_child_activity( events, goal_id=goal_id, configured_limit=int(orchestration["max_children"]), + goal_ref=goal_ref, ) if native_activity: # The shared status snapshot is bounded for Todo work. Once it @@ -1203,6 +1221,8 @@ def request_active_state_todo_fields( agent_id=native_activity["agent_id"], turn_instance_id=native_activity["turn_instance_id"], configured_limit=int(orchestration["max_children"]), + goal_ref=goal_ref, + registry_path=registry_path, ) return queue diff --git a/tests/architecture/test_goal_instance_binding_inventory.py b/tests/architecture/test_goal_instance_binding_inventory.py index 466461766d..6fbbe284ee 100644 --- a/tests/architecture/test_goal_instance_binding_inventory.py +++ b/tests/architecture/test_goal_instance_binding_inventory.py @@ -56,6 +56,7 @@ QUALIFIED_OWNER_IDS = { "attached_host_chat_session", "handoff_inbox_outbox", + "quota_settlement", "turn_journal", } TYPESCRIPT_DECLARATION = re.compile( diff --git a/tests/capabilities/test_native_child_receipts.py b/tests/capabilities/test_native_child_receipts.py index f5131f005b..5476fbdc1b 100644 --- a/tests/capabilities/test_native_child_receipts.py +++ b/tests/capabilities/test_native_child_receipts.py @@ -13,6 +13,9 @@ native_child_activity, record_native_child, ) +from loopx.control_plane.projects.registry_codec import ( + source_session_registry_transaction, +) from loopx.rollout_event_log import ( append_rollout_event, build_rollout_event, @@ -24,9 +27,15 @@ GOAL = "native-child-fixture" AGENT = "generic-coordinator" TURN = "turn-native-1" +INSTANCE_A = "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" +INSTANCE_B = "ginst_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" -def _admit(runtime_root: Path) -> None: +def _admit( + runtime_root: Path, + *, + goal_ref: dict[str, str] | None = None, +) -> None: append_rollout_event( rollout_event_log_path(runtime_root, GOAL), build_rollout_event( @@ -34,6 +43,7 @@ def _admit(runtime_root: Path) -> None: run_id=TURN, todo_id="todo_native_1", status="normal_run", details={"todo_id": "todo_native_1", "settlement_effect_id": f"{GOAL}:{AGENT}:todo_native_1:{TURN}"}, + goal_ref=goal_ref, ), ) @@ -47,6 +57,84 @@ def _record(runtime_root: Path, operation_id: str, *, stage: str, outcome: str, ) +def _write_source_registry( + registry_path: Path, + runtime_root: Path, + instance_id: str, +) -> None: + payload = { + "schema_version": "0.2", + "registry_role": "project-local", + "profile_id": "source_session_v1", + "common_runtime_root": str(runtime_root), + "projects": [], + "goals": [ + { + "id": GOAL, + "goal_instance_id": instance_id, + "status": "active", + "execution_authority": False, + } + ], + "session_bindings": [], + "session_receipts": [], + "lifetime_receipts": [], + "retired_goal_instances": [], + } + with source_session_registry_transaction( + registry_path, + operation="native_child_goal_instance_test", + create=lambda: payload, + ) as transaction: + transaction.commit(payload) + + +def test_native_child_receipts_are_partitioned_by_exact_goal_owner( + tmp_path: Path, +) -> None: + runtime = tmp_path / "runtime" + registry = tmp_path / "project" / ".loopx" / "registry.json" + goal_ref_a = {"goal_id": GOAL, "goal_instance_id": INSTANCE_A} + goal_ref_b = {"goal_id": GOAL, "goal_instance_id": INSTANCE_B} + _admit(runtime, goal_ref=goal_ref_a) + _admit(runtime, goal_ref=goal_ref_b) + _write_source_registry(registry, runtime, INSTANCE_B) + + result = record_native_child( + runtime_root=runtime, + registry_path=registry, + goal_ref=goal_ref_b, + goal_id=GOAL, + agent_id=AGENT, + turn_instance_id=TURN, + operation_id="op-b", + configured_limit=6, + stage="decision", + operation="spawn", + outcome="started", + entrypoint_id="generic_host", + execute=True, + ) + assert result["native_child_activity"]["operation_count"] == 1 + events = load_rollout_events(rollout_event_log_path(runtime, GOAL)) + assert events[-1]["goal_ref"] == goal_ref_b + assert native_child_activity( + events, + goal_id=GOAL, + agent_id=AGENT, + turn_instance_id=TURN, + configured_limit=6, + goal_ref=goal_ref_a, + )["operation_count"] == 0 + assert native_child_activity( + events, + goal_id=GOAL, + agent_id=AGENT, + turn_instance_id=TURN, + configured_limit=6, + )["operation_count"] == 0 + + def test_generic_report_adoption_is_idempotent_and_survives_restart(tmp_path: Path): _admit(tmp_path) unknown = load_native_child_activity( diff --git a/tests/control_plane/checkpoint_process.py b/tests/control_plane/checkpoint_process.py index f1001c8dcd..16ef630268 100644 --- a/tests/control_plane/checkpoint_process.py +++ b/tests/control_plane/checkpoint_process.py @@ -32,7 +32,7 @@ def wait_for(path: Path, child=None, timeout=20): time.sleep(0.01) -def refresh(registry, runtime, token): +def refresh(registry, runtime, token, *, goal_ref=None): from loopx.state_refresh import refresh_state_run from tests.control_plane.test_quota_settlement_cli import GOAL_ID, AGENT_ID, TODO_ID, TURN_ID return refresh_state_run(registry_path=Path(registry), runtime_root_override=str(runtime), @@ -41,7 +41,8 @@ def refresh(registry, runtime, token): delivery_batch_scale="implementation", delivery_outcome="outcome_progress", vision_unchanged_reason="The current basis remains applicable.", checkpoint_read_context_id=token, dry_run=False, sync_global=False, - external_delivery={"suppress": True, "resume_key": None}) + external_delivery={"suppress": True, "resume_key": None}, + goal_ref=goal_ref) def main(request): diff --git a/tests/control_plane/test_checkpoint_provider_fence.py b/tests/control_plane/test_checkpoint_provider_fence.py index b82efa20b2..3c94c84e18 100644 --- a/tests/control_plane/test_checkpoint_provider_fence.py +++ b/tests/control_plane/test_checkpoint_provider_fence.py @@ -12,12 +12,84 @@ from canonical_authority_fixture import initialize_canonical_authority, isolate_sqlite_runtime from loopx.control_plane.coordination.local_authority import read_canonical_todos_if_promoted from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection +from loopx.control_plane.effect_runtime import EffectRuntimeConflict from loopx.control_plane.goals import checkpoint_context_io as context_io +from loopx.control_plane.projects.registry_codec import ( + load_project_registry, + source_session_registry_transaction, +) from tests.control_plane.test_checkpoint_read_context import _missing from tests.control_plane.test_quota_settlement_cli import GOAL_ID, AGENT_ID, TODO_ID, TURN_ID, _run_cli, _spend_run_count from tests.control_plane.checkpoint_process import REPO, start_probe, wait_for, refresh +INSTANCE_A = "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" +INSTANCE_B = "ginst_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + + +def _stamp_runtime_owner(runtime: Path, goal_ref: dict[str, str]) -> None: + index = runtime / "goals" / GOAL_ID / "runs" / "index.jsonl" + rows = [json.loads(line) for line in index.read_text().splitlines()] + for row in rows: + row["goal_ref"] = dict(goal_ref) + json_path = row.get("json_path") + if isinstance(json_path, str) and Path(json_path).is_file(): + record = json.loads(Path(json_path).read_text()) + record["goal_ref"] = dict(goal_ref) + Path(json_path).write_text(json.dumps(record) + "\n") + index.write_text("".join(json.dumps(row) + "\n" for row in rows)) + + event_log = runtime / "goals" / GOAL_ID / "rollout-event-log.jsonl" + events = [json.loads(line) for line in event_log.read_text().splitlines()] + for event in events: + event["goal_ref"] = dict(goal_ref) + event_log.write_text("".join(json.dumps(event) + "\n" for event in events)) + + +def _replace_with_source_registry( + registry: Path, + runtime: Path, + instance_id: str, +) -> None: + legacy = json.loads(registry.read_text()) + goal = dict(legacy["goals"][0]) + goal.update( + status="active", + goal_instance_id=instance_id, + execution_authority=False, + ) + payload = { + **legacy, + "schema_version": "0.2", + "registry_role": "project-local", + "profile_id": "source_session_v1", + "common_runtime_root": str(runtime), + "projects": [], + "goals": [goal], + "session_bindings": [], + "session_receipts": [], + "lifetime_receipts": [], + "retired_goal_instances": [], + } + registry.unlink() + with source_session_registry_transaction( + registry, + operation="checkpoint_goal_instance_test", + create=lambda: payload, + ) as transaction: + transaction.commit(payload) + + +def _replace_source_goal(registry: Path, instance_id: str) -> None: + with source_session_registry_transaction( + registry, + operation="checkpoint_goal_instance_recreate", + ) as transaction: + payload = transaction.payload_copy() + payload["goals"][0]["goal_instance_id"] = instance_id + transaction.commit(payload) + + def fixture(tmp_path, monkeypatch, provider): isolate_sqlite_runtime(tmp_path, monkeypatch) project, runtime, registry, binding, delivery, original = _missing(tmp_path) @@ -86,6 +158,75 @@ def test_public_update_before_final_read_rejects_then_reread_succeeds(tmp_path, assert _spend_run_count(runtime) == 0 +def test_exact_source_checkpoint_commits_only_for_the_current_goal_instance( + tmp_path, + monkeypatch, +): + from loopx import state_refresh + + current = _missing(tmp_path / "current") + stale = _missing(tmp_path / "stale") + goal_ref = {"goal_id": GOAL_ID, "goal_instance_id": INSTANCE_A} + for _, runtime, registry, _, _, _ in (current, stale): + _stamp_runtime_owner(runtime, goal_ref) + _replace_with_source_registry(registry, runtime, INSTANCE_A) + + monkeypatch.setattr(context_io, "load_registry", load_project_registry) + monkeypatch.setattr(state_refresh, "load_registry", load_project_registry) + + _, current_runtime, current_registry, _, _, current_original = current + current_context = context_io.read_checkpoint_context( + registry_path=current_registry, + runtime_root_override=str(current_runtime), + goal_id=GOAL_ID, + agent_id=AGENT_ID, + todo_id=TODO_ID, + turn_instance_id=TURN_ID, + goal_ref=goal_ref, + ) + committed = refresh( + current_registry, + current_runtime, + current_context["read_context_id"], + goal_ref=goal_ref, + ) + assert committed["appended"] is True + assert committed["goal_ref"] == goal_ref + assert committed["settlement_identity"] == current_original["settlement_identity"] + committed_row = json.loads( + (current_runtime / "goals" / GOAL_ID / "runs" / "index.jsonl") + .read_text() + .splitlines()[-1] + ) + assert committed_row["goal_ref"] == goal_ref + + _, stale_runtime, stale_registry, _, _, _ = stale + stale_context = context_io.read_checkpoint_context( + registry_path=stale_registry, + runtime_root_override=str(stale_runtime), + goal_id=GOAL_ID, + agent_id=AGENT_ID, + todo_id=TODO_ID, + turn_instance_id=TURN_ID, + goal_ref=goal_ref, + ) + stale_index = stale_runtime / "goals" / GOAL_ID / "runs" / "index.jsonl" + before = stale_index.read_bytes() + _replace_source_goal(stale_registry, INSTANCE_B) + with pytest.raises( + EffectRuntimeConflict, + match="stale_goal_instance", + ) as rejected: + refresh( + stale_registry, + stale_runtime, + stale_context["read_context_id"], + goal_ref=goal_ref, + ) + assert rejected.value.diagnostic_code == "stale_goal_instance" + assert stale_index.read_bytes() == before + + @pytest.mark.parametrize("provider", ["file", "sqlite"]) def test_existing_cli_maintenance_guard_precedes_provider_commit(tmp_path, monkeypatch, provider): """Characterize the reviewed head accurately: normal CLI already holds M.""" diff --git a/tests/control_plane/test_quota_monitor_poll_runtime.py b/tests/control_plane/test_quota_monitor_poll_runtime.py index be3da145ac..19798b03f0 100644 --- a/tests/control_plane/test_quota_monitor_poll_runtime.py +++ b/tests/control_plane/test_quota_monitor_poll_runtime.py @@ -135,6 +135,7 @@ def test_turn_monitor_effect_identity_replays_legacy_receipt_then_scopes_new_tod turn_instance_id=turn_id, todo_id="todo_monitor_legacy", target_key="legacy-target", + goal_ref=None, ) == legacy_effect_id assert monitor_poll._monitor_poll_effect_id( runtime_root=tmp_path, @@ -143,6 +144,7 @@ def test_turn_monitor_effect_identity_replays_legacy_receipt_then_scopes_new_tod turn_instance_id=turn_id, todo_id="todo_monitor_new", target_key="new-target", + goal_ref=None, ) == ( f"quota-monitor-poll:{goal_id}:{agent_id}:{turn_id}:" "todo:todo_monitor_new" diff --git a/tests/control_plane/test_quota_rolling_window_projection.py b/tests/control_plane/test_quota_rolling_window_projection.py index 98d3ca6752..477dce99fb 100644 --- a/tests/control_plane/test_quota_rolling_window_projection.py +++ b/tests/control_plane/test_quota_rolling_window_projection.py @@ -87,3 +87,39 @@ def test_rolling_counter_can_decrease_without_replaying_or_voiding_a_spend() -> assert after["spend_event_count"] == 1 assert append_only_runs[0]["classification"] == "quota_slot_spent" assert all(run["classification"] != "quota_slot_voided" for run in append_only_runs) + + +def test_rolling_counter_isolated_by_exact_goal_instance() -> None: + goal_ref_a = { + "goal_id": GOAL_ID, + "goal_instance_id": "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + } + goal_ref_b = { + "goal_id": GOAL_ID, + "goal_instance_id": "ginst_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + } + legacy = _spend("2026-01-01T00:10:00+00:00", "turn-legacy") + spend_a = { + **_spend("2026-01-01T00:20:00+00:00", "turn-a"), + "goal_ref": goal_ref_a, + } + spend_b = { + **_spend("2026-01-01T00:30:00+00:00", "turn-b"), + "goal_ref": goal_ref_b, + } + + current = goal_quota_with_spend_ledger( + {**_goal(), "goal_instance_id": goal_ref_b["goal_instance_id"]}, + [legacy, spend_a, spend_b], + now=datetime(2026, 1, 1, 0, 59, tzinfo=timezone.utc), + ) + legacy_owner = goal_quota_with_spend_ledger( + _goal(), + [legacy, spend_a, spend_b], + now=datetime(2026, 1, 1, 0, 59, tzinfo=timezone.utc), + ) + + assert current["spent_slots"] == 1 + assert current["spend_event_count"] == 1 + assert legacy_owner["spent_slots"] == 1 + assert legacy_owner["spend_event_count"] == 1 diff --git a/tests/control_plane/test_quota_slot_accounting.py b/tests/control_plane/test_quota_slot_accounting.py index 0772c95665..101cc0497a 100644 --- a/tests/control_plane/test_quota_slot_accounting.py +++ b/tests/control_plane/test_quota_slot_accounting.py @@ -1,5 +1,6 @@ from __future__ import annotations +import hashlib import json from pathlib import Path from typing import Any @@ -9,6 +10,7 @@ from loopx.control_plane.quota.slot_accounting import ( build_quota_slot_preview_for_decision, build_quota_slot_spend_event, + record_quota_slot_spend_from_preview, ) from loopx.quota import spend_quota_slot from loopx.rollout_event_log import rollout_event_log_path @@ -273,6 +275,77 @@ def test_safe_bypass_consumes_accountable_writeback_once_across_neutral_refresh( assert _preview(runtime, before_overrides=before_overrides)["ok"] is False +def test_alias_safe_bypass_ignores_exact_delivery(tmp_path: Path) -> None: + runtime = tmp_path / "runtime" + exact_delivery = { + **_run( + "2026-01-01T00:01:00+00:00", + classification="validated_fallback", + delivery_outcome="outcome_progress", + ), + "goal_ref": { + "goal_id": GOAL_ID, + "goal_instance_id": "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + }, + } + _write_run_index(runtime, [exact_delivery]) + + preview = _preview( + runtime, + before_overrides={"state": "operator_gate", "safe_bypass_allowed": True}, + ) + + assert preview["ok"] is False + assert "requires a latest unspent Turn settlement writeback" in preview["reason"] + + +def test_alias_safe_bypass_ignores_exact_spend_and_commits_once( + tmp_path: Path, +) -> None: + runtime = tmp_path / "runtime" + legacy_delivery = _run( + "2026-01-01T00:01:00+00:00", + classification="validated_fallback", + delivery_outcome="outcome_progress", + ) + exact_spend = { + **_spent("2026-01-01T00:02:00+00:00"), + "goal_ref": { + "goal_id": GOAL_ID, + "goal_instance_id": "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + }, + } + _write_run_index(runtime, [legacy_delivery, exact_spend]) + before_overrides = { + "state": "operator_gate", + "safe_bypass_allowed": True, + } + + preview = _preview(runtime, before_overrides=before_overrides) + + assert preview["ok"] is True + assert preview["delivery_run_classification"] == "validated_fallback" + index_path = runtime / "goals" / GOAL_ID / "runs" / "index.jsonl" + preview["expected_index_digest"] = ( + f"sha256:{hashlib.sha256(index_path.read_bytes()).hexdigest()}" + ) + committed = record_quota_slot_spend_from_preview( + preview, + {"runtime_root": str(runtime)}, + goal_id=GOAL_ID, + execute=True, + ) + assert committed["ok"] is True + assert committed["appended"] is True + rows = [ + json.loads(line) + for line in index_path.read_text(encoding="utf-8").splitlines() + ] + assert rows[-1]["classification"] == "quota_slot_spent" + assert "goal_ref" not in rows[-1] + assert _preview(runtime, before_overrides=before_overrides)["ok"] is False + + def test_material_monitor_poll_builds_attributed_spend_event(tmp_path: Path) -> None: runtime = tmp_path / "runtime" material_poll = _poll( diff --git a/tests/control_plane/test_quota_spend_commit_runtime.py b/tests/control_plane/test_quota_spend_commit_runtime.py index 1f7df5be1f..af9726c61d 100644 --- a/tests/control_plane/test_quota_spend_commit_runtime.py +++ b/tests/control_plane/test_quota_spend_commit_runtime.py @@ -10,6 +10,9 @@ build_quota_slot_spend_event, record_quota_slot_spend_from_preview, ) +from loopx.control_plane.projects.registry_codec import ( + source_session_registry_transaction, +) from loopx.control_plane.testing.quota_fixtures import ( quota_status_payload, quota_todo_item, @@ -22,6 +25,8 @@ GOAL_ID = "quota-spend-commit-runtime" AGENT_ID = "codex-main-control" +INSTANCE_A = "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" +INSTANCE_B = "ginst_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" def _decision(spent_slots: int) -> dict[str, object]: @@ -67,16 +72,59 @@ def _preview(**updates: object) -> dict[str, object]: } -def _commit(runtime_root: Path, preview: dict[str, object]) -> dict[str, object]: +def _commit( + runtime_root: Path, + preview: dict[str, object], + *, + registry_path: Path | None = None, + goal_ref: dict[str, str] | None = None, +) -> dict[str, object]: return record_quota_slot_spend_from_preview( preview, {"runtime_root": str(runtime_root)}, goal_id=GOAL_ID, execute=True, source="heartbeat", + registry_path=registry_path, + goal_ref=goal_ref, ) +def _write_source_registry( + registry_path: Path, + runtime_root: Path, + instance_id: str, +) -> None: + expected = { + "schema_version": "0.2", + "registry_role": "project-local", + "profile_id": "source_session_v1", + "common_runtime_root": str(runtime_root), + "projects": [], + "goals": [ + { + "id": GOAL_ID, + "goal_instance_id": instance_id, + "status": "active", + "execution_authority": False, + } + ], + "session_bindings": [], + "session_receipts": [], + "lifetime_receipts": [], + "retired_goal_instances": [], + } + create = None if registry_path.exists() else lambda: expected + with source_session_registry_transaction( + registry_path, + operation="quota_spend_goal_instance_test", + create=create, + ) as transaction: + payload = transaction.payload_copy() + payload["goals"] = expected["goals"] + transaction.commit(payload) + + def _status(runtime_root: Path) -> dict[str, object]: todo = quota_todo_item( todo_id="todo_quota_basis", @@ -159,6 +207,52 @@ def test_python_facade_serializes_concurrent_exact_effect_retries( assert len(index_path.read_text(encoding="utf-8").splitlines()) == 1 +def test_source_spend_rejects_stale_goal_before_any_write_and_stamps_successor( + tmp_path: Path, +) -> None: + runtime_root = tmp_path / "runtime" + registry_path = tmp_path / "project" / ".loopx" / "registry.json" + goal_ref_a = {"goal_id": GOAL_ID, "goal_instance_id": INSTANCE_A} + goal_ref_b = {"goal_id": GOAL_ID, "goal_instance_id": INSTANCE_B} + _write_source_registry(registry_path, runtime_root, INSTANCE_A) + preview_a = _preview(effect_ref="provider-effect-a#quota_spend") + + _write_source_registry(registry_path, runtime_root, INSTANCE_B) + with pytest.raises(ValueError, match="stale_goal_instance"): + _commit( + runtime_root, + preview_a, + registry_path=registry_path, + goal_ref=goal_ref_a, + ) + + runs_dir = runtime_root / "goals" / GOAL_ID / "runs" + assert not (runs_dir / "index.jsonl").exists() + assert not (runs_dir / ".transactions").exists() + assert not list(runs_dir.glob("*.json")) + assert not list(runs_dir.glob("*.md")) + assert not list(tmp_path.rglob("*.ts-effect.lock")) + + written = _commit( + runtime_root, + _preview(effect_ref="provider-effect-b#quota_spend"), + registry_path=registry_path, + goal_ref=goal_ref_b, + ) + + assert written["goal_ref"] == goal_ref_b + record = json.loads(Path(written["json_path"]).read_text(encoding="utf-8")) + row = json.loads(Path(written["index_path"]).read_text(encoding="utf-8")) + receipt_paths = list(runs_dir.glob(".transactions/quota-spend/*.json")) + assert len(receipt_paths) == 1 + receipt = json.loads(receipt_paths[0].read_text(encoding="utf-8")) + assert record["goal_ref"] == goal_ref_b + assert record["quota_event"]["goal_ref"] == goal_ref_b + assert row["goal_ref"] == goal_ref_b + assert receipt["goal_ref"] == goal_ref_b + assert not list(tmp_path.rglob("*.ts-effect.lock")) + + def test_python_facade_rejects_a_second_effect_from_the_same_quota_basis( tmp_path: Path, ) -> None: diff --git a/tests/control_plane/test_quota_void_commit_runtime.py b/tests/control_plane/test_quota_void_commit_runtime.py index 671c7fb2f2..8190f1cae7 100644 --- a/tests/control_plane/test_quota_void_commit_runtime.py +++ b/tests/control_plane/test_quota_void_commit_runtime.py @@ -20,6 +20,9 @@ commit_quota_slot_void, record_quota_slot_void_from_preview, ) +from loopx.control_plane.projects.registry_codec import ( + source_session_registry_transaction, +) from loopx.history import repair_index_duplicates from loopx.presentation.renderers.quota_event_markdown import ( render_quota_slot_preview_markdown, @@ -32,6 +35,8 @@ TARGET_AT = "2026-08-31T09:00:00+08:00" VOID_AT = "2026-08-31T09:05:00+08:00" REASON = "void duplicate quota spend" +INSTANCE_A = "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" +INSTANCE_B = "ginst_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" def test_legacy_void_commit_import_paths_remain_compatible() -> None: @@ -75,7 +80,11 @@ def _decision(spent_slots: int) -> dict[str, Any]: } -def _write_spend_target(runtime_root: Path) -> tuple[Path, Path]: +def _write_spend_target( + runtime_root: Path, + *, + goal_ref: dict[str, str] | None = None, +) -> tuple[Path, Path]: runs_dir = runtime_root / "goals" / GOAL_ID / "runs" runs_dir.mkdir(parents=True, exist_ok=True) target_path = runs_dir / "target-quota-slot-spent.json" @@ -94,6 +103,9 @@ def _write_spend_target(runtime_root: Path) -> tuple[Path, Path]: "after": compact_quota_decision(_decision(2)), }, } + if goal_ref is not None: + target["goal_ref"] = goal_ref + target["quota_event"]["goal_ref"] = goal_ref target_path.write_text( json.dumps(target, ensure_ascii=False, indent=2) + "\n", encoding="utf-8", @@ -101,23 +113,55 @@ def _write_spend_target(runtime_root: Path) -> tuple[Path, Path]: index_path = runs_dir / "index.jsonl" # Cover the legacy bounded-artifact fallback instead of relying only on # quota_event being embedded in the compact index row. + index_record = { + "generated_at": TARGET_AT, + "goal_id": GOAL_ID, + "classification": "quota_slot_spent", + "agent_id": AGENT_ID, + "json_path": str(target_path), + "markdown_path": str(target_path.with_suffix(".md")), + } + if goal_ref is not None: + index_record["goal_ref"] = goal_ref index_path.write_text( - json.dumps( - { - "generated_at": TARGET_AT, - "goal_id": GOAL_ID, - "classification": "quota_slot_spent", - "agent_id": AGENT_ID, - "json_path": str(target_path), - "markdown_path": str(target_path.with_suffix(".md")), - } - ) - + "\n", + json.dumps(index_record) + "\n", encoding="utf-8", ) return index_path, target_path +def _write_source_registry( + registry_path: Path, + runtime_root: Path, + instance_id: str, +) -> None: + payload = { + "schema_version": "0.2", + "registry_role": "project-local", + "profile_id": "source_session_v1", + "common_runtime_root": str(runtime_root), + "projects": [], + "goals": [ + { + "id": GOAL_ID, + "goal_instance_id": instance_id, + "status": "active", + "execution_authority": False, + } + ], + "session_bindings": [], + "session_receipts": [], + "lifetime_receipts": [], + "retired_goal_instances": [], + } + with source_session_registry_transaction( + registry_path, + operation="quota_void_goal_instance_test", + create=lambda: payload, + ) as transaction: + transaction.commit(payload) + + def _preview(runtime_root: Path) -> dict[str, Any]: return { "ok": True, @@ -275,6 +319,91 @@ def test_real_runtime_preserves_public_payloads_and_three_artifact_write( assert rows[-1]["markdown_path"] == str(markdown_path) +@pytest.mark.parametrize("execute", [False, True]) +@pytest.mark.parametrize( + "target_goal_ref", + [ + {"goal_id": GOAL_ID, "goal_instance_id": INSTANCE_A}, + None, + ], +) +def test_source_void_rejects_foreign_and_legacy_spend_targets( + tmp_path: Path, + target_goal_ref: dict[str, str] | None, + execute: bool, +) -> None: + runtime_root = tmp_path / "runtime" + registry_path = tmp_path / "project" / ".loopx" / "registry.json" + goal_ref_b = {"goal_id": GOAL_ID, "goal_instance_id": INSTANCE_B} + _write_source_registry(registry_path, runtime_root, INSTANCE_B) + index_path, _ = _write_spend_target( + runtime_root, + goal_ref=target_goal_ref, + ) + before = index_path.read_bytes() + + with pytest.raises(ValueError, match="Goal instance"): + commit_quota_slot_void( + {"runtime_root": str(runtime_root)}, + goal_id=GOAL_ID, + voided_run_generated_at=TARGET_AT, + before=_decision(2), + execute=execute, + source="heartbeat", + registry_path=registry_path, + goal_ref=goal_ref_b, + ) + + assert index_path.read_bytes() == before + assert not list( + (index_path.parent / ".transactions" / "quota-void").glob("*.json") + ) + + +def test_source_void_stamps_the_current_goal_ref( + tmp_path: Path, +) -> None: + runtime_root = tmp_path / "runtime" + registry_path = tmp_path / "project" / ".loopx" / "registry.json" + goal_ref_b = {"goal_id": GOAL_ID, "goal_instance_id": INSTANCE_B} + _write_source_registry(registry_path, runtime_root, INSTANCE_B) + index_path, _ = _write_spend_target(runtime_root, goal_ref=goal_ref_b) + + preview = commit_quota_slot_void( + {"runtime_root": str(runtime_root)}, + goal_id=GOAL_ID, + voided_run_generated_at=TARGET_AT, + before=_decision(2), + execute=False, + source="heartbeat", + registry_path=registry_path, + goal_ref=goal_ref_b, + ) + assert preview["goal_ref"] == goal_ref_b + assert index_path.read_text(encoding="utf-8").count("\n") == 1 + + written = commit_quota_slot_void( + {"runtime_root": str(runtime_root)}, + goal_id=GOAL_ID, + voided_run_generated_at=TARGET_AT, + before=_decision(2), + execute=True, + source="heartbeat", + registry_path=registry_path, + goal_ref=goal_ref_b, + ) + + assert written["goal_ref"] == goal_ref_b + rows = [ + json.loads(line) + for line in index_path.read_text(encoding="utf-8").splitlines() + ] + assert rows[-1]["goal_ref"] == goal_ref_b + record = json.loads(Path(written["json_path"]).read_text(encoding="utf-8")) + assert record["goal_ref"] == goal_ref_b + assert record["quota_event"]["goal_ref"] == goal_ref_b + + def test_real_runtime_normalizes_ecmascript_goal_and_effect_identity( tmp_path: Path, ) -> None: diff --git a/tests/control_plane/test_refresh_external_delivery.py b/tests/control_plane/test_refresh_external_delivery.py index 3f80e5c25b..1e1a5ba8c9 100644 --- a/tests/control_plane/test_refresh_external_delivery.py +++ b/tests/control_plane/test_refresh_external_delivery.py @@ -1,10 +1,12 @@ """Exercise legacy, local and failed-persistence paths through the real backend.""" import json +from types import SimpleNamespace import pytest from loopx import cli from loopx.control_plane.quota import refresh_external_delivery as bridge +from loopx.control_plane.quota.settlement import SettlementIdentity from loopx.state_refresh import refresh_state_run from tests.control_plane.test_quota_settlement_cli import ( AGENT_ID, GOAL_ID, TODO_ID, TURN_ID, _write_fixture, @@ -101,3 +103,41 @@ def test_receipt_only_repair_preserves_pause_without_requiring_confirmation(sess assert run(args)["receipt_repaired"] is True assert index.read_bytes() == before assert run(args, expected=1)["error_code"] == "external_delivery_resume_required" + + +def test_exact_external_delivery_transition_persists_goal_ref(tmp_path): + goal_ref = { + "goal_id": GOAL_ID, + "goal_instance_id": "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + } + identity = SettlementIdentity( + GOAL_ID, + AGENT_ID, + TODO_ID, + TURN_ID, + ) + readback = SimpleNamespace( + identity=SimpleNamespace(value=identity), + external_delivery={ + "schema_version": "refresh_external_delivery_v0", + "authorized": False, + "transition": { + "state": "paused", + "resume_key": "resume-fixture", + }, + }, + ) + result = bridge.finish_external_delivery_refresh( + {"ok": True}, + readback, + tmp_path, + dry_run=False, + goal_ref=goal_ref, + ) + events = json.loads( + (tmp_path / "goals" / GOAL_ID / "rollout-event-log.jsonl") + .read_text(encoding="utf-8") + .strip() + ) + assert events["goal_ref"] == goal_ref + assert result["external_sink_delivery_authorized"] is False diff --git a/tests/control_plane_ts/auxiliary_monitor_settlement.test.ts b/tests/control_plane_ts/auxiliary_monitor_settlement.test.ts index 2fec58f10a..cfe1a76c22 100644 --- a/tests/control_plane_ts/auxiliary_monitor_settlement.test.ts +++ b/tests/control_plane_ts/auxiliary_monitor_settlement.test.ts @@ -2,11 +2,15 @@ import assert from "node:assert/strict"; import { createHash } from "node:crypto"; import { appendFile, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join } from "node:path"; +import { dirname, join } from "node:path"; import test from "node:test"; import { settlementIdentity, type JsonObject } from "../../loopx/control_plane/effect_program.ts"; import { EffectRuntimeRequestError } from "../../loopx/control_plane/effect_runtime_errors.ts"; +import { + acquireFileMutationLock, + releaseFileMutationLock, +} from "../../loopx/control_plane/effect_runtime_io.ts"; import { evaluateQuotaMonitorPollCommit, QUOTA_MONITOR_POLL_COMMIT_REQUEST_SCHEMA, } from "../../loopx/control_plane/quota/monitor_poll_commit.ts"; @@ -16,20 +20,25 @@ const agent = "agent-monitor-fixture"; const turn = "turn-completed-advancement"; const primary = "todo_primary"; const monitor = "todo_monitor"; +const instanceA = "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; const identity = settlementIdentity({ goal_id: goal, agent_id: agent, turn_instance_id: turn, todo_id: primary }); -function event(kind: string): JsonObject { +function event(kind: string, goalRef?: JsonObject): JsonObject { return { schema_version: "loopx_rollout_event_v0", event_id: `event-${kind}`, event_kind: kind, goal_id: goal, agent_id: agent, run_id: turn, + ...(goalRef ? { goal_ref: goalRef } : {}), details: { todo_id: primary, settlement_effect_id: identity.effect_id } }; } -async function fixture(t: test.TestContext): Promise<{ runtime: string; params: JsonObject }> { +async function fixture( + t: test.TestContext, + goalRef?: JsonObject, +): Promise<{ runtime: string; params: JsonObject }> { const runtime = await mkdtemp(join(tmpdir(), "loopx-auxiliary-settlement-")); t.after(() => rm(runtime, { recursive: true, force: true })); await mkdir(join(runtime, "goals", goal, "runs"), { recursive: true }); await writeFile(join(runtime, "goals", goal, "runs", "index.jsonl"), ""); - await writeFile(join(runtime, "goals", goal, "rollout-event-log.jsonl"), `${JSON.stringify(event("quota_should_run"))}\n`); + await writeFile(join(runtime, "goals", goal, "rollout-event-log.jsonl"), `${JSON.stringify(event("quota_should_run", goalRef))}\n`); const candidate = { todo_id: monitor, task_class: "continuous_monitor", claimed_by: agent, target_key: "public-watch", due: true }; return { runtime, params: { @@ -57,14 +66,71 @@ async function fixture(t: test.TestContext): Promise<{ runtime: string; params: } }; } -async function settlePrimary(runtime: string): Promise { +async function settlePrimary(runtime: string, goalRef?: JsonObject): Promise { + const ownerProjection = goalRef ? { goal_ref: goalRef } : {}; await appendFile(join(runtime, "goals", goal, "rollout-event-log.jsonl"), - [event("refresh_state"), event("quota_spend")].map(row => JSON.stringify(row)).join("\n") + "\n"); + [event("refresh_state", goalRef), event("quota_spend", goalRef)] + .map(row => JSON.stringify(row)).join("\n") + "\n"); await appendFile(join(runtime, "goals", goal, "runs", "index.jsonl"), [ { classification: "state_refreshed", delivery_outcome: "outcome_progress" }, { classification: "quota_slot_spent" }, ].map(row => JSON.stringify({ ...row, goal_id: goal, agent_id: agent, todo_id: primary, - turn_instance_id: turn, settlement_identity: identity })).join("\n") + "\n"); + turn_instance_id: turn, settlement_identity: identity, ...ownerProjection })).join("\n") + "\n"); +} + +function sourceGuardPath(registryPath: string): string { + const digest = createHash("sha256").update(goal, "utf8").digest("hex"); + return join( + dirname(registryPath), + ".loopx", + "lifecycle", + "goal-instance", + "guards", + `${digest}.guard`, + ); +} + +async function withSourceAdmission( + runtime: string, + run: (binding: JsonObject) => Promise, +): Promise { + const indexPath = join(runtime, "goals", goal, "runs", "index.jsonl"); + const registryPath = join(runtime, "project", ".loopx", "registry.json"); + const guardPath = sourceGuardPath(registryPath); + const indexLock = await acquireFileMutationLock(indexPath); + const guardLock = await acquireFileMutationLock(guardPath); + try { + return await run({ + goal_ref: { goal_id: goal, goal_instance_id: instanceA }, + source_admission: { + schema_version: "loopx_quota_source_admission_v0", + profile_id: "source_session_v1", + registry_path: registryPath, + planned_goal_ref: { goal_id: goal, goal_instance_id: instanceA }, + authority: { + kind: "present", + goal_ref: { goal_id: goal, goal_instance_id: instanceA }, + }, + locks: [ + { + role: "run_index", + target: indexPath, + pid: process.pid, + token: indexLock.token, + }, + { + role: "source_guard", + target: guardPath, + pid: process.pid, + token: guardLock.token, + }, + ], + }, + }); + } finally { + await releaseFileMutationLock(guardPath, guardLock.token, null, true); + await releaseFileMutationLock(indexPath, indexLock.token, null, true); + } } test("fresh auxiliary admission requires exact advancement identity and ordinary due-work gates before and after settlement", async t => { @@ -161,6 +227,39 @@ test(`unsettled ${status} primary does not admit a new auxiliary effect`, async }); } +test("exact auxiliary monitor borrows one admission through preflight, commit, and replay", async t => { + const goalRef = { goal_id: goal, goal_instance_id: instanceA }; + const { runtime, params } = await fixture(t, goalRef); + await settlePrimary(runtime, goalRef); + const indexPath = join(runtime, "goals", goal, "runs", "index.jsonl"); + const index = await readFile(indexPath); + + await withSourceAdmission(runtime, async binding => { + const request = { + ...params, + ...binding, + expected_index_digest: `sha256:${createHash("sha256").update(index).digest("hex")}`, + }; + const preflight = await evaluateQuotaMonitorPollCommit(request); + assert.equal(preflight.status, "provider_required", JSON.stringify(preflight)); + + const commit = { + ...request, + phase: "commit", + provider_receipt: providerReceipt(request), + }; + await assert.rejects( + evaluateQuotaMonitorPollCommit({ ...commit, provider_receipt: null }), + /requires a provider receipt/, + ); + assert.equal((await evaluateQuotaMonitorPollCommit(commit)).status, "written"); + assert.equal((await evaluateQuotaMonitorPollCommit(commit)).status, "replayed"); + }); + + const indexLock = await acquireFileMutationLock(indexPath); + await releaseFileMutationLock(indexPath, indexLock.token, null, true); +}); + test("completed primary preserves an admitted pending effect across settlement and replay reads current closeout", async t => { const { runtime, params } = await fixture(t); const preflight = await evaluateQuotaMonitorPollCommit(params); diff --git a/tests/control_plane_ts/host_todo_completion.test.ts b/tests/control_plane_ts/host_todo_completion.test.ts index c608bb392c..459fcc7db8 100644 --- a/tests/control_plane_ts/host_todo_completion.test.ts +++ b/tests/control_plane_ts/host_todo_completion.test.ts @@ -44,6 +44,30 @@ test("host context read uses the original identity and cannot carry a decision", checkpoint_read_context_id: "receipt-a"}), /only to vision recovery/); }); +test("host settlement commands preserve an exact GoalRef", () => { + const instanceId = "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + const goalRef = {goal_id: "goal", goal_instance_id: instanceId}; + const prepared = prepare({goal_ref: goalRef}); + const steps = ( + prepared.provider_effect as {steps: {args: string[]}[]} + ).steps; + for (const step of steps) { + assert.deepEqual( + step.args.slice(step.args.indexOf("--goal-instance-id"), step.args.indexOf("--goal-instance-id") + 2), + ["--goal-instance-id", instanceId], + ); + } + const context = evaluateHostTodoCompletion(request("prepare", { + schema_version: HOST_TODO_VISION_TRANSACTION_SCHEMA_VERSION, + phase: "vision_context", + goal_ref: goalRef, + })); + assert.deepEqual( + (context.args as string[]).slice(-2), + ["--goal-instance-id", instanceId], + ); +}); + test("vision decisions require v1 and cannot combine patch with unchanged", () => { assert.throws(() => prepare({vision_path: "vision.json"}), /requires v1/); assert.throws(() => prepare({schema_version: HOST_TODO_VISION_TRANSACTION_SCHEMA_VERSION, diff --git a/tests/control_plane_ts/quota_settlement_readback.test.ts b/tests/control_plane_ts/quota_settlement_readback.test.ts index fd5304638b..397f0c6342 100644 --- a/tests/control_plane_ts/quota_settlement_readback.test.ts +++ b/tests/control_plane_ts/quota_settlement_readback.test.ts @@ -1,4 +1,5 @@ import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; import { appendFile, mkdir, @@ -8,10 +9,14 @@ import { writeFile, } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join } from "node:path"; +import { dirname, join } from "node:path"; import test from "node:test"; import { settlementIdentity } from "../../loopx/control_plane/effect_program.ts"; +import { + acquireFileMutationLock, + releaseFileMutationLock, +} from "../../loopx/control_plane/effect_runtime_io.ts"; import { BLOCKED_WAIT_REQUEST_SCHEMA, prepareBlockedWait } from "../../loopx/control_plane/quota/blocked_wait.ts"; import { evaluateTodoResumeConditions, TODO_RESUME_EVALUATION_REQUEST_SCHEMA_VERSION } from "../../loopx/control_plane/todos/resume_condition.ts"; import { @@ -20,11 +25,14 @@ import { readQuotaSettlement, readQuotaSettlementSnapshot, } from "../../loopx/control_plane/quota/settlement_readback.ts"; +import { requireJsonObject } from "../../loopx/control_plane/runtime_decode.ts"; const goalId = "settlement-goal"; const agentId = "codex-settlement"; const todoId = "todo_settlement"; const turnId = "turn-settlement-1"; +const instanceA = "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; +const instanceB = "ginst_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; const identity = settlementIdentity({ goal_id: goalId, agent_id: agentId, @@ -82,10 +90,20 @@ async function fixture(options: { progressObservation?: Record; blockedRetry?: boolean | Record; visionCheckpoint?: Record; + goalRef?: Record; + turnId?: string; } = {}) { const runtimeRoot = await mkdtemp(join(tmpdir(), "loopx-settlement-readback-")); const goalRoot = join(runtimeRoot, "goals", goalId); const runsRoot = join(goalRoot, "runs"); + const ownerProjection = options.goalRef ? { goal_ref: options.goalRef } : {}; + const fixtureTurnId = options.turnId ?? turnId; + const fixtureIdentity = settlementIdentity({ + goal_id: goalId, + agent_id: agentId, + todo_id: todoId, + turn_instance_id: fixtureTurnId, + }); await mkdir(runsRoot, { recursive: true }); const events: Record[] = options.guard === false ? [] @@ -95,13 +113,14 @@ async function fixture(options: { event_kind: "quota_should_run", goal_id: goalId, agent_id: agentId, - run_id: turnId, + run_id: fixtureTurnId, + ...ownerProjection, details: { ...(options.guardUnbound ? {} : { todo_id: todoId, - settlement_effect_id: identity.effect_id, + settlement_effect_id: fixtureIdentity.effect_id, }), ...(options.workspace ? { @@ -124,8 +143,9 @@ async function fixture(options: { event_kind: "quota_should_run", goal_id: goalId, agent_id: agentId, - run_id: turnId, + run_id: fixtureTurnId, status: "action_selection_deferred", + ...ownerProjection, details: { pending_action_selection_todo_id: todoId, pending_action_selection_state: "deferred", @@ -143,8 +163,9 @@ async function fixture(options: { event_kind: "refresh_state", goal_id: goalId, agent_id: agentId, - run_id: turnId, - details: { settlement_effect_id: identity.effect_id }, + run_id: fixtureTurnId, + ...ownerProjection, + details: { settlement_effect_id: fixtureIdentity.effect_id }, }); runs.push({ classification: "state_refreshed", @@ -152,8 +173,9 @@ async function fixture(options: { goal_id: goalId, agent_id: agentId, todo_id: todoId, - turn_instance_id: turnId, - settlement_identity: identity, + turn_instance_id: fixtureTurnId, + settlement_identity: fixtureIdentity, + ...ownerProjection, ...(options.visionCheckpoint ? {vision_checkpoint: options.visionCheckpoint} : {}), ...(options.blockedRetry ? {blocked_retry: typeof options.blockedRetry === "object" ? options.blockedRetry : { schema_version: "quota_blocked_retry_v0", @@ -175,16 +197,18 @@ async function fixture(options: { event_kind: "quota_spend", goal_id: goalId, agent_id: agentId, - run_id: turnId, - details: { settlement_effect_id: identity.effect_id }, + run_id: fixtureTurnId, + ...ownerProjection, + details: { settlement_effect_id: fixtureIdentity.effect_id }, }); runs.push({ classification: "quota_slot_spent", goal_id: goalId, agent_id: agentId, todo_id: todoId, - turn_instance_id: turnId, - settlement_identity: identity, + turn_instance_id: fixtureTurnId, + settlement_identity: fixtureIdentity, + ...ownerProjection, }); } if (options.completion) { @@ -194,9 +218,10 @@ async function fixture(options: { event_kind: "todo_complete", goal_id: goalId, agent_id: agentId, - run_id: turnId, + run_id: fixtureTurnId, + ...ownerProjection, details: { - settlement_effect_id: identity.effect_id, + settlement_effect_id: fixtureIdentity.effect_id, no_followup: options.noFollowup === true, }, }); @@ -207,11 +232,12 @@ async function fixture(options: { goal_id: goalId, agent_id: agentId, todo_id: todoId, - turn_instance_id: turnId, + turn_instance_id: fixtureTurnId, material_change: true, + ...ownerProjection, quota_monitor_poll_commit: { schema_version: "quota_monitor_poll_commit_receipt_v0", - effect_id: `quota-monitor-poll:${goalId}:${agentId}:${turnId}:todo:${todoId}`, + effect_id: `quota-monitor-poll:${goalId}:${agentId}:${fixtureTurnId}:todo:${todoId}`, request_digest: "fixture", }, }); @@ -227,6 +253,19 @@ async function fixture(options: { return runtimeRoot; } +async function appendHistory(targetRoot: string, sourceRoot: string): Promise { + const relativePaths = [ + join("goals", goalId, "rollout-event-log.jsonl"), + join("goals", goalId, "runs", "index.jsonl"), + ]; + for (const relativePath of relativePaths) { + await appendFile( + join(targetRoot, relativePath), + await readFile(join(sourceRoot, relativePath), "utf8"), + ); + } +} + function request(runtimeRoot: string, overrides: Record = {}) { return { schema_version: QUOTA_SETTLEMENT_READBACK_REQUEST_SCHEMA, @@ -242,6 +281,72 @@ function request(runtimeRoot: string, overrides: Record = {}) { }; } +function sourceGuardPath(registryPath: string): string { + const digest = createHash("sha256").update(goalId, "utf8").digest("hex"); + return join( + dirname(registryPath), + ".loopx", + "lifecycle", + "goal-instance", + "guards", + `${digest}.guard`, + ); +} + +async function withSourceAdmission( + runtimeRoot: string, + plannedInstanceId: string, + currentInstanceId: string, + run: (binding: Record) => Promise, +): Promise { + const indexPath = join(runtimeRoot, "goals", goalId, "runs", "index.jsonl"); + const registryPath = join(runtimeRoot, "project", ".loopx", "registry.json"); + const guardPath = sourceGuardPath(registryPath); + const indexLock = await acquireFileMutationLock(indexPath); + const guardLock = await acquireFileMutationLock(guardPath); + try { + return await run({ + goal_ref: { + goal_id: goalId, + goal_instance_id: plannedInstanceId, + }, + source_admission: { + schema_version: "loopx_quota_source_admission_v0", + profile_id: "source_session_v1", + registry_path: registryPath, + planned_goal_ref: { + goal_id: goalId, + goal_instance_id: plannedInstanceId, + }, + authority: { + kind: "present", + goal_ref: { + goal_id: goalId, + goal_instance_id: currentInstanceId, + }, + }, + locks: [ + { + role: "run_index", + target: indexPath, + pid: process.pid, + token: indexLock.token, + }, + { + role: "source_guard", + target: guardPath, + pid: process.pid, + token: guardLock.token, + }, + ], + }, + }); + } finally { + await releaseFileMutationLock(guardPath, guardLock.token, null, true); + await releaseFileMutationLock(indexPath, indexLock.token, null, true); + } +} + test("scoped supersede settles only its exact Turn and never terminal acceptance", async t => { const cases = [ {name: "original tuple", expected: "settled", patch: {}}, @@ -298,6 +403,213 @@ test("settlement progress requires both effects and exact receipts", async t => }); }); +test("settlement readback enforces exact source ownership before identity inference", async () => { + const goalRefA = { + goal_id: goalId, + goal_instance_id: instanceA, + }; + const root = await fixture({ + writeback: true, + spend: true, + goalRef: goalRefA, + }); + try { + const unscoped = await readQuotaSettlement(request(root)); + assert.equal(unscoped.found, true); + const unscopedIdentity = requireJsonObject( + requireJsonObject(unscoped.identity, "unscoped identity").result, + "unscoped identity result", + ); + assert.equal( + requireJsonObject( + unscopedIdentity.failure, + "unscoped identity failure", + ).kind, + "receipt_missing", + ); + assert.equal(unscoped.spend_run, null); + + const inferred = await withSourceAdmission( + root, + instanceB, + instanceB, + async (binding) => await readQuotaSettlement(request(root, { + ...binding, + turn_instance_id: null, + infer_turn_instance_id: true, + })), + ); + assert.deepEqual(inferred, { + schema_version: "loopx_quota_settlement_readback_result_v0", + found: false, + }); + + await assert.rejects( + withSourceAdmission( + root, + instanceA, + instanceB, + async (binding) => await readQuotaSettlement(request(root, binding)), + ), + (error: unknown) => { + assert.equal( + requireJsonObject(error, "stale GoalRef error").code, + "stale_goal_instance", + ); + return true; + }, + ); + + const matching = await withSourceAdmission( + root, + instanceA, + instanceA, + async (binding) => await readQuotaSettlement(request(root, binding)), + ); + const matchingSpend = requireJsonObject(matching.spend, "matching spend"); + const matchingPayload = requireJsonObject( + matchingSpend.payload, + "matching spend payload", + ); + assert.equal(matchingPayload.ok, true); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("alias and exact histories remain independently readable", async () => { + const legacy = await fixture({ writeback: true, spend: true }); + const exact = await fixture({ + writeback: true, + spend: true, + goalRef: { + goal_id: goalId, + goal_instance_id: instanceA, + }, + }); + try { + await appendHistory(legacy, exact); + const aliasReadback = await readQuotaSettlement(request(legacy)); + assert.equal( + requireJsonObject( + requireJsonObject(aliasReadback.settlement, "alias settlement").payload, + "alias settlement payload", + ).ok, + true, + ); + const exactReadback = await withSourceAdmission( + legacy, + instanceA, + instanceA, + async (binding) => await readQuotaSettlement(request(legacy, binding)), + ); + assert.equal( + requireJsonObject( + requireJsonObject(exactReadback.settlement, "exact settlement").payload, + "exact settlement payload", + ).ok, + true, + ); + } finally { + await rm(legacy, { recursive: true, force: true }); + await rm(exact, { recursive: true, force: true }); + } +}); + +test("a delayed stale owner cannot replace the current owner's inferred Turn", async () => { + const current = await fixture({ + writeback: true, + spend: true, + goalRef: { + goal_id: goalId, + goal_instance_id: instanceB, + }, + }); + const delayed = await fixture({ + writeback: true, + spend: true, + turnId: "turn-stale-a", + goalRef: { + goal_id: goalId, + goal_instance_id: instanceA, + }, + }); + try { + await appendHistory(current, delayed); + const inferred = await withSourceAdmission( + current, + instanceB, + instanceB, + async (binding) => await readQuotaSettlement(request(current, { + ...binding, + turn_instance_id: null, + infer_turn_instance_id: true, + })), + ); + assert.equal(inferred.found, true); + assert.equal( + requireJsonObject( + requireJsonObject(inferred.settlement, "current settlement").payload, + "current settlement payload", + ).ok, + true, + ); + } finally { + await rm(current, { recursive: true, force: true }); + await rm(delayed, { recursive: true, force: true }); + } +}); + +test("borrowed exact admission remains valid for an enclosing transaction", async () => { + const root = await fixture({ + writeback: true, + goalRef: { + goal_id: goalId, + goal_instance_id: instanceA, + }, + }); + try { + await withSourceAdmission( + root, + instanceA, + instanceA, + async (binding) => { + const borrowed = { + ...binding, + borrow_source_admission: true, + }; + assert.equal( + (await readQuotaSettlement(request(root, borrowed))).found, + true, + ); + assert.equal( + (await readQuotaSettlement(request(root, borrowed))).found, + true, + ); + }, + ); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("legacy settlement still consumes legacy rows", async () => { + const root = await fixture({ writeback: true, spend: true }); + try { + const result = await readQuotaSettlement(request(root)); + assert.equal(result.found, true); + assert.equal( + requireJsonObject( + requireJsonObject(result.settlement, "legacy settlement").payload, + "legacy settlement payload", + ).ok, + true, + ); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + test("settlement progress preserves typed source and rejects malformed sources", async () => { const root = await fixture({writeback: true}); try { diff --git a/tests/control_plane_ts/quota_spend_commit.test.ts b/tests/control_plane_ts/quota_spend_commit.test.ts index 346c61af2c..553c25ed95 100644 --- a/tests/control_plane_ts/quota_spend_commit.test.ts +++ b/tests/control_plane_ts/quota_spend_commit.test.ts @@ -1,4 +1,5 @@ import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; import { readFile, mkdir, @@ -12,6 +13,20 @@ import { tmpdir } from "node:os"; import { dirname, join } from "node:path"; import test from "node:test"; +import { + acquireFileMutationLock, + releaseFileMutationLock, +} from "../../loopx/control_plane/effect_runtime_io.ts"; +import { + EffectRuntimeConflictError, + EffectRuntimeLockTimeoutError, + EffectRuntimeRequestError, +} from "../../loopx/control_plane/effect_runtime_errors.ts"; +import { requireJsonObject } from "../../loopx/control_plane/runtime_decode.ts"; +import { + parseQuotaAccountingOwner, + withQuotaAccountingOwner, +} from "../../loopx/control_plane/quota/source_admission.ts"; import { evaluateQuotaSpendCommit, quotaSpendIndexDigest, @@ -19,6 +34,12 @@ import { } from "../../loopx/control_plane/quota/spend_commit.ts"; const goalId = "quota-spend-transaction"; +const instanceA = "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; +const instanceB = "ginst_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; + +interface TestSourceAdmission extends Record { + locks: [Record, Record]; +} function decision( spentSlots: number, @@ -94,6 +115,76 @@ async function tempRuntime(t: test.TestContext): Promise { return runtimeRoot; } +function sourceGuardPath(registryPath: string): string { + const digest = createHash("sha256").update(goalId, "utf8").digest("hex"); + return join( + dirname(registryPath), + ".loopx", + "lifecycle", + "goal-instance", + "guards", + `${digest}.guard`, + ); +} + +async function withSourceAdmission( + runtimeRoot: string, + plannedInstanceId: string, + currentInstanceId: string, + run: (binding: Record) => Promise, + mutateAdmission?: (admission: TestSourceAdmission) => void, + ownerPid = process.pid, +): Promise { + const indexPath = join(runtimeRoot, "goals", goalId, "runs", "index.jsonl"); + const registryPath = join(runtimeRoot, "project", ".loopx", "registry.json"); + const guardPath = sourceGuardPath(registryPath); + const indexLock = await acquireFileMutationLock(indexPath, ownerPid); + const guardLock = await acquireFileMutationLock(guardPath, ownerPid); + const admission: TestSourceAdmission = { + schema_version: "loopx_quota_source_admission_v0", + profile_id: "source_session_v1", + registry_path: registryPath, + planned_goal_ref: { + goal_id: goalId, + goal_instance_id: plannedInstanceId, + }, + authority: { + kind: "present", + goal_ref: { + goal_id: goalId, + goal_instance_id: currentInstanceId, + }, + }, + locks: [ + { + role: "run_index", + target: indexPath, + pid: ownerPid, + token: indexLock.token, + }, + { + role: "source_guard", + target: guardPath, + pid: ownerPid, + token: guardLock.token, + }, + ], + }; + mutateAdmission?.(admission); + try { + return await run({ + goal_ref: { + goal_id: goalId, + goal_instance_id: plannedInstanceId, + }, + source_admission: admission, + }); + } finally { + await releaseFileMutationLock(guardPath, guardLock.token, null, true); + await releaseFileMutationLock(indexPath, indexLock.token, null, true); + } +} + function replayRequest(runtimeRoot: string, effectId: string) { return { schema_version: QUOTA_SPEND_COMMIT_REQUEST_SCHEMA, @@ -252,6 +343,287 @@ test("commit owns JSON, Markdown, index, and exact-effect replay", async (t) => assert.equal((await readFile(indexPath, "utf8")).trim().split("\n").length, 1); }); +test("source owner rejects stale Goal A before writes and stamps Goal B", async (t) => { + const runtimeRoot = await tempRuntime(t); + await assert.rejects( + withSourceAdmission(runtimeRoot, instanceA, instanceB, async (binding) => + await evaluateQuotaSpendCommit({ + ...request(runtimeRoot), + ...binding, + }) + ), + (error: unknown) => { + assert.ok(error instanceof EffectRuntimeConflictError); + assert.equal(error.code, "stale_goal_instance"); + return true; + }, + ); + const runsDir = join(runtimeRoot, "goals", goalId, "runs"); + await assert.rejects(readFile(join(runsDir, "index.jsonl")), { code: "ENOENT" }); + await assert.rejects(readdir(join(runsDir, ".transactions")), { code: "ENOENT" }); + + const written = await withSourceAdmission( + runtimeRoot, + instanceB, + instanceB, + async (binding) => await evaluateQuotaSpendCommit({ + ...request(runtimeRoot), + ...binding, + }), + ); + const expectedGoalRef = { + goal_id: goalId, + goal_instance_id: instanceB, + }; + assert.deepEqual(written.payload.goal_ref, expectedGoalRef); + const record = requireJsonObject( + JSON.parse(await readFile(String(written.payload.json_path), "utf8")), + "quota spend record", + ); + const event = requireJsonObject(record.quota_event, "quota spend event"); + const row = requireJsonObject( + JSON.parse(await readFile(String(written.payload.index_path), "utf8")), + "quota spend index row", + ); + const receiptDirectory = join(runsDir, ".transactions", "quota-spend"); + const [receiptName] = await readdir(receiptDirectory); + const receipt = requireJsonObject( + JSON.parse(await readFile(join(receiptDirectory, receiptName), "utf8")), + "quota spend receipt", + ); + assert.deepEqual(record.goal_ref, expectedGoalRef); + assert.deepEqual(event.goal_ref, expectedGoalRef); + assert.deepEqual(row.goal_ref, expectedGoalRef); + assert.deepEqual(receipt.goal_ref, expectedGoalRef); +}); + +test("source owner validates every witness target before claiming locks", async (t) => { + const runtimeRoot = await tempRuntime(t); + await assert.rejects( + withSourceAdmission( + runtimeRoot, + instanceA, + instanceA, + async (binding) => await evaluateQuotaSpendCommit({ + ...request(runtimeRoot), + ...binding, + }), + (admission) => { + admission.locks[1].target = join(runtimeRoot, "wrong-source.guard"); + }, + ), + (error: unknown) => { + assert.ok(error instanceof EffectRuntimeRequestError); + assert.equal(error.code, "quota_source_admission_invalid"); + return true; + }, + ); + await assert.rejects( + readFile(join(runtimeRoot, "goals", goalId, "runs", "index.jsonl")), + { code: "ENOENT" }, + ); +}); + +test("source owner rejects an expired lock handoff", async (t) => { + const runtimeRoot = await tempRuntime(t); + const indexPath = join(runtimeRoot, "goals", goalId, "runs", "index.jsonl"); + const registryPath = join(runtimeRoot, "project", ".loopx", "registry.json"); + const guardPath = sourceGuardPath(registryPath); + const indexLock = await acquireFileMutationLock(indexPath); + const guardLock = await acquireFileMutationLock(guardPath); + await releaseFileMutationLock(guardPath, guardLock.token, null, true); + await releaseFileMutationLock(indexPath, indexLock.token, null, true); + + await assert.rejects( + evaluateQuotaSpendCommit({ + ...request(runtimeRoot), + goal_ref: { + goal_id: goalId, + goal_instance_id: instanceA, + }, + source_admission: { + schema_version: "loopx_quota_source_admission_v0", + profile_id: "source_session_v1", + registry_path: registryPath, + planned_goal_ref: { + goal_id: goalId, + goal_instance_id: instanceA, + }, + authority: { + kind: "present", + goal_ref: { + goal_id: goalId, + goal_instance_id: instanceA, + }, + }, + locks: [ + { + role: "run_index", + target: indexPath, + pid: process.pid, + token: indexLock.token, + }, + { + role: "source_guard", + target: guardPath, + pid: process.pid, + token: guardLock.token, + }, + ], + }, + }), + (error: unknown) => { + assert.ok(error instanceof EffectRuntimeConflictError); + assert.equal(error.code, "quota_source_admission_expired"); + return true; + }, + ); +}); + +test("source owner keeps both claims through durable completion after parent loss", async (t) => { + const runtimeRoot = await tempRuntime(t); + const indexPath = join(runtimeRoot, "goals", goalId, "runs", "index.jsonl"); + const registryPath = join(runtimeRoot, "project", ".loopx", "registry.json"); + const guardPath = sourceGuardPath(registryPath); + const durableResult = join(runtimeRoot, "durable-result.json"); + const deadOwnerPid = 2_147_483_647; + + await withSourceAdmission( + runtimeRoot, + instanceA, + instanceA, + async (binding) => { + const owner = parseQuotaAccountingOwner({ + goalRefValue: binding.goal_ref, + sourceAdmissionValue: binding.source_admission, + runtimeRoot, + goalId, + }); + await withQuotaAccountingOwner(owner, async (indexLockHeld) => { + assert.equal(indexLockHeld, true); + await writeFile(durableResult, "{\"status\":\"committed\"}\n", "utf8"); + for (const target of [indexPath, guardPath]) { + await assert.rejects( + acquireFileMutationLock(target, process.pid, 0), + EffectRuntimeLockTimeoutError, + ); + } + }); + }, + undefined, + deadOwnerPid, + ); + + assert.deepEqual( + JSON.parse(await readFile(durableResult, "utf8")), + { status: "committed" }, + ); + const reacquiredIndex = await acquireFileMutationLock(indexPath); + const reacquiredGuard = await acquireFileMutationLock(guardPath); + await releaseFileMutationLock( + guardPath, + reacquiredGuard.token, + null, + true, + ); + await releaseFileMutationLock( + indexPath, + reacquiredIndex.token, + null, + true, + ); +}); + +test("unscoped replay cannot consume an exact source-owned spend", async (t) => { + const runtimeRoot = await tempRuntime(t); + const params = request(runtimeRoot); + await withSourceAdmission( + runtimeRoot, + instanceA, + instanceA, + async (binding) => await evaluateQuotaSpendCommit({ + ...params, + ...binding, + }), + ); + + const replay = await evaluateQuotaSpendCommit(params); + + assert.equal(replay.status, "conflict"); + assert.equal(replay.reason_code, "goal_instance_conflict"); +}); + +test("native replay requires the exact source GoalRef", async (t) => { + const runtimeRoot = await tempRuntime(t); + const effectId = "quota-spend-source-replay"; + await withSourceAdmission( + runtimeRoot, + instanceA, + instanceA, + async (binding) => + await evaluateQuotaSpendCommit({ + ...request(runtimeRoot, { effect_id: effectId }), + ...binding, + }), + ); + + const unscoped = await evaluateQuotaSpendCommit( + replayRequest(runtimeRoot, effectId), + ); + assert.equal(unscoped.status, "conflict"); + assert.equal(unscoped.reason_code, "goal_instance_conflict"); + + const replayed = await withSourceAdmission( + runtimeRoot, + instanceA, + instanceA, + async (binding) => + await evaluateQuotaSpendCommit({ + ...replayRequest(runtimeRoot, effectId), + ...binding, + }), + ); + assert.equal(replayed.status, "replayed"); + assert.equal(replayed.payload.replay_found, true); +}); + +test("legacy spend artifacts keep the alias-only wire shape", async (t) => { + const runtimeRoot = await tempRuntime(t); + const written = await evaluateQuotaSpendCommit(request(runtimeRoot)); + const record = requireJsonObject( + JSON.parse(await readFile(String(written.payload.json_path), "utf8")), + "quota spend record", + ); + const event = requireJsonObject(record.quota_event, "quota spend event"); + const row = requireJsonObject( + JSON.parse(await readFile(String(written.payload.index_path), "utf8")), + "quota spend index row", + ); + const receiptDirectory = join( + runtimeRoot, + "goals", + goalId, + "runs", + ".transactions", + "quota-spend", + ); + const [receiptName] = await readdir(receiptDirectory); + const receipt = requireJsonObject( + JSON.parse(await readFile(join(receiptDirectory, receiptName), "utf8")), + "quota spend receipt", + ); + + for (const value of [ + written.payload, + record, + event, + row, + receipt, + ]) { + assert.equal(Object.hasOwn(value, "goal_ref"), false); + } +}); + test("native replay validates legacy rows by goal and agent", async (t) => { const runtimeRoot = await tempRuntime(t); const runsDir = join(runtimeRoot, "goals", goalId, "runs"); @@ -440,6 +812,96 @@ test("prepared transaction repairs partial artifacts exactly once", async (t) => assert.equal(replayed.status, "replayed"); }); +test("prepared source transaction repairs only for its current GoalRef", async (t) => { + const runtimeRoot = await tempRuntime(t); + const params = request(runtimeRoot); + const written = await withSourceAdmission( + runtimeRoot, + instanceA, + instanceA, + async (binding) => + await evaluateQuotaSpendCommit({ + ...params, + ...binding, + }), + ); + const indexPath = String(written.payload.index_path); + const markdownPath = String(written.payload.markdown_path); + const transactionDir = join( + dirname(indexPath), + ".transactions", + "quota-spend", + ); + const [receiptName] = await readdir(transactionDir); + assert.ok(receiptName); + const receiptPath = join(transactionDir, receiptName); + const receipt = requireJsonObject( + JSON.parse(await readFile(receiptPath, "utf8")), + "prepared quota spend receipt", + ); + receipt.status = "prepared"; + await Promise.all([ + writeFile(receiptPath, `${JSON.stringify(receipt, null, 2)}\n`, "utf8"), + unlink(markdownPath), + ]); + + const repaired = await withSourceAdmission( + runtimeRoot, + instanceA, + instanceA, + async (binding) => + await evaluateQuotaSpendCommit({ + ...params, + ...binding, + }), + ); + assert.equal(repaired.status, "repaired"); + assert.match(await readFile(markdownPath, "utf8"), /quota_slot_spent/); + + receipt.status = "prepared"; + await Promise.all([ + writeFile(receiptPath, `${JSON.stringify(receipt, null, 2)}\n`, "utf8"), + unlink(markdownPath), + ]); + await assert.rejects( + withSourceAdmission( + runtimeRoot, + instanceA, + instanceB, + async (binding) => + await evaluateQuotaSpendCommit({ + ...params, + ...binding, + }), + ), + (error: unknown) => { + assert.ok(error instanceof EffectRuntimeConflictError); + assert.equal(error.code, "stale_goal_instance"); + return true; + }, + ); + await assert.rejects(readFile(markdownPath), { code: "ENOENT" }); + + const foreign = await withSourceAdmission( + runtimeRoot, + instanceB, + instanceB, + async (binding) => + await evaluateQuotaSpendCommit({ + ...params, + ...binding, + }), + ); + assert.equal(foreign.status, "conflict"); + assert.equal(foreign.reason_code, "goal_instance_conflict"); + await assert.rejects(readFile(markdownPath), { code: "ENOENT" }); + const foreignReceipt = requireJsonObject( + JSON.parse(await readFile(receiptPath, "utf8")), + "foreign quota spend receipt", + ); + assert.equal(foreignReceipt.status, "prepared"); +}); + test("prepared transactions keep artifact paths reserved across later spends", async (t) => { const runtimeRoot = await tempRuntime(t); const firstParams = request(runtimeRoot); diff --git a/tests/control_plane_ts/quota_void_commit.test.ts b/tests/control_plane_ts/quota_void_commit.test.ts index c81f6249ca..5b13f97fec 100644 --- a/tests/control_plane_ts/quota_void_commit.test.ts +++ b/tests/control_plane_ts/quota_void_commit.test.ts @@ -11,9 +11,15 @@ import { writeFile, } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join } from "node:path"; +import { dirname, join } from "node:path"; import test from "node:test"; +import { + acquireFileMutationLock, + releaseFileMutationLock, +} from "../../loopx/control_plane/effect_runtime_io.ts"; +import { EffectRuntimeConflictError } from "../../loopx/control_plane/effect_runtime_errors.ts"; +import { requireJsonObject } from "../../loopx/control_plane/runtime_decode.ts"; import { evaluateQuotaVoidCommit, quotaVoidIndexDigest, @@ -21,9 +27,16 @@ import { } from "../../loopx/control_plane/quota/void_commit.ts"; const goalId = "quota-void-transaction"; +const instanceA = "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; +const instanceB = "ginst_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; const targetGeneratedAt = "2026-08-25T11:59:00+08:00"; const voidGeneratedAt = "2026-08-25T12:00:00+08:00"; +interface GoalRef { + goal_id: string; + goal_instance_id: string; +} + interface TargetFixture { runtimeRoot: string; runsDir: string; @@ -56,8 +69,11 @@ function beforeDecision(spentSlots: unknown = 5): Record { }; } -function quotaSpendEvent(slots: unknown): Record { - return { +function quotaSpendEvent( + slots: unknown, + goalRef?: GoalRef, +): Record { + const event: Record = { event_type: "quota_slot_spent", source: "heartbeat", slots, @@ -65,6 +81,8 @@ function quotaSpendEvent(slots: unknown): Record { before: { spent_slots: 3 }, after: { spent_slots: 5 }, }; + if (goalRef) event.goal_ref = goalRef; + return event; } async function tempRuntime(t: test.TestContext): Promise { @@ -80,6 +98,7 @@ async function targetFixture( slots?: unknown; generatedAt?: string; jsonPath?: string; + goalRef?: GoalRef; } = {}, ): Promise { const runtimeRoot = await tempRuntime(t); @@ -91,13 +110,14 @@ async function targetFixture( runsDir, "20260825-115900-quota-slot-spent.json", ); - const event = quotaSpendEvent(options.slots ?? 2); + const event = quotaSpendEvent(options.slots ?? 2, options.goalRef); const indexRecord: Record = { generated_at: generatedAt, goal_id: goalId, classification: "quota_slot_spent", json_path: targetJsonPath, }; + if (options.goalRef) indexRecord.goal_ref = options.goalRef; if (options.inline !== false) { indexRecord.quota_event = event; } else { @@ -108,6 +128,7 @@ async function targetFixture( goal_id: goalId, classification: "quota_slot_spent", quota_event: event, + ...(options.goalRef ? { goal_ref: options.goalRef } : {}), }, null, 2)}\n`, "utf8", ); @@ -117,6 +138,81 @@ async function targetFixture( return { runtimeRoot, runsDir, indexPath, indexContent, targetJsonPath }; } +function sourceGuardPath(registryPath: string): string { + const digest = createHash("sha256").update(goalId, "utf8").digest("hex"); + return join( + dirname(registryPath), + ".loopx", + "lifecycle", + "goal-instance", + "guards", + `${digest}.guard`, + ); +} + +async function withSourceAdmission( + fixture: TargetFixture, + plannedInstanceId: string, + currentInstanceId: string, + run: (binding: Record) => Promise, +): Promise { + const registryPath = join( + fixture.runtimeRoot, + "project", + ".loopx", + "registry.json", + ); + const guardPath = sourceGuardPath(registryPath); + const indexLock = await acquireFileMutationLock(fixture.indexPath); + const guardLock = await acquireFileMutationLock(guardPath); + try { + return await run({ + goal_ref: { + goal_id: goalId, + goal_instance_id: plannedInstanceId, + }, + source_admission: { + schema_version: "loopx_quota_source_admission_v0", + profile_id: "source_session_v1", + registry_path: registryPath, + planned_goal_ref: { + goal_id: goalId, + goal_instance_id: plannedInstanceId, + }, + authority: { + kind: "present", + goal_ref: { + goal_id: goalId, + goal_instance_id: currentInstanceId, + }, + }, + locks: [ + { + role: "run_index", + target: fixture.indexPath, + pid: process.pid, + token: indexLock.token, + }, + { + role: "source_guard", + target: guardPath, + pid: process.pid, + token: guardLock.token, + }, + ], + }, + }); + } finally { + await releaseFileMutationLock(guardPath, guardLock.token, null, true); + await releaseFileMutationLock( + fixture.indexPath, + indexLock.token, + null, + true, + ); + } +} + async function rawIndexDigest(indexPath: string): Promise { const value = await readFile(indexPath); return `sha256:${createHash("sha256").update(value).digest("hex")}`; @@ -328,6 +424,71 @@ test("commit atomically owns the JSON, Markdown, and index artifacts", async (t) ); }); +test("source Goal B voids only a spend owned by B", async (t) => { + const goalRefA = { goal_id: goalId, goal_instance_id: instanceA }; + const goalRefB = { goal_id: goalId, goal_instance_id: instanceB }; + for (const targetOwner of [goalRefA, undefined]) { + const fixture = await targetFixture(t, { goalRef: targetOwner }); + await assert.rejects( + withSourceAdmission( + fixture, + instanceB, + instanceB, + async (binding) => + await evaluateQuotaVoidCommit({ + ...await request(fixture), + ...binding, + }), + ), + (error: unknown) => { + assert.ok(error instanceof EffectRuntimeConflictError); + assert.equal(error.code, "goal_instance_conflict"); + return true; + }, + ); + assert.equal(await readFile(fixture.indexPath, "utf8"), fixture.indexContent); + await assert.rejects( + readdir(join(fixture.runsDir, ".transactions")), + { code: "ENOENT" }, + ); + } + + const fixture = await targetFixture(t, { goalRef: goalRefB }); + const written = await withSourceAdmission( + fixture, + instanceB, + instanceB, + async (binding) => + await evaluateQuotaVoidCommit({ + ...await request(fixture), + ...binding, + }), + ); + + assert.equal(written.status, "written"); + assert.deepEqual(written.payload.goal_ref, goalRefB); + const record = requireJsonObject( + JSON.parse(await readFile(String(written.payload.json_path), "utf8")), + "quota void record", + ); + const event = requireJsonObject(record.quota_event, "quota void event"); + const rows = (await readFile(fixture.indexPath, "utf8")) + .trim() + .split("\n") + .map((line) => requireJsonObject( + JSON.parse(line), + "quota void index row", + )); + const receipt = await transactionReceipt( + fixture.runsDir, + String(written.effect_id), + ); + assert.deepEqual(record.goal_ref, goalRefB); + assert.deepEqual(event.goal_ref, goalRefB); + assert.deepEqual(rows[1]?.goal_ref, goalRefB); + assert.deepEqual(receipt.value.goal_ref, goalRefB); +}); + test("the same effect replays without appending a second void", async (t) => { const fixture = await targetFixture(t); const params = await request(fixture); diff --git a/tests/control_plane_ts/unsettled_host_turn_recovery.test.ts b/tests/control_plane_ts/unsettled_host_turn_recovery.test.ts index e65f268922..8ec1f750aa 100644 --- a/tests/control_plane_ts/unsettled_host_turn_recovery.test.ts +++ b/tests/control_plane_ts/unsettled_host_turn_recovery.test.ts @@ -1,7 +1,8 @@ import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; import { appendFile, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join } from "node:path"; +import { dirname, join } from "node:path"; import { performance } from "node:perf_hooks"; import test from "node:test"; @@ -9,6 +10,10 @@ import { settlementIdentity, type JsonObject, } from "../../loopx/control_plane/effect_program.ts"; +import { + acquireFileMutationLock, + releaseFileMutationLock, +} from "../../loopx/control_plane/effect_runtime_io.ts"; import { ACCEPTED_CLOSEOUTS, PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_REQUEST_SCHEMA, @@ -153,6 +158,58 @@ function preflight(runtimeRoot: string, exclude: string | null = "current-turn") }); } +async function withSourceAdmission( + runtimeRoot: string, + instanceId: string, + run: (binding: JsonObject) => Promise, +): Promise { + const indexPath = join(runtimeRoot, "goals", GOAL, "runs", "index.jsonl"); + const registryPath = join(runtimeRoot, "project", ".loopx", "registry.json"); + const guardPath = join( + dirname(registryPath), + ".loopx", + "lifecycle", + "goal-instance", + "guards", + `${createHash("sha256").update(GOAL, "utf8").digest("hex")}.guard`, + ); + const indexLock = await acquireFileMutationLock(indexPath); + const guardLock = await acquireFileMutationLock(guardPath); + try { + const goalRef = { + goal_id: GOAL, + goal_instance_id: instanceId, + }; + return await run({ + goal_ref: goalRef, + source_admission: { + schema_version: "loopx_quota_source_admission_v0", + profile_id: "source_session_v1", + registry_path: registryPath, + planned_goal_ref: goalRef, + authority: {kind: "present", goal_ref: goalRef}, + locks: [ + { + role: "run_index", + target: indexPath, + pid: process.pid, + token: indexLock.token, + }, + { + role: "source_guard", + target: guardPath, + pid: process.pid, + token: guardLock.token, + }, + ], + }, + }); + } finally { + await releaseFileMutationLock(guardPath, guardLock.token, null, true); + await releaseFileMutationLock(indexPath, indexLock.token, null, true); + } +} + function candidateFrom(result: JsonObject): JsonObject { assert.equal(result.status, "candidate"); return result.candidate as JsonObject; @@ -218,6 +275,60 @@ test("the preflight reads the persisted receipts and names the newest required c } }); +test("exact preflight filters stale owners before selecting the newest Turn", async () => { + const instanceA = "ginst_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + const instanceB = "ginst_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; + const runtime = await runtimeWith([ + receipt("turn-b", closeoutRequired("turn-b", "todo_current"), { + goal_ref: {goal_id: GOAL, goal_instance_id: instanceB}, + }), + receipt("turn-delayed-a", closeoutRequired("turn-delayed-a", "todo_stale"), { + goal_ref: {goal_id: GOAL, goal_instance_id: instanceA}, + }), + ]); + try { + const result = await withSourceAdmission( + runtime.root, + instanceB, + async (binding) => await preflightPriorHostTurnCloseout({ + schema_version: PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_REQUEST_SCHEMA, + runtime_root: runtime.root, + goal_id: GOAL, + agent_id: AGENT, + exclude_turn_instance_id: "current-turn", + ...binding, + }), + ); + assert.equal(candidateFrom(result).prior_turn_instance_id, "turn-b"); + + const onlyStale = await runtimeWith([ + receipt("turn-a", closeoutRequired("turn-a", "todo_stale"), { + goal_ref: {goal_id: GOAL, goal_instance_id: instanceA}, + }), + ]); + try { + const none = await withSourceAdmission( + onlyStale.root, + instanceB, + async (binding) => await preflightPriorHostTurnCloseout({ + schema_version: PRIOR_HOST_TURN_CLOSEOUT_PREFLIGHT_REQUEST_SCHEMA, + runtime_root: onlyStale.root, + goal_id: GOAL, + agent_id: AGENT, + exclude_turn_instance_id: "current-turn", + ...binding, + }), + ); + assert.equal(none.status, "none"); + assert.equal(none.reason, "no_prior_turn_requires_closeout"); + } finally { + await onlyStale.close(); + } + } finally { + await runtime.close(); + } +}); + test("a receipt that does not opt in never becomes a recovery obligation", async () => { const runtime = await runtimeWith([ receipt("turn-a", {todo_id: "todo_alpha", settlement_effect_id: "e"}),