Skip to content

docs(product): rework forge agent-notification to webhook-only (RIG-2732) - #634

Open
rigel-mintaka wants to merge 1 commit into
mainfrom
compass-server/rig-2732-forge-agent-notif-design
Open

docs(product): rework forge agent-notification to webhook-only (RIG-2732)#634
rigel-mintaka wants to merge 1 commit into
mainfrom
compass-server/rig-2732-forge-agent-notif-design

Conversation

@rigel-mintaka

@rigel-mintaka rigel-mintaka commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Reworks the forge → agent notification pipeline (RIG-2732 Pieces 2 + 3) from a poll driver to a webhook-only design, per Matt's transport-pivot ruling, and folds in Matt's 2026-08-26 rulings on the four carried forks (W1-W4) plus OQ-1. All forks are ruled; the record is at the freeze gate (Status: Active), pending Matt's merge.

What changed from the poll draft

GitHub/Linear webhooks are the sole change signal, delivered to the POST /webhooks ingress whose shape RIG-2717 ratified (DL-254). A per-deployment GitHub App (folded into this record) is THE credential for both read and write via forge.AppTokenSource over the existing TokenSource seam; the static PAT survives only as the no-App fallback. Reliability is a reconcile catch-up sweep at a ~30-min backstop cadence (conditional-GET, diff-only) — not a poll-as-primary loop. The fast ticker, reserve-floor, and poll-as-primary role are deleted.

Rulings folded (Matt, 2026-08-26) — see "Resolved decisions" in the record

  • W1 — App token scope: (a) UNIFIED. The App backs the one shared forge.GitHub client (read + write). App auth uses an installation token and consumes NO org seat, so separate authoring/reviewing identities = TWO Apps at zero seat cost.
  • W2 — subscription scope: (b) EXPLICIT SCOPE ENUM + provider-native containers. The number=0 sentinel is dropped (proto3 can't tell unset from 0 → a forgotten number from an LLM tool-caller would become a silent whole-repo sub). Container scope rides ForgeSubscriptionScope; the container is provider-native — GitHub REPO, Linear PROJECT (promoted from deferral to in-scope). A forgotten number under ARTIFACT stays InvalidArgument.
  • W3 — delivered-revision advance: (beta) IN-BAND ACK. Advance on a new additive ForgeNotificationAck frame (the forge sibling of DeliveryAck), emitted at turn-end flush, applied by a hub ack arm — never on dispatch success. This reopens the frozen advance-signal clause (compass-notification-delivery/design.md:894-897), so it rides the freeze-gate ratification, bundled with moving the fresh-subscription catch-up baseline to first-observed-event/sweep.
  • W4 — ingress-less adopters: (a) HARD-OFF + TUNNEL. One webhook path only; no App → no GitHub notifications (boot Warn when subs exist). Ingress-less adopters expose the webhook URL via a tunnel (Cloudflare Tunnel / ngrok), a documented runbook step — not a degraded second in-server transport.
  • OQ-1 — Linear PROJECT container-id encoding: (i) ("634 Q1 - (i) lgtm"). A dedicated project column beside a scope enum, repo kept stable — never overloaded. This was the last load-bearing fork; the T3 DDL/proto/store surface is the (i) encoding. (ii) generic container_kind/container_ref and (iii) repo-overload are rejected in the record.

Ledger-impact (applied in this PR)

Four new rows in docs/designs/DECISIONS.md (DL-264..DL-267):

  • DL-264 — the DL-053 agent-notification transport is WEBHOOK-ONLY (App + reconcile catch-up; no poll loop).
  • DL-265 — forge notifications ride the generic DispatchControl relay; the bare forge_notification = 7 variant stays reserved, no Runner dispatch arm.
  • DL-266 — the delivery cursor advances on the in-band ForgeNotificationAck (W3 beta), bundled with the catch-up-baseline move.
  • DL-267 — explicit ForgeSubscriptionScope enum + provider-native containers (GitHub REPO, Linear PROJECT via the dedicated project column, OQ-1 = (i)); adds FORGE_NOTIFICATION_KIND_REVIEW/OPENED.

Scoped supersession of DL-053 (transport only): DL-053 stays Active — the ledger row-status grammar has no partial-supersession cell, and DL-053's live clauses (the two-cursor FETCH/DELIVERY split, the push path, delivery semantics) are still cited by DL-071/DL-072 and the board lane (DL-161). DL-264's prose carries the transport-only supersession, the house partial-overturn pattern (DL-236/DL-183 "Refines … which stays Active"). No DL-053 code comment is dangling (they describe the board lane / Piece-1 store, which survive with a scope touch when that code next changes — compass-forge's Piece-2 T3).

Sequencing prerequisites (unlanded)

Tasks sequence after: the RIG-2717 impl stack (#637-639) + its /webhooks mount; PR #631 (Piece 1, forge subscription store writer, MERGED).

Ownership (post forge/compass-server lane split)

compass-server drove this record to the freeze gate (OQ-1 was its parked fork). compass-forge owns Piece 2 impl the moment #634 freezes — read the folded record for the final T3 encoding.

Ref: RIG-2732

Co-authored-by: Matt Wilkinson matt@rigel.build

@linear-code

linear-code Bot commented Aug 26, 2026

Copy link
Copy Markdown

RIG-2732

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown

Compass engineering docs preview: https://compass-server-rig-2732-forg-66q3.compass-eng-docs.pages.dev

Deployed from compass-server/rig-2732-forge-agent-notif-design at 62a8253.

Changed pages:

@mattwilkinsonn

Copy link
Copy Markdown
Contributor

I think both of these forges have webhooks? Can we just use those? Much better than polling. And also want the capability for an agent to subscribe to any new opened PR on a repo, or new issue on a repo/project, etc.

@rigelbuild-bot

Copy link
Copy Markdown
Contributor

Agreed — and the case for webhooks is now much stronger than when DL-053 punted them. DL-053 rejected webhooks only on adoption friction: "a public HTTPS ingress the Server must expose, a shared secret per repo, HMAC verification, per-repo webhook registration (admin rights on every adopted repo)" (compass-server-ownership-layer/design.md:1097-1102). A GitHub App + the Linear app dissolve every line of that: one app-level ingress, one app secret, install-to-subscribe (no per-repo admin). So this isn't overriding DL-053's reasoning — it's that the reasoning was premised on not having the apps.

Grounded what already exists before proposing the rework:

The ingress already lands in a sibling PR. RIG-2717 (#625, at your merge gate) builds the first production webhook receiver: POST /webhooks on the network TLS door (network_door.go:233-326) — HMAC-SHA256 fail-closed over the raw body, ack-200-fast / work-async, dedup riding comms client_request_id idempotency. That's exactly the reusable shape the forge side needs; the Linear app itself is already installed (RIG-2682, actor=app). And DL-053 already designed DetectChanges as the one function both poll and a webhook feed drive (ownership-layer:2259-2263), so the change-detect core is reusable as-is.

Two things genuinely don't exist yet:

  • No GitHub App. GitHub auth today is a static token via TokenSource (github.go:28-37, DL-052) — no installation id, no app JWT, no X-GitHub-Event handling anywhere. A GitHub App (registration, install flow, per-installation token minting, permissions/events) is a real new prerequisite, and it's substantial.
  • No repo/project-level subscription on the wire. SubscribeForgeRequest is per-artifact only — {repo, kind, number} (agent_gateway.proto:353-357). "Any new PR on a repo / new issue on a repo/project" is a new subscription granularity (number-absent target) + new detect/deliver semantics.

Three scope calls before I rework the record (each changes its shape materially):

1. GitHub App — its own prerequisite record, or folded into this one? I'd recommend its own record + issue: it's a standalone capability (auth model, install flow, permissions) that also unblocks anything else wanting GitHub events, and the Linear side already has its app. This record then consumes "an installed app webhook" as a dependency rather than designing it inline. (Fold-in is possible but makes this a much bigger single record.)

2. Poll — remove it, or keep as fallback? Your "if possible" reads to me as webhook-primary with poll surviving for un-app'd repos. I'd recommend webhook-primary, poll retained as degraded-mode fallback feeding the same DetectChanges — that reuses the frozen poll work instead of throwing it away, and covers repos/forges where the app isn't installed. (Alternative: drop poll entirely and require the app — simpler, but no coverage without an install.)

3. Repo/project-level subscription — this record, or a follow-up? It needs new wire (SubscribeForgeRequest gains a repo/project target) + new semantics. I'd recommend folding it into this rework, since app-webhook installation events are exactly the natural source for repo-level opened-PR / new-issue, and it's additive on the wire.

Whichever way you rule on those, I'll rework the record webhook-primary against your answers (design-critic + review pass again before it's back at your gate), flip the DL-053 transport row in the ledger, and file the GitHub-App prerequisite if you want it split out. Holding the current poll-primary content until you rule so I don't rework twice.

@mattwilkinsonn

Copy link
Copy Markdown
Contributor
  1. Github apps are simple enough i think. Just roll into this record.
  2. Remove poll. No reason for it, user just installs App at setup like they do for many other products. I think github webhooks have been slightly unreliable lately (need to check this) as the rest of github keeps having issues, so may need some kind of mitigation for that but i’d prefer just the app and webhooks if possible. But also for the App each user would need to create their own if they are doing selfhosted, which is fine and expected but we’d just need to document setup. Plenty of other selfhosted things do this like Renovate.
  3. Fold in. And any other potential subscription paths we’d want on both forges.

@rigelbuild-bot

Copy link
Copy Markdown
Contributor

All three ruled — thanks. Reworking webhook-only:

  1. GitHub App folded into this record (registration, install flow, per-installation token minting, permissions/events) alongside the already-installed Linear app.
  2. Poll removed entirely — app-install-to-subscribe is the only transport. On the reliability wobble: with no poll fallback a dropped webhook would be a permanently-missed notification, so the record will carry a lightweight reconciliation catch-up (a conditional-GET sweep on startup/reconnect and a bounded periodic backstop) that self-heals a missed delivery without reintroducing poll as the primary transport. Self-hosted: each deployment registers its own App; I'll add a setup doc (Renovate-style).
  3. Repo/project-level subscription folded in (number-absent target on the wire), and I'll enumerate the other useful subscription granularities both forges expose (GitHub App event types, Linear webhook resource types) and cover them.

Reworking now → design-critic + review pass → back at your gate. The DL-053 poll-transport row flips (superseded) on freeze.

Ref: RIG-2732

@rigel-mintaka rigel-mintaka changed the title docs(product): design forge agent-notification pipeline (RIG-2732) docs(product): rework forge agent-notification to webhook-only (RIG-2732) Aug 26, 2026
@rigel-mintaka
rigel-mintaka force-pushed the compass-server/rig-2732-forge-agent-notif-design branch 2 times, most recently from c7773f9 to 06ca31c Compare August 26, 2026 05:20
rigel-mintaka added a commit that referenced this pull request Aug 26, 2026
…ecord (RIG-2679 B)

Folds Matt's 2026-08-26 scope expansion (scope the repos an agent can clone/push/pull, not just the forge-API write path) plus a design-critic red-team into the forge-scope-enforcement record.

### Git-operation scope (new A5 + Plan T4/T4.5/T5)

Scope the credential, not the git call: mint a GitHub App installation token narrowed via the `repositories` field to the workstream repo + the account's `account_forge_scopes` set, so an out-of-scope `git push`/`clone` fails at GitHub itself — zero enforcement code inside the agent-controlled container.

- **Self-clone invariant**: the agent's own workstream repo is always clonable/pullable (else provisioning succeeds but the agent is dead on arrival); push stays write-gated; extra repos need grants; zero/insufficient-grant provision fails LOUD.
- **T4.5 (new, proto)**: the minted token must cross the Server→Runner process boundary — `ProvisionAgentWorkspaceRequest` carries no credential today; adds a server-authoritative `WorkspaceCredential` field + a server-driven refresh push on the hub control channel.
- **T5 hardened refresh**: pre-expiry margin, retry-with-backoff, keep-old-token-on-failure, atomic tmp+mv rewrite (the `GHHostsScript` shape).

### Critic dispositions folded

Loud PAT-fallback Warn (classic vs fine-grained PAT distinction corrected), half-landed cross-check Warn, x-access-token username, deploy-key + fine-grained-PAT Alternatives dismissals.

### New load-bearing forks for the freeze gate

OQ-6 gains a fine-grained-PAT option (c) + a #634-sequencing sub-fork (iii, recommend contingent split); OQ-7 added (git-op allowlist = write set + implicit workstream read, vs a distinct read superset).

Spec-impact: none (design record only; Ledger-impact declared in-record, applied at freeze). Refs RIG-2679

Co-authored-by: Matt Wilkinson <matt@rigel.build>
rigel-mintaka added a commit that referenced this pull request Aug 26, 2026
…ecord (RIG-2679 B)

Folds Matt's 2026-08-26 scope expansion (scope the repos an agent can clone/push/pull, not just the forge-API write path) plus a design-critic red-team into the forge-scope-enforcement record.

### Git-operation scope (new A5 + Plan T4/T4.5/T5)

Scope the credential, not the git call: mint a GitHub App installation token narrowed via the `repositories` field to the workstream repo + the account's `account_forge_scopes` set, so an out-of-scope `git push`/`clone` fails at GitHub itself — zero enforcement code inside the agent-controlled container.

- **Self-clone invariant**: the agent's own workstream repo is always clonable/pullable (else provisioning succeeds but the agent is dead on arrival); push stays write-gated; extra repos need grants; zero/insufficient-grant provision fails LOUD.
- **T4.5 (new, proto)**: the minted token must cross the Server→Runner process boundary — `ProvisionAgentWorkspaceRequest` carries no credential today; adds a server-authoritative `WorkspaceCredential` field + a server-driven refresh push on the hub control channel.
- **T5 hardened refresh**: pre-expiry margin, retry-with-backoff, keep-old-token-on-failure, atomic tmp+mv rewrite (the `GHHostsScript` shape).

### Critic dispositions folded

Loud PAT-fallback Warn (classic vs fine-grained PAT distinction corrected), half-landed cross-check Warn, x-access-token username, deploy-key + fine-grained-PAT Alternatives dismissals.

### New load-bearing forks for the freeze gate

OQ-6 gains a fine-grained-PAT option (c) + a #634-sequencing sub-fork (iii, recommend contingent split); OQ-7 added (git-op allowlist = write set + implicit workstream read, vs a distinct read superset).

Spec-impact: none (design record only; Ledger-impact declared in-record, applied at freeze). Refs RIG-2679

Co-authored-by: Matt Wilkinson <matt@rigel.build>
rigel-mintaka added a commit that referenced this pull request Aug 26, 2026
…ecord (RIG-2679 B)

Folds Matt's 2026-08-26 scope expansion (scope the repos an agent can clone/push/pull, not just the forge-API write path) plus a design-critic red-team into the forge-scope-enforcement record.

### Git-operation scope (new A5 + Plan T4/T4.5/T5)

Scope the credential, not the git call: mint a GitHub App installation token narrowed via the `repositories` field to the workstream repo + the account's `account_forge_scopes` set, so an out-of-scope `git push`/`clone` fails at GitHub itself — zero enforcement code inside the agent-controlled container.

- **Self-clone invariant**: the agent's own workstream repo is always clonable/pullable (else provisioning succeeds but the agent is dead on arrival); push stays write-gated; extra repos need grants; zero/insufficient-grant provision fails LOUD.
- **T4.5 (new, proto)**: the minted token must cross the Server→Runner process boundary — `ProvisionAgentWorkspaceRequest` carries no credential today; adds a server-authoritative `WorkspaceCredential` field + a server-driven refresh push on the hub control channel.
- **T5 hardened refresh**: pre-expiry margin, retry-with-backoff, keep-old-token-on-failure, atomic tmp+mv rewrite (the `GHHostsScript` shape).

### Critic dispositions folded

Loud PAT-fallback Warn (classic vs fine-grained PAT distinction corrected), half-landed cross-check Warn, x-access-token username, deploy-key + fine-grained-PAT Alternatives dismissals.

### New load-bearing forks for the freeze gate

OQ-6 gains a fine-grained-PAT option (c) + a #634-sequencing sub-fork (iii, recommend contingent split); OQ-7 added (git-op allowlist = write set + implicit workstream read, vs a distinct read superset).

Spec-impact: none (design record only; Ledger-impact declared in-record, applied at freeze). Refs RIG-2679

Co-authored-by: Matt Wilkinson <matt@rigel.build>
rigel-mintaka added a commit that referenced this pull request Aug 26, 2026
…ecord (RIG-2679 B)

Folds Matt's 2026-08-26 scope expansion (scope the repos an agent can clone/push/pull, not just the forge-API write path) plus a design-critic red-team into the forge-scope-enforcement record.

### Git-operation scope (new A5 + Plan T4/T4.5/T5)

Scope the credential, not the git call: mint a GitHub App installation token narrowed via the `repositories` field to the workstream repo + the account's `account_forge_scopes` set, so an out-of-scope `git push`/`clone` fails at GitHub itself — zero enforcement code inside the agent-controlled container.

- **Self-clone invariant**: the agent's own workstream repo is always clonable/pullable (else provisioning succeeds but the agent is dead on arrival); push stays write-gated; extra repos need grants; zero/insufficient-grant provision fails LOUD.
- **T4.5 (new, proto)**: the minted token must cross the Server→Runner process boundary — `ProvisionAgentWorkspaceRequest` carries no credential today; adds a server-authoritative `WorkspaceCredential` field + a server-driven refresh push on the hub control channel.
- **T5 hardened refresh**: pre-expiry margin, retry-with-backoff, keep-old-token-on-failure, atomic tmp+mv rewrite (the `GHHostsScript` shape).

### Critic dispositions folded

Loud PAT-fallback Warn (classic vs fine-grained PAT distinction corrected), half-landed cross-check Warn, x-access-token username, deploy-key + fine-grained-PAT Alternatives dismissals.

### New load-bearing forks for the freeze gate

OQ-6 gains a fine-grained-PAT option (c) + a #634-sequencing sub-fork (iii, recommend contingent split); OQ-7 added (git-op allowlist = write set + implicit workstream read, vs a distinct read superset).

Spec-impact: none (design record only; Ledger-impact declared in-record, applied at freeze). Refs RIG-2679

Co-authored-by: Matt Wilkinson <matt@rigel.build>
@rigel-mintaka
rigel-mintaka force-pushed the compass-server/rig-2732-forge-agent-notif-design branch from 06ca31c to 88849f3 Compare August 26, 2026 06:50
…732)

Reworks the forge → agent notification pipeline (RIG-2732 Pieces 2 + 3) from a poll driver to a webhook-primary design, per Matt's transport-pivot ruling on this record.

## What changed from the poll draft

GitHub/Linear webhooks are the primary change signal, delivered to the `POST /webhooks` ingress whose shape RIG-2717 ratified (DL-254). A per-deployment GitHub App (folded into this record) supplies both the webhook subscription and the API credential via `forge.AppTokenSource` over the existing `TokenSource` seam. Reliability is a reconcile **catch-up sweep** at tens-of-minutes cadence (conditional-GET, diff-only) that heals gaps — not a poll-as-primary loop.

Honest framing: the sweep reuses ~80% of the poll draft's machinery (DetectChanges, snapshot codec, the conditional-read endpoints, the cursor table, per-target pacing, budget abort). What the rework deletes is the fast ticker, the reserve-floor, and the poll-as-primary role — the difference from a "webhooks + slow-poll fallback" hybrid is cadence-plus-intent.

## Design-critic pass folded

Ten findings from the red-team pass: six folded (the "ingress exists" premise corrected to shape-ruled-code-unlanded with explicit RIG-2717 sequencing; a cross-producer canonicalization invariant + meeting-point test; `check_suite.completed` as a roll-up *trigger* that fetches combined checks, not per-suite truth; honest hybrid framing + a justified 30-min backstop default; repo-scope PR-filtering + pagination walk; two unlanded-dependency markers as a Global Constraint), four surfaced into the load-bearing open questions below.

## Load-bearing open questions (freeze-gate decisions for Matt)

- **W1 — GitHub App token scope**: (a) unified credential [rec] / (b) split / (c) b-then-a read-only App now, defer the PAT→App write unification + `<app>[bot]` attribution flip to its own decision.
- **W2 — repo-scope wire encoding**: (a) `number=0` sentinel [rec on churn grounds] / (b) explicit scope enum — strengthened with the proto3 absent-vs-0 silent-misfire argument (a caught `InvalidArgument` bug becomes a silent whole-repo subscription).
- **W3 — delivered-revision advance signal**: (alpha) dispatch-success advance, bounded for *synchronous* refusal only [rec] / (beta') router.complete un-advance hook, closes the async subclass with no proto work / (beta) `delivery_ack` alignment with the frozen clause. Async refusal and post-dispatch loss are unbounded-until-next-change under alpha.
- **W4 — ingress-less deployments (NEW)**: a no-public-URL adopter (the class DL-053's poll-rejection protected) gets (a) no GitHub notifications, boot Warn [rec for this record] / (b) reconcile-only degraded mode / (c) hard-off now + (b) as a ratified follow-up.

## Ledger-impact

DL-N (webhook-primary transport + reconcile catch-up; supersedes the poll premise, cites DL-053's dissolved friction legs), DL-N+1 (ride the generic DispatchControl relay; reserved `forge_notification` variant superseded), DL-N+2 (delivery cursor advances on dispatch success — subclass-bounded, pending W3). Next-free id re-verified at freeze.

## Sequencing prerequisites (unlanded)

This record's tasks sequence after: the RIG-2717 impl stack (#637-639) + its later `/webhooks` mount task (ingress shape); PR #631 (Piece 1, forge subscription store writer) — open/blocked.

Ref: RIG-2732

Co-authored-by: Matt Wilkinson <matt@rigel.build>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants