Skip to content
Merged
7 changes: 7 additions & 0 deletions .changeset/auto-advisor-decision-spacing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@capybearista/opencode-auto-advisor": minor
---

Call the Decision model far less often. The new `routing.minStepsBetweenDecisions` (default 10; `1` restores every step) spaces Decision calls across a session, and a step where the cooldown or the per-turn limit already blocks a consult no longer calls it at all; its telemetry line now has no answers. This saves a free decision model's rate limit and keeps the paid fallback from taking over.

Behavior change: with the default, a moment that falls on a skipped step gets no automatic review, and a turn shorter than 10 steps often ends without one. Lower the setting to trade calls for coverage.
5 changes: 5 additions & 0 deletions .changeset/auto-advisor-event-resubscribe.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@capybearista/opencode-auto-advisor": patch
---

Keep listening for the end of a run after the event stream fails. Before, one failed read stopped the listener until OpenCode restarted: Esc no longer ended a pending automatic review, a review of an interrupted run was delivered late into the next prompt, and finished requests stayed in memory. The plugin now subscribes again after one second.
5 changes: 5 additions & 0 deletions .changeset/auto-advisor-explicit-visibility.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@capybearista/opencode-auto-advisor": minor
---

Show which model answered the agent's own `advisor()` calls. The review panel gains a "Last advisor() call" line with the model, duration and time, and each explicit call now writes a `"kind": "explicit"` telemetry line with every Advisor model tried. `auto-advisor.json` now accepts comments and trailing commas (JSONC), and the README shows the full default configuration.
7 changes: 7 additions & 0 deletions .changeset/auto-advisor-models-key.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@capybearista/opencode-auto-advisor": minor
---

The Advisor chain key is now `advisor.models`, matching `routing.models`. Both take one `"provider/model[#variant]"` string or an ordered list; `routing.models` now accepts a single string too.

Breaking: rename `advisor.model` to `advisor.models` in `auto-advisor.json`. A file that still uses `advisor.model` is rejected as `unknown key: advisor.model`, which turns automatic reviews off and makes `advisor()` return the error.
90 changes: 42 additions & 48 deletions bun.lock

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions docs/auto-advisor/findings.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,11 @@ Each section names the date and the OpenCode version it was checked on. S1–S5
- The timeline now sends each page to a file in a private temporary directory. Against a live server it then worked 5/5, where the old script failed 3/3. `scripts/timeline.test.ts` reproduces the cut with a fake CLI that writes 3 MB pages and exits at once (20/20 failures before the fix, 0/20 after).
- Anything else that reads `opencode api` output must use a file, not a pipe.

## Decision spacing and pin bump (2026-11-02, OpenCode 2.2.5)
- Real telemetry from 2026-11-01 on: `opencode/jev-1.13-free` returned `rate_limit` with one session at 1–2 calls a minute, and on the first call after a 3 h gap; from Oct 28 to 31, 1,412 calls had none. One 429 skips the model for 60 s, so 346 of 402 answered decisions (86%) went to the paid fallback, and 115 of 408 calls (28%) ran on steps where a limit already blocked the consult.
- `routing.minStepsBetweenDecisions` (default 10) now spaces the calls, and a blocked step no longer calls the Decision model. In the sandbox, built against `@opencode/plugin` 2.2.5 and OpenTUI 0.8.0 with `advisor.models` in its config: two prompts, five agent steps, one telemetry line (step 1), as expected. That single call from a fresh sandbox was also rate limited, so the throttling isn't only our volume.
- The event re-subscribe can't be triggered live (it needs a failing stream); `src/server.test.ts` covers it with a fake stream that closes on error like `promise/adapter.ts` at 2.2.5.

## Known limitations
- `routing.requestMaxChars` caps each request, not the Decision input as a whole. The input is up to `routing.requests` × the cap plus about 12k characters of recent steps. The default is 20k (about 72k characters in total, roughly 18k–24k tokens) because Zen's free `jev-1.13` declares a 64k-token window (`models-dev` snapshot, 2.2.5). At 60k it could reach about 193k characters, and live it rejected 250k. A raised cap, more requests or a smaller first model can still overflow: the step then falls through to the next model, or gets `no-decision`, and automatic reviews pause until the large requests leave the window. Follow-up: size the requests block from each decision model's window.
- A prompt that a summary compaction absorbs before any step sees it never starts its own turn. The turn is the newest real prompt present in the request (`server.ts`); when none is present it carries on from the last one, so the earlier turn's spent budget and step count continue and the Decision requests block reads `(not in context)` until the next prompt. Accepted as rare and fail-closed (fewer consults, never more).
Expand Down
4 changes: 4 additions & 0 deletions packages/opencode-adversarial-review/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,12 +121,16 @@ Nothing is pinned — model and temperature are both inherited. Resolution order
```jsonc
{
"agents": {
// Used by /adversarial-review. Omit to inherit the invoking session's model.
"adversarial-reviewer": { "model": "openrouter/openai/gpt-6-sol" },
// Used by /constructive-review.
"constructive-reviewer": { "model": "openrouter/openai/gpt-6-sol" }
}
}
```

The plugin has no options of its own and nothing to show as defaults: with no `agents` entry, both reviewers use the invoking session's model and temperature. Their mode, hidden flag, description and read-only permissions are set by the plugin; a `system` or `color` you set on either agent is kept (see [Permissions & Security](#permissions--security)).

## Permissions & Security

Both reviewers are sandboxed and unattended; the single accepted risk is pasted arguments (below). Details:
Expand Down
2 changes: 1 addition & 1 deletion packages/opencode-adversarial-review/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
"@opencode/plugin": ">=2.0.2 <3"
},
"devDependencies": {
"@opencode/plugin": "2.2.0",
"@opencode/plugin": "2.2.5",
"@tsconfig/node22": "^22.0.6",
"@types/bun": "latest",
"typescript": "^7.0.2"
Expand Down
2 changes: 1 addition & 1 deletion packages/opencode-agents-loader/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,6 @@
"@opencode/plugin": ">=2.0.2 <3"
},
"devDependencies": {
"@opencode/plugin": "2.2.0"
"@opencode/plugin": "2.2.5"
}
}
10 changes: 5 additions & 5 deletions packages/opencode-auto-advisor/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,20 +19,20 @@

## Architecture

- `src/server.ts` wires everything: the `advisor` tool, a `prompt` hook that records real prompt ids, the `context` hook (route → consult → deliver, within the per-turn budget and the session step cooldown), an event subscription that ends stale consults, the `auto-advisor` RPC and telemetry.
- `src/server.ts` wires everything: the `advisor` tool, a `prompt` hook that records real prompt ids, the `context` hook (route → consult → deliver, within the per-turn budget and the session step cooldown; the Decision model runs only every `routing.minStepsBetweenDecisions` distinct steps and never on a step those limits block), an event subscription that ends stale consults, the `auto-advisor` RPC and telemetry.
- `src/fallback.ts`: the model-chain walker shared by the Decision router and the Advisor chain (error policy, skip map, shared deadline). `src/decisions.ts`: the two Decision questions and the router over `routing.models`. `src/policy.ts`: the pure consult verdict.
- `src/transcript.ts`: the Decision projection (the last `routing.requests` real prompts, each capped at `routing.requestMaxChars`, 20k by default (per request, not a total), the turn's recent steps without system updates, with subagent results and other user-role notices labelled `SUBAGENT`/`NOTICE`, and failure counts; no system prompt and no total cap) and the Advisor transcript (system text, tool names, messages with the same `SUBAGENT`/`NOTICE` labels; compacted, then budgeted, keeping the first real request to `min(advisor.maxTokens, 75% of the window)` at 3 chars per token).
- `src/advisor.ts`: prompts and the timeout-raced consult. `src/delivery.ts`: review text, marker, tail push.
- `src/config.ts`: strict sparse `auto-advisor.json`, re-read when its mtime changes. `src/telemetry.ts`: JSONL with one rotation; each line carries the newest message id and the Decision input's last 32k chars. `scripts/timeline.ts` (`bun run timeline <sessionID>`, not shipped) joins those lines with the session's messages from `opencode api get /api/session/<id>/message` into an HTML page. `scripts/sandbox.ts` (`bun run sandbox`, not shipped) runs the isolated OpenCode used for live checks, in the foreground until Ctrl-C.
- `src/tui.ts`: the footer status, a `sidebar.footer` widget (hidden by `tui.sidebar`), and a `session.panel` named `auto-advisor` with the full review, opened by a click or `/advisor-review`. State comes from RPC events, plus one `status` call per mounted widget or panel, so a stored review shows after a restart.
- `src/config.ts`: strict sparse `auto-advisor.json` (JSONC syntax via `Bun.JSONC.parse`, unknown keys rejected), re-read when its mtime changes. `config.test.ts` checks that the README's "Default configuration" block parses to `DEFAULTS`. `src/telemetry.ts`: JSONL with one rotation; each step line carries the newest message id and the Decision input's last 32k chars, and each explicit `advisor()` call that gets past the refusals (child session, no conversation yet) writes a `kind: "explicit"` line with only the Advisor's attempts and timing. Any Advisor record where no model answered (automatic or explicit) carries `advisor.error`. Every telemetry reader must handle both shapes: `scripts/timeline.ts` renders explicit lines as one-line rows that take no messages from the routing steps. `scripts/timeline.ts` (`bun run timeline <sessionID>`, not shipped) joins those lines with the session's messages from `opencode api get /api/session/<id>/message` into an HTML page. `scripts/sandbox.ts` (`bun run sandbox`, not shipped) runs the isolated OpenCode used for live checks, in the foreground until Ctrl-C.
- `src/tui.ts`: the footer status, a `sidebar.footer` widget (hidden by `tui.sidebar`), and a `session.panel` named `auto-advisor` with the full review, opened by a click or `/advisor-review`; it also names the model that answered the last explicit `advisor()` call (server memory, sent as `explicit` with an `explicit.finished` event). State comes from RPC events, plus one `status` call per mounted widget or panel, so a stored review shows after a restart.
- `src/rpc.ts` is the contract both halves import.

### Gotchas
- **Never edit `event.system`.** It breaks the provider's prompt cache. Reviews go to the tail of `event.messages` as a `role: "system"` message and to the `auto-advisor` instruction entry.
- **Entries put in the `context` hook land one step later.** The runner admits entry changes at the step boundary, before the hook (`core/src/session/runner/llm.ts` at v2.2.0). The tail push covers the current step, and the entry covers every later one.
- **Never append to the last message.** On a continuation step it is a `tool` message, and the Anthropic lowering accepts only tool-result parts there.
- **The `context` hook re-fires on a retried step.** The retry may already carry the entry from the failed attempt. `stateKey` ignores system messages and memoizes the outcome; the marker stops a second copy.
- **Hooks and `generate.*` calls can't be cancelled.** A consult that outlives its run is dropped by the session epoch, which is bumped on `session.execution.{succeeded,failed,interrupted}`. The listener reading those events ends for good if one read fails: the adapter closes the stream on a read error (2.2.0 through 2.2.5) (`@opencode/plugin` `dist/promise/adapter.js`), so retrying the read does nothing. Accepted (no failed read has been seen): after one, a consult that outlives its run is no longer dropped, so its review is delivered late and stored in the entry for later runs, and the finished run's request stays in memory. A fix would re-subscribe; a test needs a fake stream that closes on error, like the real one.
- **Hooks and `generate.*` calls can't be cancelled.** A consult that outlives its run is dropped by the session epoch, which is bumped on `session.execution.{succeeded,failed,interrupted}`. On a failed read, the adapter closes the stream and `next()` rejects once (2.2.0 through 2.2.5; `packages/plugin/src/promise/adapter.ts` `makeStreams`), so retrying the read does nothing; a closed stream only reports `done`, which also happens when the plugin unloads. The listener therefore subscribes again after `RESUBSCRIBE_MS` (1 s) on a rejection and stops on `done`; events sent during that wait are missed. The fake stream in `src/server.test.ts` closes on error like the real one.
- **Synthetic, skill, shell and compaction messages are `role: "user"` too**, and their metadata is dropped (`core/src/session/runner/to-llm-message.ts` at v2.2.0). A background subagent's result is one of them. They keep their stored ids in the hook. Real prompt ids come from the `prompt` hook plus every `type: "user"` message in `ctx.session.context`, read once per session (the session may predate the process). The turn starts at the newest real prompt present in the request: the hook fires at admission, before a steered or queued prompt is delivered. When a compaction drops that prompt, the turn carries on (its budget stays spent) and the requests block reads `(not in context)` until a new prompt arrives; a prompt absorbed into a compaction before any step saw it therefore never starts its own turn. Never treat other `role: "user"` messages as requests. If that history read fails, it is retried on the next step; until then, prompts from before the process started are labelled `NOTICE`. Accepted as rare and limited to one step.
- **Our own reviews come back as system updates.** The Decision input leaves out system messages; otherwise a delivered review (`moment: done`) primes the next decision.
- **A failing shell command is not a tool error.** A non-zero exit, a kill or a timeout completes the tool with a notice (`core/src/shell/result.ts`, unchanged through 2.2.5). On 2.2.5 the notice is its own text part (`tool/plugin/shell.ts`): a single part arrives as a plain string, several as `content`. The Decision input counts a failure only when a whole part of a `shell` or `bash` result is a notice (`bash` runs as `shell` on 2.2.5 and keeps its name; an alias note follows the notice), so quoted notices and other tools' output don't count. Tool results are truncated from the start so the notice survives. Recheck the strings and the part shape after an OpenCode upgrade.
Expand All @@ -46,7 +46,7 @@
- **A slot's render body runs once, untracked** (`tui/src/plugin/render.tsx` at v2.2.0); its input stays reactive through getters. Read `input.sessionID` and `input.name` inside reactive functions, and choose the panel's content from `input.name` reactively. Opening a panel hides the sidebar, footer widget included.
- **Never `insert` children into `markdown`**, not even `[]`: it removes the blocks rendered from `content`, and the panel stays blank in the real TUI until `content` changes (ram-monitor's `/ram` modal hides it: its content changes once, when the async load finishes after the insert). `testRender` parses later, so the unit tests pass anyway; only a live check catches it.
- **Read `opencode api` output from a file, never a pipe.** The CLI calls `process.exit` right after writing (`packages/cli/src/index.ts`, 2.2.5), so a piped response of a few MB arrives cut short at a random length; 350 KB still arrives whole, so small test sessions hide it. `fetchMessages` in `scripts/timeline.ts` writes each page to a file.
- **`bun run build` starts with `rm -rf dist`.** A running OpenCode watches the local plugin, reloads mid-build, and drops the plugin (server and TUI) if `dist/` is missing or the build fails. Restart OpenCode after rebuilding.
- **`bun run build` starts with `rm -rf dist`.** A running OpenCode watches the local plugin, reloads mid-build, and drops the plugin (server and TUI) if `dist/` is missing or the build fails. Restart OpenCode after rebuilding. The root `bun run test` builds too (`turbo.json`: `test` depends on `build`), so it has the same effect; to check without touching `dist/`, run `bun test` in this package.
- **The TUI can't reach the host's markdown syntax style** or the `PanelInput` type. `markdownStyle` mirrors the markdown subset of `theme/src/tui/syntax.ts` from the public `context.theme.markdown` tokens, as ram-monitor does.
- **One `opencode.json` entry loads both halves**; `cli.json` needs none. The TUI gets `context.options = {}`, so configuration stays server-side.
- **Neither hook may throw into the user's request.** Failures are swallowed.
Expand Down
Loading
Loading