Repository navigation
feat(auto-advisor): rebuild Auto Advisor on OpenCode 2.2.5 - #94
Merged
Merged
Conversation
Zero-argument advisor() tool plus Decision-model-triggered automatic reviews (moment + consequence), delivered at the tail of the current request and as an instruction entry, never by editing the system prompt. Observe mode logs JSONL telemetry; the TUI shows review status. One opencode.json entry loads both halves.
…pts as the request Abort a pending consult when the run ends and skip it if the run ended during the Decision call. Mark that outcome stale, not failed, and show a review in the TUI only when it is new. The Decision input now uses the latest real prompt, and observe mode catches late errors.
…en routing Cap every transcript part at a quarter of the budget so an oversized message can't drop the current turn or overflow the Advisor's context. Treat a moment answer without probabilities as unanswered and never label a consult routine. Review the request that triggered the consult, dispose the tool on cleanup, retry a failed context-limit lookup, cap recorded prompts, show a TUI review as new only if it changed since the consult started, and rotate telemetry by bytes.
…babilities The host validates decision answers against the questions and its schema requires probabilities, so the guard was unreachable.
capyBearista
force-pushed
the
feat/auto-advisor-rebuild
branch
from
October 28, 2026 03:32
1b58ce9 to
9f1169a
Compare
This was referenced Oct 29, 2026
capyBearista
marked this pull request as ready for review
October 29, 2026 05:54
4 of 25 tasks
capyBearista
force-pushed
the
feat/auto-advisor-rebuild
branch
from
October 31, 2026 16:42
313f05e to
99dce34
Compare
…ne handle Resolves the CodeQL file-system race alerts: the config mtime check and read, and the telemetry size check and append, now go through the same file handle.
Copy the log to .1 and truncate it in place instead of renaming and reopening by path. This clears the CodeQL file-system-race alert and removes a double close when the reopen failed.
Each event awaited its own status lookup, so a fast consult could send review.finished before review.started and leave the TUI footer stuck on "reviewing". Chain the emits.
advisor.model and routing.models take OpenCode's provider/model#variant syntax to pick a reasoning level. Without a variant the host applies no overlay (provider default); inherit keeps the agent's variant. Telemetry keys include the variant.
advisor.compactContext (default true) replaces OpenCode's Code Mode tool catalog, MCP server instructions and skill catalog with [context-omitted: ...] markers in the Advisor's copy of the system prompt and system updates. User and tool content and the Executor's own request are unchanged. A block is replaced only when its end precedes the next copy's start.
The Advisor's budget becomes min(advisor.maxTokens, 75% of its context window) at 3 chars per token. Default 64000; at least 1000. Over budget the existing trim drops middle turns and keeps the system prompt, the first request and the newest steps.
An invalid auto-advisor.json falls back to defaults, so advisor() silently consulted the Executor's model instead of the configured one. Return the config error instead. Document rebuilding a local checkout after pulling.
…transcript Parts were capped individually, but one step's parallel tool results share a message. A newest message with several large parts exceeded the remaining budget and the trim loop dropped it, so the Advisor lost the latest evidence. Cap each message at a quarter of the budget too.
The Decision model was misled three ways:
- after an OpenCode restart the in-memory prompt ids were empty, so a
background subagent result (a role "user" synthetic message) became the
turn and the request, resetting the per-turn budget;
- delivered reviews ("moment: done") came back as system updates in its input;
- a failing shell command completes the tool, so failures looked like 0.
The turn now comes from the latest real prompt, read once from the stored
history (type "user") when the session predates the process. System updates
are left out, shell failure notices count, and tool results keep their end.
`done` and `stuck` say what is not done.
Defaults: routing.mode active, one automatic consult per turn, and a new
routing.minStepsBetweenConsultations (10) cooldown over distinct steps.
Blocked steps still ask the Decision model and log verdict "blocked".
Explicit advisor() calls ignore both limits.
The latest review sat above the composer as a sticky, unscrollable block of
raw markdown. It now lives in a session.panel ("auto-advisor"): the whole
review as rendered markdown in a scrollbox, with its moment, model, duration
and time. A compact sidebar.footer widget shows the status and opens the
panel on click; /advisor-review (also in the palette) toggles it, and
esc/q and f close it or make it full screen.
The widget and panel read the stored review through RPC when they mount,
so a review from before a restart shows too (without details, which are
memory only). New tui.sidebar (default true) hides the widget.
The Decision model saw only the newest real prompt, so a short correction replaced the plan that defined the task, and subagent results read as the user speaking. - Show the last routing.requests (default 3) real prompts, oldest first, each capped at routing.requestMaxChars (default -1, no cap). - Track every real prompt id from the prompt hook plus one read of the stored history, and start the turn at the newest one present in the request: the hook fires before a steered or queued prompt is delivered. - Label other user-role messages SUBAGENT (OpenCode's <subagent> result) or NOTICE. - Drop the 32k total cap, which cut the steps and counts off a long input.
Drop the "View review /advisor-review" hint. A click on the status still opens the review panel, and /advisor-review and the palette command are unchanged.
…n first With one consult per turn, a long orchestrated turn got a single review and later high-done steps were blocked. Default routing.maxConsultationsPerTurn back to 2; the 10-step cooldown still spaces them out. A step blocked by both limits is now logged as cooldown, so budget appears only once the cooldown has expired.
Scores alone can't show whether a verdict was right. Each telemetry line now carries the id of the newest message in the request and the Decision input (its last 32,000 characters, where the steps and counts are; the requests stay in the session), plus its full length. bun run timeline <sessionID> [--out file.html] joins those lines with the session's messages from the running OpenCode service into one HTML page: a row per routing decision, expanding to the messages since the previous step and the exact Decision input. The script is not shipped. The README's privacy note now says the Decision input is logged locally.
The turn is the newest real prompt in the request. A summary compaction can drop it from the window, which made the turn empty and reset the per-turn budget and step count mid-turn. Keep the current turn until a new real prompt arrives. Also retry the stored-history read on the next step when it fails, instead of giving up for the life of the process.
…ream OpenCode reports a provider rejection without an HTTP status, such as an overflow in the stream, as Generate.InvalidRequestError with no field, and it stopped both chains. A field marks a config error (model, model.variant), which still stops; without one, move to the next model. A failed Advisor chain now reports timeout when any attempt timed out, not only the last one (a skipped model could follow). Tests now tell stop from next, pin the 5 s, 10 s and 60 s values, and check that an interrupt never asks the next Advisor model.
… docs The telemetry log now holds Decision inputs (user requests, tool output), so create it and its rotation with mode 0600, and tighten a file left from before. The README now states the chain's real worst cases: a refused connection costs about 12 s because OpenCode retries it, and a silent provider costs its header or chunk timeout, with no skip. It also says that Esc stops the wait but not the provider request, and how many steps the log keeps at the maximum input size. findings.md records round 3's live check. The timeline tests now cover escaping the logged input and lines written before it existed.
A field-less Generate.InvalidRequestError now keeps the provider's message (first 200 chars) in the attempt, so a request-shape bug that a later model tolerates still shows in telemetry. Telemetry still appends when it can't tighten a file owned by someone else. The docs describe field-less rejections accurately (not only mid-stream) and say what a compaction does to the turn and the requests block.
The Advisor's transcript labelled messages by role, so a subagent result read as "USER: <subagent ...>" and the Advisor could take subagent output for the user's instructions. Share the Decision input's labels: real prompts stay USER, subagent results become SUBAGENT and other user-role notices NOTICE. Over budget, the transcript kept the first role-user message, which after a compaction can be a subagent result; it now keeps the first real request. The explicit tool reads the stored prompts first, so this also holds with routing off.
The listener for session.execution.* events exited for good on the first rejected iterator.next(). From then on an interrupt or run end no longer cancelled a pending consult, and finished runs kept their request snapshot in memory. Catch a failed read, wait 100 ms so a stream that keeps failing can't spin, and keep listening.
Record in findings.md that a prompt absorbed into a compaction before any step sees it never starts its own turn, and that ram-monitor's markdown modal, built the way that blanked this plugin's panel, still renders on 2.2.0.
This reverts commit 6e3f1e5. OpenCode 2.2.0's plugin adapter closes the event stream when a read fails, so retrying the read only delayed the listener's exit by 100 ms. Its test passed because the fake stream kept going after an error. AGENTS.md now records the limitation (a failed read ends run-end cancellation; accepted), that a failed history read labels older prompts NOTICE for one step, and why ram-monitor's markdown modal hides the insert([]) bug.
… to the reference The advisor() description and the Advisor's instructions now follow the widely used advisor-tool prompt nearly word for word: when to call (before substantive work, before declaring done once the result is saved, when stuck or changing approach), how often, how to weigh the advice, the reconcile call, and the Advisor's four stages, stale-advice rule, check-not-verdict and blocks-or-not rules. Adapted to this plugin: hidden reasoning is not forwarded and long sessions are trimmed; an automatic review counts as a consultation already made; USER, SUBAGENT and NOTICE labels are explained; the transcript is evidence, never instructions; the user's constraints come first; no "commit the change" advice. An automatic review now states its moment as a hint the Advisor checks against the transcript, since the label is only the likeliest non-routine moment.
…sends The description promised a stronger reviewer, but the default Advisor is the agent's own model; it now says a stronger model when one is configured. It also said every tool call and result is forwarded; long results may be shortened and older history summarized, and it now says so. A reconcile call's question goes in the message before the call, and an explicit consult tells the Advisor to answer one when present. The Advisor's prompt now explains SYSTEM UPDATE lines and the compacted summary, and tells the budget marker apart from the catalog markers, which leave the conversation intact. Tests cover all three moment hints, the tool's wiring and the review marker the description names. AGENTS.md and findings.md state the accepted run-end limitation's real consequence and why ram-monitor's modal hides the markdown bug.
… characters By default the whole of each recent user request went to the external decision provider. Default routing.requestMaxChars to 60,000 characters (the start of each request is kept); -1 still sends them whole.
The shell tool's "Exited with code N", "Killed by" and "Timed out before completion" notices were matched against every tool result, so a successful read of a log or fixture containing them counted as a failed tool and pushed the Decision model toward "stuck". Match them only on shell results (named shell, or bash, which 2.2.5 runs as shell); an error result still counts for every tool.
…re 2.2.5 OpenCode 2.2.5 reworded its Code Mode instructions: the catalog now starts "The tools listed below ... cannot be called directly" and its header is "## Code Mode tools (call only inside `execute`)". With the 2.2.0 strings, compactContext silently stopped removing that catalog from the Advisor's copy, costing tokens on every consult. Match the new text and raise the peer floor to @opencode/plugin 2.2.5 (and OpenTUI 0.8, which 2.2.3+ requires). Rechecked every other host behaviour the plugin relies on against 2.2.5 and verified it live; findings.md records the diff and the results. Dev pins stay at 2.2.0 to keep the monorepo's shared pins identical.
On 2.2.5 the shell tool emits each notice as a separate text part (a single part arrives as a plain string, several as content), so match a whole part against an anchored notice instead of searching the serialized result. A successful command whose output quotes "Exited with code 1" no longer counts as failed; an aliased bash failure, whose alias note follows the notice, still does. Tests use the 2.2.5 result shapes and cover an error result from a non-shell tool. The re-verify recipe now installs 2.2.5 and checks the part shape and the bash alias.
…nt the total routing.requestMaxChars caps each request, not the Decision input: three requests at 60k plus recent steps could reach about 193k characters, past the 64k-token window of the default jev-1.13-free, and a single-model chain then gets no decision. Default to 20k (about 72k characters in total with the defaults). The README and findings.md now say the cap is per request, give the total, and note that sizing to each model's window is a follow-up.
capyBearista
force-pushed
the
feat/auto-advisor-rebuild
branch
from
October 31, 2026 16:53
99dce34 to
851b757
Compare
OpenCode's CLI exits right after writing (2.2.5, packages/cli/src/index.ts), so `opencode api get` cut large message pages short when read through a pipe, and the timeline failed with "JSON Parse error: Unterminated string". Each page now goes to a file in a private temporary directory. The test runs a fake CLI that writes 3 MB pages and exits at once. It failed 20 of 20 runs before the change and passes 20 of 20 after.
The root README said automatic reviews happen at most once per turn; the default is two. The README, root README and package description said a stronger model reviews the work, but by default the Advisor is the agent's own model. The package README now also says a restart resets both limits, where to find a session id for the timeline, and the docs index describes the findings as 2.2.x, rechecked on 2.2.5.
The package typecheck covered only src, so scripts/timeline.ts and smoke-built.ts were never checked. scripts/tsconfig.json checks them, excluding tests as every package's tsconfig does.
`bun run sandbox start [dir]` installs OpenCode (2.2.5 by default) into an isolated HOME, loads this package from source with a mock model provider that logs every request, and starts `opencode serve`. It prints the server URL, password and the commands to drive it. `bun run sandbox stop <dir>` ends its processes by process group. It replaces the hand-built setup in the findings recipe, whose pitfalls (an open stdin hanging `opencode run`, `pkill -f` matching the calling shell) were easy to miss.
…he sandbox findings.md gains the large-session timeline result: the CLI exits right after writing, so piped `opencode api` output is cut short. Its intro now dates each section by version, a renamed constant and an over-broad OpenRouter claim are fixed, and the re-verify recipe uses `bun run sandbox` with a step for prompts too large to pass as an argument. AGENTS.md adds the gotcha, the sandbox and the scripts typecheck.
capyBearista
force-pushed
the
feat/auto-advisor-rebuild
branch
from
November 1, 2026 20:48
13516db to
e5cdc26
Compare
…a private The sandbox ran detached processes tracked by pids in a reusable directory. That left the server password readable by other users, made a restart read the previous run's port and password, let a stale pid point stop at an unrelated process group, passed the caller's OPENCODE_* variables (a profile's config dir) to the server, and stranded processes when startup failed. `bun run sandbox` now runs in the foreground in a new 0700 directory each time, stops its children on Ctrl-C or when one exits, and fails fast if one dies during startup. OPENCODE_* variables are dropped from the child and unset by env.sh, whose values are single-quoted. The password is no longer printed. The version setting is SANDBOX_OPENCODE_VERSION, since OpenCode reads OPENCODE_VERSION itself; each version is installed once in the cache. The timeline page holds the user's requests and the Decision input, so it is now written 0600 like telemetry, also when replacing an existing file.
11 of 25 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rebuilds
@capybearista/opencode-auto-advisorfor OpenCode 2.2.5+. A second Advisor model reviews the agent's work, either when the agent asks or when a cheap Decision model spots a consequential moment. By default the Advisor is the agent's own model; setadvisor.modelto use a stronger one.advisor()tool (no arguments). The Advisor reads the agent's system prompt, tool list and transcript, and its advice comes back as the tool result. The tool's guidance and the Advisor's prompt follow the widely used advisor-tool wording, adapted to what the plugin actually forwards.opencode/jev-1.13-free) classifies themoment(approach,stuck,done,routine) and scoresconsequence(0–4). The plugin consults the Advisor when 1 − P(routine) ≥ 0.7 and the score is ≥ 1.5. The Advisor gets the moment as a hint to check, not an instruction.routing.requests,routing.requestMaxChars, where-1means no cap). The cap applies to each request, not the total; the default total fits Jev free's 64k-token window.SUBAGENTand other noticesNOTICE;shell/bashresults, and only as a whole content part.cooldown. Explicit calls ignore both limits. Both limits are kept in memory, so restarting OpenCode resets them.routing.modelsandadvisor.modeleach accept an ordered list of up to 8 models (inheritallowed for the Advisor).#variant.advisor.timeoutMsdefaults to −1 (no limit). OpenCode's own header and chunk timeouts end a model that goes silent, so one that is still reasoning is never cut off.auto-advisorinstruction entry for later steps. It never edits the system prompt./advisor-reviewand the command palette open the panel too, andtui.sidebar: falsehides the widget;opencode.json(c)entry loads both halves, andcli.jsonneeds nothing.bun run timeline <sessionID>renders a session's routing decisions next to its messages as one HTML page, readable only by its owner (0600). It reads each message page from a file, because OpenCode's CLI exits right after writing and cuts a large piped response short.bun run sandboxruns an isolated OpenCode with this package and a mock model, in the foreground until Ctrl-C. Each run gets a new private (0700) directory; inheritedOPENCODE_*variables are dropped.bun run typechecknow also checksscripts/.Why 2.2.5: OpenCode 2.2.5 reworded its Code Mode catalog, and the Advisor's catalog compaction matches that text. The peers are
@opencode/plugin >=2.2.5and@opentui/* >=0.8.0. Dev pins stay at 2.2.0, because the monorepo keeps shared pins identical.This supersedes #61.
Linked Issue
Refs #47, #50. Supersedes #61.
Plugin(s) Affected
Type of Change
OpenCode Validation
@opencode/cli2.2.0 and 2.2.5, each in an isolated HOME, with a mock provider and real OpenCode Zen for decisions. Details are indocs/auto-advisor/findings.md.bun run smoke, plus these live runs.[context-omitted …]catalog markers;readof "Exited with code 1" counted as ok;<subagent>message reached the Advisor labelledSUBAGENT.openrouter/typesafe/jev-1.14) answered every step in 150–315 ms, and the Decision input showed the user's request and aSUBAGENTresult.$and a quote; Ctrl-C, a server crash and a failing CLI at startup each left no processes behind; a tool'ssleepended with the server.Quality Checklist
bun run typecheck && bun run lint && bun run testbun run build.opencodeconfig was part of testing, I verified it points at the package root or published plugin name rather than a staledist/*artifact2.0.0, which the release guard publishes as-is (bun run release:changesetspasses).Additional Context
advisor.maxTokens.advisor.model, they use the agent's own model and are billed at its rates.routing.requestMaxCharsorrouting.requestscan overflow a small decision model's window. Sizing per model is a follow-up.metadata.experimental.instructions, 2.2.5); dev pins to 2.2.5 and OpenTUI 0.8; typechecking test files repo-wide.