feat: allow switching provider/model mid-conversation#3799
Conversation
Threads were pinned to their initial provider once started: the server rejected cross-driver/continuation switches and the UI locked the picker. This lets a thread hand off to a different provider mid-conversation. - Server: replace the switch-rejection guards in ensureSessionForThread with a handoff path that starts a fresh session on the new provider (dropping the old resume cursor), persists the new modelSelection, and tolerates the transient binding mismatch during rebind. - Handoff context: new providerHandoffTranscript renderer replays prior user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) into the first turn on the new provider, so it inherits the actual work done, not just prose. - Notice: emit a thread.model-changed activity on model/provider change, rendered as an inline "Switched from X to Y" timeline row above the triggering user message. - UI: unlock the model picker on started threads; keep the in-thread block only for providers that require a new thread for a model change. Adds no DB migrations. Server + web suites, typecheck, and lint pass; verified end-to-end (cross-provider switch carries context, incl. hidden command output and non-zero exit codes). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b747679. Configure here.
ApprovabilityVerdict: Needs human review This PR introduces a significant new feature enabling provider/model switching mid-conversation, changing previously blocked behavior to a complex handoff flow. The changes span orchestration, session management, and UI layers (~1300 additions), introducing new runtime behavior that warrants human review. You can customize Macroscope's approvability policy. Learn more. |
From Cursor Bugbot + Macroscope on the PR: - Handoff replay no longer requires a prior assistant message; the renderer already returns undefined when there's nothing to carry, so user-only / tool-only threads keep their context on switch. - Handoff detection prefers the last-bound session's provider over the thread's stored modelSelection, so a switch is still detected after the session is stopped/restarted (or the selection was advanced early). - The "Switched model" notice is emitted only after the session is confirmed (re)started, not before — a failed start no longer leaves a stale success notice. - Transcript extraction also reads stderr, data.rawOutput.exitCode, and data.toolCallId (dedup key), so failed-command diagnostics and exit codes survive the handoff and lifecycle events collapse correctly. - listSessions dedupes by threadId, preferring the binding-matched session, so a mid-handoff stale session can't produce two entries for one thread. - getStartedThreadModelChangeBlockReason only treats a cross-instance switch as an unblocked handoff when the driver/continuation actually differ; same-driver+continuation instances stay blocked for providers that require a new thread. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
collectToolTrailEntries kept the longest line per tool-call group, so a verbose in-progress tool.updated could shadow the shorter tool.completed/ task.completed line that actually carries the exit code and final output. The next provider then inherited stale partial tool state. Prefer the terminal lifecycle event, falling back to longest line only on ties. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 333d44eeebb21d41f0f6d18af7153f52727c77a2)

Closes #3797.
What changed
Allows switching provider/model mid-conversation, without resuming the old provider's session (which #2365 found doesn't work across providers).
ensureSessionForThreadstarts a fresh session on the target provider (dropping the old resume cursor) and persists the newmodelSelection, instead of rejecting the switch.providerHandoffTranscriptrenders the prior conversation — user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) — and prepends it to the first turn on the new provider, so it inherits the actual work, not just prose.Why this differs from prior attempts
Notes
Before / after
Note
High Risk
Changes core turn-start/session binding and provider input shaping in orchestration; incorrect handoff detection or transcript handling could mis-route turns or leak stale resume state, though behavior is heavily covered by new tests.
Overview
Mid-conversation provider/model switches are allowed when the target instance uses a different driver or continuation key. The server no longer rejects those turns; it starts a fresh provider session (no old
resumeCursor), persists the newmodelSelection, and prepends a[Conversation handoff]transcript to the first turn on the new provider via newrenderProviderHandoffPrelude(messages plus deduped tool/command/MCP trail, size-capped).ProviderCommandReactordetects handoffs, skips in-session model-change rules for them, appendsthread.model-changedactivities after a successful (re)start, and resolves “current” instance from the bound session when needed.ProviderServicetolerates transient binding mismatches during handoff and dedupes overlapping live sessions per thread.The web UI unlocks the model picker on started threads, only blocks same-driver/same-continuation switches for
requiresNewThreadForModelChangeproviders, and shows inline “Switched from X to Y” timeline notices from those activities.Reviewed by Cursor Bugbot for commit 853c8d5. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Allow switching provider/model mid-conversation with transcript handoff
[Conversation handoff]transcript prelude to the next turn so the new provider has prior context.thread.model-changedactivity is appended to the thread on handoff; the UI renders this as a pill-styled notice row in the conversation timeline via a newNoticeTimelineRowcomponent.listSessionsinProviderServiceno longer dies on provider/instance binding mismatches; it returns at most one session per thread, preferring the authoritative binding match.getStartedThreadModelChangeBlockReasonlogic is relaxed so cross-instance switches are allowed unless both instances share the same driver and continuation group.Macroscope summarized 853c8d5.