Skip to content

feat: allow switching provider/model mid-conversation#3799

Open
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation
Open

feat: allow switching provider/model mid-conversation#3799
Brechard wants to merge 3 commits into
pingdotgg:mainfrom
Brechard:pr/provider-switch-midconversation

Conversation

@Brechard

@Brechard Brechard commented Jul 8, 2026

Copy link
Copy Markdown

Closes #3797.

Heads up on scope: this is a large change (~1.1k lines) and overlaps the in-flight orchestrator-v2 work. I'm opening it for visibility per the linked issue and CONTRIBUTING's "issues first" — happy to close, split, or defer if it's not wanted as-is.

What changed

Allows switching provider/model mid-conversation, without resuming the old provider's session (which #2365 found doesn't work across providers).

  • On a cross-provider switch, ensureSessionForThread starts a fresh session on the target provider (dropping the old resume cursor) and persists the new modelSelection, instead of rejecting the switch.
  • providerHandoffTranscript renders the prior conversation — user/assistant messages plus a structured tool/command trail (commands, file edits, MCP/skill calls, exit codes, output) — and prepends it to the first turn on the new provider, so it inherits the actual work, not just prose.
  • An inline "Switched from X to Y" activity is rendered as a timeline notice above the triggering message.
  • The picker is unlocked on started threads; the in-thread block is kept only for providers that require a new thread for a model change.

Why this differs from prior attempts

Notes

  • Adds no DB migrations.
  • Same-driver model switches are unchanged; this only adds the cross-driver handoff path.
  • Verified end-to-end: a cross-provider switch carries context, including command output and non-zero exit codes the previous assistant never narrated.
  • Tradeoff: the new provider sees a rendered transcript, not native tool-call state, so per-turn rollback/checkpointing doesn't span the switch boundary.

Before / after

pr3799-provider-AFTER-switched-notice

Note

High Risk
Changes core turn-start/session binding and provider input shaping in orchestration; incorrect handoff detection or transcript handling could mis-route turns or leak stale resume state, though behavior is heavily covered by new tests.

Overview
Mid-conversation provider/model switches are allowed when the target instance uses a different driver or continuation key. The server no longer rejects those turns; it starts a fresh provider session (no old resumeCursor), persists the new modelSelection, and prepends a [Conversation handoff] transcript to the first turn on the new provider via new renderProviderHandoffPrelude (messages plus deduped tool/command/MCP trail, size-capped).

ProviderCommandReactor detects handoffs, skips in-session model-change rules for them, appends thread.model-changed activities after a successful (re)start, and resolves “current” instance from the bound session when needed. ProviderService tolerates transient binding mismatches during handoff and dedupes overlapping live sessions per thread.

The web UI unlocks the model picker on started threads, only blocks same-driver/same-continuation switches for requiresNewThreadForModelChange providers, and shows inline “Switched from X to Y” timeline notices from those activities.

Reviewed by Cursor Bugbot for commit 853c8d5. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Allow switching provider/model mid-conversation with transcript handoff

  • When a user switches to a provider with a different driver or continuation group, the orchestrator starts a fresh session on the new provider instead of rejecting the change, prepending a [Conversation handoff] transcript prelude to the next turn so the new provider has prior context.
  • A thread.model-changed activity is appended to the thread on handoff; the UI renders this as a pill-styled notice row in the conversation timeline via a new NoticeTimelineRow component.
  • listSessions in ProviderService no longer dies on provider/instance binding mismatches; it returns at most one session per thread, preferring the authoritative binding match.
  • The getStartedThreadModelChangeBlockReason logic is relaxed so cross-instance switches are allowed unless both instances share the same driver and continuation group.
  • Resume cursors are cleared when rebinding to a different provider instance to prevent stale cursor reuse across drivers.
  • Risk: the handoff prelude is capped at 80,000 chars and will silently omit older history when truncated, which may cause the new provider to miss earlier context.

Macroscope summarized 853c8d5.

Threads were pinned to their initial provider once started: the server
rejected cross-driver/continuation switches and the UI locked the picker.
This lets a thread hand off to a different provider mid-conversation.

- Server: replace the switch-rejection guards in ensureSessionForThread
  with a handoff path that starts a fresh session on the new provider
  (dropping the old resume cursor), persists the new modelSelection, and
  tolerates the transient binding mismatch during rebind.
- Handoff context: new providerHandoffTranscript renderer replays prior
  user/assistant messages plus a structured tool/command trail (commands,
  file edits, MCP/skill calls, exit codes, output) into the first turn on
  the new provider, so it inherits the actual work done, not just prose.
- Notice: emit a thread.model-changed activity on model/provider change,
  rendered as an inline "Switched from X to Y" timeline row above the
  triggering user message.
- UI: unlock the model picker on started threads; keep the in-thread
  block only for providers that require a new thread for a model change.

Adds no DB migrations. Server + web suites, typecheck, and lint pass;
verified end-to-end (cross-provider switch carries context, incl. hidden
command output and non-zero exit codes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: da904db3-b1f6-4b05-b0f9-69c5e67e6d8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 8, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b747679. Configure here.

Comment thread apps/server/src/orchestration/Layers/ProviderCommandReactor.ts
Comment thread apps/server/src/orchestration/Layers/ProviderCommandReactor.ts
Comment thread apps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment thread apps/web/src/components/ChatView.logic.ts Outdated
Comment thread apps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment thread apps/server/src/orchestration/providerHandoffTranscript.ts
Comment thread apps/server/src/provider/Layers/ProviderService.ts Outdated
Comment thread apps/server/src/orchestration/providerHandoffTranscript.ts
Comment thread apps/server/src/orchestration/Layers/ProviderCommandReactor.ts Outdated
Comment thread apps/server/src/orchestration/providerHandoffTranscript.ts
@macroscopeapp

macroscopeapp Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This PR introduces a significant new feature enabling provider/model switching mid-conversation, changing previously blocked behavior to a complex handoff flow. The changes span orchestration, session management, and UI layers (~1300 additions), introducing new runtime behavior that warrants human review.

You can customize Macroscope's approvability policy. Learn more.

From Cursor Bugbot + Macroscope on the PR:
- Handoff replay no longer requires a prior assistant message; the renderer
  already returns undefined when there's nothing to carry, so user-only /
  tool-only threads keep their context on switch.
- Handoff detection prefers the last-bound session's provider over the
  thread's stored modelSelection, so a switch is still detected after the
  session is stopped/restarted (or the selection was advanced early).
- The "Switched model" notice is emitted only after the session is
  confirmed (re)started, not before — a failed start no longer leaves a
  stale success notice.
- Transcript extraction also reads stderr, data.rawOutput.exitCode, and
  data.toolCallId (dedup key), so failed-command diagnostics and exit codes
  survive the handoff and lifecycle events collapse correctly.
- listSessions dedupes by threadId, preferring the binding-matched session,
  so a mid-handoff stale session can't produce two entries for one thread.
- getStartedThreadModelChangeBlockReason only treats a cross-instance switch
  as an unblocked handoff when the driver/continuation actually differ;
  same-driver+continuation instances stay blocked for providers that
  require a new thread.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread apps/server/src/orchestration/providerHandoffTranscript.ts
collectToolTrailEntries kept the longest line per tool-call group, so a
verbose in-progress tool.updated could shadow the shorter tool.completed/
task.completed line that actually carries the exit code and final output.
The next provider then inherited stale partial tool state. Prefer the
terminal lifecycle event, falling back to longest line only on ties.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 333d44eeebb21d41f0f6d18af7153f52727c77a2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL 500-999 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Switch provider/model mid-conversation via transcript handoff

2 participants