Skip to content

fix(models): preserve and configure per-model reasoning efforts - #1026

Merged
crispinhu27 merged 14 commits into
mainfrom
feature/model-reasoning-effort
Nov 18, 2026
Merged

crispinhu27 merged 14 commits into
mainfrom
feature/model-reasoning-effort

Conversation

@crispinhu27

@crispinhu27 crispinhu27 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

Fetching available models could return only IDs and names, leaving reasoning levels missing or lost when the personal catalog was saved or refreshed. Models now carry reasoning levels through discovery, persistence, chat validation, and role selection, with a per-model editor for automatic or custom levels.

  • Read advertised reasoningEfforts / supported_reasoning_efforts; use the existing CLI catalog when metadata is absent and restrict choices to levels the CLI supports.
  • Preserve explicit overrides and Cursor's effort-to-model-ID mappings. Save capability-only changes when model IDs and names remain the same.
  • Keep model and effort pickers synchronized, including delayed catalog loads, the legacy executor catalog endpoint, and the claude-code alias.
  • Add GPT-6.2 Sol and GPT-6.2 Astra to the built-in Codex model list used by chat and the role editor. Codex model documentation.
  • Add Claude Fable 5.2 (claude-fable-5-2) to the built-in Claude Code model list. Claude Code model documentation.
  • Restore the last CLI, model, and reasoning effort from browser storage. Save approval modes, including full permissions, for the signed-in account in that browser.
  • Hide the CodeBuddy CLI default from model pickers. New chats restore an available saved model or select the first listed model. CodeBuddy uses its CLI default when the list is empty.
  • Refresh discovered effort levels independently of explicit overrides, including default-only overrides. Turn validation can rediscover capabilities on a cold API instance.
  • Keep a saved role's effort when editing other fields, including when the editor's personal catalog offers a different set of efforts.
  • Refresh model and effort choices after saving CLI credentials. Cursor's account catalog updates in the existing chat when its background probe completes.
  • Bind browser turn approval to the authenticated account. Reject submissions from a stale tab using another account's preference, and carry the approval account through queued messages.
  • Validate shared CodeBuddy turns against the shared model capabilities after a personal key is cleared.
  • Use the Codex model catalog as the fallback for each model's reasoning levels. GPT-5.5 offers xhigh and the supported levels for newer models can include max or ultra. Explicit preferences and discovered capabilities take precedence.
  • Use Claude Code's per-model effort table when explicit and discovered settings are absent. Fable 5.2 offers xhigh and max, with on-demand installation pinned to Claude Code 2.1.321. Claude Code model configuration.
  • Apply per-model effort controls for Kimi Code, DeepSeek Harness, and ZCode on new and resumed turns. The Kimi K2.7 Highspeed picker offers Default, Off, and On. Off follows Kimi's documented routing to K2.8 Preview. Manual Kimi controls use its native provider.

Configuration behavior references DeepSeek Harness's per-model reasoning editor.

Scope

  • Platform core / API / runner
  • Public adapter / deployment
  • Classic
  • Portable
  • Shared behavior contract
  • Installer / CLI
  • Documentation only

Compatibility

reasoningEfforts is an optional field inside existing model-catalog JSON. Omission keeps the previous automatic behavior; an explicit empty list permits only the model's native default. Existing Cursor mappings remain independent. No database schema migration is required.

CLI and model preferences reuse the browser keys from main. Per-model effort preferences use the same browser storage. Approval preferences use account-specific browser keys, with confirmation for accounts that have not made a selection. Existing chats use their recorded model and effort. The CodeBuddy execution default is part of the catalog metadata.

discoveredReasoningEfforts stores automatic capabilities in the existing catalog JSON. reasoningEfforts stores explicit overrides.

Validation

Passed:

  • Focused Go model, effort, catalog, credential, Codex, Cursor, Agent CLI, and embedded-web regressions in internal/httpapi, internal/repo, internal/runtime, and web.
  • Chatbot tests, the new effort regression tests, and npm --prefix chatbot run build.
  • Web tests for model catalogs, credentials, runner model editors, and role pickers, including capability-only saves and delayed capabilities preserving unsaved edits.
  • go vet ./... and python3 scripts/check-public.py.
  • Gitleaks 8.30.1 scanned an export of the candidate files with .gitleaks.toml and found no leaks. Used the checksum-verified official macOS ARM64 binary because the repository script downloads a Linux executable.
  • TestProviderUserCredentials against a disposable local Docker MySQL 8.4 database, including explicit empty reasoning levels.
  • Browser acceptance against a disposable local platform and model-list fixture: discovery, save/reload, chat and role catalogs, desktop/mobile light and dark themes, keyboard interaction, reduced motion, and no page errors or horizontal overflow.
  • Built-in catalog regression verifies the GPT-6.2 Sol, GPT-6.2 Astra, and Claude Fable 5.2 model IDs and display names. The chat catalog, personal catalog, model probe, and effort regressions passed on the updated branch.
  • Composer preferences passed all 389 chatbot tests, the production build, the full web package, and focused HTTP API regressions. Browser checks covered desktop and narrow layouts, reloads across a login change, fresh and existing chats, and the model, effort, and approval mode in intercepted generation requests.
  • Review follow-up passed local chatbot, role-editor, catalog, and cold-discovery regressions. The sponsor default regression and all four executor browser cases passed on a fresh local platform backed by Docker MySQL with an unconfigured CodeBuddy catalog.
  • Verified a live Cursor key against the local platform and Docker MySQL. Discovery populated 75 models. Grok 4.7 accepted the xhigh selection in the original chat frame, and the intercepted generation request carried the selected level. The 392 chatbot tests, production build, Go web suite, and focused Go regressions for account binding, shared CodeBuddy, and the Codex 6.2 catalogs passed. In Chromium, a stale tab retained its old display after a second tab changed the authenticated account. The generation request received HTTP 409 before executing.
  • Verified the first available CodeBuddy selection and restoration of main's browser keys. The 393 chatbot tests, frontend build, orchestration and schedule regressions, Go web tests, and desktop and narrow Chromium checks passed.
  • Built-in Codex reasoning regressions cover picker responses, model probes, overrides, discovery, session writes, and turn validation. Focused Go tests and vet passed for the HTTP API, runtime, and domain packages.
  • Claude Code 2.1.321 sent both xhigh and max for Fable 5.2 to a local fixture gateway. The Claude Code native gateway tests, focused Go regressions, and go vet ./... passed.
  • Verified the Kimi thinking switch and selected effort on resumed turns with Kimi Code 2.6.3, DeepSeek Harness 0.2.2-rc.1, and ZCode 0.17.1 against local fixture gateways. The 394 chatbot tests, 212 web tests, production build, focused Go regressions, and go vet ./... passed.

The full go test -p=2 ./... -count=1 -timeout=180s run was attempted and is not green locally: macOS /proc-dependent deployment tests fail, a systemd notification socket test exceeds the macOS socket-path limit (passes when rerun with TMPDIR=/tmp), the host's safe.bareRepository=explicit setting rejects Git fixtures, engine gate tests time out, and the full HTTP API package exceeds the 180-second limit. Runner/runtime Git checkout and index-lock fixtures also fail on this host. Focused regressions listed above pass; the PR's Linux CI will provide the full-suite result.

The full public browser suite and paid-provider live tests were not run. Local-edition checks are not applicable to this platform change.

Safety

  • I did not include credentials, internal endpoints, production data, or organization-specific infrastructure.
  • I preserved user files and unrelated changes.
  • I updated generated Classic host bundles when applicable. (Not applicable.)
  • I documented third-party content and licensing changes when applicable. (Behavioral reference linked above; no third-party code or assets imported.)

@Robert0622 Robert0622 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed b4bacef7. The live test used CodeBuddy Code 2.168.0 and an isolated LoopForge instance. It covered iOA model discovery, saving and reloading the personal effort settings, the chat and role catalogs, rejecting an unavailable effort, and one successful deepseek-v4.1-flash generation with low. The saved conversation retained both the reply and the effort.

The chatbot suite passed 389 tests and the production build passed. The existing web test suite passed. I reproduced TestOwnKeyKeepsItsEnvironmentDefaultWhileAKeyIsShared locally. The same test fails in the platform core job, and three executor-picker cases fail in the browser job, including a CodeBuddy catalog response with zero models on the unconfigured fixture.

Comment thread chatbot/src/PermissionPicker.jsx Outdated
} catch {
// Without storage the pick still applies to this page.
}
if (validApprovalMode(value)) writeComposerPreference("approvalMode", value);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Scope the full-permission choice to the signed-in account

auto is now saved in browser storage under an origin-wide key. After one account selects Full permissions and signs out, another account using that browser starts its next chat in auto and submits the turn with its own credentials. Platform and external writes can then run without that account opting in. Key this setting by the authenticated user and default to confirmation until that user makes a choice.

Comment thread internal/httpapi/assistant_models.go Outdated
}
catalog.Models = selectableCodeBuddyModels(catalog.Models)
if catalog.Default == assistantDefaultModel && len(catalog.Models) > 0 {
catalog.Default = catalog.Models[0].ID

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve the execution default when hiding the picker entry

With the platform default unset and no personal model list or remembered choice, real iOA discovery made claude-sonnet-5 the new catalog default. The composer also picks the first concrete model and sends its ID explicitly, so users with a fresh browser or a saved default choice can start spending on a different model without selecting it. Requests that omit model still take the original CLI-default path. Resolve the default separately from the options shown in the picker.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Product follow-up on this thread. A potentially higher-cost first model is acceptable as the initial selection when the composer shows its concrete name before sending and the user can switch. With that policy, I’m withdrawing the P1.

preferredModel on 7c23f96 still falls back to the CLI default when a valid preference is absent, even with concrete models available. For a new CodeBuddy chat, use the remembered model for that CLI if it is still selectable. Otherwise preselect the first selectable concrete model, show its name, and submit that ID.

Keep backend support for stored conversations and API clients that send default or omit model. Those compatibility paths can still handle requests with no concrete model.

Comment thread web/app/workbench.js Outdated
kept.add(id);
if(picked.has(id)&&discovered.has(id)){
const current=credentialCatalogRowModel(row),fresh=discovered.get(id);
row.outerHTML=credentialCatalogRowHTML({...current,...fresh,name:current.name,reasoningEfforts:current.reasoningEfforts??fresh.reasoningEfforts},catalog.dataset.catalogEngine);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Distinguish discovered capabilities from a user override

I imported a model whose provider advertised ["low"] and saved without touching the effort controls. The next probe of the same ID advertised ["high"], but the editor retained ["low"] because the first probe had already populated current.reasoningEfforts. This makes capability-only refreshes ineffective for previously discovered models. Track the discovered levels separately so a refresh updates them while preserving deliberate overrides.

Comment thread internal/httpapi/assistant_effort.go Outdated
if len(catalog) == 0 {
key, baseURL := assistantEngineCredential(user, engine)
anthropic := assistantEngineAnthropicModels(engine, assistantEngineProtocol(user, engine))
if cached, ok := assistantDiscoveryCache.Load(assistantDiscoveryKey(anthropic, key, baseURL)); ok {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Validate automatic efforts on a cold API process

A custom Codex gateway can advertise xhigh for a model with no saved override. I reproduced the effort being offered and accepted with a warm discovery cache, then rejected by the same validation after clearing that process-local cache. A restart or another API replica can make the generation return HTTP 400 for the effort just offered by the picker. Resolve automatic capabilities through the same discovery path when validating a turn.

Comment thread web/app/agents.js Outdated
const items=agentEffortItems(currentAgentExecutorId());
const current=picker.querySelector('[name="effort"]').value;
const field=picker.querySelector('[name="effort"]');
if(executorModelCatalogs()[currentAgentExecutorId()]&&!items.some(item=>item.id===field.value))field.value='';

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Keep a saved role effort during unrelated edits

The arriving personal catalog can clear the role's stored effort as soon as the editor opens. I reproduced opening a role saved with xhigh while the editor's personal catalog allowed only the model default, changing the role name, and getting an empty effort in the save request. A personal catalog should constrain new selections without silently rewriting a shared role's saved setting.

@Robert0622 Robert0622 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed f4e1c9f. The earlier CodeBuddy default, catalog refresh, cold discovery, and role-edit cases pass on this revision. A fresh page loads the approval preference for its own account.

The iOA retest made one generation on deepseek-v4.1-flash with low. The reply and effort survived reopening the session. The latest GitHub platform, e2e, and validate workflows passed. Platform CI

The inline threads cover an already-open tab during an account switch, shared-key effort validation, and official Codex model capabilities.

Comment thread chatbot/src/App.jsx
),
[theme, setTheme] = useState(() => readThemePreference(localStorage)),
[environmentVisibility, setEnvironmentVisibility] = useState(() => readJSON(environmentVisibilityKey, {}));
const approvalUserId = realChat.user?.isGuest ? null : realChat.user?.id ?? null;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Bind full permissions to the account authenticating the turn

In Chromium, account A selected Full permissions and reloaded its chat tab. After signing into account B in a second tab, B's fresh composer showed Request approval. Submitting from the still-open first tab sent approvalMode: "auto" with B's authenticated cookie. That tab gets its approval owner from the user loaded at mount, while the request uses the current cookie. Bind the approval choice to the authenticated account at submission so a stale tab cannot silently authorize B's writes.

func assistantModelEfforts(ctx context.Context, user domain.User, model string) []string {
engine, _ := assistantModelEngine(model)
if engine == assistantEngineCodeBuddy || engine == runtime.AgentCLICursor {
return assistantModelEffortsIn(model, assistantEngineCustomModels(user, engine))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Use the same effort rules for the shared catalog and generation

Save a default-only override for deepseek-v4.1-flash, then clear the personal CodeBuddy key on a platform that shares that model. I reproduced /api/assistant/models offering low on the shared key, followed by the generation validator rejecting it because it still reads the account's old personal override. Resolve effort validation from the same shared catalog the composer uses whenever the turn borrows the platform key.

Comment thread internal/httpapi/executor_models.go Outdated
} else if engine != assistantEngineCodeBuddy {
id = engine + ":" + id
}
catalog.Models[i].Efforts = effortOptions(assistantModelEffortsIn(id, nil))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Advertise the reasoning levels supported by the built-in Codex 6.2 models

The Codex CLI catalog advertises xhigh, max, and ultra for both GPT-6.2 Sol and Astra. Without a custom gateway catalog, the new built-in entries fall through to low, medium, and high. I confirmed that the role catalog omits xhigh for both models and chat validation rejects it. Add model-specific Codex defaults so automatic mode exposes the supported levels. OpenAI model selection guide

@Robert0622

Copy link
Copy Markdown
Collaborator

[P2] 内置 Codex 模型应保留所支持的思考强度

在 7c23f96 重新验证。个人配置未提供强度元数据时,Codex 会走到 CodeBuddy 的模型分支。gpt-5.5 的角色编辑器只提供 low / medium / high,原来可选的 xhigh 消失,聊天校验也拒绝该值。新提交为两款 GPT-6.2 单独补上了默认强度,gpt-5.5 仍可复现。请按 Codex 的模型能力回退,继续优先使用显式配置和已发现的能力。相关代码 · GPT-5.5 官方强度说明

本次复验包含账号审批与模型相关的 Go 回归,以及 CodeBuddy iOA 的 DeepSeek V4.1 Flash 实际调用。

@Robert0622

Copy link
Copy Markdown
Collaborator

[P2] 为内置 Claude Fable 5.2 提供 xhigh 和 max 强度

已复验内置 Codex 的 gpt-5.5 恢复了 xhigh。在没有自定义或发现的强度元数据时,内置 claude-fable-5-2 仍走通用的 low / medium / high 回退。我用本地目录和校验测试复现,角色编辑器缺少 xhigh、max,对话校验也直接拒绝这两个值。请增加 Claude Code 对应的按模型默认能力,并继续优先采用显式和发现的配置。相关代码 · Anthropic 模型强度文档

@Robert0622 Robert0622 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

用 DeepSeek V4.1 Flash 实测 low 生成并保存回复通过。补充回归发现历史对话改名失败,已标在代码行。

Comment thread internal/httpapi/assistant_session.go Outdated
if engine, _ := assistantModelEngine(model); engine == assistantEngineCodeBuddy {
editor, _, _ = h.assistantChatUser(editor)
}
if err := validateAssistantEffort(c.Request.Context(), editor, model, conv.Effort); err != nil {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 允许在模型能力变化后为历史对话改名

我用 deepseek-v4.1-flash 保存了一个 low 对话,再把该模型配置为「仅使用默认」。只改标题的 PUT 会拿旧的 low 按新目录验证,返回 HTTP 400,导致用户无法改名。建议仅在调用方实际修改模型或推理强度时校验此设置,发起生成时继续按当前能力验证。

Robert0622
Robert0622 previously approved these changes Nov 18, 2026

@Robert0622 Robert0622 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

用 DeepSeek V4.1 Flash 的 low 完成真实生成后,把模型能力改为「仅使用默认」,历史对话改名成功并保留 low。真正修改模型、修改推理强度或发起新一轮生成仍会校验当前能力。复测通过。

@Robert0622 Robert0622 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

复审 59e9015。用 CodeBuddy iOA 的 DeepSeek V4.1 Flash 实测生成中即时发送,新回复正常入库并保留 low 思考强度。历史会话改名、旧账号的审批校验、CLI 选择及前后端回归通过。最新提交的 CI 全绿。

@crispinhu27
crispinhu27 merged commit 5c37c33 into main Nov 18, 2026
16 checks passed
@crispinhu27
crispinhu27 deleted the feature/model-reasoning-effort branch November 18, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants