Repository navigation
fix(models): preserve and configure per-model reasoning efforts - #1026
Conversation
Robert0622
left a comment
There was a problem hiding this comment.
Reviewed b4bacef7. The live test used CodeBuddy Code 2.168.0 and an isolated LoopForge instance. It covered iOA model discovery, saving and reloading the personal effort settings, the chat and role catalogs, rejecting an unavailable effort, and one successful deepseek-v4.1-flash generation with low. The saved conversation retained both the reply and the effort.
The chatbot suite passed 389 tests and the production build passed. The existing web test suite passed. I reproduced TestOwnKeyKeepsItsEnvironmentDefaultWhileAKeyIsShared locally. The same test fails in the platform core job, and three executor-picker cases fail in the browser job, including a CodeBuddy catalog response with zero models on the unconfigured fixture.
| } catch { | ||
| // Without storage the pick still applies to this page. | ||
| } | ||
| if (validApprovalMode(value)) writeComposerPreference("approvalMode", value); |
There was a problem hiding this comment.
[P1] Scope the full-permission choice to the signed-in account
auto is now saved in browser storage under an origin-wide key. After one account selects Full permissions and signs out, another account using that browser starts its next chat in auto and submits the turn with its own credentials. Platform and external writes can then run without that account opting in. Key this setting by the authenticated user and default to confirmation until that user makes a choice.
| } | ||
| catalog.Models = selectableCodeBuddyModels(catalog.Models) | ||
| if catalog.Default == assistantDefaultModel && len(catalog.Models) > 0 { | ||
| catalog.Default = catalog.Models[0].ID |
There was a problem hiding this comment.
[P1] Preserve the execution default when hiding the picker entry
With the platform default unset and no personal model list or remembered choice, real iOA discovery made claude-sonnet-5 the new catalog default. The composer also picks the first concrete model and sends its ID explicitly, so users with a fresh browser or a saved default choice can start spending on a different model without selecting it. Requests that omit model still take the original CLI-default path. Resolve the default separately from the options shown in the picker.
There was a problem hiding this comment.
Product follow-up on this thread. A potentially higher-cost first model is acceptable as the initial selection when the composer shows its concrete name before sending and the user can switch. With that policy, I’m withdrawing the P1.
preferredModel on 7c23f96 still falls back to the CLI default when a valid preference is absent, even with concrete models available. For a new CodeBuddy chat, use the remembered model for that CLI if it is still selectable. Otherwise preselect the first selectable concrete model, show its name, and submit that ID.
Keep backend support for stored conversations and API clients that send default or omit model. Those compatibility paths can still handle requests with no concrete model.
| kept.add(id); | ||
| if(picked.has(id)&&discovered.has(id)){ | ||
| const current=credentialCatalogRowModel(row),fresh=discovered.get(id); | ||
| row.outerHTML=credentialCatalogRowHTML({...current,...fresh,name:current.name,reasoningEfforts:current.reasoningEfforts??fresh.reasoningEfforts},catalog.dataset.catalogEngine); |
There was a problem hiding this comment.
[P2] Distinguish discovered capabilities from a user override
I imported a model whose provider advertised ["low"] and saved without touching the effort controls. The next probe of the same ID advertised ["high"], but the editor retained ["low"] because the first probe had already populated current.reasoningEfforts. This makes capability-only refreshes ineffective for previously discovered models. Track the discovered levels separately so a refresh updates them while preserving deliberate overrides.
| if len(catalog) == 0 { | ||
| key, baseURL := assistantEngineCredential(user, engine) | ||
| anthropic := assistantEngineAnthropicModels(engine, assistantEngineProtocol(user, engine)) | ||
| if cached, ok := assistantDiscoveryCache.Load(assistantDiscoveryKey(anthropic, key, baseURL)); ok { |
There was a problem hiding this comment.
[P2] Validate automatic efforts on a cold API process
A custom Codex gateway can advertise xhigh for a model with no saved override. I reproduced the effort being offered and accepted with a warm discovery cache, then rejected by the same validation after clearing that process-local cache. A restart or another API replica can make the generation return HTTP 400 for the effort just offered by the picker. Resolve automatic capabilities through the same discovery path when validating a turn.
| const items=agentEffortItems(currentAgentExecutorId()); | ||
| const current=picker.querySelector('[name="effort"]').value; | ||
| const field=picker.querySelector('[name="effort"]'); | ||
| if(executorModelCatalogs()[currentAgentExecutorId()]&&!items.some(item=>item.id===field.value))field.value=''; |
There was a problem hiding this comment.
[P2] Keep a saved role effort during unrelated edits
The arriving personal catalog can clear the role's stored effort as soon as the editor opens. I reproduced opening a role saved with xhigh while the editor's personal catalog allowed only the model default, changing the role name, and getting an empty effort in the save request. A personal catalog should constrain new selections without silently rewriting a shared role's saved setting.
Robert0622
left a comment
There was a problem hiding this comment.
Re-reviewed f4e1c9f. The earlier CodeBuddy default, catalog refresh, cold discovery, and role-edit cases pass on this revision. A fresh page loads the approval preference for its own account.
The iOA retest made one generation on deepseek-v4.1-flash with low. The reply and effort survived reopening the session. The latest GitHub platform, e2e, and validate workflows passed. Platform CI
The inline threads cover an already-open tab during an account switch, shared-key effort validation, and official Codex model capabilities.
| ), | ||
| [theme, setTheme] = useState(() => readThemePreference(localStorage)), | ||
| [environmentVisibility, setEnvironmentVisibility] = useState(() => readJSON(environmentVisibilityKey, {})); | ||
| const approvalUserId = realChat.user?.isGuest ? null : realChat.user?.id ?? null; |
There was a problem hiding this comment.
[P1] Bind full permissions to the account authenticating the turn
In Chromium, account A selected Full permissions and reloaded its chat tab. After signing into account B in a second tab, B's fresh composer showed Request approval. Submitting from the still-open first tab sent approvalMode: "auto" with B's authenticated cookie. That tab gets its approval owner from the user loaded at mount, while the request uses the current cookie. Bind the approval choice to the authenticated account at submission so a stale tab cannot silently authorize B's writes.
| func assistantModelEfforts(ctx context.Context, user domain.User, model string) []string { | ||
| engine, _ := assistantModelEngine(model) | ||
| if engine == assistantEngineCodeBuddy || engine == runtime.AgentCLICursor { | ||
| return assistantModelEffortsIn(model, assistantEngineCustomModels(user, engine)) |
There was a problem hiding this comment.
[P2] Use the same effort rules for the shared catalog and generation
Save a default-only override for deepseek-v4.1-flash, then clear the personal CodeBuddy key on a platform that shares that model. I reproduced /api/assistant/models offering low on the shared key, followed by the generation validator rejecting it because it still reads the account's old personal override. Resolve effort validation from the same shared catalog the composer uses whenever the turn borrows the platform key.
| } else if engine != assistantEngineCodeBuddy { | ||
| id = engine + ":" + id | ||
| } | ||
| catalog.Models[i].Efforts = effortOptions(assistantModelEffortsIn(id, nil)) |
There was a problem hiding this comment.
[P2] Advertise the reasoning levels supported by the built-in Codex 6.2 models
The Codex CLI catalog advertises xhigh, max, and ultra for both GPT-6.2 Sol and Astra. Without a custom gateway catalog, the new built-in entries fall through to low, medium, and high. I confirmed that the role catalog omits xhigh for both models and chat validation rejects it. Add model-specific Codex defaults so automatic mode exposes the supported levels. OpenAI model selection guide
|
[P2] 内置 Codex 模型应保留所支持的思考强度 在 本次复验包含账号审批与模型相关的 Go 回归,以及 CodeBuddy iOA 的 DeepSeek V4.1 Flash 实际调用。 |
|
[P2] 为内置 Claude Fable 5.2 提供 已复验内置 Codex 的 |
Robert0622
left a comment
There was a problem hiding this comment.
用 DeepSeek V4.1 Flash 实测 low 生成并保存回复通过。补充回归发现历史对话改名失败,已标在代码行。
| if engine, _ := assistantModelEngine(model); engine == assistantEngineCodeBuddy { | ||
| editor, _, _ = h.assistantChatUser(editor) | ||
| } | ||
| if err := validateAssistantEffort(c.Request.Context(), editor, model, conv.Effort); err != nil { |
There was a problem hiding this comment.
[P2] 允许在模型能力变化后为历史对话改名
我用 deepseek-v4.1-flash 保存了一个 low 对话,再把该模型配置为「仅使用默认」。只改标题的 PUT 会拿旧的 low 按新目录验证,返回 HTTP 400,导致用户无法改名。建议仅在调用方实际修改模型或推理强度时校验此设置,发起生成时继续按当前能力验证。
Robert0622
left a comment
There was a problem hiding this comment.
用 DeepSeek V4.1 Flash 的 low 完成真实生成后,把模型能力改为「仅使用默认」,历史对话改名成功并保留 low。真正修改模型、修改推理强度或发起新一轮生成仍会校验当前能力。复测通过。
Robert0622
left a comment
There was a problem hiding this comment.
复审 59e9015。用 CodeBuddy iOA 的 DeepSeek V4.1 Flash 实测生成中即时发送,新回复正常入库并保留 low 思考强度。历史会话改名、旧账号的审批校验、CLI 选择及前后端回归通过。最新提交的 CI 全绿。
What changed
Fetching available models could return only IDs and names, leaving reasoning levels missing or lost when the personal catalog was saved or refreshed. Models now carry reasoning levels through discovery, persistence, chat validation, and role selection, with a per-model editor for automatic or custom levels.
reasoningEfforts/supported_reasoning_efforts; use the existing CLI catalog when metadata is absent and restrict choices to levels the CLI supports.claude-codealias.claude-fable-5-2) to the built-in Claude Code model list. Claude Code model documentation.xhighand the supported levels for newer models can includemaxorultra. Explicit preferences and discovered capabilities take precedence.xhighandmax, with on-demand installation pinned to Claude Code2.1.321. Claude Code model configuration.Configuration behavior references DeepSeek Harness's per-model reasoning editor.
Scope
Compatibility
reasoningEffortsis an optional field inside existing model-catalog JSON. Omission keeps the previous automatic behavior; an explicit empty list permits only the model's native default. Existing Cursor mappings remain independent. No database schema migration is required.CLI and model preferences reuse the browser keys from main. Per-model effort preferences use the same browser storage. Approval preferences use account-specific browser keys, with confirmation for accounts that have not made a selection. Existing chats use their recorded model and effort. The CodeBuddy execution default is part of the catalog metadata.
discoveredReasoningEffortsstores automatic capabilities in the existing catalog JSON.reasoningEffortsstores explicit overrides.Validation
Passed:
internal/httpapi,internal/repo,internal/runtime, andweb.npm --prefix chatbot run build.go vet ./...andpython3 scripts/check-public.py..gitleaks.tomland found no leaks. Used the checksum-verified official macOS ARM64 binary because the repository script downloads a Linux executable.TestProviderUserCredentialsagainst a disposable local Docker MySQL 8.4 database, including explicit empty reasoning levels.xhighselection in the original chat frame, and the intercepted generation request carried the selected level. The 392 chatbot tests, production build, Go web suite, and focused Go regressions for account binding, shared CodeBuddy, and the Codex 6.2 catalogs passed. In Chromium, a stale tab retained its old display after a second tab changed the authenticated account. The generation request received HTTP 409 before executing.2.1.321sent bothxhighandmaxfor Fable 5.2 to a local fixture gateway. The Claude Code native gateway tests, focused Go regressions, andgo vet ./...passed.2.6.3, DeepSeek Harness0.2.2-rc.1, and ZCode0.17.1against local fixture gateways. The 394 chatbot tests, 212 web tests, production build, focused Go regressions, andgo vet ./...passed.The full
go test -p=2 ./... -count=1 -timeout=180srun was attempted and is not green locally: macOS/proc-dependent deployment tests fail, a systemd notification socket test exceeds the macOS socket-path limit (passes when rerun withTMPDIR=/tmp), the host'ssafe.bareRepository=explicitsetting rejects Git fixtures, engine gate tests time out, and the full HTTP API package exceeds the 180-second limit. Runner/runtime Git checkout and index-lock fixtures also fail on this host. Focused regressions listed above pass; the PR's Linux CI will provide the full-suite result.The full public browser suite and paid-provider live tests were not run. Local-edition checks are not applicable to this platform change.
Safety