[BUG] Stock harness re-bills ~15k fresh inputTokens on every call — base prefix never caches (deepseek-v4-flash)

[JeanRGW]

Summary

Stock harness (zero mods, --no-skills, no MCP) re-bills ~15–19k fresh inputTokens on every model call in the same session. A trivial same-session follow-up (hi → hi again) bills 15207 fresh + 15104 cache — the ~15k base never enters cache while history deltas do. Same task via OpenCode harness on the same model through the same gateway bills deltas (~100–500 fresh on warm steps). Measured cost gap ~16x.

Related but distinct from #762 (that is mod-authored appendSystemPrompt churn; this repro has no mods), #626, #702.

Repro

mkdir -p /tmp/cc-cache/src && cd /tmp/cc-cache
echo 'export function add(a:number,b:number){return a+b}' > src/a.ts
# call 1
commandcode -p -t --yolo --model deepseek/deepseek-v4-flash --no-skills --output-format json "List files in src/ only, no edits." | grep model_request_end
# call 2, same session (use sessionId from call 1 result)
commandcode -p -t --yolo --model deepseek/deepseek-v4-flash --no-skills --session <SES> --output-format json "hi again, reply with one word" | grep model_request_end

Minimal proof (empty history, no tools, greeting only):

call	inputTokens (fresh)	cacheReadTokens	outputTokens
hi (new session)	15185	7296	45
hi again (same session)	15207	15104	10

Fresh is flat; cache grows by roughly the previous call's content. The ~15k base is re-billed fresh every call and never cached.

Full task (read 3 tiny .ts files → write REPORT.md → JSDoc edit + grep, 2 user turns), 3 trials, deepseek/deepseek-v4-flash:

trial	CC fresh in	CC cacheR	CC out	CC billed (costUsd sum)
1	171883	151680	1484	$0.0399
2	171737	151680	1035	$0.0395
3	133066	112768	1039	$0.0307

Same task via opencode2 run -m commandcode/deepseek/deepseek-v4-flash (same gateway, same model): fresh ~6.7k + cache ~38k total for both turns, est ~$0.0023 at list rates. Per warm step OC bills ~100–500 fresh vs CC ~18–19k.

Billing check: costUsd matches list rates under disjoint accounting exactly, e.g. 18325*0.22 + 7936*0.007 + 80*0.66 = $0.004139852 — so the gap is billed, not a reporting artifact.

Ablations (first-call fresh inputTokens, same prompt)

flags	fresh in	cacheR
--tools-all (full)	18287	9344
baseline -p default tools	15253	512
--no-skills	16717	7424
default tools + --no-skills	13569	5888
minimal --tools-enable read_file,glob,grep + --no-skills	13564	5888

Floor with everything removable stripped is still ~13.5k vs OpenCode's ~5.4k cold. Remaining ~8k is base system + base tool schemas.

Suspected cause

Something in the built-in per-request prefix (~15k) changes every call so DeepSeek automatic prefix caching never matches it, while accumulated history (stable) does cache — hence flat fresh + growing cache. Candidates in dist/cli.mjs: systemPrompt({sessionId, state, permissionMode}) is state-dependent; every message carries fresh meta.createdAt: Date.now() + new UUIDs (queuedToMessages, buildUserMessage); run_end.nextState ships modState.workspace (cwd/platform/branch/gitStatus/recentCommits) + compaction basis (~15k, matches compaction.tokens: 15254 seen on the bare-hi call). Any of these re-serialized into the request prefix breaks it. The 0.33.0 "relocate volatile dynamic context block" fix improved history caching but the base block itself still looks volatile.

Environment

• commandcode@1.49.1, linux x64, deepseek/deepseek-v4-flash, --effort max (default from config)
• Zero mods (mods list empty), no MCP servers, taste file 86 bytes
• OpenCode2 v0.0.0-beta-19135 control via api.commandcode.ai/provider/v1, same model, same fixture

Impact

Same work costs ~16x more ($0.037 vs $0.0023) and each call re-pays the full base prefix, so long sessions scale as O(calls × base) instead of O(base + deltas). A dev-mode diagnostic that warns when the built-in system-prefix bytes change between turns (as proposed in #762 for mods, but applied to the stock prefix) would have caught this.

[ahmadbilaldev]

Let me investigate and get back to you. Thanks for reporting.

[Acid37]

Same problem here, going through the Provider API endpoint (api.commandcode.ai/provider/v1) with deepseek-v4-flash. My chat framework sends 2-3 requests per chat turn, with ~25K tokens of growing conversation history per request.

My numbers today:

• 402 requests total
• 8.7M input tokens
• Average 21,641 input tokens per request (should be ~3K-4K with proper caching)
• Token consumption is roughly 6x what it should be

Deep analysis of ~69K requests from my LLM stats DB:

• Same conversation, consecutive requests 8 seconds apart → 25K tokens all missed cache (should have hit >20K)
• 36.4% of requests had miss >> prompt increment — not just new content missed, but previously-sent stable history also missed
• 155 full-cache-miss requests happened within 30 seconds of the previous request
• cache_write_tokens is consistently 0 across all requests

Setup:

• Chat framework → api.commandcode.ai/provider/v1 → DeepSeek v4-flash
• Sends stable system prompt + growing history + user prompt with seconds-level timestamp
• Even accounting for the timestamp (should only affect hundreds of tokens max), having the ENTIRE 25K prefix miss cache on consecutive requests makes no sense

This mirrors what agarwalbikram said on #392:

"it's not a CLI problem but rather how they handle requests"

Has anyone confirmed whether v0.33.0's dynamic prefix cache thrashing fix was applied to the Provider API gateway, or only to the CLI client? From these numbers it looks like the Provider API path still has the same thrashing behavior — or the gateway isn't forwarding upstream cache hits properly.