[zijiexia]

The SGLang DSpark drafter stores its KV cache in bf16 by default. On B300 TP2 that took 21 GB of HBM and 71 GB of the 370 GB HiCache host pool per rank, leaving about 10% less GPU KV than vLLM, whose DSpark drafter uses FP8 KV. At high concurrency the smaller GPU cache lowered the GPU prefix hit rate and cost throughput against vLLM.

This PR sets speculative-draft-kv-cache-dtype: fp8_e4m3 in the base args of the B200 MiniMax-M3.1-Flash SGLang AgentX recipe, so the freed HBM and host memory go to target KV. The changelog entry is not append-only, so the whole curve reruns on the new recipe. The same change for B300 is in #4524.

Risk: FP8 draft KV can lower real DSpark acceptance. Throughput runs use the golden synthetic AL, so the sweep's eval is what checks this.

中文

SGLang 的 DSpark drafter 默认用 bf16 存 KV cache。在 B300 TP2 上，它每个 rank 占用 21 GB HBM 和 370 GB HiCache host 池中的 71 GB，GPU KV 容量比 vLLM 少约 10%（vLLM 的 DSpark drafter 用 FP8 KV）。高并发时 GPU cache 更小，GPU prefix 命中率下降，吞吐落后于 vLLM。

本 PR 在 B200 MiniMax-M3.1-Flash SGLang AgentX recipe 的 base args 中设置 speculative-draft-kv-cache-dtype: fp8_e4m3，把省下的 HBM 和 host 内存留给 target KV。changelog 条目不是 append-only，整条曲线会用新 recipe 重跑。B300 的同样改动在 #4524。

风险：FP8 draft KV 可能降低真实的 DSpark 接受率。吞吐测试使用 golden synthetic AL，因此需要由 sweep 中的 eval 来验证。

AI model disclosure

Prepared with Claude Code using claude-opus-5-5. Reviewed by the human author before submission.

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

🔴 Every point of this B200 SGLang submission now runs the DSpark draft cheaper than the pinned image's default, which CONTRIBUTING.md forbids. The override sets the draft KV to fp8_e4m3, while the PR says SGLang's drafter default is bf16. The target KV dtype stays unset. CONTRIBUTING.md:86-87 forbids "KV-cache dtype overrides aimed at the draft that lower its precision below the framework's default". Fix: drop speculative-draft-kv-cache-dtype and keep the image's default draft KV. Free HBM some other way, for example a target kv-cache-dtype that the draft inherits consistently.

Why this was flagged

The recipe base args at agentic.yaml:37 apply to every override (tp4_c1 through tp4_ep4_dram_c80), and nvidia-master.yaml:5635-5639 binds all of them for minimaxm31flash-fp4-b200-sglang-agentic-dspark. The recipe sets no target kv-cache-dtype, so only the draft's KV is quantized. That is not the "applied consistently to target and draft" allowance at CONTRIBUTING.md:60-62. On the base branch the draft KV uses the pinned image's default, bf16 per the PR's own account. After this change, all published points come from a draft run below shipped precision. CONTRIBUTING.md:100-104 says matching an upstream recipe (here, vLLM's FP8 drafter) or passing evals does not exempt this. It also says synthetic AgentX acceptance hides the hit. These throughput runs use golden synthetic AL, so the acceptance loss the PR names as a risk never shows up in the measured numbers. The codeowner Check 13 should FAIL this as a demonstrated draft-precision violation.

Verification: In agentic.yaml, line 37 of the diff adds speculative-draft-kv-cache-dtype: fp8_e4m3 under base.roles.agg.args, so all 13 overrides inherit it. No override sets a target kv-cache-dtype. The PR description says the default is bf16. So this is a draft-only override that sets precision below the framework default. CONTRIBUTING.md:86-87 forbids exactly that.

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40739120064

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40739120064

[claude]

Code review found no new issues

No new issues were found in this update; 1 finding from earlier reviews is still open above.

Still open from earlier reviews (1):

• 🔴 inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm31flash/sglang/b200-fp4-mtp/agentic.yaml:37 — Every point of this B200 SGLang submission now runs the DSpark draft cheaper than the pinned image's default, which CON…

If you have decided not to act on one of these findings, resolve its thread (a reply alone leaves it open) and the next review stops counting it. To review this commit again now, use Re-run on its "Claude Code Review" check.

[nvpohanh]

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (minimaxm31flash-fp4-b200-sglang-agentic-dspark) will happen in sgl-project/sglang#47027, in the Agentic Long-Context Deployment section of the MiniMax-M3.1-Flash cookbook page.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (0185c5373) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

[nvpohanh]

[by Claude Code] sgl-project/sglang#47027 has been merged. This PR's AgentX config (minimaxm31flash-fp4-b200-sglang-agentic-dspark) is now in the Agentic Long-Context Deployment section of the SGLang MiniMax-M3.1-Flash cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (0185c5373) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

[zijiexia]

/use 40739120064

[github-actions]

@zijiexia staged run 40739120064.

• MiniMax-M3.1-Flash · agentic-traces · FP4 · b200_sglang · 2026-10-28

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

[github-actions]

Ingest report — ✅ Results ingested into the production database.

Ingest workflow · Merge Ingest run · Benchmark source run

Category	new	duplicate	skipped	unmapped	failed
benchmark	13	0	0	0	0
eval	3	3	3	0	0
trace	13	0	0	0	0
power	13	0	0	0	0

_{benchmarkFiles 13, rawBenchmarkRows 13, coveredByPerJobFiles 0, purgedPoints 0, failedRunRows 0, runStatsNew 0, runStatsDuplicate 0, evalSamplesNew 3957, gpuMetricSamplesNew 258696, powerPublicationPoints 13, changelogEntriesWritten 1}

Counts cover only this run. Rows: benchmark_results, eval_results, agentic rows linked to a trace, gpu_metrics series. This comment updates if the ingest is rerun.

中文

Ingest 报告 — ✅ 结果已写入 production 数据库。

上表仅统计本次运行写入的数据，各行单位依次为 benchmark_results 行、eval_results 行、已关联 trace 的 agentic 行和 gpu_metrics 序列。重新运行 ingest 时本评论会原地更新。