[ispobock]

Add TP2 points with DRAM KV offload at conc 32 to 48 to the existing B300 SGLang AgentX curve, and store the DSpark draft KV cache in FP8 on every point. The whole curve is rerun.

• New TP2 points: the new agentic-tp2.yaml recipe adds conc 32 to 48. The host cache is 370 GB per rank, inside the 749 GB allocation of a two-GPU point at DRAM utilization 1.0. Conc 44 to 48 use longest-prefix-match scheduling.
• Existing TP4 points: the existing agentic.yaml recipe also gains speculative-draft-kv-cache-dtype: fp8_e4m3, so the existing TP4 points change too. FP8 replaces the bf16 default for the draft KV cache, which frees about 10 GB of HBM per rank. All TP4 points are rerun with this setting.

中文

为现有的 B300 SGLang AgentX 曲线新增 conc 32 到 48 的 TP2 + DRAM KV offload 点，并在所有点上将 DSpark draft KV cache 存为 FP8。整条曲线会重新运行。

• 新增 TP2 点： 新的 agentic-tp2.yaml recipe 新增 conc 32 到 48。每个 rank 的 host cache 为 370 GB，在 DRAM utilization 1.0 下两卡点 749 GB 的分配额度以内。conc 44 到 48 使用 longest-prefix-match 调度。
• 现有 TP4 点： 现有的 agentic.yaml recipe 也加入了 speculative-draft-kv-cache-dtype: fp8_e4m3，因此现有的 TP4 点也会变化。draft KV cache 由默认的 bf16 改为 FP8，每个 rank 可节省约 10 GB HBM。所有 TP4 点都会以此设置重新运行。

AI model disclosure

Prepared with Claude Code using claude-opus-5-5. Reviewed by the human author before submission.

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

[claude]

🟡 nit (optional): New override and search-space comments are section headers that restate code, which AGENTS.md forbids. sweep:^# TP2 with L2 HiCache|^# Higher-concurrency points|# TP2 with L2 HiCache: 295 GB Examples: agentic-tp2.yaml:72 repeats schedule-policy: lpm, and nvidia-master.yaml:9484 repeats the tp, hicache size and conc-list values. Fix: delete these header comments. Keep only a one-line why at a specific line where needed, such as next to max-running-requests for the 2x CONC choice. [also at: inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm31flash/sglang/b300-fp4-mtp/agentic-tp2.yaml:72 - nit: Two new comments only restate the config next to them, which the repo's comment rules forbid. "Higher-concurrency points use longest-prefix-match scheduling." just repeats schedule-policy: lpm, and the master-config comment "TP2 with L2 HiCache: 295 GB per rank at conc 32-48." repeats the row below it.]

Why this was flagged

AGENTS.md "Comments (mandatory, every file)" bans "header comments describing a file, topology, config, recipe, override, or section" and says "never restate the code". agentic-tp2.yaml:52-53 sits above zip_override_tp2_hicache and repeats write_back and 295 GB, which are already in the override's hicache-write-policy and hicache-size lines. agentic-tp2.yaml:72 sits above zip_override_tp2_hicache_lpm and only restates schedule-policy: lpm. nvidia-master.yaml:9484 "# TP2 with L2 HiCache: 295 GB per rank at conc 32-48." restates the search-space row below it. The sibling agentic.yaml has similar section comments, which is precedent but not a sanctioned exception. Nothing breaks at runtime; this is a convention slip that reviewers following AGENTS.md would flag.

Verification: AGENTS.md lines 54-58 say: "No header comments describing a file, topology, config, recipe, override, or section" and "never restate the code". agentic-tp2.yaml:51-52 and :72 are section headers that repeat hicache-size: 295 and schedule-policy: lpm. nvidia-master.yaml:9484 restates the row below it. Nothing functional breaks.

[nvpohanh]

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (minimaxm31flash-fp4-b300-sglang-agentic-dspark-tp2) will happen in sgl-project/sglang#46506, in the Agentic Long-Context Deployment section of the MiniMax-M3.1-Flash cookbook page.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (2e12ce08a) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

[nvpohanh]

[by Claude Code] sgl-project/sglang#46506 has been merged. This PR's AgentX config (minimaxm31flash-fp4-b300-sglang-agentic-dspark-tp2) is now in the Agentic Long-Context Deployment section of the SGLang MiniMax-M3.1-Flash cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (2e12ce08a) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40739166138

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40739166138

[nvpohanh]

[by Claude Code] The SGLang cookbook update of this PR's AgentX config (minimaxm31flash-fp4-b300-sglang-agentic-dspark-tp2) will happen in sgl-project/sglang#46547, in the Agentic Long-Context Deployment section of the MiniMax-M3.1-Flash cookbook page. The config has been live in that section since sgl-project/sglang#46506; sgl-project/sglang#46547 refreshes it with the pin below.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (8eddada3c) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

[nvpohanh]

[by Claude Code] sgl-project/sglang#46547 has been merged. This PR's AgentX config (minimaxm31flash-fp4-b300-sglang-agentic-dspark-tp2) is now in the Agentic Long-Context Deployment section of the SGLang MiniMax-M3.1-Flash cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (8eddada3c) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

[nvpohanh]

[by Claude Code] The SGLang cookbook update of this PR's AgentX config (minimaxm31flash-fp4-b300-sglang-agentic-dspark) will happen in sgl-project/sglang#46948, in the Agentic Long-Context Deployment section of the MiniMax-M3.1-Flash cookbook page. The config has been live in that section since sgl-project/sglang#46506; sgl-project/sglang#46948 refreshes it with the pin below.

This PR's /use comment pins run 40345805111, which measured 6481cd6b3. That commit does not have this PR's configurations under this config key (the PR moved or added them since), so the cookbook shows the recipe at this PR's head (5f2ebd584) as "in progress". A later cookbook PR marks it measured once a run of this key is pinned or this PR merges.

[nvpohanh]

[by Claude Code] sgl-project/sglang#46948 has been merged. This PR's AgentX config (minimaxm31flash-fp4-b300-sglang-agentic-dspark) is now in the Agentic Long-Context Deployment section of the SGLang MiniMax-M3.1-Flash cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1#agentic-long-context-deployment

This PR's /use comment pins run 40345805111, which measured 6481cd6b3. That commit does not have this PR's configurations under this config key (the PR moved or added them since), so the cookbook shows the recipe at this PR's head (5f2ebd584) as "in progress". A later cookbook PR marks it measured once a run of this key is pinned or this PR merges.

[nvpohanh]

[by Claude Code] The SGLang cookbook update of this PR's AgentX config (minimaxm31flash-fp4-b300-sglang-agentic-dspark) will happen in sgl-project/sglang#47027, in the Agentic Long-Context Deployment section of the MiniMax-M3.1-Flash cookbook page. The config has been live in that section since sgl-project/sglang#46506; sgl-project/sglang#47027 refreshes it with the pin below.

This PR's /use comment pins run 40345805111, which measured 6481cd6b3. That commit does not have this PR's configurations under this config key (the PR moved or added them since), so the cookbook shows the recipe at this PR's head (3c964a225) as "in progress". A later cookbook PR marks it measured once a run of this key is pinned or this PR merges.

[nvpohanh]

[by Claude Code] sgl-project/sglang#47027 has been merged. This PR's AgentX config (minimaxm31flash-fp4-b300-sglang-agentic-dspark) is now in the Agentic Long-Context Deployment section of the SGLang MiniMax-M3.1-Flash cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1#agentic-long-context-deployment

This PR's /use comment pins run 40345805111, which measured 6481cd6b3. That commit does not have this PR's configurations under this config key (the PR moved or added them since), so the cookbook shows the recipe at this PR's head (3c964a225) as "in progress". A later cookbook PR marks it measured once a run of this key is pinned or this PR merges.

[nvpohanh]

/use 40739166138

[github-actions]

@nvpohanh staged run 40739166138.

• MiniMax-M3.1-Flash · agentic-traces · FP4 · b300_sglang · 2026-10-28

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

[nvpohanh]

[by Claude Code] The SGLang cookbook update of this PR's AgentX config (minimaxm31flash-fp4-b300-sglang-agentic-dspark) will happen in sgl-project/sglang#47214, in the Agentic Long-Context Deployment section of the MiniMax-M3.1-Flash cookbook page. The config has been live in that section since sgl-project/sglang#46506; sgl-project/sglang#47214 refreshes it with the pin below.

The commands mirror the launch measured in run 40739166138, pinned by this PR's /use comment.

[nvpohanh]

[by Claude Code] sgl-project/sglang#47214 has been merged. This PR's AgentX config (minimaxm31flash-fp4-b300-sglang-agentic-dspark) is now in the Agentic Long-Context Deployment section of the SGLang MiniMax-M3.1-Flash cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1#agentic-long-context-deployment

The commands mirror the launch measured in run 40739166138, pinned by this PR's /use comment.

[nvpohanh]

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
• Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• Verified that every draft model and draft head is served as it ships: the draft that ships with the served checkpoint, at its stored precision, through the pinned upstream image's default handling, with the shipped and effective draft precision recorded in the additional detail section. No submission-side quantization, dtype override, checkpoint substitution, or patch may lower draft precision below that default, regardless of eval results or AL. Explicitly verified that SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not enabled in the effective recipe, including inherited settings; enabling it is prohibited going forward, and historical runs do not grant an exception. See Draft-model precision for what counts as the default and the MLPerf comparison.
• For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in infx/golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
• Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; target/verifier FLOPs at lower precisions is fine, given that the config passes private evals, but this does not permit lowering draft-model or draft-head precision below what ships. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• Verified that new models disable integer all-reduce, with SGLang ROCm explicitly setting ROCM_QUICK_REDUCE_QUANTIZATION=NONE. Existing models are grandfathered per MODELS.md.
• For affected Dynamo TRT-LLM benchmarks, verified that iteration/KV-cache metrics are collected and published under the pinned Dynamo implementation and effective engine settings. Resolve the installed Dynamo version separately from the container tag; use supported metrics controls without requiring KV events. Record the version and configuration evidence in the additional detail section. See verifier Check 17. N/A for unaffected serving paths or frameworks without these Dynamo settings.
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• For official InferenceX (non-preview), verified that established models use upstream engine images built from master (or the equivalent default development branch), with new models transitioning within a reasonable time after release. See MODELS.md.
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that every affected vLLM/SGLang production serving configuration in this PR, including Dynamo deployments, is documented in the official vLLM recipes and/or the SGLang cookbook, for both single-node and multi-node aggregated and disaggregated serving. The documentation covers production-useful flags, environment variables, and launch commands (including prefill, decode, and router/frontend commands for disaggregation); InferenceX-only benchmark/harness settings are excluded with reasoning in the additional detail section:
  • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang cookbook source repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
• Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
• If this PR uses append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it.
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
• Reported measured throughput/E2EL Pareto counts and evidence per affected curve (≥5 points strongly recommended). Below 5 or unverifiable: tag a core maintainer for review; recorded admin bypass required before merge. N/A if no curves are affected. Details.

Additional detail section:

• Scope: config key minimaxm31flash-fp4-b300-sglang-agentic-dspark (MiniMax-M3.1-Flash FP4, B300, single-node aggregated SGLang, AgentX agentic-coding). The PR adds TP2 HiCache points (conc 32-48, new agentic-tp2.yaml) and sets speculative-draft-kv-cache-dtype: fp8_e4m3 on the existing TP4 recipe agentic.yaml, rerunning the whole curve. The assessed head is 51bb400f.
• PR validation: pinned by /use 40739166138 → Run Sweep 40739166138 on head 51bb400f, completed with success. All 16 agentic / jobs passed (TP2 c32/36/40/44/46/48 with DRAM HiCache; TP4 c1 ×2, c4, c8, c16, c32, plus TP4 c32/64/80/96 with DRAM HiCache). The skipped jobs are fixed-sequence, multi-node, canary and compare lanes that this AgentX single-node config does not generate. Result artifacts (bmk_agentic_*, eval_results_all) are unexpired.
• Evals: (gsm8k em_strict, threshold 0.90 under default in infx/evals/thresholds.yaml; MiniMax-M3.1-Flash has no model-specific override)
  • Pinned run 40739166138, agentic eval / on head 51bb400f. This includes the FP8 draft KV cache. TP4 c1 scored 0.9629 and TP4 c96 HiCache scored 0.9666.
  • TP2 recipe: eval dispatch 40543116004 (linked in an earlier PR comment), 6/6 agentic eval / jobs passed. TP2 c32 0.9636, c36 0.9621, c40 0.9591, c44 0.9636, c46 0.9651, c48 0.9644.
  • Caveat: dispatch 40543116004 ran on commit 9b45b789, which is outside this PR and predates the FP8 draft-KV change. The FP8 draft-KV path is eval-validated on TP4 in the pinned run. The TP2 recipe is eval-validated in the dispatch.
• Chat template: AgentX trace replay uses the shared AIPerf chat-completions client, which this PR does not change. The TP2 points use reasoning-parser/tool-call-parser: minimax-m3.
• Draft precision:
  • Shipped: MiniMaxAI/MiniMax-M3.1-Flash bundles the DSpark draft at dspark/. Its config.json has quantization_config.quant_method: mxfp8 with block size [1, 32], and the ignored layers (confidence head, fc, markov head, MoE gates) are kept in bf16.
  • Effective: server log of TP2 c46 in run 40739166138 shows speculative_draft_model_path='/model/dspark', Load weight end ... type=DSparkMiniMaxDraftModel, quant=mxfp8. That is the image's default handling, with no draft quantization, dtype or checkpoint override.
  • Draft KV cache: speculative-draft-kv-cache-dtype: fp8_e4m3 stores the draft's KV cache in FP8 instead of bf16. It does not change the draft model's weight or compute precision. I confirmed offline with SA admins that changing the draft KV-cache dtype is allowed, and that only the draft weight and compute dtypes must stay as shipped.
  • SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE does not appear in the recipes, in the launch environment, or in any server or benchmark log of the pinned run.
• Golden AL: the server logs of the pinned run show SGLANG_SIMULATE_ACC_LEN=4.04, SGLANG_SIMULATE_ACC_METHOD=match-expected and SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token. This matches minimaxm31flash_dspark.yaml thinking_on[7] = 4.04 for DSpark block size 7.
• MODELS.md: minimaxm31flash was added 2026-10-23 with Agentic coding active and has no deprecations.
• Architecture hacks: none. There is no --hf-overrides or model-config edit. TP2, HiCache, the LPM schedule policy and flashinfer_trtllm_routed leave the computation unchanged.
• Integer all-reduce: this is a new model on NVIDIA CUDA SGLang. Custom all-reduce is not integer-quantized, and no *_QUICK_REDUCE_QUANTIZATION control is set. The ROCm-specific requirement does not apply.
• Dynamo TRT-LLM metrics: N/A. This is direct SGLang serving with no Dynamo TRT-LLM path.
• Image:
  • The image is upstream lmsysorg/sglang:dev-minimax-m31-flash@sha256:614b1509…, unchanged by this PR.
  • Master-branch policy: MiniMax-M3.1-Flash is a new model (added 2026-10-23), so the model-enablement image is within the new-model transition window.
  • Engine order: SGLang is the in-tree engine submitted here, and this PR adds no other frameworks.
• Upstream recipe:
  • sgl-project/sglang#47214 is MERGED (2026-10-29). It mirrors this PR's head 51bb400f and pinned run 40739166138 into the published MiniMax-M3.1 cookbook, Agentic Long-Context Deployment section.
  • That section covers the B300 "Agg TP2" entry (agentic-tp2.yaml, conc 32-48, --hicache-size 370, write_back, LPM at conc 44-48, per-point --max-running-requests/--cuda-graph-max-bs-decode) and the B300 TP4 entries, including the FP8 draft KV cache. The TP2 entry was first added in sgl-project/sglang#46807 (MERGED).
  • Excluded settings are InferenceX harness-only: health_check, observability, and AIPERF_* / CONC / KV_OFFLOADING benchmark env.
• Engine patches: none. Neither recipe has a setup script, patch, source edit or wheel install, and no waiver is needed.
• append-only: N/A. The new changelog entry does not set append-only: true; it reruns the whole curve.
• Pareto coverage:
  • Curve: MiniMax-M3.1-Flash / agentic-traces / b300_sglang / FP4, run 40739166138, P90 E2EL vs total_tput_tps per GPU.
  • Count: 16 measured points, 0 invalid, 8 frontier points (PASS via infx.workflows.pareto_coverage).
  • Frontier: TP4 c16 and TP4 c32 without offload, plus TP2 HiCache c32/36/40/44/46/48 (the TP2 points alone give 6).

Signed: nvpohanh

[github-actions]

❌❌❌ REJECTED ❌❌❌

@nvpohanh Blocking: both recipes override only the DSpark draft's KV cache to FP8 (--speculative-draft-kv-cache-dtype fp8_e4m3). The target KV stays at the bf16 default, so the draft runs below its baseline. The "confirmed offline with SA admins" note in the sign-off is not a recognized exception under Check 13.

❌ Check 13 (Draft precision): FAIL — Draft KV-cache dtype override below the framework default, aimed only at the draft. minimaxm31flash-fp4-b300-sglang-agentic-dspark (b300-fp4-mtp/agentic.yaml and agentic-tp2.yaml at 51bb400f) sets speculative-draft-kv-cache-dtype: fp8_e4m3, and the launch command in run 40739166138 (TP2 c46 job) passes it. The target gets no --kv-cache-dtype. The checkpoint config.json has torch_dtype: bfloat16 and no KV quantization, and the PR's own changelog says FP8 replaces "the bf16 default". So the FP8 draft KV is neither the framework default nor a dtype applied to both target and draft. To fix: remove the override, or set the same upstream-supported KV dtype on target and draft, then rerun. Draft weights are as shipped: dspark/ mxfp8. SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not set in either recipe or the launch command.

Passed and not applicable checks

✅ Check 0 (CODEOWNER): PASS — nvpohanh is a listed owner of inferencex-e2e/configs/nvidia-master.yaml and /inferencex-e2e/benchmarks/*_node/srt-slurm-recipes/**/b300-*/. perf-changelog.yaml falls under the catch-all only.

✅ Check 1 (In-PR sweep + evals): PASS — run 40739166138 on the pinned head 51bb400f passed 16/16 agentic / jobs and 2/2 agentic eval / jobs. The head has not moved and there are no later commits.

✅ Check 2 (Eval accuracy): PASS — gsm8k em_strict on the pinned image: TP4 c1 0.9629 and TP4 c96 HiCache 0.9666. Both are within 2σ of each other and of the ingested cohort (run 40178345296, 0.9636). Both evals ran the FP8 draft-KV path. TP2 has only an off-PR eval (40543116004 on 9b45b789, 6/6 passed), and its backends match the evaluated TP4 HiCache path.

✅ Check 3 (Upstream recipe): PASS — sgl-project/sglang#47214 merged 2026-10-29. The minimax-m3.1.jsx snippet on main pins PR 4524 at 51bb400f. It covers the B300 TP2 HiCache settings (--hicache-size 370, write_back, LPM) and the B300 TP4 settings (flashinfer_trtllm_routed, --speculative-draft-kv-cache-dtype fp8_e4m3). The sign-off justifies the harness-only exclusions.

✅ Check 4 (Reuse command): PASS — COLLABORATOR nvpohanh posted /use 40739166138 on 2026-10-29. That is the Check 1 run.

✅ Check 5 (Checklist template): PASS — all 20 items of the current PR_REVIEW_CHECKLIST.md are present and checked.

✅ Check 6 (Upstream image/order/provenance): PASS — the image is upstream lmsysorg/sglang:dev-minimax-m31-flash@sha256:614b1509… and is unchanged by this PR. It is a model-enablement image for a new model added 2026-10-23, which the new-model transition window allows. The PR adds only SGLang entries.

✅ Check 7 (Deprecated models): PASS — minimaxm31flash Agentic coding is active in MODELS.md and has no deprecation.

✅ Check 8 (Architecture hacks): PASS — there is no --hf-overrides or model-config edit. This is a new model on CUDA SGLang with no *QUICK_REDUCE_QUANTIZATION control set, so the ROCm requirement does not apply.

✅ Check 9 (Chat template): PASS — AgentX trace replay goes through the shared AIPerf chat-completions client, with the minimax-m3 reasoning and tool parsers. The PR does not change the client.

✅ Check 10 (Engine patches): PASS — neither recipe has a setup script, patch, source edit or wheel install. The launch command is a stock sglang.launch_server.

✅ Check 11 (Golden AL): PASS — the job logs show SGLANG_SIMULATE_ACC_LEN=4.04, match-expected and real-draft-token. This equals minimaxm31flash_dspark.yaml thinking_on[7]=4.04 for DSpark block size 7.

➖ Check 12 (Append-only): N/A — the new changelog entry does not set append-only: true.

✅ Check 14 (Pareto coverage): PASS — curve minimaxm31flash/agentic-coding/b300_sglang/fp4, run 40739166138 attempt 1, P90 E2EL vs total_tput_tps per GPU (bmk_agentic_* artifacts): 16 measured points, 0 invalid, 8 frontier points (TP4 c16, TP4 c32, and TP2 HiCache c32/36/40/44/46/48).

✅ Check 15 (PR description): PASS — the body matches the pinned code: TP2 conc 32-48, 370 GB HiCache per rank, the 749 GB allocation matches allocated_cpu_dram_gb, LPM at conc 44-48, and the FP8 draft KV on the TP4 recipe with a full-curve rerun.

✅ Check 16 (NVIDIA overview): PASS — the NVIDIA overview lists MiniMax-M3.1-Flash with SGLang populated.

➖ Check 17 (Metrics publication): N/A — this is direct SGLang serving with no Dynamo TRT-LLM path.

✅ Check 18 (Recipe/image/sweep consistency): PASS — the master TP2 entry (conc 32-48, agentic-tp2.yaml, dram-utilization: 1.0) matches the recipe's gpus: 2 and zip CONC lists. There are no generation errors and no new images. There are no undominated removals; 9 TP4 points are re-tuned (c64 HiCache −9.6% throughput, c96 +5.2%, the others within ±4.4%). The changelog accurately describes the FP8 draft-KV change.

Assessed commit: 51bb400fde2167c9a265ec9d7e19f61a61d7b7ee.