[NV] Add GB300 AgentX Qwen3.5 recipes

[csahithi]

• 9 Qwen3.5-397B-A17B-NVFP4 GB300 sglang AgentX recipes (agg + disagg pareto)
• configs/nvidia-master.yaml: qwen3.5-fp4-gb300-dynamo-sglang{,-agentic-agg,-agentic-disagg}
• perf-changelog.yaml: agentic pareto entry
• runners/launch_gb300-nv.sh: qwen3.5 launcher branch + dynamo-wheels cache

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28975134400
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=28975134400

[claude]

Additional findings (outside current diff — PR may have been updated during review):

• 🔴 configs/nvidia-master.yaml:10940-10969 — 9 new sweep entries in configs/nvidia-master.yaml (agentic-agg + agentic-disagg, lines 10940–11085) declare spec-decoding: none, but every one of the 9 corresponding recipe files enables NEXTN MTP (speculative-algorithm: NEXTN + speculative-num-steps: 3 + eagle-topk/num-draft-tokens; the recipe filenames also encode -mtp-). The engine runs MTP correctly because srtctl applies the recipe directly, but SPEC_DECODING propagates verbatim into RESULT_FILENAME (_spec-none_… in artifact names, benchmark-multinode-tmpl.yml:225) and into the aggregated result JSON's spec_decoding field (utils/agentic/aggregation/process_agentic_result.py:139), so downstream dashboards / Pareto comparisons will silently misclassify these MTP runs as non-MTP. Fix: change spec-decoding: none → spec-decoding: "mtp" on all 9 entries in this block (matches the file's existing MTP convention — every other qwen3.5/dsr1/dsv4 MTP entry uses mtp; the parallel dsv4 agentic entry using none is self-consistent because its vLLM recipes have no speculative-* fields).

  Extended reasoning...

  What the bug is

  All 9 newly added sweep entries under qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg and qwen3.5-fp4-gb300-dynamo-sglang-agentic-disagg (configs/nvidia-master.yaml lines 10943, 10957, 10986, 11000, 11014, 11029, 11044, 11059, 11074) set:

  spec-decoding: none

  But every one of the 9 corresponding recipe files under benchmarks/multi_node/srt-slurm-recipes/sglang/qwen3.5/gb300-fp4/agentic/*.yaml enables NEXTN MTP:

  speculative-algorithm: NEXTN
  speculative-num-steps: 3
  speculative-eagle-topk: 1
  speculative-num-draft-tokens: 4

  The recipe filenames themselves also encode -mtp- (e.g. agg-gb300-tp4-c1-mtp-hicache-jid2191933.yaml, disagg-gb300-1p1d-tp4-tp4-c32-mtp-hicache-convaware-jid2212783.yaml).

  Why the engine still runs MTP but the metadata is wrong

  srtctl apply reads the recipe file directly, so the engine gets --speculative-algorithm NEXTN … on its command line and NEXTN MTP runs as intended. The spec-decoding field in nvidia-master.yaml is not a live engine setting — it feeds SPEC_DECODING into the workflow env, which has two downstream consumers, and both are silently corrupted by this mislabel:

  1. Artifact filenames — .github/workflows/benchmark-multinode-tmpl.yml:225 bakes _spec-${{ env.SPEC_DECODING }}_ directly into RESULT_FILENAME. So uploaded result JSONs land as …_spec-none_conc*.json despite the run actually using MTP.

  2. Aggregated result JSON body — utils/agentic/aggregation/process_agentic_result.py:139 writes "spec_decoding": os.environ.get("SPEC_DECODING", "none") verbatim into the aggregated result JSON. Downstream dashboards / Pareto plots that filter or facet on spec_decoding will silently misclassify these MTP runs as non-MTP.

  (Some verifiers also flagged utils/compare_results.py:51 reading result['spec_decoding'] for baseline lookup — worth double-checking, but the two consumers above are sufficient on their own.)

  Convention check

  Every other MTP recipe in nvidia-master.yaml (dsr1-mtp, dsv4-b200-vllm-mtp, qwen3.5 fixed-seq-len MTP tiers, etc.) uses spec-decoding: "mtp". The one other agentic entry that legitimately uses spec-decoding: none — dsv4-fp4-gb300-dynamo-vllm-agentic — is self-consistent because its vLLM recipes at benchmarks/multi_node/srt-slurm-recipes/vllm/deepseek-v4/agentic/*.yaml contain no speculative-* fields. So this is a real inconsistency introduced by this PR, not the file's existing style.

  Step-by-step proof (agentic-agg conc=1)

  1. Sweep matrix expands the spec-decoding: none / conc-list: [1] entry into a GHA job with SPEC_DECODING=none, CONFIG_FILE=recipes/sglang/qwen3.5/gb300-fp4/agentic/agg-gb300-tp4-c1-mtp-hicache-jid2191933.yaml.
  2. srtctl apply -f $CONFIG_FILE reads the recipe, which contains speculative-algorithm: NEXTN + speculative-num-steps: 3 — the sglang server starts with NEXTN MTP enabled. ✅ Engine correct.
  3. benchmark-multinode-tmpl.yml:225 computes RESULT_FILENAME=…_spec-none_conc1_… from env.SPEC_DECODING=none. ❌ Artifact filename claims non-MTP.
  4. process_agentic_result.py:139 reads os.environ.get("SPEC_DECODING", "none") → "none" and writes "spec_decoding": "none" into the aggregated result JSON. ❌ Result body claims non-MTP.
  5. A downstream MTP-vs-non-MTP Pareto plot that groups by spec_decoding puts this run in the non-MTP bucket, distorting the frontier.

  Fix

  Mechanical 9-line change — replace spec-decoding: none with spec-decoding: "mtp" on all 9 new entries in this block (lines 10943, 10957, 10986, 11000, 11014, 11029, 11044, 11059, 11074). Nothing else in the recipe files needs to change; the change is purely to keep the master-config label truthful to what the recipe actually runs.

[claude]

🔴 The new perf-changelog entry for qwen3.5-fp4-gb300-dynamo-sglang-agentic-{agg,disagg} (perf-changelog.yaml:4366-4373) will block the changelog gate on two independent grounds: (1) it's missing the required pr-link field, and (2) it's inserted mid-file rather than appended to the end. Fix: move the 8-line block to the end of the file and add pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2121 (or the XXX placeholder while WIP).

Extended reasoning...

What breaks. utils/validate_perf_changelog.py (invoked from .github/workflows/run-sweep.yml on every PR push) enforces three independent checks that this new entry violates. Any one of them is enough to fail the changelog gate; the entry fails all three.

1. Missing pr-link — schema-level rejection. In utils/matrix_logic/validation.py:683, ChangelogEntry declares pr_link: str = Field(alias="pr-link") with no default and extra="forbid". parse_changelog (validate_perf_changelog.py:128-133) calls ChangelogEntry.model_validate(entry) on every entry; Pydantic raises ValidationError on the new entry, which is re-raised as ChangelogValidationError('… fails ChangelogEntry validation'). Even if that were bypassed, validate_added_pr_link (lines 144-160) rejects it a second time: str(entry.get('pr-link') or '') == '', which is neither in PR_LINK_PLACEHOLDERS = {'XXX', 'https://github.com/SemiAnalysisAI/InferenceX/pull/XXX'} nor equal to the canonical PR URL, so it raises ChangelogValidationError('new PR entry must use …').

2. Mid-file insertion — positional rejection. The new 8-line block sits at lines 4366-4373, between an existing pull/1931 entry and the pre-existing minimaxm3-fp4-mi355x-vllm-mtp entry — with ~30 more historical entries after it. compare_entries (lines 163-208) walks base_entries and head_entries in lockstep by index; at the insertion index, head_entries[i] is the new qwen3.5 entry but base_entries[i] is the pre-existing minimaxm3 entry, and without_pr_link differs, so it raises ChangelogValidationError('entry N changed; existing entries are immutable except for pr-link-only corrections').

3. Byte-level prefix check. validate_raw_change (lines 211-235) additionally requires head_raw.startswith(base_raw) when there are additions. A mid-file insertion breaks that even before compare_entries runs, raising 'appended entries changed historical perf-changelog.yaml bytes; restore the base file byte-for-byte and append at the end'. AGENTS.md line 78 codifies the same rule: 'new entries MUST be appended to the END, never inserted in the middle or prepended.'

Step-by-step proof.

1. CI runs python3 utils/validate_perf_changelog.py --base-ref origin/main --head-ref HEAD.
2. parse_changelog reads perf-changelog.yaml and iterates entries. When it reaches the qwen3.5 entry, ChangelogEntry.model_validate({config-keys: [...], description: [...]}) fails because pr-link is required and absent → ChangelogValidationError → exit non-zero → the Validate perf-changelog check turns red → PR cannot merge. If pr-link were the only issue, adding pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2121 at the end of the block would clear check 1.
3. Even then, validate_raw_change computes base_raw = git show origin/main:perf-changelog.yaml and head_raw = the current file; because ~30 entries follow the insertion point in head_raw but not at the same byte offset in base_raw, head_raw.startswith(base_raw) is false → raises. Only after moving the 8-line block to the file end (after the current last entry) does this pass.
4. compare_entries then compares base and head entries index-by-index; index-of-first-difference is now len(base_entries), the appended entry is treated as an addition, validate_added_pr_link re-validates the pr-link, and the gate passes.

Fix. Two edits:

• Move the qwen3.5-fp4-gb300-dynamo-sglang-agentic-{agg,disagg} entry (perf-changelog.yaml:4366-4373) to the end of perf-changelog.yaml, after the current last entry.
• Add pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2121 to the moved block (or pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX for early WIP).

[csahithi]

Resolved both the above comments

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28977280171
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=28977280171

[functionstackx]

started about discussion about AgentX MTP +viz @kedarpotdar-nv

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28977625116
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=28977625116

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29055853595
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=29055853595

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29056354557
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=29056354557

[Ankur-singh]

/reuse-sweep-run

[Ankur-singh]

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
• Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that the single-node recipes are similar to the official vLLM recipes and/or theSGLang cookbook:
  • If they are not, I have verified that a PR has been opened in vLLM recipe repo or SGLang repo and linked it below in the additional detail section:
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.

Additional detail section:

• PR validation: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29056354557 / PR 验证：https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29056354557
• Evals: not applicable to this AgentX-only submission; the eval jobs are intentionally skipped. / 评估：本提交仅包含 AgentX 基准测试，因此不适用；评估任务按设计跳过。
• This AgentX multi-node submission includes disaggregated configurations, so no single-node recipe update is required. / 本 AgentX 多节点提交包含分离式配置，因此无需更新单节点 recipe。
• AgentX uses /v1/chat/completions, satisfying the MTP chat-template requirement. / AgentX 使用 /v1/chat/completions，满足 MTP 的 chat template 要求。
• The golden acceptance length is applied using real-draft-token, preserving real draft-token timing under the AgentX fairness convention. / 按照 AgentX 公平性约定，固定 acceptance length 使用 real-draft-token 模式，保留真实 draft token 的时序。

Signed: ankur-singh

[Klaud-Cold]

✅✅✅ Verdict: PASS ✅✅✅

✅ Check 0 (CODEOWNER): PASS — @Ankur-singh is a listed owner of configs/nvidia-master.yaml (the only specifically-owned changed path); remaining paths fall to the catch-all, which a recognized CODEOWNER satisfies.
✅ Check 1 (sweep on in-PR commit): PASS — full sweep 29056354557 ran on the PR head f644fd7 itself; all 9 executed multi-node agentic / benchmark jobs concluded success. single-node */ / eval / lanes were skipped because they don't apply to this agentic-only config set (run-sweep.yml hardcodes run-eval: false for agentic lanes), not because of a reuse-skip.
➖ Check 2 (evals pass): N/A — evals are structurally never run for agentic scenarios (verified in run-sweep.yml; find_reusable_sweep_run.py accepts bmk_agentic_* sources without evals). Sign-off leaves the eval box unchecked with this reasoning, per the checklist's exception item.
➖ Check 3 (recipe link): N/A — exclusively multi-node/disagg-stack submission (benchmarks/multi_node/srt-slurm-recipes/**, entries multinode: true, framework dynamo-sglang); the recipe-link requirement covers single-node recipes only.
✅ Check 4 (reuse command): PASS — /reuse-sweep-run posted by Ankur-singh (COLLABORATOR).
✅ Check 5 (latest template): PASS — all current PR_REVIEW_CHECKLIST.md items present; the two unchecked items (evals, recipe link) are each justified in the additional detail section.
✅ Check 6 (upstream image / engine-first): PASS — both new entries pin upstream lmsysorg/sglang:nightly-dev-cu13-20260709-074bb928; the engine is upstream SGLang (dynamo is only the disagg frontend; the agg entry serves via a pure sglang frontend), and a same-model/SKU qwen3.5-fp4-gb300-dynamo-sglang entry already exists. Informational: the perf-changelog entry cites image ...20260624-b2c8f7a2 while the config pins ...20260709-074bb928 — worth aligning, not blocking.
✅ Check 7 (no architecture hacks): PASS — no --hf-overrides/model-config edits; all flags are parallelism/quant/kernel selection. Informational: SGLANG_SIMULATE_ACC_LEN=3.39 (match-expected, real-draft-token) pins spec-decode acceptance length per the declared AgentX fairness convention; draft and verify computation still execute, so no architecture FLOPs are removed.
✅ Check 8 (spec-decode chat template): PASS — all MTP/NEXTN configs benchmark through build_replay_cmd, which sets --endpoint /v1/chat/completions --endpoint-type chat (benchmarks/benchmark_lib.sh:1448-1449).

[Ankur-singh]

@functionstackx @adibarra