[AMD][AgentX] MINIMAX-M3 FP4 MI355X agentX vLLM

[ajith-sirra-amd]

Summary

• Add MiniMax-M3 FP4 single-node agentic benchmark support on MI355X using vLLM
• New script: benchmarks/single_node/agentic/minimaxm3_fp4_mi355x.sh with lmcache (LMCache MP server + LMCacheMPConnector)
• New master config entry minimaxm3-fp4-mi355x-vllm-agentic with TP4 search space
• Image: vllm/vllm-openai-rocm:nightly-69715823df89b11ee684b84066390cbb9092d5c1
• Model: amd/MiniMax-M3-MXFP4

[claude]

Additional findings (outside current diff — PR may have been updated during review):

• 🔴 perf-changelog.yaml:4607 — perf-changelog.yaml is missing its trailing newline after pr-link: To-Be-Added (the diff footer shows \ No newline at end of file). utils/validate_perf_changelog.py explicitly rejects this: parse_changelog (line 89) and validate_matrix_compatible_change (line 352-354) both raise ChangelogValidationError("{path} does not end with a newline") — so the changelog-validation job will fail and block merge. Also worth updating pr-link: To-Be-Added to the actual PR URL before merge; every other entry uses a real link.

  Extended reasoning...

  What the bug is

  The new entry appended to perf-changelog.yaml terminates without an LF byte. Byte-level inspection (od -c) confirms the file ends with the raw bytes T o - B e - A d d e d — no trailing \n. The diff itself makes this explicit with the \ No newline at end of file marker after pr-link: To-Be-Added.

  Why existing code doesn't tolerate it

  utils/validate_perf_changelog.py has two independent hard checks for this exact condition:

  • parse_changelog (line 88-90): if not raw.endswith(b"\n"): raise ChangelogValidationError(f"{label} does not end with a newline")
  • validate_matrix_compatible_change (line 350-354) re-runs the same check on the head-ref bytes: if not head_raw.endswith(b"\n"): raise ChangelogValidationError(f"{path} at {head_ref} does not end with a newline")

  AGENTS.md line 176 explicitly warns about this file: "Never delete or modify whitespace in perf-changelog.yaml — CI depends on exact whitespace (including trailing spaces on blank separator lines). Altering it breaks CI."

  Impact

  The changelog-validation job will refuse this PR — this is a merge-blocking CI failure, not a cosmetic issue. It's also the failure mode AGENTS.md explicitly warns about: a future append to this file (via >> or a non-POSIX editor that doesn't insert an LF before writing) would physically concatenate the next - config-keys: onto the same line as To-Be-Added, silently corrupting the YAML.

  Step-by-step proof

  1. git show HEAD -- perf-changelog.yaml | tail -3 shows: pr-link: To-Be-Added followed by \ No newline at end of file.
  2. tail -c 30 perf-changelog.yaml | od -c returns ...9 2 d 5 c 1 " \n p r - l i n k : T o - B e - A d d e d — the final byte is d, not \n.
  3. Running validation: python utils/validate_perf_changelog.py (or its equivalent CI entrypoint) enters parse_changelog, which reads the file as bytes and immediately hits if not raw.endswith(b"\n") on line 89 → raises ChangelogValidationError: perf-changelog.yaml does not end with a newline.
  4. Every other entry in the file terminates with \n\n before the next - config-keys: — this is the only entry that breaks the invariant.

  Fix

  Append a single \n byte to the end of the file (e.g., printf '\n' >> perf-changelog.yaml or ensure the editor's "insert final newline" setting is on). Also update pr-link: To-Be-Added to https://github.com/SemiAnalysisAI/InferenceX/pull/2129 to match every other entry's convention before merge.

[claude]

🟡 The LMCache install at minimaxm3_fp4_mi355x.sh:93 clones https://github.com/LMCache/LMCache.git with no pinned tag/SHA (--branch/--depth), so each benchmark run installs whatever HEAD of the LMCache default branch happens to be at that moment — silently invalidating cross-run perf comparability, which is the whole point of this recipe. The clone is also non-idempotent under set -euo pipefail: a retried run in the same workspace dies at git clone with 'destination path already exists'. Prefer a versioned pip install (as in kimik2.5_fp4_b200.sh:99: agentic_pip_install --quiet --no-cache-dir lmcache), or pin the clone (git clone --depth 1 --branch <tag> ...) with an rm -rf LMCache guard first (as in minimaxm3_fp8_mi325x.sh:87 for mooncake).

Extended reasoning...

What the bug is

At benchmarks/single_node/agentic/minimaxm3_fp4_mi355x.sh:91-97, the lmcache branch of the offload-backend case builds LMCache from source like this:

lmcache)
    unset VLLM_USE_SIMPLE_KV_OFFLOAD
    git clone https://github.com/LMCache/LMCache.git
    cd LMCache
    pip install -r requirements/build.txt
    CXX=hipcc BUILD_WITH_HIP=1 pip install -e .   --no-build-isolation
    cd ..

There is no --branch <tag>, no --depth 1, no post-clone git checkout <sha> — the clone resolves to whatever HEAD of the LMCache default branch (main) is at run time.

Impact — reproducibility

The entire purpose of a recipe in this repo is to produce numerically comparable perf points that get ingested into perf-changelog.yaml and rolled up across time. The single search-space row in configs/amd-master.yaml for this recipe is:

- { tp: 4, kv-offloading: dram, kv-offload-backend: lmcache, conc-list: [1, 4, 8, 16, 32] }

i.e. every configuration this recipe measures goes through the lmcache branch and therefore through the LMCache library. LMCache is an active project — LMCache/main moves. Two runs of this recipe on different days will install different LMCache revisions and can produce different tokens/s, TTFT and cache-hit numbers without any change to this repository. That is the exact anti-pattern the codebase avoids everywhere else: configs/amd-master.yaml pins every vLLM/SGLang image to a digest-suffixed nightly tag (see the dsv4-fp4-mi355x-vllm block comment that explicitly warns "pin to a digest-suffixed nightly tag rather than the floating :nightly"), and the sibling Mooncake source builds in minimaxm3_fp8_mi325x.sh:87 and minimaxm3_fp8_mi300x.sh:87 use git clone --depth 1 --branch "$mooncake_tag" ... with mooncake_tag='v0.3.11.post1'.

Impact — idempotency

The script begins with set -euo pipefail at line 2. git clone <url> into an existing non-empty directory fails with fatal: destination path 'LMCache' already exists and is not an empty directory, exit code 128. There is no rm -rf LMCache guard or [[ -d LMCache ]] || ... check. If a prior run in the same workspace didn't clean up (retried run, workspace reuse across configs in the sweep, or actions/checkout not pruning untracked directories), the second attempt aborts before the LMCache server ever starts.

Regression from the pre-existing pattern

The sibling NVIDIA recipe benchmarks/single_node/agentic/kimik2.5_fp4_b200.sh:99 uses a versioned PyPI install through the isolated agentic venv:

agentic_pip_install --quiet --no-cache-dir lmcache

This was the established pattern for installing LMCache in an agentic recipe. Switching to an unpinned HEAD clone is a strict regression. (The MI355X build does need a HIP source build — no ROCm wheel on PyPI — so falling back to a git clone is fine, but it must be pinned.)

Step-by-step proof of the non-idempotency failure

1. Sweep dispatches the config { tp: 4, kv-offloading: dram, kv-offload-backend: lmcache, conc: 1 }; the script cds into the workspace, runs git clone https://github.com/LMCache/LMCache.git, builds, benchmarks, exits 0. LMCache/ remains in the workspace.
2. Sweep dispatches the next config { tp: 4, ..., conc: 4 }; the script cds into the same workspace again, hits git clone at line 93.
3. Git prints fatal: destination path 'LMCache' already exists and is not an empty directory and returns exit 128.
4. set -e immediately aborts the script with exit 128 — wait_for_lmcache_ready, vllm serve, and run_agentic_replay_and_write_outputs never run. The config is recorded as failed.

(Whether the workspace persists across configs depends on the launcher; the current launch_mi355x-amds.sh mounts $GITHUB_WORKSPACE at /workspace, so in practice fresh CI containers usually paper over this — but any local re-run or workspace-reuse path trips the failure.)

Fix

A one-line change, either

agentic_pip_install --quiet --no-cache-dir lmcache==<version>

or, keeping the HIP source build:

rm -rf LMCache
git clone --depth 1 --branch <tag-or-sha> https://github.com/LMCache/LMCache.git

Severity

Nit — a clean container run in CI passes, the numerical result is some valid measurement, and there is no user-visible crash on the green path. But this is directly identified as an anti-pattern relative to the sibling recipes in the same directory and it silently erodes benchmark comparability over time, so worth calling out before merge.

[claude]

🟡 The docstring, check_env_vars call, and variable defaults in the new script are internally inconsistent: (1) the header lists OFFLOADING but check_env_vars actually requires KV_OFFLOADING; (2) the header omits DURATION, EP_SIZE, DP_ATTENTION, and KV_OFFLOAD_BACKEND, all of which check_env_vars enforces; (3) the DURATION=${DURATION:-1800} and EP_SIZE=${EP_SIZE:-1} defaults at lines 19-20 are dead code (check_env_vars has already exited if either was empty); and (4) the header enumerates none / cpu / lmcache as OFFLOADING values but the case statement switches on $KV_OFFLOAD_BACKEND and only implements the lmcache arm — cpu and none silently fall through to --no-enable-prefix-caching. The launcher passes the correct vars so CI is unaffected, but the header will actively mislead anyone reading or running this stand-alone. Fix by syncing the docstring with the actual required vars and either dropping the unreachable defaults or removing DURATION/EP_SIZE from check_env_vars.

Extended reasoning...

The script header at lines 5-20 sets up a runtime interface that contradicts itself on four related points. This is doc/dead-code inconsistency in a brand-new file, so nothing breaks in the CI-driven flow — but this recipe is very likely to be copy-pasted (it is the first vLLM+LMCache MI355X agentic entry), and future readers will be misled.

(1) Wrong variable name in the required-vars list. Line 7 documents OFFLOADING, but line 14 calls check_env_vars MODEL TP CONC KV_OFFLOADING TOTAL_CPU_DRAM_GB RESULT_DIR DURATION EP_SIZE DP_ATTENTION. Sibling recipes (dsv4_fp4_b300_sglang.sh:7, kimik2.5_fp4_b300.sh:10, qwen3.5_fp8_h100.sh:15, minimaxm3_fp8_mi325x.sh) all consistently use the KV_OFFLOADING name — the header here is out of sync with the codebase convention.

(2) Missing required vars. The header lists 6 vars; check_env_vars enforces 9. DURATION, EP_SIZE, and DP_ATTENTION are all required at line 14 but never mentioned in the header. A user reading only the header, exporting the 6 documented vars, and running the script will hit Error: The following required environment variables are not set: DURATION EP_SIZE DP_ATTENTION.

(3) Dead-code defaults. Lines 19-20 set DURATION=${DURATION:-1800} and EP_SIZE=${EP_SIZE:-1}. But check_env_vars (benchmarks/benchmark_lib.sh:234) uses [[ -z "${!var_name:-}" ]] to detect unset OR empty and exits 1 if any listed var is missing. So by the time execution reaches line 19, DURATION and EP_SIZE are guaranteed non-empty — the :- fallbacks are unreachable. Compare minimaxm3_fp8_mi325x.sh:7, which requires the same vars in check_env_vars and correctly does NOT repeat the defaults.

(4) Values list documents a third variable name. Lines 9-12 enumerate none / cpu / lmcache as OFFLOADING values. The case statement at line 90 switches on $KV_OFFLOAD_BACKEND — a third, undocumented name — and only implements the lmcache arm. Setting KV_OFFLOAD_BACKEND=cpu (or =none) falls through to the initial OFFLOAD_ARGS=(--no-enable-prefix-caching) from line 47, which is neither vLLM native CPU offload nor GPU-KV-only in any meaningful sense.

Step-by-step proof. Take the header at face value and run: MODEL=amd/MiniMax-M3-MXFP4 TP=4 CONC=8 OFFLOADING=lmcache TOTAL_CPU_DRAM_GB=3000 RESULT_DIR=/tmp/x ./minimaxm3_fp4_mi355x.sh. set -u is on, so line 14 checks KV_OFFLOADING (which the user did not set because the header said OFFLOADING); check_env_vars reports 4 missing vars: KV_OFFLOADING, DURATION, EP_SIZE, DP_ATTENTION. Even if the user then also exports KV_OFFLOADING=lmcache DURATION=1800 EP_SIZE=1 DP_ATTENTION=false, the case at line 90 needs KV_OFFLOAD_BACKEND=lmcache — still-undocumented — otherwise LMCache never starts and vLLM runs with --no-enable-prefix-caching only.

Fix. Two clean options: (a) sync the header — replace OFFLOADING with KV_OFFLOADING, add KV_OFFLOAD_BACKEND, DURATION, EP_SIZE, DP_ATTENTION to the required list, drop the invented cpu/none values (or implement them), and delete lines 19-20; or (b) treat DURATION/EP_SIZE as optional-with-defaults — remove them from check_env_vars, keep the defaults, and add them to the header as optional. The sibling minimaxm3_fp8_mi325x.sh follows option (a); either is fine so long as the docstring and code match. Non-blocking (nit) — CI passes because configs/amd-master.yaml supplies the right variables via the launcher, so this only bites stand-alone users and future readers.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29024988092
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=29024988092

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29032224772
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=29032224772

[ajith-sirra-amd]

/reuse-sweep-run

[seungrokj]

@ajith-sirra-amd
#2118
looks better than this. Why don't we go with PR2118 ?

[functionstackx]

thanks for the contribution @seungrokj @ajith-sirra-amd

can u please update recipes.vllm.ai with ur new configs?

vllm recipes doesn't currently support offloading tho the inferact guys said they are open to contributions to support mooncake store, lmcache, vllm native offloading support in vllm recipes

as in sglang cookbook recipes , they also have support for cpu offloading recipes