diff --git a/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm3/vllm/mi300x-fp8-mtp/agentic.yaml b/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm3/vllm/mi300x-fp8-mtp/agentic.yaml index ad1cb5bd46..9a8677070d 100644 --- a/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm3/vllm/mi300x-fp8-mtp/agentic.yaml +++ b/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm3/vllm/mi300x-fp8-mtp/agentic.yaml @@ -5,7 +5,7 @@ base: name: minimaxm3-fp8-mi300x-vllm-agentic model: path: hf:MiniMaxAI/MiniMax-M3-MXFP8 - container: vllm/vllm-openai-rocm:v0.30.0 + container: vllm/vllm-openai-rocm:nightly-5555a2d8f7116b2b4877f66516f4128176cf722a@sha256:e0be00f1b99da608cc3c8571c7c86bae0c05f19e1e61a59c1d70aeea6bca6140 precision: fp8 resources: gpu_type: mi300x diff --git a/inferencex-e2e/configs/amd-master.yaml b/inferencex-e2e/configs/amd-master.yaml index 0a2ea83d01..c678038c0e 100644 --- a/inferencex-e2e/configs/amd-master.yaml +++ b/inferencex-e2e/configs/amd-master.yaml @@ -1033,7 +1033,7 @@ dsr1-fp4-mi355x-sglang-disagg-mtp: decode: {num-worker: 1, tp: 8, ep: 8, dp-attn: true} minimaxm3-fp8-mi300x-vllm-agentic-mtp: - image: vllm/vllm-openai-rocm:v0.30.0 + image: vllm/vllm-openai-rocm:nightly-5555a2d8f7116b2b4877f66516f4128176cf722a@sha256:e0be00f1b99da608cc3c8571c7c86bae0c05f19e1e61a59c1d70aeea6bca6140 model: MiniMaxAI/MiniMax-M3-MXFP8 model-prefix: minimaxm3 runner: cluster:mi300x-amd diff --git a/inferencex-e2e/perf-changelog.yaml b/inferencex-e2e/perf-changelog.yaml index 4a40f3a1ed..4d38545f0e 100644 --- a/inferencex-e2e/perf-changelog.yaml +++ b/inferencex-e2e/perf-changelog.yaml @@ -9683,3 +9683,10 @@ - "TP2 c16/c32: 16K prefill chunks with decoder SWA bounded replay and the prefill CUDA graph disabled, a 96/48-step prefill-decode interval, PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True and 4 tokenizer workers; SGLANG_NUMA_BIND_V2=0 at c16 (previously 4K chunks with the default 16-step interval). Local one-hour runs on the new image: c16 55,560 tok/s/GPU at 264.6 tok/s/user P90 (324.9 median); c32 113,613 at 183.1 (242.7)." - "c64 and c128 move from TP2 to DEP2 (TP2/DP2/EP2 with DP attention, DP LM head and MegaMoE) behind sglang-router (cache_aware, dp-aware, routing key from the AIPerf correlation id), with total_requests load balancing, shortest-prefill-first scheduling, 16K chunks, 4096 SWA prefix tails and 4 tokenizer workers. c64 uses a 24-step prefill-decode interval and admits 128; c128 a 16-step interval, mem-fraction-static 0.85, admits 256 with a 128 graph batch. Local one-hour runs: c64 200,265 tok/s/GPU at 114.3 tok/s/user P90 (132.7 median), versus 149,666 at 99.4 median for the TP2 point it replaces; c128 271,364 at 81.6 P90 (93.0 median), versus 147,763 at 37.9 median." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3859 + +- config-keys: + - minimaxm3-fp8-mi300x-vllm-agentic-mtp + description: + - Pin minimaxm3 MI300X FP8 vllm mtp AgentX to the 2026-10-13 image vllm/vllm-openai-rocm:nightly-5555a2d8f7116b2b4877f66516f4128176cf722a@sha256:e0be00f1b99da608cc3c8571c7c86bae0c05f19e1e61a59c1d70aeea6bca6140. + - Preserve deployment topology, speculative mode, concurrency points, KV-offload settings and evaluation coverage. + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3986