Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ base:
name: minimaxm3-fp8-mi300x-vllm-agentic
model:
path: hf:MiniMaxAI/MiniMax-M3-MXFP8
container: vllm/vllm-openai-rocm:v0.30.0
container: vllm/vllm-openai-rocm:nightly-5555a2d8f7116b2b4877f66516f4128176cf722a@sha256:e0be00f1b99da608cc3c8571c7c86bae0c05f19e1e61a59c1d70aeea6bca6140
precision: fp8
resources:
gpu_type: mi300x
Expand Down
2 changes: 1 addition & 1 deletion inferencex-e2e/configs/amd-master.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1033,7 +1033,7 @@ dsr1-fp4-mi355x-sglang-disagg-mtp:
decode: {num-worker: 1, tp: 8, ep: 8, dp-attn: true}

minimaxm3-fp8-mi300x-vllm-agentic-mtp:
image: vllm/vllm-openai-rocm:v0.30.0
image: vllm/vllm-openai-rocm:nightly-5555a2d8f7116b2b4877f66516f4128176cf722a@sha256:e0be00f1b99da608cc3c8571c7c86bae0c05f19e1e61a59c1d70aeea6bca6140
model: MiniMaxAI/MiniMax-M3-MXFP8
model-prefix: minimaxm3
runner: cluster:mi300x-amd
Expand Down
7 changes: 7 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9683,3 +9683,10 @@
- "TP2 c16/c32: 16K prefill chunks with decoder SWA bounded replay and the prefill CUDA graph disabled, a 96/48-step prefill-decode interval, PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True and 4 tokenizer workers; SGLANG_NUMA_BIND_V2=0 at c16 (previously 4K chunks with the default 16-step interval). Local one-hour runs on the new image: c16 55,560 tok/s/GPU at 264.6 tok/s/user P90 (324.9 median); c32 113,613 at 183.1 (242.7)."
- "c64 and c128 move from TP2 to DEP2 (TP2/DP2/EP2 with DP attention, DP LM head and MegaMoE) behind sglang-router (cache_aware, dp-aware, routing key from the AIPerf correlation id), with total_requests load balancing, shortest-prefill-first scheduling, 16K chunks, 4096 SWA prefix tails and 4 tokenizer workers. c64 uses a 24-step prefill-decode interval and admits 128; c128 a 16-step interval, mem-fraction-static 0.85, admits 256 with a 128 graph batch. Local one-hour runs: c64 200,265 tok/s/GPU at 114.3 tok/s/user P90 (132.7 median), versus 149,666 at 99.4 median for the TP2 point it replaces; c128 271,364 at 81.6 P90 (93.0 median), versus 147,763 at 37.9 median."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3859

- config-keys:
- minimaxm3-fp8-mi300x-vllm-agentic-mtp
description:
- Pin minimaxm3 MI300X FP8 vllm mtp AgentX to the 2026-10-13 image vllm/vllm-openai-rocm:nightly-5555a2d8f7116b2b4877f66516f4128176cf722a@sha256:e0be00f1b99da608cc3c8571c7c86bae0c05f19e1e61a59c1d70aeea6bca6140.
- Preserve deployment topology, speculative mode, concurrency points, KV-offload settings and evaluation coverage.
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3986
Loading