Skip to content

[AgentX nightly] dsv41flash MI325X FP4 vllm mtp / [AgentX nightly] dsv41flash MI325X FP4 vllm 升级每日构建 #3987

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
functionstackx wants to merge 2 commits into from
Closed

[AgentX nightly] dsv41flash MI325X FP4 vllm mtp / [AgentX nightly] dsv41flash MI325X FP4 vllm 升级每日构建 #3987

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
239a219
[AgentX nightly] dsv41flash MI325X FP4 vllm mtp
functionstackx Oct 13, 2026
437ca03
Record canonical PR link for #3987
functionstackx Oct 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...-e2e/benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/mi325x-fp4-mtp/agentic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ base:
name: dsv41flash-fp4-mi325x-vllm-agentic
model:
path: hf:deepseek-ai/DeepSeek-V4.1-Flash
container: vllm/vllm-openai-rocm:nightly-rocm100-3df4ae153eb385e27b52f26c81f8edb9e20b9984
container: vllm/vllm-openai-rocm:nightly-rocm100-5555a2d8f7116b2b4877f66516f4128176cf722a@sha256:e7d4d8643501172acc5e9a6fcc3739680359613c2904ec977cde9e721e798e82
precision: fp4
resources:
gpu_type: mi325x
Expand Down
2 changes: 1 addition & 1 deletion inferencex-e2e/configs/amd-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1120,7 +1120,7 @@ minimaxm3-fp8-mi325x-vllm-agentic-mtp:
# golden AL 3.51 for throughput, real block verification for evals, adaptive
# verification off on ROCm.
dsv41flash-fp4-mi325x-vllm-agentic-dspark:
image: vllm/vllm-openai-rocm:nightly-rocm100-3df4ae153eb385e27b52f26c81f8edb9e20b9984
image: vllm/vllm-openai-rocm:nightly-rocm100-5555a2d8f7116b2b4877f66516f4128176cf722a@sha256:e7d4d8643501172acc5e9a6fcc3739680359613c2904ec977cde9e721e798e82
model: deepseek-ai/DeepSeek-V4.1-Flash
model-prefix: dsv41flash
runner: cluster:mi325x-amd
Expand Down
7 changes: 7 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9683,3 +9683,10 @@
- "TP2 c16/c32: 16K prefill chunks with decoder SWA bounded replay and the prefill CUDA graph disabled, a 96/48-step prefill-decode interval, PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True and 4 tokenizer workers; SGLANG_NUMA_BIND_V2=0 at c16 (previously 4K chunks with the default 16-step interval). Local one-hour runs on the new image: c16 55,560 tok/s/GPU at 264.6 tok/s/user P90 (324.9 median); c32 113,613 at 183.1 (242.7)."
- "c64 and c128 move from TP2 to DEP2 (TP2/DP2/EP2 with DP attention, DP LM head and MegaMoE) behind sglang-router (cache_aware, dp-aware, routing key from the AIPerf correlation id), with total_requests load balancing, shortest-prefill-first scheduling, 16K chunks, 4096 SWA prefix tails and 4 tokenizer workers. c64 uses a 24-step prefill-decode interval and admits 128; c128 a 16-step interval, mem-fraction-static 0.85, admits 256 with a 128 graph batch. Local one-hour runs: c64 200,265 tok/s/GPU at 114.3 tok/s/user P90 (132.7 median), versus 149,666 at 99.4 median for the TP2 point it replaces; c128 271,364 at 81.6 P90 (93.0 median), versus 147,763 at 37.9 median."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3859

- config-keys:
- dsv41flash-fp4-mi325x-vllm-agentic-dspark
description:
- Pin dsv41flash MI325X FP4 vllm mtp AgentX to the 2026-10-13 image vllm/vllm-openai-rocm:nightly-rocm100-5555a2d8f7116b2b4877f66516f4128176cf722a@sha256:e7d4d8643501172acc5e9a6fcc3739680359613c2904ec977cde9e721e798e82.
- Preserve deployment topology, speculative mode, concurrency points, KV-offload settings and evaluation coverage.
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3987
Loading