Repository navigation
ROCMAI-556: populate the AIM catalog and warn on stale entries - #794
juhovainio wants to merge 10 commits into
Conversation
nowycondro
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · a2ae9e5
Review — needs work
Full review of the whole change (bbc3a3f...a2ae9e5, 10 files, 2491/-54).
Implements ROCMAI-556 per its own description (catalog population, staleness warning, HF-token gating, gfx/board/multi-GPU checks). Ticket ROCMAI-556 is named but not retrievable from this review environment (no Jira tool available here) — relationship not independently verified against the ticket itself.
Blocking
-
This PR's own change breaks the required "Sphinx docs build (-W)" CI check —
README.md:12322
The new README.md line... (see [docs/aim-catalog.md](docs/aim-catalog.md))is a markdown link, and the live CI run for this exact commit fails withREADME.md:12322: WARNING: 'myst' cross-reference target not found: 'docs/aim-catalog.md'→build finished with problems, 1 warning (with warnings treated as errors)→ exit 1. This is exactly the pitfall this repo's own AGENTS.md §5 documents: README.md/CONTRIBUTING.md are{include}d into the separate Sphinx docs site, where a relative link to a file outside that site failssphinx-build -W; the convention is a bare backtick path instead (which CONTRIBUTING.md's own new paragraph in this same PR correctly uses:see `docs/aim-catalog.md`). Confidence 100 · mechanical · Fix: change the README.md citation to a bare backtick path, matching the CONTRIBUTING.md citation added in the same commit. -
validate_model_recipe_engine_record's "must not be empty" guard was not extended for 5 of the 6 new optional fields —crates/rocm-core/src/model_catalog.rs(final guard invalidate_model_recipe_engine_record, just beforebail!("... must not be empty"))
Of the six new fields (generated_at_unix_ms,supported_gfx_families,profile_ids,requires_hf_token,supported_boards,min_gpu_counts), onlysupported_gfx_familieswas added to the empty-check condition. A minimalModelRecipeEngineRecord { engine: "aim", requires_hf_token: true, ..Default::default() }(or one carrying onlygenerated_at_unix_msorsupported_boards) passes every earlier check and then incorrectlybail!s "must not be empty" even though it carries real information. Not reachable through today's catalog data (every realaimentry also hasnotes/image_digest), but it is a latent trap for the next contributor adding a minimal entry. Confidence 85 · logic · Fix: extend the condition with&& engine_recipe.generated_at_unix_ms.is_none() && engine_recipe.profile_ids.is_empty() && !engine_recipe.requires_hf_token && engine_recipe.supported_boards.is_empty() && engine_recipe.min_gpu_counts.is_empty(). -
The new AIM gfx-family refusal is not skipped for
--device cpu_only, unlike every other GPU-availability check this PR and its neighbors added —apps/rocm/src/serve_cmd.rs(computation ofaim_host_family/ call toengine_recipe_with_aim_gfx_profile, ahead of wherecpu_onlyis parsed) vs themin_gpu_countsenforcement a few hundred lines later, which is explicitly gatedif let (Some(needed), false) = (aim_min_gpus, cpu_only)at both of its call sites
aim_host_family/engine_recipe_with_aim_gfx_profile(which canbail!("the AIM image for ... has no profile for this GPU ...")) run unconditionally, structurally beforedevice_policy/cpu_onlyare even computed, while every sibling GPU-hardware check in the same function is gated on!cpu_only. A user runningrocm serve <model> --engine aim --device cpu_onlyon a host whose GPU family isn't insupported_gfx_familiesis refused with a GPU-compatibility error despite asking not to use the GPU. No test exercisescpu_onlytogether with the gfx-family check. Confidence 72 · logic · Fix: gate the gfx-family check on!cpu_onlythe same way the multi-GPU check is gated, or add an explicit, clearer refusal for--engine aim --device cpu_onlyif AIM can never serve CPU-only at all. -
The new
rocm model --verbosestaleness-warning line has no test confirming it is actually rendered —apps/rocm/src/main.rs(new loop inrender_model_registry_verbose_text_with_context_and_hostthat callsrocm_core::model_recipe_staleness_warningper engine recipe)
model_recipe_staleness_warningitself is well unit-tested as a pure function inmodel_catalog.rs, but no test inmain.rsfeeds a stalegenerated_at_unix_msfixture through the verbose-rendering path and asserts the warning text appears in the output; the one touched verbose-rendering test exercises only the (fresh) built-in catalog. If the rendering loop were deleted or its condition broken, nothing in the suite would fail — the PR text itself says as much ("the rendering hook inrocm modelis not directly tested"). Confidence 78 · mechanical · Fix: add a test mirroring the file's existing pattern of injecting a craftedModelRecipeEngineRecordwith a stale timestamp and asserting the rendered text contains the warning. -
Four of the five new user-observable
rocm servebehaviors have no Gherkin/e2e coverage, and the PR text only justifies the omission for one of them — no changes undertests/e2e-cucumber/features/
This repo's AGENTS.md §3 requires a Gherkin scenario (or an explicit stated reason) for any change altering user-observable CLI behavior — command output, exit codes, or which runtime/engine is selected — and explicitly says a unit test "proves the function, not the behavior." The PR text justifies skipping a scenario only for the staleness warning (no clock injection). It says nothing about why the HF-token refusal, the gfx-family refusal, the board-mismatch warning, or the multi-GPU refusal — all new refusal/warning paths reachable before any network pull, so none obviously need real hardware or a pulled image to exercise end-to-end — have no scenario. Confidence 80 · architectural · Fix: add e2e scenarios for these four paths (they fail-fast before pulling, so a scenario against the catalog data alone should suffice), or state in the PR text why each is exempt.
Non-blocking
docs/aim-catalog.mdstep 2 tells a maintainer to checkAIM_PROFILE_IDinextra_env, but this PR moves that pin toprofile_ids—docs/aim-catalog.md(refresh step 2)
The Qwen3.5-9B entry's profile pin became aprofile_idsentry in this same PR, and new Radeon entries useprofile_idsexclusively, yet the refresh instructions still point a future maintainer atextra_env. Confidence 72 · mechanical · Fix: update step 2 to sayprofile_ids.
Decisions for the author
aim_board_note's board-name matching is a coarse substring check, not scoped to the specific gfx family actually matched — non-blocking-improvement
name_key.contains(&squash(board))could in principle over/under-match a future board whose name is a superset of an existing one, and the warning fires off the wholesupported_boardslist rather than the family that was matched. Today's real board names (W7900/R9700 vs. RX 7900 XTX) are verified correct by the included test, so this is forward-looking rather than a present defect.- Host GPU/gfx state is probed through at least three separate code paths within one
serve()call (detect_host_gfx_target(),detect_host_gpu_summary(), and the existing GPU-index/VRAM probes) — non-blocking-improvement
host_gpu_summaryis already deliberately skipped in some branches for performance, yetaim_gpu_namere-probes it independently whenever an AIM recipe pins profiles. Consider caching one probe perserve()invocation. validate_model_recipe_engine_recorddoes not requiresupported_gfx_familiesto be non-empty whenengine == "aim"— non-blocking-improvement
Today's invariant ("everyaimentry names its platforms") holds only because a test asserts it over the fixed built-in data, not because the schema enforces it. A futureaimrecipe added withoutsupported_gfx_familieswould silently land under whichever platform happens to listaimfirst ("AMD Radeon"), regardless of its real target hardware.- 37 of the 38
aimentries share the identicalgenerated_at_unix_msmillisecond timestamp — tradeoff
Consistent with a single bulk-authoring pass rather than genuinely independent per-image verification against the upstream registry, which sits a little in tension with the field's stated meaning ("when a maintainer last checked this entry"). Worth the author confirming each image really was individually checked rather than stamped in bulk, since the whole staleness mechanism's value depends on that being true.
Positive signals
- Every new refusal/warning helper in
serve_cmd.rs(ensure_aim_hf_token,ensure_aim_gpu_count,aim_min_gpu_count,aim_board_note,engine_recipe_with_aim_gfx_profile) has a dedicated unit test that asserts on the actual branch outcome rather than just string presence, and each would fail if its guard were weakened. validate_model_recipe_engine_recordadds real cross-field schema checks for the new data (profile_ids/min_gpu_countskeys must be a subset ofsupported_gfx_families,min_gpu_countsmust be ≥ 2, the new fields are rejected on non-aimengines) — this is the right place to catch the exact class of key-mismatch bug these new BTreeMaps invite, and it's exercised by its own tests.docs/aim-catalog.mdis a clear, step-by-step maintenance runbook that a future, different maintainer (the PR itself says the AIM relationship owner, not necessarily a rocm-cli maintainer) could follow without additional context.
Deployment notes
- The PR branch currently has real merge conflicts against the current tip of
main(confirmed withgit merge-tree:CONTRIBUTING.md,README.md, andapps/rocm/src/main.rsall conflict), matching GitHub's ownmergeable_state: dirtyfor this PR. It needs a rebase before it can land, independent of the findings above. - This adds
requires_hf_tokenenforcement that is a hard behavior change for 10 existing-catalog-adjacent gated models:rocm servenow refuses them outright withoutHF_TOKEN, where previously (per the PR) nothing enforced this.
What this covered
Every file in the diff, read completely: crates/rocm-core/src/model_catalog.rs, crates/rocm-core/src/model_catalog.json (structurally validated as a whole plus a representative field-by-field sample of new/changed entries), crates/rocm-core/src/lib.rs, apps/rocm/src/serve_cmd.rs, apps/rocm/src/main.rs, docs/aim-catalog.md, docs/manual-testing.md, docs/testing.md, README.md, CONTRIBUTING.md, against bbc3a3f9fb2fc52254df9a59f4f39d95942f67a9...a2ae9e59d1931be7d1bc74a6033455c0782ba867. Two independent fan-out workers each covered one coupled file group (serve_cmd.rs+main.rs; model_catalog.rs+json+lib.rs); the driver (this context) independently cross-read both groups' diffs in full as well as ran the agent-instruction-adherence, impact-analysis, and CI-status passes directly, including fetching the actual failing-check log and running git merge-tree against current main.
Did not run: "prior changes" pass (whether review comments on earlier changes to these files apply here) — not checked, no practical way to retrieve it in this environment; a git log/history pass was run instead and showed only routine prior extraction/refactor commits, nothing contradicting this change. Clippy/full test suite were not re-run locally; CI's own clippy and test jobs are taken at face value (both green) per the project's own CI-authoritative convention, except for the Sphinx job noted above. Not reconciled against any PR discussion or review thread — none was read.
tomastola
left a comment
There was a problem hiding this comment.
Blocking
1. Hugging Face ids that served on vLLM now route to AIM, and on Instinct some are refused. crates/rocm-core/src/model_catalog.json (the 35 new AIM-only recipes plus the renamed Qwen/Qwen3.5-9B); apps/rocm/src/serve_cmd.rs select_serve_engine
Before this PR, ids such as meta-llama/Llama-3.1-8B-Instruct, openai/gpt-oss-20b, Qwen/Qwen3-32B and meta-llama/Llama-3.3-70B-Instruct had no recipe. On an Instinct host, recipe.is_none_or(..) was therefore true, and select_serve_engine picked vLLM. Each of these ids is now the canonical_model_id of a recipe whose preferred_engines and engine_recipes contain only aim. model_recipe_supports_engine(recipe, "vllm") is false, so selection falls through to aim. The effects:
- Six ids are refused on MI300X. Their AIM records cover only
gfx110X-all/gfx120X-all:Qwen/Qwen3.5-9B,meta-llama/Llama-3.1-8B-Instruct,google/gemma-3n-E4B-it,Qwen/Qwen3-VL-8B-Instruct,Qwen/Qwen3.6-35B-A3B,zai-org/GLM-4.7-Flash.rocm serve <id>on MI300X now fails with "no profile for this GPU". - The Instinct-class ids change engine without notice. They move from a vLLM pull of the Hugging Face weights to a Docker AIM image. The gated ones also start requiring
--engine-env HF_TOKEN. - Other callers shift too.
dash.rs,freeform_planner.rsandmodel_readiness.rsall readpreferred_engines.first(). - The description doesn't mention it. It says the default
rocm modellist is unchanged, but not thatrocm serveresolves these ids to a different engine.
Suggested fixes, either one:
- add
vllmto these recipes'preferred_engines/engine_recipes, so the Instinct preference to vLLM still applies; or - key the AIM-only recipes on non-HF ids (e.g.
aim-<model>, with the image name as an alias), so a plain HF id keeps resolving as it did.
Please also add a select_serve_engine test: a newly catalogued HF id on a gfx94X host still selects vLLM.
2. The Sphinx docs build fails on a README link. README.md:803
[docs/aim-catalog.md](docs/aim-catalog.md) sits inside the README range that docs/rocm-docs/commands.md pulls in with {include}. Sphinx docs build (-W) fails on a2ae9e5 with 'myst' cross-reference target not found: 'docs/aim-catalog.md'. AGENTS.md §5 calls out this case for README.md. Use an absolute GitHub URL, as README already does elsewhere.
3. The staleness warning only appears with --verbose, but the text says plain rocm model shows it. apps/rocm/src/main.rs render_model_registry_verbose_text_with_context_and_host; README.md:801-803; docs/aim-catalog.md ("Freshness")
The loop calling model_recipe_staleness_warning exists only in the verbose renderer. The new AIM entries aren't featured, so they never reach the default list anyway. Two ways to make code and text agree:
- emit the warning wherever these entries are used (
rocm serveresolution is the natural place); or - narrow the README and doc text to
rocm model --verbose.
Non-blocking
- The multi-GPU refusal fires late, and its first message misleads. (
serve_cmd.rs, the twoensure_aim_gpu_countcalls)select_auto_gpu_indexreturns at most one GPU, so the second check refuses every multi-GPU AIM image. It only does so afterensure_self_managed_engine_readyand theResolveModelRPC have run, although the outcome is already known at the first check.- The first check's "only N usable GPUs were found" implies that more GPUs would help. They won't, until multi-GPU selection lands.
- Suggest refusing at the first check with the "does not support multi-GPU serving yet" message, and dropping the second check.
min_gpu_mem_gbcontradictsmin_gpu_countsin the same record.openai/gpt-oss-120bgets 281 GB. That is the ×2 rule applied to an MXFP4 model whose AIM profiles run on one MI300X.zai-org/GLM-4.7gets 426 GB, where one gfx950 GPU is implied.- As a result,
rocm model --verboseshows "doesn't fit" on hardware AIM supports. Suggest capping each estimate atmin_gpu_counts× per-GPU VRAM, or leaving it unset where the dtype isn't fp8/bf16.
- A validator error omits a field it checks. The "only apply to aim" error in
model_catalog.rschecksmin_gpu_counts, but the message doesn't list it. - Some doc text doesn't match the code.
docs/aim-catalog.mdstep 2 says to checkAIM_PROFILE_IDinextra_env. No AIM entry uses that any more, and the validator rejects it alongsideprofile_ids.- "Open a PR" is step 5, but steps 6–10 then add more fields.
docs/manual-testing.mdsays--verboseshowsrequires_hf_token, which it never prints.
- The staleness warning sends end users to a repository path. An installed binary prints "see docs/aim-catalog.md to refresh". That's a maintainer action, and the warning also applies to entries from an external signed index. Suggest telling users to update rocm-cli, and keeping the doc pointer for maintainers.
Add a per-engine-recipe generated_at_unix_ms, warn in `rocm model` once it is over 180 days old, and document the hand-maintained AIM refresh process. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Adds 38 GPU chat AIM images, each pinned to the newest stable tag's digest and stamped with generated_at_unix_ms. All are hidden from the default list. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Name each AIM recipe by its model (e.g. Qwen/Qwen3-VL-8B-Instruct) as the catalog page lists it, with the image as an alias. Models that already have a recipe (Llama-3.2-3B, gemma-4-31B) gain an aim engine record instead of a duplicate recipe. aim-qwen3.5-9b is now Qwen/Qwen3.5-9B and keeps its old id as an alias. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Ten catalog images are gated per the AIM catalog's model pages. Their engine recipe notes now say to export HF_TOKEN and pass --engine-env HF_TOKEN. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
The platform list predates AIM, and the only AIM image at the time was Radeon-targeted. The catalog now carries Instinct-class images. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Record supported_gfx_families per AIM engine recipe from the image's profile table and pin AIM_PROFILE_ID per gfx family for the Radeon images, whose auto-selection is unreliable. rocm serve refuses a GPU with no profile, rocm model places recipes by the platform covering those families, and Strix Halo no longer lists aim since no profile supports gfx1151. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
…r AIM entries Gated AIM images carry requires_hf_token and rocm serve refuses them before launch without HF_TOKEN. Radeon entries record supported_boards and serve warns when the detected GPU is none of them. 32 AIM entries get an estimated min_gpu_mem_gb. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Record min_gpu_counts per gfx family from each image's profile table. rocm serve refuses an image that needs several GPUs when --gpu pins one device, too few GPUs are usable, or serve would select a single GPU, instead of letting the container fail inside the image. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
… and GPU checks - keep vllm as a second preferred engine on AIM-catalogued HF ids so Instinct hosts still select vLLM - refuse multi-GPU AIM images at the first check and drop the late second check - skip the AIM gfx-family check for --device cpu_only - cap/unset min_gpu_mem_gb estimates that contradicted min_gpu_counts - extend the engine-recipe empty check and the aim-only error to the new fields - point end users at updating rocm-cli in the staleness warning - scope README/docs to rocm model --verbose; use a bare path for the Sphinx include - test the staleness rendering helper and select_serve_engine routing Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Signed-off-by: Juho Vainio <juho.vainio@amd.com>
a2ae9e5 to
5846184
Compare
|
Pushed 5846184: the branch is rebased onto current Blocking
Non-blocking, addressed
Tests
Left as is (author decisions)
Local gates: clippy and fmt clean. The pre-push hook's full run failed only on the known-flaky |
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review (Claude Code) · 5846184
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — needs work
Full review of the whole change (cc56179...5846184, 12 files, +2637/-57), as it stands at this exact head commit.
Implements the staleness-tracking and hand-maintained-update-process scope of ROCMAI-556 ("Define AIM catalog freshness/update mechanism"), plus substantial additional work the ticket's own text does not call out (populating 35 new AIM catalog entries, HF-token gating, gfx-family/board/multi-GPU enforcement at serve time) — fully disclosed in the PR body, not a misstatement.
This commit is a revision of an earlier version of this PR that was reviewed before (at a2ae9e5, by this same automation and by a human reviewer). Several of those earlier findings are fixed here: the README Sphinx-build link, the validator's "must not be empty" guard now covering all six new fields, the --device cpu_only AIM gfx-check now properly skipped, the staleness-warning render path now directly tested, the HF-id-routes-to-AIM-on-Instinct regression now fixed and tested, and two of the four previously-missing Gherkin scenarios now added (with an explicit, code-verified justification for the other two).
Blocking
- CI
clippyfails on this exact commit, caused by code this PR adds —tests/e2e-cucumber/tests/e2e/serving_steps.rs:1364-1366
Theclippycheck's "Clippy (e2e harness + steps)" step (cargo clippy --locked -p e2e-cucumber --test e2e -- -D warnings) fails with two errors, both inside the newassert_no_service_startedstep added in this PR:let records = std::fs::read_dir(&services_dir) .map(|entries| entries.count()) .unwrap_or(0);
clippy::map_unwrap_or("calledmap(<f>).unwrap_or(<a>)on aResultvalue") andclippy::redundant_closure_for_method_calls(the closure can be the method itself). Confirmed directly from the live job log for head commit58461849(job id 124145420973):error: could not compile e2e-cucumber (test "e2e") due to 2 previous errors, exit code 101. The same step passed on the prior base commit, so this is newly introduced by this PR, not a pre-existing flake. The PR body itself flags the risk ("Clippy--all-targetswas clean on the first commit, not re-run after the catalog commit"), which is exactly what happened here. Confidence 100 · mechanical · Fix:.map_or(0, |entries| entries.count())(or.map_or(0, Iterator::count)to also satisfy the redundant-closure lint), per clippy's own suggested edit.
Non-blocking
docs/aim-catalog.md's refresh-process numbered list skips "5." —docs/aim-catalog.md(the ordered list under "Refreshing an entry")
The list runs 1, 2, 3, 4, 6, 7, 8, 9, 10, 11 — item 5 was evidently dropped during editing and the rest never renumbered. Markdown renders ordered lists sequentially regardless of the literal numbers typed (so the rendered page looks fine), but the raw source is misleading to anyone reading the diff, the file in a non-rendering context, or cross-referencing "step N" later. Confidence 90 · mechanical · Fix: renumber 6-11 to 5-10, or restore whatever step was intended at 5.
Decisions for the author
aim_board_note's board match is a case-insensitive alphanumeric substring check, not scoped to the matched gfx family — tradeoff, already disclosed by the author
The PR's own "Known gaps" section states this plainly ("The board check matches the GPU name by substring and only warns; the detected name is the host's primary GPU, not necessarily the one selected with--gpu"), so this isn't a finding so much as a confirmation that the stated limitation matches the code exactly as described.validate_model_recipe_engine_recorddoes not requiresupported_gfx_familiesto be non-empty whenengine == "aim"— non-blocking-improvement
Every current built-inaimentry does carry a non-empty list (confirmed by the newaim_recipes_are_matched_to_the_platforms_their_profiles_supporttest, which asserts this over today's data), but that's a test over the fixed catalog, not a schema rule. A futureaimentry added withoutsupported_gfx_familieswould silently land under whichever platform listsaimfirst and never trigger the gfx refusal at all.- Host GPU/gfx state is still probed through multiple independent code paths within one
serve()call — non-blocking-improvement
detect_host_gfx_target()(foraim_host_family) anddetect_host_gpu_summary()(for the pre-existinghost_gpu_summary, and again foraim_gpu_namewhenhost_gpu_summarywasn't already computed) are each called fromserve(). Consider caching one probe per invocation. - 37 of the 38
aimentries share the identicalgenerated_at_unix_msmillisecond timestamp — tradeoff
Consistent with a single bulk-authoring pass rather than independently verifying each image against the upstream registry at a distinct time, which sits in some tension with the field's stated meaning ("when a maintainer last checked this entry"). Worth the author confirming each was individually checked.
Positive signals
aim_catalogued_hf_ids_still_select_vllm_on_instinct_and_aim_elsewhere(serve_cmd.rs) directly asserts the exact regression a prior round of review raised — that newly-catalogued HF ids (e.g.meta-llama/Llama-3.1-8B-Instruct,openai/gpt-oss-20b) still selectvllmon an Instinct host — by checkingpreferred_enginesnow lists bothaimandvllm, traced end-to-end againstselect_serve_engine.- The
--device cpu_onlyAIM gfx-check skip is now structural (aim_host_familyis filtered to!cpu_onlybefore the gfx-support check ever runs) rather than bolted on, and theNone-family path it relies on is covered byaim_gfx_check_is_skipped_when_the_gpu_is_unknown_or_the_engine_is_not_aim. - The new Gherkin scenarios' skip-justification comment for the gfx-family/board/multi-GPU paths holds up under independent check: unlike
rocm install sdk --family <arch>,rocm serve's host-gfx detection has no override flag, so those three paths genuinely can't be driven from the mock e2e lane the way the HF-token path can. - Every full catalog cross-check I ran (profile_ids/min_gpu_counts keys ⊆ supported_gfx_families, min_gpu_counts ≥ 2, requires_hf_token ⇔ the HF_TOKEN note, aim-only fields never set on non-aim engines) came back with zero violations across all 38
aimengine recipes.
Deployment notes
requires_hf_tokenenforcement is a hard, user-facing behavior change for the same 10 gated models:rocm servenow refuses them outright withoutHF_TOKEN, where nothing enforced this before.- Catalog population changes default engine routing for previously recipe-less ids on non-Instinct hosts: e.g.
rocm serve Qwen/Qwen3-32Bwith no--engineon a Radeon host now resolves toaim(viapreferred_engines.first()) and is refused immediately ("no profile for this GPU") rather than falling through to whatever the old recipe-less default was. - The heavy
Test (affected crates)/build-and-test/windows-build-and-testjobs are skipped on this PR's own CI run and only execute on the merge-queue push (confirmed by comparing this run against a recent merge-queue run of the same workflow) — this is pre-existing, repo-wide CI design, not specific to this PR, but it meansclippyplus the Sphinx/lint/dash-coverage jobs are the only compiler-verified signals available before merge.
What this covered
Every file in the diff, read completely, against cc56179ced9dd1be42a3000b2592ace9394dbff1...5846184904c98b6165b0b9b8db89cdfe0d798946: CONTRIBUTING.md, README.md, apps/rocm/src/examine_report_text.rs, apps/rocm/src/serve_cmd.rs, crates/rocm-core/src/lib.rs, crates/rocm-core/src/model_catalog.json (every one of its 49 recipes and 38 aim engine recipes checked programmatically for cross-field consistency, plus manual sampling), crates/rocm-core/src/model_catalog.rs, docs/aim-catalog.md, docs/manual-testing.md, docs/testing.md, tests/e2e-cucumber/features/model_serving.feature, tests/e2e-cucumber/tests/e2e/serving_steps.rs. CI was read directly for the exact head commit (get_check_runs plus the actual job logs for clippy, the changes job, and Coverage (rocm-dash crates, ratcheted)), and git merge-tree confirmed no merge conflicts against current main.
This ran as a single-context pass rather than a multi-worker fan-out, per explicit instruction for this run to avoid a background-worker-hang failure mode seen in earlier attempts — the review-code skill's own documented fallback for when fan-out is unavailable. Per that skill's measurements, a single-context pass typically surfaces a smaller fraction of the findings a fan-out would; depth was partially compensated for here by running structural/consistency checks programmatically over the full catalog data and by reading CI logs directly rather than taking job conclusions at face value.
Did not run as a strictly blind review: this task required a mandatory pre-review claim-check (get_reviews) that incidentally surfaced the text of two earlier reviews of a prior commit on this same PR before any code was read. Every substantive claim carried forward from that context was independently re-verified against the code, the live catalog data, or the CI logs at this exact head commit before being included above (each finding above states what was actually traced); nothing was taken on the earlier reviews' word alone, and several of their claims were checked and found already fixed. "Prior changes" (review comments on earlier changes to these specific files, independent of this PR's own prior rounds) was not separately checked — no practical way to retrieve that here. Not reconciled against any PR discussion thread beyond the two prior formal reviews surfaced by the mandatory claim-check.
Implements ROCMAI-556
What
Catalog: adds the AMD Enterprise AI AIM images to
model_catalog.json. Recipes use the real model name from the catalog page as the canonical id (e.g.Qwen/Qwen3-VL-8B-Instruct) with the image as an alias; 35 are new recipes, and models that already had a recipe (meta-llama/Llama-3.2-3B-Instruct,google/gemma-4-31B-it) gain anaimengine record instead.aim-qwen3.5-9bis nowQwen/Qwen3.5-9Band keeps the old id as an alias. Where the catalog has both a Radeon and a generic image for one model (Llama-3.1-8B-Instruct), the Radeon image is used. Each is pinned to the digest of its newest stable tag (read withoras) and stamped withgenerated_at_unix_ms. None are featured, so the defaultrocm modellist is unchanged; they resolve viarocm serveand show inrocm model --verbose.Staleness: adds an optional per-engine-recipe
generated_at_unix_ms(when a maintainer last verified the pinned image).rocm modelwarns when it is over 180 days old; entries without it are never flagged.Docs: adds docs/aim-catalog.md (hand-maintained refresh process using
oras, AMD Enterprise AI Suite releases as the trigger), linked from README and CONTRIBUTING.Gated models: ten images the AIM catalog marks as requiring a Hugging Face token (Llama 3.1/3.2/3.3, Gemma 3/3n, MedGemma, Command A Reasoning) set
requires_hf_token.rocm serverefuses them before anything is pulled unlessHF_TOKENis passed with--engine-env(bare name with the variable exported, orNAME=value), and the error names that command.GPU matching: each AIM engine recipe records
supported_gfx_families, taken from the profile table on the image's model page (Instinct images: MI250X/MI300X/MI325X/MI350X/MI355X; Radeon images: W7900 and R9700 only).rocm serverefuses a detected GPU outside that list with the supported families; if the GPU can't be detected nothing is enforced.rocm modelplaces a recipe under the platform whose gfx families it supports, which also fixes AIM recipes being labelled Strix Halo.Profile pinning: Radeon images get
profile_ids(gfx family toAIM_PROFILE_ID, the lowest-GPU-count optimized latency profile), applied automatically because their auto-selection is unreliable; two need several GPUs and say so in their notes. Instinct images leave AIM's auto-selection alone.--engine-env AIM_PROFILE_ID=...still wins. The Qwen3.5-9Bextra_envpin becameprofile_ids.Board check: Radeon entries record
supported_boards(W7900, R9700). Since a gfx family covers cards the pinned profile was not built for (for example RX 7900 XTX),rocm serveprints a warning, without refusing, when the detected GPU name matches none of them.Multi-GPU: 19 images need more than one GPU on at least one gfx family (smallest tensor-parallel size in the family's profiles, recorded as
min_gpu_counts; for example DeepSeek-R1 needs 8 on MI300X, GLM-4.7-Flash needs 2 on W7900 and 4 on R9700).rocm serverefuses such an image before anything is pulled when--gpupins one device, fewer usable GPUs exist, or serve would select a single GPU, because the container would otherwise start with too few devices and fail inside the image. It does not add multi-GPU launching.Min VRAM: 32 new entries get an estimated
min_gpu_mem_gb(total parameters x 1 byte, or 2 when the image has no fp8 profile, +20%, rounded up). The AIM docs publish GPU counts but no VRAM, so these are estimates, stated as such in docs/aim-catalog.md. DeepSeek-V4-Flash, MiniMax-M3 and GLM-5.2 are left unset because their parameter counts are not published.Platforms: AMD Instinct lists
aim(label... (vLLM / AIM)). Strix Halo no longer listsaim: no AIM profile supports gfx1151.Not included
aim-epyc-*images (rocm-cli never falls back to CPU).task: chatand I did not check that the serve path handles other tasks.0.12.0-previewbecause itsAIM_PROFILE_IDpin is not verified against the newer2026.11.0tag.Known gaps
min_gpu_mem_gbvalues are estimates, not measurements.--gpu.rocm serveselects one GPU. They are refused up front (see Multi-GPU above); multi-GPU selection is tracked in ROCMAI-751.Testing
cargo test -p rocm-core model_andcargo test -p rocm --bin rocm model_pass; unit tests cover the staleness threshold and that the built-in AIM entry carries a timestamp. Clippy--all-targetswas clean on the first commit, not re-run after the catalog commit.rocm modelis not directly tested.