Repository navigation
refactor(serve): move the engine-recipe overrides out of serve_cmd.rs - #891
Conversation
serve_cmd.rs carried about 1,700 production lines, most of them the pure transforms serve() applies to the selected engine's launch recipe hint. Move them to a new serve_recipe.rs: protocol_engine_recipe_hint and render_serve_engine_recipe_lines, the --tool-call-parser, --gpu-memory-utilization, --temperature/--top-p/--max-tokens and --engine-env overrides with their private rewriters and RESERVED_ENGINE_ENV_KEYS, engine_env_ignored_note and engine_recipe_enables_tool_choice. This is a pure move. The function bodies and the 28 tests that exercise them are byte-identical and keep their leaf names. Visibility is widened to pub(crate) only for what serve() and the AIM tests in serve_cmd.rs call (ServeGenerationDefaults and its three fields included); the private rewriters and the reserved-key list stay private. The AIM pre-flight and its tests stay in serve_cmd.rs, since their subject is the AIM checks and they only use protocol_engine_recipe_hint and engine_recipe_with_extra_env to build their input. Docs follow in the next commit. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Rewrite the serve_cmd.rs entry in docs/architecture.md, add one for the new serve_recipe.rs, and update the one clause that called ServeGenerationDefaults private. Update serve_cmd.rs's module doc, which still named the engine-recipe overrides as part of that file, and the two comments that cite the moved code by file (parse_gpu_memory_utilization's "elsewhere in this file" and the fix.rs test comment naming serve_cmd.rs). Comments and docs only. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
The serve_cmd.rs entry in docs/architecture.md said protocol_engine_recipe_hint and engine_recipe_with_extra_env are pub(crate) because the AIM tests use them. All the entry points are pub(crate) because serve() calls them; those two are also used by the AIM tests. Say so. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are recorded for the recipe-helper module extraction.
No open issues 🎉
What changed in this PR
Extracts the engine-recipe helpers from serve_cmd.rs into serve_recipe.rs, continuing the CLI’s module-splitting work without changing serving behavior.
Changes:
- Moves recipe mapping, flag overrides, generation defaults, and 28 existing tests into the new module.
- Preserves
serve()call sites through named imports, while keeping engine selection and AIM pre-flight logic inserve_cmd.rs. - Updates architecture documentation and source comments to reflect the new ownership.
| File | Description |
|---|---|
docs/architecture.md |
Documents the new module and revised responsibilities and visibility. |
crates/rocm-core/src/fix.rs |
Updates a test comment to cite the parser’s new location. |
apps/rocm/src/serve_recipe.rs |
Houses the relocated recipe helpers, defaults type, and tests. |
apps/rocm/src/serve_cmd.rs |
Imports the extracted helpers and removes their former definitions and tests. |
apps/rocm/src/main.rs |
Declares the private serve_recipe module. |
Review effort: Balanced
The only conflict was docs/architecture.md: the runtimes_cmd.rs entry is taken from main (the engine_install_env.rs and examine_cmd.rs notes), and the serve_cmd.rs and serve_recipe.rs entries from this branch. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
juhovainio
left a comment
There was a problem hiding this comment.
I read through the diff and this looks good. It is a pure move: the code removed from serve_cmd.rs matches what is added in serve_recipe.rs, apart from the license header, module docs, imports, the test scaffold and two doc-comment rewordings. No function body, message, flag or default changes. The architecture.md module map and its links are updated correctly, and CI is green.
Two small non-blocking notes:
- The "Relocated... no behavior change" history note in the
serve_recipe.rsmodule doc will age; consider dropping it later. architecture.mddrops "(Phase 6a)" from theserve_cmd.rsentry, which loses a bit of history.
This is based on a read of the diff plus the CI results.
…e 6a label Review notes on the serve_recipe.rs split. The module doc no longer says the code was relocated with no behavior change, since that sentence describes the move rather than the module and will age. The serve_cmd.rs entry in docs/architecture.md gets its "(Phase 6a)" label back, which the split had dropped from the entry. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Signed-off-by: Jussi Elo <jussi.elo@amd.com>
|
Both notes are addressed in e8740e7, with |
Summary
apps/rocm/src/serve_cmd.rshad grown to about 1,718 production lines (test module excluded, brace-matched), well past the 1,500-line size gate this series of module splits is working towards. Roughly 400 of those lines were one self-contained cluster: the pure transformsserve()applies, in order, to the selected engine's launch recipe hint (map the catalog record onto the protocol hint, then layer--tool-call-parser,--gpu-memory-utilization,--temperature/--top-p/--max-tokensand--engine-envon top). This PR moves that cluster out ofserve_cmd.rsinto a newserve_recipe.rs.serve()is the only production caller, and nothing in the moved span calls back intoserve_cmd.rs, so the seam is clean.serve_cmd.rsdrops to 1,318 production lines.It is a pure move. No function body, message, flag or default changes, and the 28 tests that exercise the moved functions move with them, byte-identical and with the same leaf names.
What moved
All of these moved verbatim from
serve_cmd.rstoserve_recipe.rs. The Visibility column is the only change:pub(crate)for every itemserve()calls (two of them are also used by the AIM tests that stay inserve_cmd.rs), otherwise unchanged.serve_recipe.rsrender_serve_engine_recipe_linespub(crate)protocol_engine_recipe_hintpub(crate)(also used by the AIM tests that stay)engine_recipe_with_tool_call_overridepub(crate)set_vllm_tool_call_parserServeGenerationDefaults(and its three fields)pub(crate), becauseserve()builds itengine_recipe_with_generation_defaultspub(crate)set_lemonade_generation_defaults,set_vllm_override_generation_configengine_recipe_with_gpu_memory_utilization_overridepub(crate)set_vllm_gpu_memory_utilizationparse_gpu_memory_utilizationpub(crate)RESERVED_ENGINE_ENV_KEYSengine_recipe_with_extra_envpub(crate)(also used by the AIM tests that stay)engine_env_ignored_note,engine_recipe_enables_tool_choicepub(crate)Other than the moved items, the diff is:
mod serve_recipe;inmain.rs; oneuse crate::serve_recipe::{...}inserve_cmd.rs(imported by name, so theserve()call sites are untouched); the now-unused imports removed fromserve_cmd.rs(std::fmt::Write,ENGINE_RECIPE_CONTRACT_VERSION,EngineRecipeEndpointHint,EngineRecipeUnsupportedCombinationHint, andBTreeMapin its test module). I re-derived the claim "verbatim" by diffing the moved line ranges onorigin/mainagainst the new file withpub(crate)stripped: all 1,019 moved lines match in order, and the only other lines inserve_recipe.rsare the license header, module doc, imports and themod testsscaffold.Not moved, and why: the AIM group that sits between the two moved spans in the old file (
engine_recipe_with_aim_gfx_profile,aim_gpu_preflight,aim_engine_record,ensure_aim_hf_token,aim_min_gpu_count,ensure_aim_multi_gpu_supported,aim_board_note) is a separate seam, the AIM pre-flight, so it stays inserve_cmd.rswith its tests. It is also the reason the cluster is not contiguous in the old file.Test assignment
Tests were assigned by primary subject. The 28 that moved each test one of the moved functions on its own:
protocol_engine_recipe_hint_maps_selected_engine_metadata, the fourtool_call_override_*, the fivegpu_memory_utilization_*, the sevengeneration_defaults_*, theextra_env_*andengine_env_*set (ten), andengine_recipe_enables_tool_choice_reflects_flags.These tests call both a moved function and something that stays, and they stay in
serve_cmd.rsbecause their subject is the AIM pre-flight, not the override they use to build their input:catalog_profile_pin_reaches_the_aim_hint_and_engine_env_overrides_it(subject:engine_recipe_with_aim_gfx_profilepinning the catalog's profile; it then usesengine_recipe_with_extra_envto show an explicit override wins),aim_serve_refuses_a_gpu_the_image_has_no_profile_for,gated_aim_model_needs_an_hf_token_before_launch,aim_board_note_only_fires_when_the_catalog_profile_is_what_runs,aim_gpu_preflight_probes_only_for_a_restricted_aim_record_off_cpu_onlyandaim_gpu_preflight_pins_the_profile_and_then_refuses_multi_gpu_images. They reachprotocol_engine_recipe_hintandengine_recipe_with_extra_envthrough theuse crate::serve_recipe::{...}import viause super::*; no shared fixture needed widening. The twoserve_notes_*tests stay too (they testcollect_serve_notes).Size
Production lines are counted with
#[cfg(test)] modblocks excluded by brace matching.serve_cmd.rsonorigin/mainserve_cmd.rsafter this PRserve_recipe.rs(new)main.rsonorigin/main/ afterThe 1,318 is under the 1,500 gate. (The planning figure of about 1,640 for
serve_cmd.rscame from an older audit; the same count on currentorigin/mainis 1,718, because that count includes comment and blank lines.)Crate edges
None added.
serve_recipe.rsdepends only onrocm_core,rocm_engine_protocolandanyhow, all already used byserve_cmd.rs;cargo xtask check-crate-edgesis unchanged.Verification
Ran, on each commit separately unless noted (commit 1 and commit 2 each compile and pass on their own):
cargo fmt --all --check: clean.cargo clippy -p rocm -p xtask --all-targets -- -D warnings: clean.cargo test -p rocm --bin rocm -- --listand-- --list --ignored, before (fromorigin/main) and after, compared by leaf name with the module prefix stripped: empty diff in both (1,475 tests, 8 ignored, before and after). Only the 28 moved tests change their full path (tests::xinserve_cmdtotests::xinserve_recipe); nothing is keyed to the old path (no--exactfilter,include_str!,file!()ormodule_path!()use, andapps/rocm/proptest-regressionshas noserve_cmdfile).cargo test -p rocm --no-fail-fast: all pass (1,467 passed, 8 ignored in the bin target, plus the other targets).cargo test -p rocm --bin rocm serve --no-fail-fast: 132 passed.cargo test -p rocm advised_commands: 20 passed, 1 ignored.cargo test -p rocm-core --lib the_utilization_hint_states_the_bound_the_parser_accepts(its comment cites the moved parser): passes.cargo xtask manifest --check,cargo xtask check-crate-edges,cargo test -p xtask ascii: all pass.serve_cmdacrossdocs/,AGENTS.md,CONTRIBUTING.md,xtask/src,tests/,.github/and.pre-commit-config.yaml. Thextask/src/ascii_output_contract.rsentries onserve_cmd.rs("Starting {model} on {selected_engine}…"and"Running smoke test…") are correct as they are because both literals live inserve().advised_commands.rsscans source by content, with noserve_cmd.rsentry. The only stale path was a comment incrates/rocm-core/src/fix.rs, fixed in commit 2.Not run:
--target x86_64-pc-windows-msvc): cannot run on this host. The build ofaws-lc-sysfails compiling C for the MSVC target (unknown type name 'pthread_rwlock_t') before reaching any workspace code. Nothing moved iscfg(windows)-gated (the onlycfgin the new file is#[cfg(test)], and thecfg!(windows)use inselect_serve_enginestays inserve_cmd.rs), so the Windows CI lane is the check.cargo xtask coverage: not run. Tests moved within the same crate and none were deleted or weakened.cargo xtask e2eandscripts/smoke_local.py: not run; CI carries them.User-visible behaviour
None. This is a refactor with no change to command output, exit codes, files or engine selection, so no Gherkin scenario is added or needed. The behaviour the moved code produces is already covered by the existing serve scenarios and by the moved unit tests, which still pass unchanged.
Notes for reviewers
Read commit by commit, then the merge. Commit 1 is the move (new
serve_recipe.rs,serve_cmd.rsshrink, onemodline inmain.rs);git diff --color-movedshows the production block as moved text. Commit 2 is docs and comments only: theserve_cmd.rsentry indocs/architecture.mdis rewritten, aserve_recipe.rsentry is added, and the one clause there that calledServeGenerationDefaultsprivate is updated;serve_cmd.rs's module doc no longer lists the overrides as its own; theparse_gpu_memory_utilizationdoc comment that said "elsewhere in this file" now namesserve_cmd.rs; and a test comment incrates/rocm-core/src/fix.rsthat cites the parser by file now namesserve_recipe.rs.A third commit merges
origin/maininto the branch (a merge rather than a rebase, so the existing commits keep their signatures). The only textual conflict wasdocs/architecture.md, where the already-merged engine-helper move (#872) and this PR both edited theruntimes_cmd.rsandserve_cmd.rsneighbourhood: theruntimes_cmd.rsentry is main's version, and theserve_cmd.rsandserve_recipe.rsentries are this PR's. #872 also edited the imports ofserve_cmd.rs(engine_manages_own_runtimenow comes fromengine_install_env.rs); git merged that without a conflict, and both of its lines are still present. The leaf-name test list of therocmcrate is the same on the merge result as onorigin/main.serve_cmd.rsis a hot file. I checked each open PR that touches it by merging its head with this branch (git merge-tree, read-only) and comparing against the same merge withorigin/main; that survey predates the merge oforigin/maininto this branch and has not been repeated. No open PR edited a moved line: none of theirserve_cmd.rshunks mention a moved symbol or test. Conflicts that already existed againstorigin/main(inserve()and the managed-service code) are unchanged by this PR: #351, #437, #854, #883, #723, #724 and #725. New conflicts introduced by this PR are one-line and mechanical:mod serve_recipe;next tomod serve_cmd;inmain.rsfor #817 and #243 (keep both lines). #857 and #678 merged cleanly inserve_cmd.rs. #724 and #725 also rewrap therocm_engine_protocolimport inserve_cmd.rs; whichever PR lands second needs to drop the three names this PR makes unused (ENGINE_RECIPE_CONTRACT_VERSION,EngineRecipeEndpointHint,EngineRecipeUnsupportedCombinationHint) when resolving that already-conflicting hunk, or clippy flags unused imports.