Repository navigation
basert: install the ROCm engine on Linux x86-64 with a Strix Halo or Strix Point GPU - #118
Conversation
…Strix Point GPU BaseRT 0.5.0 publishes basert-engine-linux-x64-rocm-<v>.tar.gz, one bundle for gfx1151 and gfx1150, linked against ROCm 10 at /opt/rocm/core-10.0. superfluid refused Linux x86-64 outright, so install.sh and `superfluid runtime install basert` could not reach it. - Host reads the AMD GPUs' LLVM targets from the KFD topology (gfx_target_version) and the ROCm trees under /opt/rocm/core-<version>, not counting the core/core-10 alternatives links. - The basert recipe maps Linux x64 with gfx1151/gfx1150 to linux-x64-rocm. A release's engine is refused without ROCm 10 at /opt/rocm/core-10.0, and another or no AMD GPU is refused with the supported targets named. - devices() says ROCm on x86-64 and names the GPU "AMD gfx1150". - install.sh tries the engine on x86-64 when /dev/kfd exists, and the x86-64 release build checks the basert worker starts, as arm64 does. - the_pins_are_whole accepts a linux-x64-rocm asset, so the 0.5.0 pin needs no test change at release time.
…d elsewhere
For baseRT --backend chooses nothing: each machine runs one engine, and the
option only checks it. The new test pins that: on gfx1151 and gfx1150,
--backend rocm plans exactly the plain install; on Apple silicon and a GB10
it is refused ("runs on metal/cuda here, not rocm"); on Linux x86-64
without a Strix GPU it gets the supported-platforms refusal.
IsuraManchanayake
left a comment
IsuraManchanayake
left a comment
There was a problem hiding this comment.
Codex Review
Reviewed ff4e82c157ee200e4bc1a72df0d8e7dd27d6e1a8 against af2b95ce1897ec0cca371a8d2ef360ce30d25b56, including all eight changed files, runtime installation/loading, worker environment inheritance, and the target engine's device selection and sidecar lookup. There were no existing review comments.
Two findings are attached inline:
- P2: finding a supported AMD GPU somewhere in the machine does not ensure the engine selects it.
- P3: the ROCm device report can use an unrelated NVIDIA GPU's name.
Validation in an isolated checkout on macOS:
- Adapter library tests: 28 basert + 32 kit + 36 llama.cpp passed; 2 llama.cpp tests ignored.
- Daemon library tests with
--no-default-features: 410 passed, 1 ignored. - Packaging scripts: 28 passed.
- Clippy for the kit, basert and llama.cpp adapters, all targets, with warnings denied: passed.
- Shell syntax checks and six fixture scenarios for the actual installer branch under
dash -eu: passed, including AMD absent/present, opt-out, and engine-install failure. - Compiled the Linux GPU-name/discovery function bodies with filesystem paths redirected to fixtures: AMD-only reported
AMD gfx1150; adding an NVIDIA information entry changed the reported name toNVIDIA GeForce RTX 4060 Laptop GPUwhile the AMD target remainedgfx1150. git diff --checkpassed.
Limits: no ROCm hardware inference was run. The mixed-AMD selection finding is based on the planner test and inspection of the engine's device-0 selection/rejection path, not a physical multi-GPU reproduction. CI run 285 was still running at submission; docs and packaging jobs had passed.
| match (host.os.as_str(), host.arch.as_str()) { | ||
| ("macos", "arm64") => Ok(("macos-arm64", "metal")), | ||
| ("linux", "arm64") if host.nvidia_driver.is_some() => Ok(("linux-arm64-cuda", "cuda")), | ||
| ("linux", "x64") if host.amd_gpus.iter().any(|g| ROCM_GPUS.contains(&g.as_str())) => Ok(("linux-x64-rocm", "rocm")), |
There was a problem hiding this comment.
Codex Review — [P2] Match admission to the device the engine will actually use. On a mixed-AMD host where HIP device 0 is an unsupported GPU (for example gfx1100) and another device is gfx1150/gfx1151, this any() returns a tested ROCm install plan, but neither the plan nor worker startup selects that supported device. The target engine opens HIP device 0 and rejects an unsupported architecture before loading a model, so the newly admitted host still cannot serve without manual device filtering. The new [gfx1100, gfx1151] test only checks the plan and therefore misses this failure. Select the supported device consistently for checking and serving, or return an actionable device-selection requirement; cover an unsupported effective device 0 rather than only the presence of a supported KFD node.
There was a problem hiding this comment.
Accepted, fixed in 6c434a5. The plan now judges HIP device 0, not any KFD node.
Host.amd_gpusnow lists the AMD GPUs the way HIP numbers them. That's KFD's GPU nodes in node order, narrowed byROCR_VISIBLE_DEVICESand then byHIP_VISIBLE_DEVICES(orCUDA_VISIBLE_DEVICESwhen that is unset), following AMD's GPU isolation rules. Only index lists are read. A list that names devices by UUID leaves the GPUs as they are, and the engine's own device-0 check decides.- A Strix GPU behind another AMD GPU is refused, with its device number and the variable to set, for example:
baseRT's ROCm engine runs on HIP device 0, which is gfx1100 here; the gfx1151 is HIP device 1. Make it device 0 for superfluid, when it installs and when it serves: HIP_VISIBLE_DEVICES=1 (...). superfluid's workers inherit its environment (launch.rsonly adds the plan's variables), so starting superfluid with that variable gives the install plan and the serving engine the same device 0. - Regression test: the
[gfx1100, gfx1151]host inlinux_x86_64_with_a_strix_gpu_takes_the_rocm_enginenow expects that refusal. On the previous code it failed withcalled Result::unwrap_err() on an Ok value: [Plan { ... backend: "rocm", ... why: "Linux x64, AMD gfx1100 + gfx1151, ROCm 10.0: baseRT's ROCm engine" ... }]. A[gfx1150, gfx1100]host, whose device 0 is supported, is still admitted. The kit testhip_numbers_the_gpus_the_visibility_lists_leavecovers how the lists narrow: reordering, ROCR's list before HIP's, an out-of-range index ending the list, repeats, UUIDs and empty values.
I didn't make superfluid pick the device itself. The engine works on HIP device 0, so choosing another device belongs in the engine, not in a variable superfluid would set without the user knowing. Limit: this wasn't run on a machine with two AMD GPUs (we have none). The ordering follows ROCm's documented rules.
| #[cfg(target_os = "linux")] | ||
| if let Some(gfx) = superfluid_adapter_kit::install::amd_gpus().into_iter().next() { | ||
| return format!("AMD {gfx}"); |
There was a problem hiding this comment.
Codex Review — [P3] Choose the GPU name according to the backend. This AMD lookup is only reached after nvidia_model() has returned nothing. A Strix Point laptop with an NVIDIA discrete GPU therefore reports a device such as {backend: "ROCm", name: "NVIDIA GeForce RTX 4060 Laptop GPU", ...}, with the AMD engine's memory budget attached to the NVIDIA name. I reproduced that result using the Linux function bodies and KFD/NVIDIA filesystem fixtures. For the x86-64 ROCm backend, obtain the AMD device name directly instead of giving the NVIDIA probe precedence; retain the NVIDIA lookup for the CUDA backend.
There was a problem hiding this comment.
Accepted, fixed in 6c434a5. devices() now passes its backend to gpu_name. On Linux the name comes from linux_gpu_name(backend, ...):
- the ROCm backend names AMD device 0 (
amd_gpus().first(), with the same numbering the plan uses); - the CUDA backend names the NVIDIA model;
- neither falls back to the other vendor's GPU.
Regression test the_device_is_named_by_the_gpu_the_engine_runs_on covers a Strix Point laptop with an NVIDIA GPU. With the previous order it failed: left: Some("NVIDIA GeForce RTX 4060 Laptop GPU"), right: Some("AMD gfx1150").
… backend Review fixes (Codex P2, P3). The ROCm engine opens HIP device 0 and refuses a GPU it has no kernels for, but the plan admitted a Strix GPU found anywhere in KFD: with a gfx1100 as device 0 and a gfx1151 behind it, the install went ahead and the engine then refused device 0. Host.amd_gpus now lists the GPUs as HIP numbers them (KFD node order, narrowed by ROCR_VISIBLE_DEVICES, then HIP_VISIBLE_DEVICES or CUDA_VISIBLE_DEVICES), and the plan judges device 0: a Strix GPU behind another one gets a refusal that names its device number and HIP_VISIBLE_DEVICES=<n>. The device report took the NVIDIA model whenever the NVIDIA driver answered, so a Strix Point laptop with an NVIDIA GPU reported the ROCm engine's budget under the NVIDIA name. The name now follows the backend: AMD device 0 for ROCm, the NVIDIA model for CUDA.
|
Codex review of ff4e82c: I accepted both findings, and they're fixed in 6c434a5. Each has a regression test that fails on ff4e82c; the details are in the threads.
Checks run on macOS: kit 33/33 and basert 29/29 library tests, and |
IsuraManchanayake
left a comment
IsuraManchanayake
left a comment
There was a problem hiding this comment.
Codex Review
Re-reviewed 6c434a595b21eeb28355ffccee8982997d5a6e22 together with public baseRT #92 and the engine-side packaging changes, including both previous findings and the author's replies.
The previous P3 GPU-vendor naming issue is fixed: the ROCm path now names the AMD device and the CUDA path names NVIDIA. The unfiltered mixed-AMD example also now produces the intended refusal. However, two P2 findings remain in the new visibility handling and the proposed recovery command; both are attached inline and reproduced independently. The same cases affect baseRT's updated installer.
Validation on this head:
- Adapter library tests: 29 basert + 33 kit + 36 llama.cpp passed; 2 llama.cpp tests ignored.
- Clippy for these three adapters, all targets with warnings denied: passed.
- An external probe linked against this checkout's actual
hip_devicesandrecipe::plan_withreproduced UUID-based false refusals, acceptance with GPUs hidden, and a recovery command that selects the unsupported GPU. The valid numeric selector control plans the ROCm engine successfully. - Full PR
git diff --checkpasses. CI run 286 and Docs run 180 passed.
These are host-fact/planner reproductions, with expected visibility semantics checked against AMD's documentation and runtime source. No physical multi-GPU inference was run. I did not repeat the unchanged daemon/packaging suites from the earlier review.
| let Some(list) = list.filter(|l| !l.trim().is_empty()) else { | ||
| return gpus; | ||
| }; | ||
| let Ok(indices) = list.split(',').map(|i| i.trim().parse::<usize>()).collect::<Result<Vec<_>, _>>() else { | ||
| return gpus; |
There was a problem hiding this comment.
Codex Review — [P2] Resolve visibility masks before using this list as an install gate. The fallback to raw GPUs is not deferred to the engine: recipe::platform rejects on the first entry before an engine can run. I linked a probe against this head: raw [gfx1100, gfx1151] with ROCR_VISIBLE_DEVICES=<UUID of gfx1151> still returns both GPUs and plan_with refuses installation, even though ROCm exposes the supported GPU as device 0. Adding HIP_VISIBLE_DEVICES=0 makes this helper keep only the wrong gfx1100. Conversely, ROCR_VISIBLE_DEVICES="" and HIP_VISIBLE_DEVICES=-1 are treated as unfiltered and produce a successful ROCm plan when the runtime exposes no GPU. AMD supports UUID selection; ROCr and HIP also handle empty/invalid selectors differently. Preserve UUID identity and unset-versus-empty state and match the two parsers' semantics, or carry an explicit unknown result rather than treating an unresolved mask as proof of device 0. Update the tests that currently assert UUID/empty values leave every GPU visible.
There was a problem hiding this comment.
Accepted, fixed in 06e8072. I reproduced all three cases on 6c434a5 through hip_devices and plan_with:
ROCR_VISIBLE_DEVICES=<UUID>withHIP_VISIBLE_DEVICES=0gave[gfx1100]and a refusal.ROCR_VISIBLE_DEVICES=""gave[gfx1151]andOk(rocm).HIP_VISIBLE_DEVICES=-1gave[gfx1151]andOk(rocm).
hip_devices is replaced by hip_view. It reads each list the way its runtime does, per the two sources you linked:
- ROCr (
ROCR_VISIBLE_DEVICES): unset and set-but-empty are now different; empty shows no GPU. Entries are trimmed and read up to the number of GPUs, parsed withstrtolin base 0 (so0x1counts). An entry that doesn't parse, an index out of range, or a repeat ends the list. - HIP (
HIP_VISIBLE_DEVICES, orCUDA_VISIBLE_DEVICESwhen that is empty): only the plain decimal form counts (01,1and-1don't). An invalid entry ends the list, a repeat is skipped, and the indices count what ROCr left. - UUIDs: a UUID entry that either runtime would actually reach makes the order unknown (
HipView.knownfalse,Host.amd_order_unknown). The plan then admits a Strix GPU and leaves device 0 to the engine's own check when it starts. That's the "explicit unknown" option. I didn't match UUIDs here because ROCr and HIP match them differently: ROCr uppercases and matches a prefix ofGPU-%016X, skipping devices with a zero unique ID, while HIP looks for a case-sensitive substring ofGPU-%016x, orGPU-XXfor a zero ID. A wrong guess would bring back the false refusal. The host JSON now reportsamd_order_unknown.
Tests:
hip_numbers_the_gpus_as_rocr_then_hip_read_their_listsreplaces the test that locked in the old UUID and empty behaviour. It covers each rule above, including entries past the GPU count, a final comma,,1(ROCr would throw on the empty entry, so it counts as unknown), and a UUID after an invalid HIP entry, which is never reached.the_visibility_variables_decide_device_0_as_rocm_reads_themchecks your three cases at the plan level. On 6c434a5 it failed on the first one:assertion failed: plan(&strix_seeing(&["gfx1100", "gfx1151"], Some("GPU-1a2b3c4d5e6f7a8b"), Some("0"))).is_ok().
Not run: a real machine with UUID-selected or multiple AMD GPUs.
| let i = host.amd_gpus.iter().position(strix).unwrap_or(0); | ||
| Err(format!( | ||
| "baseRT's ROCm engine runs on HIP device 0, which is {} here; the {} is HIP device {i}. Make it device 0 for \ | ||
| superfluid, when it installs and when it serves: HIP_VISIBLE_DEVICES={i} (where HIP_VISIBLE_DEVICES or \ | ||
| ROCR_VISIBLE_DEVICES is set already, list it first there)", |
There was a problem hiding this comment.
Codex Review — [P2] Do not use the filtered ordinal as a replacement HIP selector. i is an index in Host.amd_gpus after HIP/CUDA filtering, while a new HIP_VISIBLE_DEVICES value indexes the pre-HIP list. With raw [gfx1151, gfx1100] and HIP_VISIBLE_DEVICES=1,0, the current planner tells the user to set HIP_VISIBLE_DEVICES=1. Applying that instruction leaves only gfx1100, so the planner and engine still reject the machine; the correct replacement is HIP_VISIBLE_DEVICES=0. I reproduced both the error and the successful 0 control through the actual hip_devices/plan_with functions. Keep the original identity or pre-HIP index through host detection and derive the instruction from it, accounting for any ROCR filter; the parenthetical to reorder existing lists does not correct the explicit command.
There was a problem hiding this comment.
Accepted, fixed in 06e8072. You were right: the offered value was the GPU's place after the HIP list, but HIP counts the list before it. With raw [gfx1151, gfx1100] and HIP_VISIBLE_DEVICES=1,0, 6c434a5 offered 1, which selects the gfx1100.
HipView.hip_indices (and Host.amd_hip_indices) now keeps each GPU's index among the GPUs ROCr leaves, which is the number HIP_VISIBLE_DEVICES reads. The refusal offers that index and drops the old "list it first there" note: ... which is gfx1100 here; the gfx1151 is HIP device 1. Run superfluid with HIP_VISIBLE_DEVICES=0 to make it device 0, when it installs and when it serves.
the_visibility_variables_decide_device_0_as_rocm_reads_them applies each offered command and plans again:
- your case:
1,0is offered0, and with0the plan admits; - a ROCr list that reorders: raw
[gfx1151, gfx1100, gfx1150]withROCR_VISIBLE_DEVICES=1,0,2andHIP_VISIBLE_DEVICES=0,1is offered1(the gfx1151's ROCr index), and with1the plan admits; - a HIP list that leaves no Strix GPU gets the plain refusal, since there's nothing to select.
The ROCR_VISIBLE_DEVICES value is left as it is, because the index is counted after it.
…he selector HIP reads Review fixes (Codex P2 x2 on 6c434a5). Visibility: hip_devices treated an empty, a UUID or an invalid list as no filter, so ROCR_VISIBLE_DEVICES=<UUID of the gfx1151> with HIP_VISIBLE_DEVICES=0 was refused (it saw only the gfx1100), and ROCR_VISIBLE_DEVICES="" or HIP_VISIBLE_DEVICES=-1 planned the ROCm engine although ROCm shows no GPU. hip_view now reads each list as its runtime does (ROCr: set-but-empty shows no GPU, strtol base 0, a repeat or bad entry ends the list; HIP: plain decimals only, a repeat is skipped, a bad entry ends the list, counting what ROCr left). A UUID entry the runtime would reach makes the order unknown (Host.amd_order_unknown): the plan then admits a Strix GPU and leaves device 0 to the engine's own check instead of guessing from the node order. Selector: the refusal offered the GPU's place after the HIP list as the new HIP_VISIBLE_DEVICES, but HIP counts the list before it: raw [gfx1151, gfx1100] with HIP_VISIBLE_DEVICES=1,0 was told 1, which selects the gfx1100. Host.amd_hip_indices keeps each GPU's index among the GPUs ROCr leaves, and the refusal offers that; the test applies it and checks the plan.
|
Codex re-review of 6c434a5: thanks for confirming the P3 fix. I accepted both new P2 findings, and they're fixed in 06e8072; details are in the threads.
Checks run on macOS: kit 33/33 and basert 30/30 library tests, and |
IsuraManchanayake
left a comment
IsuraManchanayake
left a comment
There was a problem hiding this comment.
Codex Review
Re-reviewed 06e80721fb36af53651765b97d20fee61d8b5e1e with public baseRT #92 at 778091c and the synchronized engine-side packaging changes, including both replies to the last review.
The two previous P2 findings are addressed. UUID visibility now has an explicit unknown state that the install gate respects; empty/disabled masks no longer produce an install plan. Recovery instructions retain the pre-HIP index after ROCr filtering, and applying the offered value admits the correct GPU. The earlier cross-vendor GPU-name fix remains intact.
One P3 reporting issue remains, attached inline: amd_gpus() discards the new unknown-order flag, so the device-name path still labels raw KFD node 0 as the running GPU for UUID selection. This does not undo the installation fixes, but can misidentify the device and attach its budget to the wrong model name.
Validation on this head:
- Adapter library tests: 30 basert + 33 kit + 36 llama.cpp passed; 2 llama.cpp tests ignored.
- Clippy on these three adapters, all targets with warnings denied: passed.
- External probe linked to this checkout's
hip_viewandplan_with: 12 admission/ordering scenarios passed, covering the prior reproducers, both recovery commands, duplicate/invalid termination, hex ROCr indices, and UUID uncertainty. Combining those results with the unchanged Linux name-selection function reproduces the P3 finding. - Full PR
git diff --checkpassed. CI run 287 and Docs run 181 passed.
Limits: local planner/selection probes on macOS, not a physical UUID-selected or multi-GPU execution. Unchanged daemon and packaging suites were not repeated.
| pub fn amd_gpus() -> Vec<String> { | ||
| amd_view().gpus |
There was a problem hiding this comment.
Codex Review — [P3] Preserve unknown device order in the GPU-name path. amd_view() correctly marks UUID selection as unknown, but this helper drops known and returns the raw KFD list. Its consumer in the basert adapter's gpu_name() immediately takes .first() and reports that as the ROCm device. For raw [gfx1100, gfx1151] with ROCR_VISIBLE_DEVICES=<UUID of gfx1151>, the fixed planner now admits the configuration, but the reporting path still produces AMD gfx1100 even though the engine uses gfx1151; devices() pairs that incorrect name with the actual device's memory budget. A probe against this head's hip_view and the Linux name-selection function reproduces known=false, a successful ROCm plan, and AMD gfx1100. Have the reporting caller retain HipView.known and use a generic/unknown name when it is false, or obtain the selected device's identity from the runtime. Add a UUID-selected second-GPU reporting test, since the current test only checks AMD versus NVIDIA.
BaseRT 0.5.0 publishes a ROCm engine for Linux x86-64: one bundle,
basert-engine-linux-x64-rocm-<v>.tar.gz(+.sha256), with kernels for both AMD Strix Halo (gfx1151) and Strix Point (gfx1150), linked against the ROCm 10 runtime at/opt/rocm/core-10.0.basertalready installs superfluid's x86-64 build beside that engine. Until now, someone who installs superfluid first (install.sh, orsuperfluid runtime install basert) was refused on Linux x86-64. This PR lets that path install the engine.What changes
superfluid-adapter-kit):Hostgainsamd_gpusandrocm_runtimes.amd_gpuslists the LLVM targets the KFD driver reports:/dev/kfdplus each/sys/class/kfd/kfd/topology/nodes/*/propertieslinegfx_target_version(110501 → gfx1151, 110500 → gfx1150; the CPU node's 0 is skipped).rocm_runtimeslists the real/opt/rocm/core-<version>/libtrees. Thecoreandcore-10links that AMD's ROCm 10 packages add through/etc/alternativesare not counted twice.describe()(Linux x64, AMD gfx1150, ROCm 10.0) andto_json().("linux", "x64")with a gfx1151 or gfx1150 GPU now maps to("linux-x64-rocm", "rocm")./opt/rocm/core-10.0is missing, with a message saying what to install.--fromis not checked for ROCm 10, since it may be built against another ROCm.whysays "ROCm engine".devices()reported every Linux engine asCUDA. It now saysROCmon x86-64, and the GPU name falls back toAMD gfx1151when there is no NVIDIA model.scripts/install.sh: onx86_64-unknown-linux-gnu, runssuperfluid runtime install basertwhen/dev/kfdexists. If the GPU is not one the engine supports, superfluid says so, and the script continues as it does on arm64.scripts/build_runtime_adapters.sh: the x86-64 release build now runs the basert worker's "starts and reports" check, as arm64 does. The builder has no AMD GPU, so the check covers starting and reporting only.installation.md, the basert paragraph inruntimes.md, and the kernel-sidecar note inhosts.md(the.hsacosits besidelibbaseRT.soin the bundle, and the engine finds it there).Not in this PR
recipe.json: there is no release yet. The pin is added at release time with the ROCm bundle's size and SHA-256.the_pins_are_wholenow accepts a third,linux-x64-rocm, asset, so adding that pin needs no test change. Until then, a Strix box installs the newest listed release that carries the ROCm bundle. Before 0.5.0 there is none, and the refusal saysbaseRT publishes no engine this superfluid drives (...) for linux-x64-rocm yet.Tests
linux_x86_64_with_a_strix_gpu_takes_the_rocm_engine, covering:0.5.0-rocm), listed or pinned.--backend cudais refused.linux-x64-rocm.--fromstill installs.a_bundle_names_its_version_and_other_machines_are_refusednow checks that both Linux x86-64 and Linux arm64 without a supported GPU are refused, with the new message.an_amd_gpu_is_named_by_the_target_its_kfd_node_statesandrocm_runtimes_are_the_core_trees_with_libraries_under_the_root(including the/etc/alternativeslinks).cargo clippy --all-targets -- -D warnings; clippy on the kit, basert, llamacpp and mlx adapters;cargo clippy -p superfluid --no-default-features; the kit, basert and llamacpp lib tests; the table tests (-p superfluid --no-default-features --lib);cargo test --no-fail-fast. One test failed locally,runtime_workers::a_model_id_names_its_runtime_and_nothing_else_is_needed, and only because of where it ran. It asserts that the error does not contain "basert", and the error quotes the worker's path, which was inside a local worktree namedbasert-rocm-engine. The rest of the message is unchanged by this PR./opt/rocm/core-10.0,coreandcore-10as alternatives links):cargo clippy -p superfluid-adapter-kit -p superfluid-adapter-basert --all-targets -- -D warningsis clean, which covers the Linux-only code. The kit (32) and basert (26) lib tests pass.superfluid-worker-basert plandetects"amd_gpus": ["gfx1150"], "rocm_runtimes": ["10.0"]and pickslinux-x64-rocm. With no published release carrying that bundle yet, it saysbaseRT publishes no engine this superfluid drives ... for linux-x64-rocm yet.superfluid-worker-basert checkwithBASERT_LIBset to a local ROCm build of libbaseRT reportsavailable: trueand the device{"backend": "ROCm", "name": "AMD gfx1150", "memory": 51539607552}. The old code labelled any Linux deviceCUDAand, with no NVIDIA model, named itGPU.install.shchecked withdash -n, and its new branch underset -euin dash: with no/dev/kfdit skips the engine and reaches the end.