Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 16 additions & 11 deletions gpustack_runtime/deployer/cdi/ascend.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,22 @@
https://gitcode.com/Ascend/mind-cluster/blob/master/docs/en/scheduling/references/appendix.md.
"""

_HCCN_CONF_PATH = "/etc/hccn.conf"
_ASCEND_DRIVER_PATH = "/usr/local/Ascend/driver"
"""
The device NIC addresses, as configured on the host. A transport that builds
RoCE endpoints reads them from this file or from hccn_tool, which lives in a
driver directory none of the mounts below covers; without either, the engine
fails to initialize with "Failed to get device ip from hccn.conf and
hccn_tool". A container that talks to another node over Device RoCE therefore
needs this file, and one that does not is unaffected by its presence.
The host driver tree, injected whole rather than as the three subdirectories
the libraries and the topology file sit in.

What the wider mount adds is hccn_tool: a transport that builds Device RoCE
endpoints reads each NPU's NIC address through it, and a container holding the
driver libraries but not the tool cannot open that transport at all -- it
fails with "Failed to get device ip from hccn.conf and hccn_tool" and every
process that opens it dies at startup. Reading the addresses out of
/etc/hccn.conf instead works too, but only on a host that keeps them there,
while the tool answers from the driver on every host that has one.

The driver belongs to the host on every Ascend containerization path -- the
image supplies the toolkit -- so injecting more of it shadows nothing a
container ships with its own image.
"""

_A5_RANKTABLE_VERSION = "2.0"
Expand Down Expand Up @@ -170,10 +178,7 @@ def generate(

mount_paths = [
_HCCL_RANKTABLE_PATH,
_HCCN_CONF_PATH,
"/usr/local/Ascend/driver/topo",
"/usr/local/Ascend/driver/lib64",
"/usr/local/Ascend/driver/include",
_ASCEND_DRIVER_PATH,
"/usr/local/dcmi",
"/usr/local/bin/npu-smi",
"/var/queue_schedule",
Expand Down
4 changes: 1 addition & 3 deletions tests/gpustack_runtime/deployer/test_ascend_ranktable.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,7 @@ def test_a5_omits_the_host_ranktable():
targets = _mount_targets(cfg)
assert cdi_ascend._HCCL_RANKTABLE_PATH not in targets
# The rest of the vendor's named list is untouched.
assert "/usr/local/Ascend/driver/topo" in targets
assert "/usr/local/Ascend/driver/lib64" in targets
assert "/usr/local/Ascend/driver/include" in targets
assert cdi_ascend._ASCEND_DRIVER_PATH in targets
assert "/usr/local/dcmi" in targets
assert "/usr/local/bin/npu-smi" in targets

Expand Down
Loading