gsp: don't advertise RTD3 GC6/GC8/GCOFF on external (eGPU) boards

[arc-uri-el]

Summary

_kgspInitGpuProperties() advertises RTD3 GC6/GC8/GCOFF support for external (eGPU) boards straight from the GSP silicon capability, with no IS_EXTERNAL_GPU check. These are host-platform-coordinated deep power-down states (the host power-gates the GPU's PCIe slot), which is physically impossible for an external GPU in a Thunderbolt/OCuLink enclosure with its own PSU. After the RTD3 idle timeout the GPU attempts the power-down, the GSP firmware faults, and the GPU falls off the PCIe bus.

Crash (RTX 5090 / GB202 over Thunderbolt, 610.43.02)

NVRM: GPU0 _issueRpcAndWait: rpcSendMessage failed status 0x0000000f
NVRM: ... GPU_IN_FULLCHIP_RESET
nvidia-smi: No devices were found

Occurs a few minutes after the desktop goes idle. NVreg_DynamicPowerManagement=0 does not help (it does not override this GSP-set property). The only prior workaround was pinning clocks to max (nvidia-smi -lgc/-lmc) to keep the GPU out of idle, at a large constant idle-power cost.

Fix

Gate PDB_PROP_GPU_RTD3_{GC6,GC8,GCOFF}_SUPPORTED on !PDB_PROP_GPU_IS_EXTERNAL_GPU. IS_EXTERNAL_GPU is already set by RmCheckForExternalGpu before _kgspInitGpuProperties runs. Orthogonal to dynamic clock scaling: external GPUs keep full dynamic idle clocking (verified P0<->P8, ~38 W idle) while no longer attempting the impossible deep power-down. Zero GSP crashes over a sustained idle test after the change.

Relates to #979.

First-time contributor, happy to sign the CLA or adjust to your process.

[CLAassistant]

All committers have signed the CLA.

[Binary-Eater]

Hi @arc-uri-el,

Thank you for your contribution. First, I would like to go over RTD3 in a bit more detail, specifically, about D3hot and D3cold, just to make the conversation easier for myself.

RTD3 -> Runtime PCIe Device-state 3 (suspend)

D3hot -> PCIe is on but device is put into lowest power state over PCIe
D3cold -> Vcc for the PCIe line is actually controllable and disabled. ACPI on motherboards with D3cold support take care of powering back on.

These are host-platform-coordinated deep power-down states (the host power-gates the GPU's PCIe slot), which is physically impossible for an external GPU in a Thunderbolt/OCuLink enclosure with its own PSU.

My understanding is that this is true for D3cold but not D3hot. I think the problem here is that we are disabling FBSR with the current change proposal (framebuffer self refresh) by disabling GPU GC6 entry, which is a bit unfortunate.

References:

• https://download.nvidia.com/XFree86/Linux-x86_64/610.43.03/README/dynamicpowermanagement.html
• https://docs.kernel.org/gpu/xe/xe_pm.html

I think what likely needs to happen here is that the NVIDIA Linux driver actually needs to call pci_d3cold_disable for the root pcie device for the eGPU to prevent the transition from D3Hot to D3Cold for eGPU dock GPUs. I think however we can safely drop eGPUs to D3hot power savings levels when idle as long as we prevent D3cold transitions.

• https://elixir.bootlin.com/linux/v4.15/source/drivers/pci/pci.c#L2384

This would ideally be done somewhere in src/nvidia/arch/nvalloc/unix/src/dynamic-power.c.
My initial intuition, though I have not given this my full attention, would be to take care of this pci_d3cold_disable in RmInitPowerManagement, which is called per-RM device init for any eGPU devices.

Before this change

Internal GPU (if NVIDIA) on RTD3 capable laptop/system

Could enter D3hot -> Good
Could enter D3cold -> Good

NVIDIA eGPU on RTD3 capable laptop/system

Could enter D3hot -> Good
Could enter D3cold -> Bad

After this change

Internal GPU (if NVIDIA) on RTD3 capable laptop/system

Could enter D3hot -> Good
Could enter D3cold -> Good

NVIDIA eGPU on RTD3 capable laptop/system

Cannot enter D3hot -> Loses some power saving opportunity
Cannot enter D3cold -> Good

The ideal situation

Internal GPU (if NVIDIA) on RTD3 capable laptop/system

Could enter D3hot -> Good
Could enter D3cold -> Good

NVIDIA eGPU on RTD3 capable laptop/system

Could enter D3hot -> Good
Cannot enter D3cold -> Good

[Binary-Eater]

@arc-uri-el looking at this a bit more on the kernel side, I would expect the eGPU enclosure to be handled more gracefully by the Linux kernel, https://elixir.bootlin.com/linux/v7.1.3/source/drivers/pci/pci.c#L2945

Do you mind sharing what eGPU enclosure you are using to reproduce the issue?

[arc-uri-el]

Thank you for the detailed analysis.

The enclosure is a Razer Core X V2 (Thunderbolt 5) with an Intel JHL9480 bridge. The full reproduction setup is:

• GPU: NVIDIA RTX 5090 / GB202, PCI ID 10de:2b85, 32 GB
• Enclosure: Razer Core X V2, Thunderbolt 5, Intel JHL9480
• Host: ASUS ROG Flow Z13 GZ302EA, AMD Ryzen AI MAX+ 395
• OS: NixOS 26.05
• Kernel: Linux 7.0.13
• Driver: NVIDIA open kernel modules 610.43.02
• Link: PCIe Gen4 x4 through the Thunderbolt path
• Display: the external display is driven directly by the eGPU

Preserving D3hot and FBSR while preventing only D3cold would be preferable to the current proposal if it resolves the failure. We will implement an alternative patch following your suggestion, using pci_d3cold_disable() for the external-GPU path during RM power-management initialization, and test it against the current GC6/GC8/GCOFF gate under the same idle and subsequent load cycle.

We will collect the PCI power-state and link-state observations, GSP/kernel logs, idle power, and resume/load result, then report back here once we have the results.