GPU passthrough to Windows 11 VM

Incus
, , ,
1

So i followed ScottyBytes easy Windows 11 VM tutorial and was able to create a VM without issue. On my server i have a Nvidia A6000 that I would like to passthrough, but when i try to add the GPU (after shutting down the VM) it saves and it all looks well. But once i try to start the VM it just hangs in a starting process forever. The GPU is not being used in the host environment at all (its a brand new install) . The Nvidia driver is loaded when host boots up but other than that , thats it.

Has anyone else experienced these issues? I see from the forums that the VM’s for most users does at least starts.

This is the output of dmesg when the incus “start” operation is hanging.

[  630.424655] tap46d90a04: entered promiscuous mode
[  630.692464] VFIO - User Level meta-driver version: 0.3
[  630.725106] [drm] [nvidia-drm] [GPU ID 0x00008100] Removing device
[  630.739908] [drm] [nvidia-drm] [GPU ID 0x00008100] Unloading driver
[  630.892078] NVRM: Attempting to remove device 0000:81:00.0 with non-zero usage count!
[  775.008032] nvidia_uvm: module uses symbols nvUvmInterfaceDisableAccessCntr from proprietary module nvidia, inheriting taint.
2

You should setup your system so vfio-pci gets that PCI address assigned to it. You really don’t want the nvidia host driver binding the GPU, even if you’re not using it.

[  630.892078] NVRM: Attempting to remove device 0000:81:00.0 with non-zero usage count!

That’s the NVIDIA driver telling you that your GPU is in use. If this is your only GPU in the system, it may be used by the text console on the system.

3

@stgraber its on a gigabyte server, I believe its using aspeed (or the generic video driver). So i need to figure out how to get vfio-pci working to make this work ? would it be possible to just do a rmmod on the nvidia driver, and then push it to the VM so that its not being used ? is that a safe practice?

4

The easier/safer approach in your case would be to put the nvidia in the modprobe blacklist so it doesn’t get loaded in the first place.