HI,
I’m trying to add the only gpu (gtx1080ti) on a system to a windows VM and it just causes incus to pretty much freeze on vm launch (@stgraber), after an extended period of time (half an hour or an hour, hard to say) the operation may fail with error status but doesn’t fully recover many operations including a simple incus ls take longer than it would prior to launching the VM. Any help would be greatly appreciated! (bare in mind there are no other VGA/GPU devices on this system at all nothing on the mobo or cpu).
incus 6.5-2 and qemu 9.1.0-2
kernel (6.6.52-1-lts) cmdline:
skew_tick=1 zswap.enabled=0 intel_pstate=no_hwp intel_iommu=on iommu=pt rdblacklist=nouveau pci-stub.ids=10de:1b06,10de:10ef
incus info --resources:
GPU:
NUMA node: 1
Vendor: NVIDIA Corporation (10de)
Product: GP102 [GeForce GTX 1080 Ti] (1b06)
PCI address: 0000:81:00.0
Driver: vfio-pci (6.6.51-1-lts)
incus config show win10:
devices:
gtx1080ti:
gputype: physical
pci: 0000:81:00.0
type: gpu
And the GPU appears to be in an isolated IOMMU group on its own;
IOMMU Group 2:
81:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] [10de:1b06] (rev a1)
81:00.1 Audio device [0403]: NVIDIA Corporation GP102 HDMI Audio Controller [10de:10ef] (rev a1)
Update
The issue is somehow related to memory. With limits.memory=32Gib the VM doesn’t start and causes incus to freeze but with limits.memory=30GiB everything seems to work fine.
Memory:
NUMA nodes:
Node 0:
Free: 1.10GiB
Used: 30.90GiB
Total: 32.00GiB
Node 1:
Free: 31.34GiB
Used: 675.16MiB
Total: 32.00GiB
Free: 61.52GiB
Used: 2.48GiB
Total: 64.00GiB