Cannot add Intel graphic(SR-IOV enabled) to Windows 11 VM

awin · March 31, 2024, 3:05am

Host: Ubuntu server 22.04
GPU: i915, Intel Corporation AlderLake-S GT1

I already enable sr-iov of graphic card on host. I have 8 VGA, check by lspci | grep 00:02, and output:

00:02.0 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
00:02.1 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
00:02.2 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
00:02.3 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
00:02.4 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
00:02.5 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
00:02.6 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
00:02.7 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)

Below is my GPU resource by sudo incus info --resources:

GPU:
  NUMA node: 0
  Vendor: Intel Corporation (8086)
  Product: AlderLake-S GT1 (4680)
  PCI address: 0000:00:02.0
  Driver: i915 (6.5.0-26-generic)
  DRM:
    ID: 0
    Card: card0 (226:0)
    Control: controlD64 (226:0)
    Render: renderD128 (226:128)
  SR-IOV information:
    Current number of VFs: 7
    Maximum number of VFs: 7
    VFs: 7
    - NUMA node: 0
      Vendor: Intel Corporation (8086)
      Product: AlderLake-S GT1 (4680)
      PCI address: 0000:00:02.1
      Driver: i915 (6.5.0-26-generic)
      DRM:
        ID: 1
        Card: card1 (226:1)
        Render: renderD129 (226:129)
    - NUMA node: 0
      Vendor: Intel Corporation (8086)
      Product: AlderLake-S GT1 (4680)
      PCI address: 0000:00:02.2
      Driver: i915 (6.5.0-26-generic)
      DRM:
        ID: 2
        Card: card2 (226:2)
        Render: renderD130 (226:130)
    - NUMA node: 0
      Vendor: Intel Corporation (8086)
      Product: AlderLake-S GT1 (4680)
      PCI address: 0000:00:02.3
      Driver: i915 (6.5.0-26-generic)
      DRM:
        ID: 3
        Card: card3 (226:3)
        Render: renderD131 (226:131)
    - NUMA node: 0
      Vendor: Intel Corporation (8086)
      Product: AlderLake-S GT1 (4680)
      PCI address: 0000:00:02.4
      Driver: i915 (6.5.0-26-generic)
      DRM:
        ID: 4
        Card: card4 (226:4)
        Render: renderD132 (226:132)
    - NUMA node: 0
      Vendor: Intel Corporation (8086)
      Product: AlderLake-S GT1 (4680)
      PCI address: 0000:00:02.5
      Driver: i915 (6.5.0-26-generic)
      DRM:
        ID: 5
        Card: card5 (226:5)
        Render: renderD133 (226:133)
    - NUMA node: 0
      Vendor: Intel Corporation (8086)
      Product: AlderLake-S GT1 (4680)
      PCI address: 0000:00:02.6
      Driver: i915 (6.5.0-26-generic)
      DRM:
        ID: 6
        Card: card6 (226:6)
        Render: renderD134 (226:134)
    - NUMA node: 0
      Vendor: Intel Corporation (8086)
      Product: AlderLake-S GT1 (4680)
      PCI address: 0000:00:02.7
      Driver: i915 (6.5.0-26-generic)
      DRM:
        ID: 7
        Card: card7 (226:7)
        Render: renderD135 (226:135)

And then I created a Windows 11 virtual machine.
But when I tried to add VGA 00:02.1 to the Windows 11 VM. I got error.
If I add gpu with this command:

sudo incus config device add Win11VM gpu gpu gputype=sriov pci=0000:00:02.1
incus start Win11VM --console=vga

and get

Error: Failed to start device "gpu": Couldn't find a matching GPU with available VFs

If I add gpu with this command:

sudo incus config device add Win11VM gpu pci address=0000:00:02.1
incus start Win11VM --console=vga

VM can start, but show up:
20240331_105429_remote-viewer_692x585

Still not work. Any suggestion to achieve to add VGA to VM?
Thank you for your help.

awin · March 31, 2024, 8:02am

I found the virtual machine actually continue to boot even lxc display “Display output is not active”.
After I set GPU to VM by sudo incus config device add Win11VM gpu pci address=0000:00:02.1

If you wait long engough, in my case, I have to wait maybe 1 ot 3 minutes(I didn’t measure it), the boot screen will show up, then you can see Windows is booting, then you can install driver of Intel UHD 770.

Hope this can help someone who want to use SR-IOV of Intel graphic in Windows VM.

stgraber · April 1, 2024, 4:09am

The reason why Incus couldn’t find any VF when using gputype=sriov is because all your VFs are attached to the i915 host driver. That’s a bit of an odd behavior as normally the VFs should only be attached to the host driver if they’re in use by the host.

I don’t have any SR-IOV capable Intel GPUs here (only got the mdev capable ones) but it may be a module parameter to control whether the VFs get attached to the host driver or if not, you may be able to use a udev rule to detach them from the driver when they appear.