Trouble getting iGPU passthrough or SRIOV working

Incus
, , ,
1

So I’ve tried this on Ubuntu Server 24.04 and then decided to upgrade to 24.10 in order to get the newer linux kernel which I thought might make this work.
There was an error in dmesg that eventually led me to do the upgrade.

BACKGROUND: Essentially I’ve moved from Windows with docker desktop (WSL2) on a NUC to Ubuntu as the host and LXD as the hypervisor. I’m primarily a Windows user but do have a number of Linux services that I want running such as Nextcloud, Home Assistant etc. So far however I only have Ubuntu 24.10 running and a clean install of Windows 11.

The new hardware is an Intel Meteor Lake-P SOC and I believe supports SRIOV for the iGPU

lxc info --resources gives:

GPUs:
  Card 0:
    NUMA node: 0
    Driver: simple-framebuffer (6.11.0-8-generic)
    DRM:
      ID: 0
      Card: card0 (226:0)
      Control: controlD64 (226:0)
  Card 1:
    NUMA node: 0
    Vendor: Intel Corporation (8086)
    PCI address: 0000:00:02.0
    Driver: i915 (6.11.0-8-generic)
    DRM:
      ID: 1
      Card: card1 (226:1)
      Control: controlD65 (226:1)
      Render: renderD128 (226:128)
    SR-IOV information:
      Current number of VFs: 0
      Maximum number of VFs: 7

lspci -k gives:

00:02.0 VGA compatible controller: Intel Corporation Meteor Lake-P [Intel Graphics] (rev 08)
        DeviceName: Onboard - Video
        Subsystem: Intel Corporation Device 2212
        Kernel driver in use: vfio-pci
        Kernel modules: i915, xe

dmesg | grep i915 gives:

[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.11.0-8-generic root=UUID=8ec35089-8f08-4527-a8be-2ffc34330a3f ro quiet intel_iommu=on iommu=pt i915.enable_guc=3 crashkernel=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M
[    0.052908] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.11.0-8-generic root=UUID=8ec35089-8f08-4527-a8be-2ffc34330a3f ro quiet intel_iommu=on iommu=pt i915.enable_guc=3 crashkernel=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M
[    1.955896] i915 0000:00:02.0: [drm] Found METEORLAKE (device ID 7dd5) display version 14.00 stepping C0
[    1.956916] i915 0000:00:02.0: [drm] VT-d active for gfx access
[    1.956923] i915 0000:00:02.0: vgaarb: deactivate vga console
[    1.956948] i915 0000:00:02.0: [drm] Using Transparent Hugepages
[    1.956963] caller init_stolen_lmem+0x6d/0x110 [i915] mapping multiple BARs
[    1.969266] i915 0000:00:02.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[    1.978071] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/mtl_dmc.bin (v2.23)
[    2.049962] i915 0000:00:02.0: [drm] GT0: GuC firmware i915/mtl_guc_70.bin version 70.29.2
[    2.060792] i915 0000:00:02.0: [drm] GT0: GUC: submission enabled
[    2.060794] i915 0000:00:02.0: [drm] GT0: GUC: SLPC enabled
[    2.061003] i915 0000:00:02.0: [drm] GT0: GUC: RC enabled
[    2.065225] mei_gsc_proxy 0000:00:16.0-0f73db04-97ab-4125-b893-e904ad0d5464: bound 0000:00:02.0 (ops i915_gsc_proxy_component_ops [i915])
[    2.065560] i915 0000:00:02.0: [drm] GT1: GuC firmware i915/mtl_guc_70.bin version 70.29.2
[    2.065562] i915 0000:00:02.0: [drm] GT1: HuC firmware i915/mtl_huc_gsc.bin version 8.5.4
[    2.087948] i915 0000:00:02.0: [drm] GT1: HuC: authenticated for clear media
[    2.088300] i915 0000:00:02.0: [drm] GT1: GUC: submission enabled
[    2.088301] i915 0000:00:02.0: [drm] GT1: GUC: SLPC enabled
[    2.088365] i915 0000:00:02.0: [drm] GT1: GUC: RC enabled
[    2.090485] i915 0000:00:02.0: [drm] Protected Xe Path (PXP) protected content support initialized
[    2.107036] [drm] Initialized i915 1.6.0 for 0000:00:02.0 on minor 1
[    2.109065] snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
[    2.162313] fbcon: i915drmfb (fb1) is primary device
[    2.229468] i915 0000:00:02.0: [drm] GT1: Loaded GSC firmware i915/mtl_gsc_1.bin (cv1.0, r102.0.10.1878, svn 1)
[    2.249535] i915 0000:00:02.0: [drm] GT1: HuC: authenticated for all workloads
[    3.393428] i915 0000:00:02.0: [drm] fb1: i915drmfb frame buffer device

I could not tell if I needed these lines in the grub config because they seemed to be causing issues: quiet intel_iommu=on iommu=pt i915.enable_guc=3

Let me know what else you’d like.
Ideally I’d get this working with SRIOV. When I attempt to start the VM with this configured then I get the error:
“All virtual functions on device are already in use”.

Thanks.
Chris

2

Look for a sriov_numvfs file in /sys for the PCI address of your GPU and try to do echo 4 > /path/to/sriov_numvfs.

That should then get that “Current number of VFs” entry to go up to 4 and allow Incus to use it, assuming that your system has functional SR-IOV, otherwise that echo will likely fail.

3

so that doesn’t work. I get a dmesg that i915 doesn’t support setting sr-iov via sysfs.
Do I need any grub configuration mentioned above?
thanks.

4

What’s the output of incus version?

5

I get 6.1 for ´lxd version

6

You can’t mix both GPU passthrough and srvio at the same time on one single GPU.

Kernel driver in use: vfio-pci will block srv-io, as it’s for GPU passthrough. I use this case, for example, to pass an Nvidia card for a “dedicated” gaming machine with a separate monitor, keyboard, and mouse (it works really nice; my daughter plays games occasionally, and I continue to work).

The host uses the Intel GPU for the main display and srv-io for a few VMS. But d**b me, I have no idea how to connect with a GUI yet. That Looking Glass project seems too overcomplicated compared to making things simple and easy, and from its docs, I did not really understand the process.

7

Hi @madalinignisca I understand that I can’t mix passthrough and sr-iov. I would just like one of them to work.
With Passthrough I get the dreaded Code 43 error in Windows.
With SR-IOV it seems it is bleeding edge of Linux drivers with Meteor Lake right now. I’m trying to get the dkms driver to work but with Kernel 6.11 I’m getting lots of errors. I’ll keep working at it.