How to configure QEMU options (for passthru, etc)

Incus
1

I am trying to configure a VM with passthru devices (gpu, etc). I am starting with a QEMU command line, essentially this is a working example:

qemu-system-x86_64 -m 8192 -bios /usr/share/ovmf/x64//OVMF.fd -drive file=/dev/zvol/zos/qemu-disk,format=raw,index=0,media=disk --cpu host,kvm=off --enable-kvm -vga none -display none -device vfio-pci,host=d8:00.0 -device vfio-pci,host=d8:00.1 -device vfio-pci,host=3b:00.0

I’m passing 3 devices into the vm there - video, audio and a usb host card. I can do this with libvirt, and also with raw QEMU but fall short trying to use Incus. I just want to replicate that command line using Incus so that I can manage my instances in one place.

I have tried to RTFM but fall short… I think I should use raw.qemu.conf but I cannot find any definition or example describing the INI file format expected by QEMU readconfig option.

From what I’ve read the QEMU INI format is not documented and its implementation is incomplete however I expect it isn’t the case if Incus uses it under the hood.

If I could get a pointer in the right direction, I’d truly appreciate it. Is there a worked example somewhere I might have missed?

2

Why are you passing this stuff directly to QEMU rather than just using pci or gpu devices in your Incus config?

3

I’m not, sorry. I am trying to work out how to do it. If I can drive it with incus config directly then all the better, I just don’t know how. Also what about other options like -bios ?

How would I replicate the above QEMU command with Incus configuration? Is there an example?

4

incus config device add NAME my-gpu gpu gputype=physical pci=XYZ
or
incus config device add NAME my-pci pci address=XYZ

5

Have a look at these keys from the Incus configuration, Instance options - Incus documentation

  1. raw.qemu
  2. raw.qemu.conf
  3. The QMP parameters.

Also, the next section about Override QEMU configuration.

You can set these keys either with something like incus launch images:debian/12/cloud mycontainer -c raw.qemu=...., or through an Incus profile.

6

Thanks! I’d looked at the Incus documention page you linked to but it doesn’t link to any defintion of what the config format is or how to use it. I realise that’s really a QEMU thing but there isn’t anything documented there either.

I can see from looking at /run/incus/<vm>/qemu.conf how Incus writes this config but I can’t find anywhere that explains the format. I tried to export config from QEMU but the -writeconfig option has been removed; it seems like there is little love for this format.

The --writeconfig option was not able to serialize the entire contents of the QEMU command line. It is thus considered a failed experiment and removed without a replacement.

Hopefully, as I get my head around Incus some more, I won’t need to use raw.qemu.conf.

7

This got me going, thanks. It turns out that I “knew too much” having previously implemented this directly in QEMU. Using incus config device add NAME my-gpu gpu gputype=physical pci=d8:00.0 implies the vfio-pci device type that I was looking for but could not find.

I can launch a vm and have it use dedicated graphics card. I have other problems but I’ll post about those separately.

1 Like
8

There is one more thing I think that I am missing here… When my vm boots, the physical GPU shows the boot screen however once Linux kicks in, the physical displays go black and the primary display changes to be the virtual (spice?) console. I need to disable that so that only the physical displays are used (in QEMU this is done via -vga none -display none). I can’t see an obvious option for that… does that mean I need to use raw.qemu ?

I did try incus config set myvm raw.qemu="-vga none -display none" without success.