How to customize LXD VM qemu configuration?

While trying to migrate my VMs from libvirt to LXD, I’ve found that the virtual hardware created by qemu is somewhat different between the libvirt/lxd. Can I control how /var/log/lxd/[VMNAME]/qemu.conf is generated?

You get to append command line parameters through raw.qemu but unlike libvirt where you get to choose your PCI layout, device type, … LXD is far far more opinionated and will not offer you as much flexibility.

LXD is far far more opinionated and will not offer you as much flexibility.

Is this by design or would you accept a PR that changes this? Not looking for something very complex like overriding individual entries in qemu.conf, but maybe we could expose a new config key (eg “raw.qemu_cfg”) where the user can inline a new qemu.conf directly in the yaml to override the generated qemu.conf.

At that point, it feels like you may be better off just running QEMU by hand.

It’d certainly be possible and even trivial to detect that raw.qemu is a INI config and allow replacing the entire config that way, but LXD interacts with QEMU over QMP during the lifetime of the instances and so will potentially fail in dramatic ways if there are additional devices or device names don’t line up with what it would have generated itself.

There is also the extra difficulty that at least for users of the LXD snap, we’ve been restricting the feature set of QEMU to match what LXD supports, so if using the snap, you won’t get to use non-virtio devices, use a non-UEFI bios, … as those features are simply not built at all into the QEMU binary.

All that being said, did you play with passing -readconfig /path/to/config through raw.qemu? That should make QEMU read that separate config file.

1 Like

All that being said, did you play with passing -readconfig /path/to/config through raw.qemu? That should make QEMU read that separate config file.

I will give it a shot, thanks.

Anyone coming here from a search engine, there is now a well supported and robust mechanism using raw.qemu.conf: Instance options - Canonical LXD documentation