LXD VM's on CentOS

LXD
1

I read the announcement and saw the video about creating a VM running a complete desktop distribution under LXD and decided to try it. I’ve created containers with Fedora and CentOS and updated them to run KDE, but that takes a good bit of time. (My attempt at updating a Ubuntu container to run KDE was not successful.)

I run LXD on a CentOS 8 server. When I resolved the several naming differences between Ubuntu and CentOS (OVMF and qemu), I ran into a problem: virtio-9p-pci is not supported on CentOS. There are old messages on RedHat’s forum saying that it isn’t supported in RHEL 7 (and presumably RHEL 8).

I believe that virtio-9p is used to share file systems between the host and the VM. Is this required? Is there a way to disable this?

Are there other ways to share file systems between the host and VM? IIRC VirtualBox and VMware can share file systems, and both run on CentOS, although I’ve never used that feature.

2

It’s odd that RedHat would have disabled the feature in QEMU…
I know they’ve got it disabled in the guest kernel (but support virtiofs), just strikes me as odd that they’d also disable it in qemu userspace…

If there was a good way to detect the missing feature in QEMU prior to generating the VM config, we could in theory skip the 9p support, but this would then result in various images being unable to run the agent, so not sure it’s something we’d really want to do (would make support quite tricky).

Might be worth noting that the LXD snap does work on CentOS (we test 6/7/8) and that it would contain a QEMU build which supports everything LXD uses.

3

I’m running the LXD snap, but LXD is using the system QEMU (thus the name disconnect). When I list the snap commands, there’s only lxd.lxc (alias lxc). Is there something which I need to do after installing the LXD snap to export the LXD build of QEMU?

I believe that that virtio-9p requires a kernel config feature (CONFIG_NET_9P) that is disabled in CentOS (for reasons which are not completely clear). There may be other kernel config options required (CONFIG_9P_FS?). So, it seems that even using a QEMU from the LXD build would not work.

What does the agent do?

Is there a way that I can edit the VM config to disable virtio-9p?

4

The LXD snap includes the QEMU binary, so it does not use the one from your system.

Can you show lxc info and the error you’re getting when doing:

  • lxc init test --vm --empty
  • lxc start test
5

[eager@oak ~]$ which lxc
/var/lib/snapd/snap/bin/lxc
[eager@oak ~]$ lxc init test --vm --empty
Creating test
[eager@oak ~]$ lxc start test
Error: Failed to run: /usr/bin/lxd forklimits limit=memlock:unlimited:unlimited – /usr/bin/qemu-system-x86_64 -S -name test -uuid 91f81cdc-93f8-443d-ac55-d678239f2df6 -daemonize -cpu host -nographic -serial chardev:console -nodefaults -no-reboot -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=deny,resourcecontrol=deny -readconfig /var/log/lxd/test/qemu.conf -pidfile /var/log/lxd/test/qemu.pid -D /var/log/lxd/test/qemu.log -chroot /var/lib/lxd/virtual-machines/test -runas lxd: : exit status 1
Try lxc info --show-log test for more info
[eager@oak ~]$ lxc info --show-log test
Name: test
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2021/09/11 23:27 PDT

Log:

qemu-system-x86_64:/var/log/lxd/test/qemu.conf:104: Could not open ‘/var/lib/lxd/virtual-machines/test/qemu.nvram’: No such file or directory

=== That error is because qemu.nvram is a symlink to OVMF_VARS.fd, which is missing. After copying /usr/share/OVMF/OVMF_VARS.fd:

[eager@oak ~]$ lxc start test
Error: Failed to run: /usr/bin/lxd forklimits limit=memlock:unlimited:unlimited – /usr/bin/qemu-system-x86_64 -S -name test -uuid 91f81cdc-93f8-443d-ac55-d678239f2df6 -daemonize -cpu host -nographic -serial chardev:console -nodefaults -no-reboot -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=deny,resourcecontrol=deny -readconfig /var/log/lxd/test/qemu.conf -pidfile /var/log/lxd/test/qemu.pid -D /var/log/lxd/test/qemu.log -chroot /var/lib/lxd/virtual-machines/test -runas lxd: : exit status 1
Try lxc info --show-log test for more info
[eager@oak ~]$ lxc info --show-log test
Name: test
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2021/09/11 23:27 PDT

Log:

qemu-system-x86_64:/var/log/lxd/test/qemu.conf:142: ‘virtio-9p-pci’ is not a valid device model name

[eager@oak ~]$ which qemu-system-x86_64
/usr/bin/qemu-system-x86_64
[eager@oak ~]$ ls -l /usr/bin/qemu-system-x86_64
lrwxrwxrwx 1 root root 21 Sep 11 04:40 /usr/bin/qemu-system-x86_64 → /usr/libexec/qemu-kvm*
[eager@oak ~]$ rpm -qf /usr/libexec/qemu-kvm
qemu-kvm-core-4.2.0-48.module_el8.4.0+885+5e18b468.3.x86_64

6

The above shows that you are not using the snap package.

The snap does not use files in /usr or in /var/lib/lxd, so this shows you’re using some other type of package or manually compiled LXD, which then indeed do use your system’s QEMU and may run into issues when that QEMU build is missing some features.

7

I can see that running /var/lib/snapd/snap/bin/lxc is invoking /usr/bin/lxd, but I am invoking the LXD snap package, not the system version (3.22).

[eager@oak ~]$ /var/lib/snapd/snap/bin/lxc --version
4.18

I installed the snap in the standard fashion: “sudo snap install lxd”. I did not build LXD. I don’t know any other way to invoke a snap program.

Is the $PATH set up correctly in the snap?

8

There’s something about the LXD snap which doesn’t look right to me.

$ snap info lxd
name: lxd
summary: LXD - container and VM manager
publisher: Canonical✓
store-url: https://snapcraft.io/lxd
contact: https://github.com/lxc/lxd/issues
license: unset
description: |
LXD is a system container …
commands:

  • lxd.benchmark
  • lxd.buginfo
  • lxd.check-kernel
  • lxd.lxc
  • lxd.lxc-to-lxd
  • lxd
  • lxd.migrate

The ‘lxc’ command is not exported. It should be a link to lxd.lxc or /bin/snap.

9

It’s likely not exported because of the pre-existing lxc command from your system’s LXD. You can probably add it with snap alias lxd.lxc lxc.