LXD does not start due to missing /dev/vhost

LXD
1

Hi! Since upgrading from Ubuntu 20.04 to 22.04, LXD does not start anymore:

lxd.daemon[16762]: => Starting LXD
lxd.daemon[16899]: time="2022-09-01T08:43:34+03:00" level=warning msg="Unsupported CGroup setup detected, V1 controllers on top of V2 root"
lxd.daemon[16899]: time="2022-09-01T08:43:34+03:00" level=warning msg=" - Couldn't find the CGroup network priority controller, network priority will be ignored"
lxd.daemon[16899]: time="2022-09-01T08:43:34+03:00" level=error msg="Failed to start the daemon" err="listen vsock: open /dev/vsock: no such file or directory"
lxd.daemon[16899]: Error: listen vsock: open /dev/vsock: no such file or directory
lxd.daemon[16762]: => LXD failed to start

Indeed, the /dev/vhost file is not there. I do have vhost-net and vhost-vsock, though. The vhost kernel module is loaded, and if I unload it (after unloading a couple of modules depending on it) and reload it, then /dev/vhost appears and LXD can start.

So this is most likely not an issue with LDX itself, but hopefully here in the forums there are some experts that can help me understand what’s happening? I’m a bit at a loss, since searching in the internet for similar issues didn’t lead to many results, so it looks like this is a rare case. FWIW, in the same machine I do also have VirtualBox installed.

2

I believe that VirtualBox conflicts with LXD due to the different vsock kernel modules needed.

What version of LXD are you running?

3

I’m using 5.4/stable, but previously I was running 5.5/stable and it had the same issue. I’ll now try to disable the virtualbox.service and reboot, to see if anything changes.

EDIT: nope, no luck, the virtualbox service did not start, but still no /dev/vhost is present.

4

OK please stick with LXD 5.5 as thats the current version.

After rebooting without virtual box installed what does sudo lsmod | grep vhost show?

5

It says:

vhost_vsock            28672  0
vmw_vsock_virtio_transport_common    40960  1 vhost_vsock
vhost                  53248  1 vhost_vsock
vhost_iotlb            16384  1 vhost
vsock                  49152  3 vmw_vsock_virtio_transport_common,vhost_vsock,vmw_vsock_vmci_transport
6

On my host I don’t have vmw_vsock_vmci_transport loaded, which seems to be related to VMWare.

Can you unload that?

1 Like
7

Oh, it looks like I had one intallation of vmware-player from a few years ago. After removing that, and rebooting, that module is gone (yet, there’s still a vmw_* module left; is that normal?):

vhost_vsock            28672  0
vmw_vsock_virtio_transport_common    40960  1 vhost_vsock
vhost                  53248  1 vhost_vsock
vhost_iotlb            16384  1 vhost
vsock                  49152  2 vmw_vsock_virtio_transport_common,vhost_vsock

However, /dev/vhost is still not there, yet LXD now starts successfully. I’m a bit puzzled, but I guess this is all fine now? :slight_smile:

8

It may be that was preventing LXD from loading the module it needed.

1 Like
9

I removed open-vm-tools (details in another post here and rebooted).

Of course that’s not ideal, but at least it works. It’d be good if LXD could coexist without problems. It seems if some modules can’t be loaded by open-vm-tools that can create other issues for the VM.

10

I understand it to be a limitation of the vsock module we use with QEMU to communicate with the lxd-agent.

1 Like
11

Yeah, it’s a kernel restriction, there can only be one implementation of vsock on the system.

1 Like