Can't run Libvirt/QEMU-KVM in an unprivileged domain anymore, "Unable to set XATTR"

1

Hey guys,

I have an awesome OpenStack Ansible (Ussuri, on top of Ubuntu 20.04.1) deployment that not only my Compute Nodes (Libvirt/QEMU) are LXD containers, but also, the Neutron-*-Agents (Linux Bridges) and even Ceph OSDs!

This week something happened and I don’t know what exactly but, Libvirt started to throw the following error while trying to launch a virtual machine:

Nov 13 16:09:40 coscmpt-3 nova-compute[478]: 2020-11-13 16:09:40.991 478 ERROR nova.virt.libvirt.driver [req-a337a254-b9bc-4d86-ba6f-44e3afe5a21e b602d41580b94810bb41f2d752789b81 280b9117b1a74d389f232aff1b8cee98 - default default] [instance: 2c2dfde7-4750-4cab-961e-ae7c153c3765] Failed to start libvirt guest: libvirt.libvirtError: internal error: child reported (status=125): Unable to set XATTR trusted.libvirt.security.dac on /var/lib/libvirt/qemu/domain-1-instance-00000003: Operation not permitted

To fix this, I had to enable security.privileged: "true", then it works again! Thing is, I’ve always used Libvirt/QEMU (with KVM) inside of LXD containers without any problem!

Any idea about what is happening?

Worth to mention that I also have security.nesting: "true", so the ip netns (i.e., the Neutron Agents) can create the required Linux Namespaces inside of the LXD containers (my Network Nodes are running the same Compute Nodes containers as well similar to an OVN topology with OpenvSwitch).

I also have exposed /dev/kvm as unix-char to the LXD Container (like I always do, otherwise the KVM wouldn’t ever work).

Cheers!
Thiago

1 Like
2

Must have been something new on the libvirt side as such extended attributes can only be written by real root and so should never have worked inside unprivileged containers.

Maybe there’s a way to configure libvirt not to do that or maybe your filesystem supports a mount option to disable extended attributes altogether, making libvirt fallback to a different behavior?

3

Libvirt uses XATTRS to remember the original file owner in some cases. There is an option to disable this behavior in /etc/libvirt/qemu.conf: remember_owner = 0

I was able to start a VM in an unprivileged container after making this config change.

2 Likes