Libvirt uses XATTRS to remember the original file owner in some cases. There is an option to disable this behavior in /etc/libvirt/qemu.conf: remember_owner = 0
I was able to start a VM in an unprivileged container after making this config change.