Fedora CoreOS and LXD snap confinement

LXD
1

Hello there !

I’m trying to play with Fedora CoreOS virtual machine on my LXD host, but I encounter a problem with the Ignition file to pass to the instance for the bootstrap. I already create the image without any problems but I can’t launch it due to snapd confinement I guess.

I need to pass an Ignition file to the instance using a special device in QEMU (same as with Libvirt), but due to the nature of LXD package, I can’t use it :

$ lxc launch fcos/36 fedora-coreos -p vm \
-c raw.qemu="-fw_cfg name=opt/com.coreos/config,file=/var/lib/snapd/hostfs/lxd/tmp/default.ign"

Error: Failed to run: forklimits limit=memlock:unlimited:unlimited fd=3 -- /snap/lxd/23193/bin/qemu-system-x86_64 -S -name fedora-coreos -uuid 0911de70-e16c-4825-97a4-37680c5bd7d7 -daemonize -cpu host,hv_passthrough -nographic -serial chardev:console -nodefaults -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=allow,resourcecontrol=deny -readconfig /var/snap/lxd/common/lxd/logs/fedora-coreos/qemu.conf -spice unix=on,disable-ticketing=on,addr=/var/snap/lxd/common/lxd/logs/fedora-coreos/qemu.spice -pidfile /var/snap/lxd/common/lxd/logs/fedora-coreos/qemu.pid -D /var/snap/lxd/common/lxd/logs/fedora-coreos/qemu.log -smbios type=2,manufacturer=Canonical Ltd.,product=LXD -runas lxd -fw_cfg name=opt/com.coreos/config,file=/var/lib/snapd/hostfs/lxd/tmp/default.ign: : Process exited with non-zero value 1
Try `lxc info --show-log fedora-coreos` for more info

$ lxc info --show-log fedora-coreos                                                                                                                                                        Name: fedora-coreos
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2022/06/19 15:57 UTC

Log:

qemu-system-x86_64: -fw_cfg name=opt/com.coreos/config,file=/var/lib/snapd/hostfs/lxd/tmp/default.ign: can't load /var/lib/snapd/hostfs/lxd/tmp/default.ign: Failed to open file “/var/lib/snapd/hostfs/lxd/tmp/default.ign”: Permission denied

Note: the path to the file is world-readable

I tried without the /var/lib/snapd/hostfs prefix, but it seems obvious that it wasn’t going to work due path to the file not available inside the snap mount namespace.

Is there any known solution to let QEMU access file outside of the snap confinement (I doubt about it but who knows :slight_smile: ) ? If not, I guess the only solution is to accomplish an ISO installation with a remote Ignition accessible from an S3 bucket or HTTP server, but this is not very convenient :frowning:

Thanks !

2

Can you check dmesg to see if it’s apparmor blocking it?
If it is, then you can use raw.apparmor to allow it.

3

Ah yes, I always forget about AppArmor… indeed the file load was denied.

[334537.614355] audit: type=1400 audit(1655658464.067:39848): apparmor="DENIED" operation="open" profile="lxd-fedora-coreos_</var/snap/lxd/common/lxd>" name="/var/lib/snapd/hostfs/lxd/tmp/default.ign" pid=461143 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=0 ouid=1000

It works way better now with the raw.apparmor option :

$ lxc launch fcos/36 fedora-coreos -p vm \
-c raw.qemu="-fw_cfg name=opt/com.coreos/config,file=/var/lib/snapd/hostfs/lxd/tmp/default.ign" \
-c raw.apparmor="/var/lib/snapd/hostfs/lxd/tmp/default.ign r,"

And if I try to connect to the instance, it works as expected :

$ ssh core@10.0.122.120
Fedora CoreOS 36.20220522.3.0
Tracker: https://github.com/coreos/fedora-coreos-tracker
Discuss: https://discussion.fedoraproject.org/tag/coreos

[core@fcos-lxd ~]$

Thank you @stgraber !

1 Like