Fail to bind mount with Debian Buster (AppArmor issue?)

1

Hi,

With my host running LXD 3.22 under Debian Buster, I was using bind mount without issue for a container running Debian Stretch (with subuid/subgid to enable write access on host). I upgraded that container to Buster (9.x to 10.x so), and the bind mount doesn’t work any more. The host’s syslog indicates an issue with AppArmor (see below) but despite me reading numerous posts on the subject (I don’t know AppArmor well), I’m lost about what to do…Any help? Thank you!

My host has apparmor version 2.13.2-10

Mar 21 20:48:12 hoster kernel: [36075.232132] lxdbr0: port 4(vethcd7573cd) entered disabled state
Mar 21 20:48:13 hoster kernel: [36075.370286] vethf93ab26e: renamed from eth0
Mar 21 20:48:13 hoster kernel: [36075.423954] lxdbr0: port 4(vethcd7573cd) entered disabled state
Mar 21 20:48:13 hoster kernel: [36075.424065] device vethcd7573cd left promiscuous mode
Mar 21 20:48:13 hoster kernel: [36075.424067] lxdbr0: port 4(vethcd7573cd) entered disabled state
Mar 21 20:48:13 hoster kernel: [36076.209438] audit: type=1400 audit(1584820093.837:186): apparmor=“STATUS” operation=“profile_remove” profile=“unconfined” name=“lxd-cloud_</var/snap/lxd/common/lxd>” pid=17361 comm=“apparmor_parser”
Mar 21 20:48:13 hoster kernel: [36076.218346] lxdbr0: port 4(veth9d925da8) entered blocking state
Mar 21 20:48:13 hoster kernel: [36076.218348] lxdbr0: port 4(veth9d925da8) entered disabled state
Mar 21 20:48:13 hoster kernel: [36076.218407] device veth9d925da8 entered promiscuous mode
Mar 21 20:48:13 hoster kernel: [36076.218441] lxdbr0: port 4(veth9d925da8) entered blocking state
Mar 21 20:48:13 hoster kernel: [36076.218442] lxdbr0: port 4(veth9d925da8) entered forwarding state
Mar 21 20:48:13 hoster kernel: [36076.234138] lxdbr0: port 4(veth9d925da8) entered disabled state
Mar 21 20:48:13 hoster kernel: [36076.261068] audit: type=1400 audit(1584820093.889:187): apparmor=“STATUS” operation=“profile_load” profile=“unconfined” name=“lxd-cloud_</var/snap/lxd/common/lxd>” pid=17404 comm=“apparmor_parser”
Mar 21 20:48:13 hoster kernel: [36076.303159] eth0: renamed from veth3b717e17
Mar 21 20:48:13 hoster kernel: [36076.322247] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Mar 21 20:48:13 hoster kernel: [36076.323288] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Mar 21 20:48:13 hoster kernel: [36076.323325] lxdbr0: port 4(veth9d925da8) entered blocking state
Mar 21 20:48:13 hoster kernel: [36076.323326] lxdbr0: port 4(veth9d925da8) entered forwarding state
Mar 21 20:48:15 hoster kernel: [36078.091382] audit: type=1400 audit(1584820095.721:188): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-cloud_</var/snap/lxd/common/lxd>” name=“/run/systemd/unit-root/srv/nextcloud/data/fdn/” pid=17668 comm=“(s-server)” flags=“ro, remount, noatime, bind”
Mar 21 20:48:15 hoster kernel: [36078.208598] audit: type=1400 audit(1584820095.837:189): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-cloud_</var/snap/lxd/common/lxd>” name=“/run/systemd/unit-root/bin/” pid=17729 comm=“(mysqld)” flags=“ro, remount, noatime, bind”
Mar 21 20:48:16 hoster kernel: [36078.615802] audit: type=1400 audit(1584820096.245:190): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-cloud_</var/snap/lxd/common/lxd>” name=“/run/systemd/unit-root/srv/nextcloud/data/fdn/” pid=17732 comm=“(s-server)” flags=“ro, remount, noatime, bind”
Mar 21 20:48:16 hoster kernel: [36079.340253] audit: type=1400 audit(1584820096.969:191): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-cloud_</var/snap/lxd/common/lxd>” name=“/run/systemd/unit-root/srv/nextcloud/data/fdn/” pid=17735 comm=“(s-server)” flags=“ro, remount, noatime, bind”
Mar 21 20:48:17 hoster kernel: [36080.083568] audit: type=1400 audit(1584820097.713:192): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-cloud_</var/snap/lxd/common/lxd>” name=“/run/systemd/unit-root/srv/nextcloud/data/fdn/” pid=17738 comm=“(s-server)” flags=“ro, remount, noatime, bind”
Mar 21 20:48:18 hoster kernel: [36080.872807] audit: type=1400 audit(1584820098.501:193): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-cloud_</var/snap/lxd/common/lxd>” name=“/run/systemd/unit-root/srv/nextcloud/data/fdn/” pid=17741 comm=“(s-server)” flags=“ro, remount, noatime, bind”

2

I discovered the containers’ profiles in /var/snap/lxd/common/lxd/security/apparmor/profiles. Are we supposed to play directly with them? Or could/should we do it via a “lxc config edit” ?
I can join the existing profile if needed (I haven’t changed them right now).

3

@stgraber : I found a hack for the bind-mount issue, but I think this is a bug in LXD

I noticed the error message on the bind mount was about flags=“ro, remount, noatime, bind”, whereas apparmor profiles generated by LXD in in /var/snap/lxd/common/lxd/security/apparmor/profiles speficy “mount options=(ro,remount,bind,noatime)”.

Duplicating the lines with “mount options=(ro,remount,bind,noatime)” into “mount options=(ro,remount,noatime,bind)” fixed the bin-mount issue. It works, but that’s dirty and as for me that"s a bug in LXD, isn’t it? Shouldn’t LXD generate the right apparmor profile and/or generate bind-mount commands with arguments in the right order?

4

Well, actually, it’s a bug in apparmor :slight_smile:
The order in which the flags are specified has no importance whatsoever, so forcing us to put every single combination is ridiculous.

5

https://github.com/lxc/lxd/pull/7070 as yet another workaround for this apparmor mess.

6

Hi Stéphane. I agree - order should not matter. Anway, the fix/hack is welcome in the meantime.