Can an lxc container with AppArmor be run inside an lxd managed container (nested)?
I cannot get proxmox’s lxc-start inside the lxd container to work if I am using lxc.apparmor.profile = generated.
Any tips or hints if this is possible are appreciated.
dmesg:
[21952.109905] audit: type=1400 audit(1648839251.313:1885): apparmor="STATUS" info="failed to unpack end of profile" error=-71 namespace="root//lxd-deb11pve_<var-lib-lxd>" profile="/usr/bin/lxc-start" name="lxc-103_</var/lib/lxc>" pid=947129 comm="apparmor_parser" name="lxc-103_</var/lib/lxc>" offset=151
root@deb11pve:~# lxc-start -n 103 -F -l DEBUG -o /tmp/lxc-103.log
lxc-start: 103: cgroups/cgroup2_devices.c: bpf_program_load_kernel: 332 Operation not permitted - Failed to load bpf program: (null)
lxc-start: 103: sync.c: sync_wait: 34 An error occurred in another process (expected sequence number 4)
lxc-start: 103: start.c: __lxc_start: 2068 Failed to spawn container "103"
lxc-start: 103: lsm/apparmor.c: remove_apparmor_namespace: 883 No such file or directory - Error removing AppArmor namespace
lxc-start: 103: tools/lxc_start.c: main: 306 The container failed to start
lxc-start: 103: tools/lxc_start.c: main: 311 Additional information can be obtained by setting the --logfile and --logpriority options
root@deb11pve:~# aa-status
apparmor module is loaded.
8 profiles are loaded.
8 profiles are in enforce mode.
/usr/bin/lxc-start
lsb_release
lxc-container-default
lxc-container-default-cgns
lxc-container-default-with-mounting
lxc-container-default-with-nesting
nvidia_modprobe
nvidia_modprobe//kmod
0 profiles are in complain mode.
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.
Background:
I am essentially trying to run proxmox inside an unprivileged lxd container (https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_11_Bullsey).
Virtual machines in proxmox run without any problems and so do lxc containers if either I hide AppArmor or switch the lxc container in proxmox from lxc.apparmor.profile = generated to lxc.apparmor.profile: lxc-container-default-cgns.
I am aware that this is an unusual setup and that can cause quite a lot of problems but I do want to test the limits of nested container and AppArmor stacking.
EDIT:
I test a few more setups:
Archlinux host with AppArmor and running an lxd arch Linux container.
Inside this lxd container, I got both Alpine as well as Debian 11 working inside lxc (with lxc.apparmor.profile = generated).
lxc-create -n alp -t /usr/share/lxc/templates/lxc-download -- --dist alpine --release 3.15 --arch amd64
lxc-create -n deb -t /usr/share/lxc/templates/lxc-download -- --dist debian --release bullseye --arch amd64 --no-validate
Same setup but Debian 11 running inside lxd arch Linux container.
Alpine Linux does work but Debian does not since systemd fails:
Failed to mount tmpfs at /dev/shm: Permission denied
apparmor="DENIED" operation="mount" info="failed flags match"