I am able to successfully install docker and run some containers inside of a container created by LXD. How can I do the same if just using LXC? (It would be a large project to migrate to LXD at this time for me)
The docker.io package installs fine, but there are errors when pulling an image.
root@xen:/# docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
67a0688b88df: Extracting [==================================================>] 722.8 kB/722.8 kB
failed to register layer: ApplyLayer exit status 1 stdout: stderr: permission denied
root@xen:/# journalctl -u docker
No journal files were found.
-- No entries --
In the lxc.conf I have the following line to allow container nesting. lxc.include = /usr/share/lxc/config/nesting.conf
From dmesg I see the apparmor error, do I need to allow some additional mounting for docker to work properly? Does LXD add this automatically when passing -c security.nesting=true ?
The reason why it works in LXD is because LXD has logic to dynamically create apparmor namespaces, making it possible for the container to then load its own profiles, stacked on top of the base profile that LXD sets up.
Doing that with LXC directly is a pretty manual process, which boils down to:
Create a new directory in /sys/kernel/security/apparmor/policy/namespaces, lets use “lxc-xen” as an example here.
Set the lxc.aa_profile or lxc.apparmor.profile config key (dependent on LXC version) to lxc-container-default//&:lxc-xen:
Override lxc.cap.drop to only include sys_time sys_module sys_rawio as mac_override mac_admin are capabilities you’ll now need in your container
Start the container and confirm that cat /proc/self/attr/current inside the container reports unconfined
At that point you’ll have a container with a working apparmor namespace. You may still run into apparmor failures related to that base lxc-container-default profile though which may not be permissive enough for this setup. If that’s the case, you’ll need to either modify it in /etc/apparmor.d/lxc/ or create a new one in there and use that instead.
Thank you! I made some progress but think I ended up in nested apparmor hell. I will document how far I got here in case anyone else attempts this.
From inside LXC container: root@myserver:/# docker run -it busybox docker: Error response from daemon: AppArmor enabled on system but the docker-default profile could not be loaded..
Docker complains about loading the docker apparmor profile so I installed apparmor inside the container and attempted to start it.
root@myserver:/# service apparmor restart
Job for apparmor.service failed because the control process exited with error code. See "systemctl status apparmor.service" and "journalctl -xe" for details.
root@myserver:/# systemctl status apparmor.service
â—Ź apparmor.service - LSB: AppArmor initialization
Loaded: loaded (/etc/init.d/apparmor; bad; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2018-02-08 22:56:17 UTC; 4s ago
Docs: man:systemd-sysv-generator(8)
Process: 3025 ExecStart=/etc/init.d/apparmor start (code=exited, status=123)
Feb 08 22:56:16 myserver apparmor[3025]: /sbin/apparmor_parser: Unable to replace "lxc-container-default". Permission denied; attempted to load a profile while confined?
Feb 08 22:56:16 myserver apparmor[3025]: Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Feb 08 22:56:16 myserver apparmor[3025]: /sbin/apparmor_parser: Unable to replace "/usr/sbin/tcpdump". Permission denied; attempted to load a profile while confined?
Feb 08 22:56:17 myserver apparmor[3025]: /sbin/apparmor_parser: Unable to replace "mount-namespace-capture-helper". Permission denied; attempted to load a profile while confined?
Feb 08 22:56:17 myserver apparmor[3025]: /sbin/apparmor_parser: Unable to replace "/usr/lib/snapd/snap-confine". Permission denied; attempted to load a profile while confined?
Feb 08 22:56:17 myserver apparmor[3025]: ...fail!
Feb 08 22:56:17 myserver systemd[1]: apparmor.service: Control process exited, code=exited status=123
Feb 08 22:56:17 myserver systemd[1]: Failed to start LSB: AppArmor initialization.
Feb 08 22:56:17 myserver systemd[1]: apparmor.service: Unit entered failed state.
Feb 08 22:56:17 myserver systemd[1]: apparmor.service: Failed with result 'exit-code'.
root@myserver:/#
The following errors show up in the kernel log on the host:
Thanks, you were correct. The subsequent error is for oom_score_adj
# docker run -it busybox
docker: Error response from daemon: invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:295: setting oom score for ready process caused \\\"write /proc/2913/oom_score_adj: permission denied\\\"\"\n".
After playing around with this more I’ve gotten it to work. service docker stop (unsure why the daemon has issues). Starting dockerd on its own and can run a container.
EDIT: The above error was caused by setting oom_score_adj to 10 before starting the container. Setting it to 0 has fixed this.
I’ve had some success setting an unconfined apparmor profile in lxd containers, and then running docker run commands appending --security-opt "apparmor:unconfined"
I’m still trying to troubleshoot the Rancher RKE tool for spinning up Kubernetes clusters. It really wants me to have the docker-default apparmor profile.
The LXD profile I use contains the following lines:
