Docker cannot write to devices.allow


(Jan Delius) #1

I've a issue whilst try to run docker containers inside an lxc container(using lxd).

I tried to setup the following docker-compose project:
https://mailcow.github.io/mailcow-dockerized-docs/

When running the "docker-compose up -d " command, the following error appears:

ERROR: for fail2ban-mailcow  Cannot start service fail2ban-mailcow: OCI runtime create failed: container_linux.go:296: starting container process caused "process_linux.go:398: container init caused \"process_linux.go:365: setting cgroup config for procHooks process caused \\\"failed to write a *:* rwm to devices.allow: write /sys/fs/cgroup/devices/docker/9cc4fa5ee410af11cdaaffd1a046979be3a9e0db3beaf53988dfbef8ad614999/devices.allow: operation not permitted\\\"\"": unknown

ERROR: for ipv6nat  Cannot start service ipv6nat: OCI runtime create failed: container_linux.go:296: starting container process caused "process_linux.go:398: container init caused \"process_linux.go:365: setting cgroup config for procHooks process caused \\\"failed to write a *:* rwm to devices.allow: write /sys/fs/cgroup/devices/docker/a23565f8a46dae7e69164f2041793bb3958a273891573ec0028440ffa27598a9/devices.allow: operation not permitted\\\"\"": unknown

The lxc container is running ubuntu-daily:17.10 currently. The error occours on ubuntu:17.10 too.

ubuntu:16.04 and images:ubuntu/xenial fails earlier.

I tried docker via package(docker.io) and current master from git.

After setting a raw.lxc config like here, docker cannot start anymore.
The configuration security.nesting and security.privileged are true.


(Jan Delius) #2

After setting raw.lxc following error stops docker from working:

Dec 30 19:09:58 mail dockerd[1245]: time="2017-12-30T19:09:58.858154568Z" level=error msg="There are no more loopback devices available."
Dec 30 19:09:58 mail dockerd[1245]: time="2017-12-30T19:09:58.858341501Z" level=error msg="[graphdriver] prior storage driver devicemapper failed: loopback attach failed"
Dec 30 19:09:58 mail dockerd[1245]: Error starting daemon: error initializing graphdriver: loopback attach failed

When i apply linux.kernel_modules from the other thread, nothing is changed.


(Andrew Ernst) #3

Hi there! I'm not sure if this helps you out, but I fired up a lxd instance on my system this evening to test the mailcow deployment. It went pretty smoothly, so hopefully my experiences can help you get it deployed similarly.

The entirety of my deployment is at https://asciinema.org/a/yfaU6Gc5aRuASglBC38wtriR3

I had to run a privileged base lxd container to get this working correctly, though, so keep that in mind from a security perspective.

What isn't shown in that asciinema output, is when I created my docker profile. I did that with lxc profile copy default docker and then edited with lxc profile edit docker adding the raw.lxc configuration as follows:

config:
  raw.lxc: |-
    lxc.aa_profile = unconfined
    lxc.cgroup.devices.allow = a
    lxc.mount.auto=proc:rw sys:rw
    lxc.cap.drop=

When you deploy the lxd instance, make sure you enable the configuration settings that allow Docker to run embedded, which are `security.nesting=true` and `security.privileged=true`. You can do this at `lxc launch` time with the `-c` flag or after-the-launch with a combination of:

lxc config set mailcow security.nesting true
lxc config set mailcow security.privileged true
lxc restart mailcow

Good luck, and I hope you manage to get everything working as expected.

Cheers,

  • Andrew


(Ruben Barkow) #4

Did you find a solution? I have the same Problem in a XEN vm: