I am a bit lost. I have found plenty of questions from people trying but failing to run overlayfs in a container to run a docker image, but so far, I have not found anywhere that explains how I would get it to run. I am not using zfs, so that part is not an issue for me. Am I missing a treasure trove of useful howtos, or how would I go about figuring out what is needed? So far, I added the kernel module to the lxd profile for autoloading, but what would the next step be?
That should be it. Overlayfs, at least on Ubuntu kernels works fine inside of unprivileged containers.
You could try to force Docker to use overlay2 if it’s somehow still selecting vfs.
So I added
# cat /etc/docker/daemon.json
{
"storage-driver": "overlay2"
}
test-docker#
And I am getting the following output when running /usr/bin/dockerd:
INFO[2022-04-19T20:52:00.632989159Z] Starting up
INFO[2022-04-19T20:52:00.634453031Z] libcontainerd: started new containerd process pid=437
INFO[2022-04-19T20:52:00.634506959Z] parsed scheme: "unix" module=grpc
INFO[2022-04-19T20:52:00.634525120Z] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2022-04-19T20:52:00.634566491Z] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2022-04-19T20:52:00.634930776Z] ClientConn switching balancer to "pick_first" module=grpc
WARN[0000] deprecated version : `1`, please switch to version `2`
INFO[2022-04-19T20:52:00.665233400Z] starting containerd revision=3df54a852345ae127d1fa3092b95168e4a88e2f8 version=v1.5.11
INFO[2022-04-19T20:52:00.705013090Z] loading plugin "io.containerd.content.v1.content"... type=io.containerd.content.v1
INFO[2022-04-19T20:52:00.705155293Z] loading plugin "io.containerd.snapshotter.v1.aufs"... type=io.containerd.snapshotter.v1
INFO[2022-04-19T20:52:00.706356193Z] skip loading plugin "io.containerd.snapshotter.v1.aufs"... error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: can't change directory to '/lib/modules': No such file or directory\\n\"): skip plugin" type=io.containerd.snapshotter.v1
INFO[2022-04-19T20:52:00.706405986Z] loading plugin "io.containerd.snapshotter.v1.btrfs"... type=io.containerd.snapshotter.v1
INFO[2022-04-19T20:52:00.706672841Z] skip loading plugin "io.containerd.snapshotter.v1.btrfs"... error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
INFO[2022-04-19T20:52:00.706706904Z] loading plugin "io.containerd.snapshotter.v1.devmapper"... type=io.containerd.snapshotter.v1
WARN[2022-04-19T20:52:00.706750739Z] failed to load plugin io.containerd.snapshotter.v1.devmapper error="devmapper not configured"
INFO[2022-04-19T20:52:00.706774872Z] loading plugin "io.containerd.snapshotter.v1.native"... type=io.containerd.snapshotter.v1
INFO[2022-04-19T20:52:00.706827695Z] loading plugin "io.containerd.snapshotter.v1.overlayfs"... type=io.containerd.snapshotter.v1
INFO[2022-04-19T20:52:00.714541198Z] loading plugin "io.containerd.snapshotter.v1.zfs"... type=io.containerd.snapshotter.v1
INFO[2022-04-19T20:52:00.714803589Z] skip loading plugin "io.containerd.snapshotter.v1.zfs"... error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
INFO[2022-04-19T20:52:00.714890616Z] loading plugin "io.containerd.metadata.v1.bolt"... type=io.containerd.metadata.v1
WARN[2022-04-19T20:52:00.714928801Z] could not use snapshotter devmapper in metadata plugin error="devmapper not configured"
INFO[2022-04-19T20:52:00.714952071Z] metadata content store policy set policy=shared
INFO[2022-04-19T20:52:00.715086923Z] loading plugin "io.containerd.differ.v1.walking"... type=io.containerd.differ.v1
INFO[2022-04-19T20:52:00.715123789Z] loading plugin "io.containerd.gc.v1.scheduler"... type=io.containerd.gc.v1
INFO[2022-04-19T20:52:00.715185587Z] loading plugin "io.containerd.service.v1.introspection-service"... type=io.containerd.service.v1
INFO[2022-04-19T20:52:00.715941906Z] loading plugin "io.containerd.service.v1.containers-service"... type=io.containerd.service.v1
INFO[2022-04-19T20:52:00.716012241Z] loading plugin "io.containerd.service.v1.content-service"... type=io.containerd.service.v1
INFO[2022-04-19T20:52:00.716053648Z] loading plugin "io.containerd.service.v1.diff-service"... type=io.containerd.service.v1
INFO[2022-04-19T20:52:00.716092186Z] loading plugin "io.containerd.service.v1.images-service"... type=io.containerd.service.v1
INFO[2022-04-19T20:52:00.716129468Z] loading plugin "io.containerd.service.v1.leases-service"... type=io.containerd.service.v1
INFO[2022-04-19T20:52:00.716166984Z] loading plugin "io.containerd.service.v1.namespaces-service"... type=io.containerd.service.v1
INFO[2022-04-19T20:52:00.716207617Z] loading plugin "io.containerd.service.v1.snapshots-service"... type=io.containerd.service.v1
INFO[2022-04-19T20:52:00.716245443Z] loading plugin "io.containerd.runtime.v1.linux"... type=io.containerd.runtime.v1
INFO[2022-04-19T20:52:00.716385994Z] loading plugin "io.containerd.runtime.v2.task"... type=io.containerd.runtime.v2
INFO[2022-04-19T20:52:00.716481453Z] loading plugin "io.containerd.monitor.v1.cgroups"... type=io.containerd.monitor.v1
INFO[2022-04-19T20:52:00.716981010Z] loading plugin "io.containerd.service.v1.tasks-service"... type=io.containerd.service.v1
INFO[2022-04-19T20:52:00.717033348Z] loading plugin "io.containerd.internal.v1.restart"... type=io.containerd.internal.v1
INFO[2022-04-19T20:52:00.717136517Z] loading plugin "io.containerd.grpc.v1.containers"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717195269Z] loading plugin "io.containerd.grpc.v1.content"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717245453Z] loading plugin "io.containerd.grpc.v1.diff"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717289241Z] loading plugin "io.containerd.grpc.v1.events"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717334400Z] loading plugin "io.containerd.grpc.v1.healthcheck"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717374736Z] loading plugin "io.containerd.grpc.v1.images"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717421747Z] loading plugin "io.containerd.grpc.v1.leases"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717456179Z] loading plugin "io.containerd.grpc.v1.namespaces"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717504581Z] loading plugin "io.containerd.internal.v1.opt"... type=io.containerd.internal.v1
INFO[2022-04-19T20:52:00.717592062Z] loading plugin "io.containerd.grpc.v1.snapshots"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717658946Z] loading plugin "io.containerd.grpc.v1.tasks"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717740697Z] loading plugin "io.containerd.grpc.v1.version"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.717902691Z] loading plugin "io.containerd.grpc.v1.introspection"... type=io.containerd.grpc.v1
INFO[2022-04-19T20:52:00.718209726Z] serving... address=/var/run/docker/containerd/containerd-debug.sock
INFO[2022-04-19T20:52:00.718299935Z] serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
INFO[2022-04-19T20:52:00.718482964Z] serving... address=/var/run/docker/containerd/containerd.sock
INFO[2022-04-19T20:52:00.718534681Z] containerd successfully booted in 0.055487s
INFO[2022-04-19T20:52:00.731357757Z] parsed scheme: "unix" module=grpc
INFO[2022-04-19T20:52:00.731407287Z] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2022-04-19T20:52:00.731446789Z] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2022-04-19T20:52:00.731466680Z] ClientConn switching balancer to "pick_first" module=grpc
INFO[2022-04-19T20:52:00.732785229Z] parsed scheme: "unix" module=grpc
INFO[2022-04-19T20:52:00.732819985Z] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2022-04-19T20:52:00.732855476Z] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2022-04-19T20:52:00.732875406Z] ClientConn switching balancer to "pick_first" module=grpc
ERRO[2022-04-19T20:52:00.741381502Z] failed to mount overlay: invalid argument storage-driver=overlay2
INFO[2022-04-19T20:52:00.742106708Z] stopping event stream following graceful shutdown error="context canceled" module=libcontainerd namespace=plugins.moby
INFO[2022-04-19T20:52:00.742108338Z] stopping healthcheck following graceful shutdown module=libcontainerd
failed to start daemon: error initializing graphdriver: driver not supported
Same thing when trying to change the overlay2 to just overlay. Any suggestions on what I might be missing?
What kernel are you running? (uname -a)
% uname -a
Linux zerus 5.15.32-gentoo-r1-x86_64-mw #4 SMP Wed Apr 20 23:06:07 CEST 2022 x86_64 Intel(R) Atom(TM) CPU C3758 @ 2.20GHz GenuineIntel GNU/Linux
Your kernel doesn’t have support for unprivileged overlayfs.
It’s currently not a mainline feature and it looks like only the Ubuntu and ArchLinux kernels carry the patch needed.