Hi, quick question: is docker’s overlay2 supposed to work on btrfs host?
Longer version: In order to avoid the known issues with docker’s btrfs driver (notably when it comes to exporting or moving), I thought I’d give overlay2 a try and proceeded like in the official docker documentation, but docker startup fails with level=error msg="failed to mount overlay: invalid argument" storage-driver=overlay2.
The container has been created using the following line: lxc launch images:ubuntu/22.04 photoview -c security.nesting=true -c security.syscalls.intercept.mknod=true -c security.syscalls.intercept.setxattr=true
Docker version is docker.io/jammy-updates,now 20.10.21-0ubuntu1~22.04.2 from 22.04’s repository.
Host is Ubuntu 20.04 with btrfs everywhere. Kernel is standard ubuntu 5.15.0-67-generic.
I guess I could just go for vfs or use a lxd vm instead of a container, but first I’d like to make sure it’s not some user error on my side
(Or I’ll go with btrfs after all. If I understand correctly its “failure” mode is similar to what vfs does anyway.)
root@photoview:~# dmesg | grep overl
[270044.387623] overlayfs: idmapped layers are currently not supported
[270046.650667] overlayfs: idmapped layers are currently not supported
[270048.896203] overlayfs: idmapped layers are currently not supported
[270242.494895] overlayfs: idmapped layers are currently not supported
[270244.647997] overlayfs: idmapped layers are currently not supported
[270246.901194] overlayfs: idmapped layers are currently not supported
root@photoview:~# mount | grep idmap
/dev/sda4 on / type btrfs (rw,relatime,idmapped,
So maybe sometime in the future, but docker on lxd has quite a few gotchas. Maybe it would work if the host were running 22.04 and its newer kernel? This comment sounds as if it’s a solved issue by now.
@Gilles
mknod doesn’t work with overlay2 though so you’ll have to use btrfs as backend storage driver for docker if that’s something you need (e.g. some docker containers rely on mknod)
you can test this as follows (assuming you have a kernel version > 5.19):
lxc init images:<whatever image you like> docker-ct
lxc config set docker-ct security{.nesting=true,.syscalls.intercept{.mknod=true,.setxattr=true}}
lxc start docker-ct
lxc exec docker-ct -- su -l
# here proceed to install docker then perform these steps
# vim /etc/docker/daemon.json
# {
# "storage-driver":"overlay2"
# }
# systemctl restart docker.service
docker run -it --rm busybox
# mknod /root/test c 1 3
# > mknod: /root/test: Operation not permitted
# replace overlay2 above with btrfs and repeat the test
# and mknod should work now
I’ve tried it just now with the current kernel (5.19.0-38-generic #39~22.04.1-Ubuntu), but it’s still not working correctly. But given this report on launchpad, I suppose you are aware of it
@amikhalitsyn
Great! So I’ll just have to wait a few days(?) till it’s available via the regular repositories? Because right now the latest one I can get is 5.19.0-38.
unfortunately, I have no information about release queue and when this fix will be available in production. I think it’s worth to install -proposed kernel.
