I’m working on a project with the goal of running multiple applications in unprivileged containers on a system. Each application should “believe” that it is running natively, but in actuality, all file writes are done to an overlayfs, rather than directly to the rootfs.
The end target is an embedded device running OpenWrt; however, I am currently testing on an Ubuntu 18 machine. That said, the goal is to use LXC without LXD.
This plan “should be simple” given the available documentation, but I seem to be missing something.
My initial attempt was to add a line such as this to the configuration file
lxc.rootfs.path = overlayfs:/:/home/user/delta0
This gives a “too many levels of symbolic links” error. ‘dmesg’ clarifies the reason as being that the upperdir is within the lowerdir, which is not allowed. Is there a way around this? It seems to be a typical use case, and I suspect there is a simple solution I’ve missed. I also tried doing a read-only bind-mount of / into a directory below, but lxc is not fooled.
Without specifying lxc.rootfs, it defaults to the host’s rootfs, as expected – however in this case, I get an error ‘failed to mount “proc” onto “/proc”’ (which isn’t surprising).
I can mount with an overlayfs if the ‘lowerdir’ is a separate rootfs (not the host’s). (as a separate issue, everything in this case is owned by ‘nobody’ within the container – but I haven’t looked into that yet - probably an error in the lxc.idmap).
There are a number of questions in the old mailing list archives addressing this, but they are all many years old and the options and usage has changed since those answers were posted.
Thanks for the help!