Proposal:
Add a new device type to container configurations and/or profiles that allows specifying a network filesystem mount (e.g. NFS, CephFS, etc.) on an unprivileged container. The goal would be to provide a safe and portable way to have the container host mount a shared network resource into an unprivileged container before or at container launch.
Device Examples:
The device configuration would closely resemble what is typically placed in /etc/fstab:
devices:
nfs-www:
path: /var/www
source: nfs-server:/nfs/www
fs-type: nfs
options:
- vers: 4
- noexec
type: net-fs
cephfs-debian:
path: /var/www/debian
source: ceph-mon:/debian
fs-type: ceph
options:
- name: cephfs.debian.ro
- secretfile: /etc/ceph/client.cephfs.debian.ro.secret
- noatime
type: net-fs
Benefits:
- No need for ‘security.privileged=true’ to get mounts working internally in the container.
- Avoids manually replicating the fstab entries across all host machines that need to be able to host the container.
- Minimizes the number of client connections to the external resource because mounts can happen JIT at container launch instead of having to be pre-mounted on all hosts.
Disadvantages:
- The device configuration parameters and options would need to be sanitized by LXD.
- Possible duplicate mounts if multiple containers on the same host shared the same device configuration?
- Moving a container between hosts might require extra logic for mount/umount?
Motivation:
It feels like I am regularly encountering cases where I would like to use LXD and containers in lieu of KVM and virtualization but can’t because the solution depends on mounting shared/common resources into an otherwise generic instance. A few such use cases:
- Multiple webservers that all need to serve up the same contents under /var/www. Preferably, /var/www should be mounted from a NAS/SAN/NFS/CephFS server to ease deployment of code updates and to make it simpler to spin up/down LXD webserver instances on demand.
- A Debian/Ubuntu package mirror where the contents of the repository is stored outside of the container (e.g. on a CephFS pool). Doing so is almost mandatory to allow for easy export, migration and restoration of the container without many TBs of package data in the rootfs image.
I understand that there are at least a few ways to work around these LXC/LXD constraints, but none of them feel right in my experience. I certainly think mounting external resources onto the container host(s) and then bind mounting it to the container is not intuitive and definitely a pain point when it comes to administration and configuration of LXD hosts and containers.
I wanted to scribble down my thoughts and experiences on the point and see what others think. Maybe a feature such as this could help with adoption of LXD? Or maybe I am the only one having qualms about implementing shared resource mounts within the current constraints of LXD.
Is there already a more elegant way to do this that I just haven’t found from perusing the documentation?