Proposal: New container device type to support network mounts on unprivileged containers

Proposal:

Add a new device type to container configurations and/or profiles that allows specifying a network filesystem mount (e.g. NFS, CephFS, etc.) on an unprivileged container. The goal would be to provide a safe and portable way to have the container host mount a shared network resource into an unprivileged container before or at container launch.

Device Examples:

The device configuration would closely resemble what is typically placed in /etc/fstab:

devices:
  nfs-www:
    path: /var/www
    source: nfs-server:/nfs/www
    fs-type: nfs
    options:
      - vers: 4
      - noexec
    type: net-fs
  cephfs-debian:
    path: /var/www/debian
    source: ceph-mon:/debian
    fs-type: ceph
    options:
      - name: cephfs.debian.ro
      - secretfile: /etc/ceph/client.cephfs.debian.ro.secret
      - noatime
    type: net-fs

Benefits:

  • No need for ‘security.privileged=true’ to get mounts working internally in the container.
  • Avoids manually replicating the fstab entries across all host machines that need to be able to host the container.
  • Minimizes the number of client connections to the external resource because mounts can happen JIT at container launch instead of having to be pre-mounted on all hosts.

Disadvantages:

  • The device configuration parameters and options would need to be sanitized by LXD.
  • Possible duplicate mounts if multiple containers on the same host shared the same device configuration?
  • Moving a container between hosts might require extra logic for mount/umount?

Motivation:

It feels like I am regularly encountering cases where I would like to use LXD and containers in lieu of KVM and virtualization but can’t because the solution depends on mounting shared/common resources into an otherwise generic instance. A few such use cases:

  1. Multiple webservers that all need to serve up the same contents under /var/www. Preferably, /var/www should be mounted from a NAS/SAN/NFS/CephFS server to ease deployment of code updates and to make it simpler to spin up/down LXD webserver instances on demand.
  2. A Debian/Ubuntu package mirror where the contents of the repository is stored outside of the container (e.g. on a CephFS pool). Doing so is almost mandatory to allow for easy export, migration and restoration of the container without many TBs of package data in the rootfs image.

I understand that there are at least a few ways to work around these LXC/LXD constraints, but none of them feel right in my experience. I certainly think mounting external resources onto the container host(s) and then bind mounting it to the container is not intuitive and definitely a pain point when it comes to administration and configuration of LXD hosts and containers.

I wanted to scribble down my thoughts and experiences on the point and see what others think. Maybe a feature such as this could help with adoption of LXD? Or maybe I am the only one having qualms about implementing shared resource mounts within the current constraints of LXD.

Is there already a more elegant way to do this that I just haven’t found from perusing the documentation?

We’re already doing that:

Right now the focus is on CEPH (both rbd and fs) but the same syntax would be suitable for someone to pretty easily contribute nfs too.

Very nice! Thanks for the links. I will definitely keep an eye on those threads.