Synergies between snaps and LXD containers

Yes, exactly, in the long term, these constructs could be unified.

For the present, perhaps you can already use LXD containers as application containers.

I do something similar by separating the OS from application data. The OS is a container. The application data is a separate filesystem (or more) that I attach to the container. When I need to upgrade the OS, I delete the container, I rebuild it, and I reattach the data.

This way I can backup and restore my data independently of the container. I consider the containers expendable/replaceable and I don’t even back them up. I backup the zfs filesystems with the application data. When I need to migrate my containers to a new LXD host, I copy the data filesystems, and rebuild the containers on the new host.

I prefer using alpine containers for this, whenever possible, typically for website software, but not for databases. Isn’t alpine the preferred base of docker containers? Thanks to docker, alpine has up-to-date versions of many software packages.

I don’t do OS updates on alpine containers. When a new alpine version comes out, I delete my old alpine containers and rebuild them, starting with the latest alpine image. More precisely, I rebuild a template container with the packages and configuration I want for a particular type of application (e.g. nginx, or apache+php), I make a snapshot of this template container, and I then replace each website container by deleting it and cloning it from the template snapshot:
lxc copy {template}/{snapshot} {container}

I do some additional things that bring the OS to the same state as before: I retain the host ssh keys and recreate users (typically the user I use to ssh to the container). I restore certain service initialization files, etc. I make sure the new container has the same local ip address as the old container. I do this by preserving the hwaddr of the old container and configuring it in the new container.

Software packages typically mix their files with user files, some more than others. I modify their configuration a bit to completely separate my application files from the software package files.

For example, the nginx alpine package expects users to put their nginx configuration files in /etc/nginx/http.d/
I typically add a file there with this contents:

include /etc/opt/nginx/*.conf;

I have arranged for /etc/opt/ to be an attached LXD device, separate from the container. It belongs to a zfs filesystem outside of LXD. I do the same with /var/opt, /opt, /usr/local/bin, and /home. These directories are typically empty or don’t exist in an OS image. I put my files there to keep them separate from the OS.

I also attach a second zfs filesystem to /var/log, so that I preserve the OS logs when I replace the container.

Going back to the nginx container, I put application files in /var/opt, configuration files in /etc/opt, custom binaries in /usr/local/bin (or in separate read-only filesystem that I attach to multiple containers). There are some OS files that I want to preserve (such as host ssh/rsa key files or custom service start-up files in /etc/init.d/). I make copies of these files in /etc/opt/etc/ or /etc/opt/copy/ When I rebuild the OS, I copy /etc/opt/etc/ to /etc/ or /etc/opt/copy/ to /.

I have done this with debian containers too, but not as much. The steps to do this are different between the two OSes.

The procedure that I described requires a lot of automation on top of LXD. For each container template, I have template configuration file that specifies how to build it from scratch, or from another template, including creating and attaching disk devices, installing packages and files, running scripts in the container, creating users, etc. I also have an instance configuration file (same format as the template configuration file) with instructions for how to create a container from the template. I use this second configuration file to create multiple application containers (instances) from the template.

Here’s a previous post that I wrote for this: