Can an LXD container be readonly? I mean that the root (zfs) filesystem is readonly.
I tried it and I found that the answer is yes and no. Yes, because I was able to create a container that runs with a readonly root filesystem. No, because LXD fails to start or stop such a container.
First of all, I attached writable disk devices to the directories that the container might need to write to, such as /var/log, and /tmp.
Having done this, I made the zfs filesystem of the container readonly, while the container was running, and the container continued to run. I was able to start a shell in it and download a file from the internet (to a writable directory).
I used an images:alpine/3.17 image, with just the dhcpcd package added on top of the image.
But when I tried to start or stop such a container, the LXD operations failed. That’s because LXD modifies metadata files in the container filesystem, and also makes some other changes. These fail if the filesystem is readonly.
For the examples below, I use z/lxd as the zfs storage pool filesystem.
Stopping a container (b) with a readonly root filesystem:
sudo zfs set readonly=on z/lxd/containers/b
lxc stop b
Error: Failed clearing ownership: chown /var/snap/lxd/common/lxd/containers/b: read-only file system
Try `lxc info --show-log b` for more info
lxc list -f compact b
NAME STATE IPV4 IPV6 TYPE SNAPSHOTS
b STOPPED CONTAINER 0
The container seems to have stopped.
Starting a container (b) with a readonly root filesystem:
sudo zfs set readonly=on z/lxd/containers/b
lxc start b
Error: saving config file for the container failed
Try `lxc info --show-log b` for more info
The container did not start.