One of the (many) reasons I choose LXC + LXD over Docker + K8S is the straight forward (traditional) approach to storage that LXD uses.
When I first started learning about LXD, I found ZFS to be about the most practical thing you could pair with LXD, so I used it for almost 2 years, and I am very happy with it, especially being able to snapshot containers on the fly and launching those snapshots as containers on a different “backup server” and being able to recover containers by just bringing back the copy from the backup host almost instantly.
Now I find myself re-evaluating the usage of ZFS for the database containers due to performance concerns. On dedicated hardware, where I can give ZFS direct control over full physical devices, it is less of an issue, however, when using networked block devices on the cloud, I’ve tried all sorts of combinations and the performance seems not to improve much if you add several disks (or even to degrade the more disks you add), so I tend to stick to one disk (block device).
I run many instances of a PHP APP on containers and then I run the one corresponding database for those instances on another container. Meaning I have n copies of the APP with exactly the same number of database containers (1 to 1). This setup gives me a few very important advantages over the traditional approaches of running one RDMS for all the databases (and a few drawbacks of course), so I would like to keep using it (but I am not closed to alternatives).
So the question is, what would you recommend to use as FS for MySQL/MariaDB database containers?
- In the cloud (AWS mostly).
- On bare-metal