ZFS has a very special place in Ubuntu, and will (likely) be the filesystem of choice.
Future versions of Ubuntu Desktop will be able to be installed and boot on ZFS. In Ubuntu 20.04/20.10 you can try it out already as an experimental feature. I have tried it and used it a lot. The cute outcome is that you can just create a dataset for LXD and tell lxd init to use that dataset. No need for extra partitioning!
Obviously, apart from Ubuntu Desktop, you can do the same with Ubuntu Server, etc.
Ever since we added support for multiple storage pools in LXD 2.1 or so, zfs has been our default pick. What has changed is that in the past zfs would need you to install the zfsutils-linux package in order for LXD to be able to use it, without that package, we’d default to btrfs.
With the LXD snap, the ZFS tools are always present, so as long as your kernel supports ZFS, we’ll default to it now.
Systems that would have defaulted to btrfs were systems where either the zfs kernel module or the zfs tools couldn’t be found during lxd init. zfs is always listed as the first option when available and so is picked as the default by lxd init.
LXD as pre-installed on Ubuntu 18.04 does not have the zfs tools available despite the kernel supporting it, so those systems and other prior Ubuntu releases would default to btrfs unless the user had preinstalled the zfs tools.
All releases after that use the snap which comes with the tools built-in, so on any LXD snap install, if the kernel has ZFS support, LXD will default to it during init.
Without being an expert on file systems, I got to the same question: btrfs or ZFS? The documentation says that btrfs has two features that ZFS does not have: storage driver usable inside a container and restore from older snapshots (not latest). On the other hand, the documentation recommends using ZFS:
The two best options for use with LXD are ZFS and btrfs.
They have about similar functionalities but ZFS is more reliable if available on your particular platform.
I would expect a reference in the documentation supporting the statement regarding the better reliability of ZFS.
We’ve never had a production user lose data on ZFS, we have had many production users lose data on btrfs.
Btrfs can be made somewhat reliable when you stay away from compression and RAID features but even then, its completely broken quota support makes it a no-go for any environment where you need quotas to actually be enforced.
In my use case I stay away from compression and RAID. Also, I do not need support for quotas because I use the computer to run benchmark and containers to improve the repeatability of the experiments. Thus, because the aforementioned two additional features, I preferred to use btrfs.
I guess that the feature of being the storage driver usable inside a container is relevant to use nested containers. Otherwise, the nested containers would not take advantage of the storage driver optimizations. Am I right?
I have deployed LXD on low-end commodity hardware. At the moment I have 4 machines with Ubuntu20.04 server on the bare metal with LXD installed using all defaults. The four machines are successfully clustered.
Two machines have a 16GB RAM; two machines are at 8BG RAM. Each has at least one dedicated HD.
My goal is to setup a development environment and tool train in my garage.
I can dedicate another machine strictly for ZFS as NAS; what I do not know is: Is doing so a best practice; or is the best practice to have a dedicated disk and/or partition at each local node.
@stgraber has done a series of videos covering each of the lxd storage drivers along with a discussion of their pros and cons you may find helpful.
Keep in mind also that because lxd supports custom volumes and these can be attached to instances at particular Mount points, it means you can use different storage pools within a single instance to best suit the particular requirements of a directory.
For example, you could have your docker directory on a btrfs, dir or lvm (with ext4 for overlayfs support and effective quotas) custom volume , and the rest of the container on zfs.