In terms of instance IO performance, when switching from ZFS or BTRFS to the DIR backend, how much will performance suffer? I’m asking since I may combine multiple btrfs pools with mergerfs to create a jbod-like system.
It depends a LOT on setup and workload unfortunately.
dir on top of ext4 or xfs tends to often be the fastest as there’s none of the copy-on-write overhead that comes with ZFS or btrfs, but then you also don’t get fast snapshots as a consequence of that.
ZFS and btrfs can both be reasonably fast with tweaking but the amount of tweaking is workload dependent. Running a large database on either will often suffer because of copy-on-write so specific files or volumes may need additional config options to improve performance (sometimes at the cost of some of the added reliability).
Then you also have the whole question about what’s backing those pools. ZFS and btrfs will both perform better on dedicated disks or partitions (with ZFS particularly liking disks).
3 Likes
Thank you for the (very quick I might add) response! I think I can turn off BTRFS’s copy-on-write(idk about ZFS), would that help with the speed? I guess my big question is BTRFS dir vs direct BTRFS speed(or ZFS dir vs direct ZFS ), has anyone done benchmarks on that?
Using dir on top of a btrfs filesystem or using btrfs on top of a btrfs filesystem or block device won’t make a difference at run time, as far as the container is concerned, it’s the same thing.
The main difference will be at the LXD level as dir won’t give you:
- Copy on write images (each new instance will have the image unpacked fresh)
- No fast snapshots (snapshots will be a fully copy of the instance)
- Slow migration and exports (rsync/full tarball instead of native send/receive)
- No quotas
2 Likes