LXD currently doesn’t support ZFS block mode. Supporting this will allow the use of btrfs, ext4, and xfs filesystems on top of ZFS.
Specification
Design
The following new configuration keys will be added for ZFS storage pools:
volume.zfs.block_mode
volume.block.mount_options
volume.block.filesystem
Setting volume.zfs.block_mode to true on the storage pool will turn on ZFS block mode. This setting can be changed per-volume, setting zfs.block_mode on the volume itself. This will use ZFS zvols instead of ZFS datasets. Zvols need to be formatted with a filesystem. The default FS will be ext4 but can be configured using the volume.block.filesystem configuration key.
Once ZFS block mode is turned on, new instances will be created as zvols. Existing instances which use datasets will not be converted to zvols. Also, copying dataset based instances will not create zvols but instead create new datasets.
Zvols will follow the same naming pattern as the existing datasets except for images. These will be suffixed with the filesystem they were formatted with.
Just a bit of food for thought since i don’t have many instances with a requirement for this because Docker Swarm doesn’t like being in an unprivileged container.
If possible, it would be good to have these settings available at the profile/instance level because not all instances will need zvols on a storage pool.
As this allows for the coexistence of filesystem zvols (volumes created after zfs.block_mode was enabled) as well as filesystem datasets in a single pool (volumes create before zfs.block_mode was enabled), would it be more consistent to have this as a volume.zfs.block_mode setting?
I think I agree but rather than special case this one volume setting, I think I’d prefer we think through a way to apply storage volume setting through the root disk device definition.
The same could then be used to set a different filesystem, quota mode, …
So out of scope for this work, but something that I think would make sense in general.
Apologize for jumping into an ongoing team development.
This new feature is promising, yet when I create a container with a block device as storage backend, I also eventually would want to attach another block device as second hdd …
Can in this context, the below feature request be revisited?
The old one: with LVM controller
The new one: with ZFS driver, zfs.block_mode = true
When I copy or move containers from LVM storage to ZFS storage, the zfs.block_mode flag is not honored and the containers are created on ZFS storage as a dataset.
Also if I export containers from the LVM storage and I import them into ZFS storage they are created as dataset instead of zvols.
I understand that “existing instances using datasets will not be converted to zvols” when they are copied or moved between zfs data set, but why zfs.block_mode = true can’t be honored in previous described cases?
I found no way to migrate my db/docker containers (that are not compatible with zfs datasets) from LVM to ZFS and create all of them from scratch is not an option, is there any solution?