[LXD] ZFS block mode

Specifications
#1
Project LXD
Status Draft
Author(s) @monstermunchkin
Approver(s) @stgraber @tomp
Release 5.10
Internal ID LXXXX

Abstract

This adds support for ZFS block mode.

Rationale

LXD currently doesn’t support ZFS block mode. Supporting this will allow the use of btrfs, ext4, and xfs filesystems on top of ZFS.

Specification

Design

The following new configuration keys will be added for ZFS storage pools:

  • volume.zfs.block_mode
  • volume.block.mount_options
  • volume.block.filesystem

Setting volume.zfs.block_mode to true on the storage pool will turn on ZFS block mode. This setting can be changed per-volume, setting zfs.block_mode on the volume itself. This will use ZFS zvols instead of ZFS datasets. Zvols need to be formatted with a filesystem. The default FS will be ext4 but can be configured using the volume.block.filesystem configuration key.

Once ZFS block mode is turned on, new instances will be created as zvols. Existing instances which use datasets will not be converted to zvols. Also, copying dataset based instances will not create zvols but instead create new datasets.

Zvols will follow the same naming pattern as the existing datasets except for images. These will be suffixed with the filesystem they were formatted with.

Example:

default/images/999c7ff6585ab0bcc6ff21736407cfd494dcdde2c68907512fef2b307a25c36b_ext4

If there are both a zvol and a dataset for the same image, this will not create an additional database record.

When deleting image storage volumes, it will attempt to delete both datasets and zvols.

API changes

No API changes.

CLI changes

No CLI changes.

Database changes

No database changes.

Upgrade handling

No upgrading handling.

Further information

No further information.

Synergies between snaps and LXD containers
#2

Hi

Just a bit of food for thought since i don’t have many instances with a requirement for this because Docker Swarm doesn’t like being in an unprivileged container.

If possible, it would be good to have these settings available at the profile/instance level because not all instances will need zvols on a storage pool.

Thanks

#3

As this allows for the coexistence of filesystem zvols (volumes created after zfs.block_mode was enabled) as well as filesystem datasets in a single pool (volumes create before zfs.block_mode was enabled), would it be more consistent to have this as a volume.zfs.block_mode setting?

#4

Please could you provide some detail on storage DB record changes and ZFS zvol naming/layout?

#5

I think I agree but rather than special case this one volume setting, I think I’d prefer we think through a way to apply storage volume setting through the root disk device definition.

The same could then be used to set a different filesystem, quota mode, …

So out of scope for this work, but something that I think would make sense in general.

#6

Done.

#7

Apologize for jumping into an ongoing team development.
This new feature is promising, yet when I create a container with a block device as storage backend, I also eventually would want to attach another block device as second hdd …

Can in this context, the below feature request be revisited?

Add lxc config device add disk --type=block for containers · Issue #10077 · lxc/lxd (github.com)