Understanding ZFS dataset mounting

I’m adding a second storage pool to my Incus setup, and it’s on an existing ZFS pool.

I created a ZFS dataset for Incus to use first:

❯ sudo zfs create -o encryption=on -o keylocation=file:///root/key -o keyformat=hex -o atime=off -o recordsize=1M pool/incus

❯ zfs list -o name,mountpoint,mounted,encryption                                                                            
NAME        MOUNTPOINT   MOUNTED  ENCRYPTION
pool        /pool        yes      off
pool/data   /pool/data   yes      aes-256-gcm
pool/incus  /pool/incus  yes      aes-256-gcm

When I tell Incus to use this dataset to create a new pool, it does, but it also unmounts it and changes the mount type to “legacy”:

❯ sudo incus storage create test-pool zfs source=pool/incus                      
Storage pool test-pool created

❯ zfs list -o name,mountpoint,mounted           
NAME                                 MOUNTPOINT  MOUNTED
pool                                 /pool       yes
pool/data                            /pool/data  yes
pool/incus                           legacy      no
pool/incus/buckets                   legacy      no
pool/incus/containers                legacy      no
pool/incus/custom                    legacy      no
pool/incus/deleted                   legacy      no
pool/incus/deleted/buckets           legacy      no
pool/incus/deleted/containers        legacy      no
pool/incus/deleted/custom            legacy      no
pool/incus/deleted/images            legacy      no
pool/incus/deleted/virtual-machines  legacy      no
pool/incus/images                    legacy      no
pool/incus/virtual-machines          legacy      no

Is this expected behaviour? And additionally, I’m kinda curious why? After looking at it a bit, I’m suspecting it’s so the files are mounted under /var/lib/incus/storage-pools/ like they are for my existing pool, but I couldn’t find a definitive answer on Google or these forums so I thought I would ask :slight_smile:

It might be helpful to document this somewhere, initially this confused me a bit. I was also a little surprised by the multiple datasets (e.g. “pool/incus/containers”) and wondering if I did something wrong.

EDIT: Also, if anyone is reading this, don’t do the exact thing I did above — the legacy mount won’t automount on next boot due to the encryption. What I’m doing now (which may or may not be correct) is creating a new dataset for Incus below the dataset I created:

❯ sudo zfs list
NAME         USED  AVAIL  REFER  MOUNTPOINT
pool        1.59M  38.4G  42.0K  /pool
pool/data    217K  38.4G   217K  /pool/data
pool/incus   118K  38.4G   118K  /pool/incus

❯ sudo incus storage create test-pool zfs source=pool/incus/test-pool
Storage pool test-pool created

❯ zfs list -o name,mountpoint,mounted,encryption
NAME                                   MOUNTPOINT   MOUNTED  ENCRYPTION
pool                                   /pool        yes      off
pool/data                              /pool/data   yes      aes-256-gcm
pool/incus                             /pool/incus  yes      aes-256-gcm
pool/incus/test-pool                   legacy       no       aes-256-gcm
pool/incus/test-pool/buckets           legacy       no       aes-256-gcm
... etc ...