"ZFS doesn't support restoring from snapshots other than the latest one"

Yeah, thats right. But keep in mind that the zfs.clone_copy setting also applies to “normal” instance creation as well (from an image), and not just copies of one instance to another.

The normal creation of an instance on a ZFS pool is to “copy” the image volume to a new instance volume. Normally this uses zfs.clone_copy=true which means it just takes a snapshot of the source image to use as the instance volume. With zfs.clone_copy=false the full image volume is duplicated for the new instance volume. With zfs.clone_copy=rebase a snapshot is taken (same as zfs.clone_copy=true), but on subsequent copies of the instance to a new instance, the new instance volume is duplicated based on only the differences between the source instance and the source image.

So zfs.clone_copy=rebase is only different to zfs.clone_copy=true when copying instances to another instance. Whereas zfs.clone_copy=false will affect normal instance creation too.

@stgraber does this all sound right to you?

hmm. so its now going to change how images are created as well.

I just want to standardise the process, offering the best integrity without having to carry out so many hacks.

So btrfs does not have these issues? If not what setting would be emulate this on ZFS pool which is the recommended one.


No, zfs.clone_copy=false wont change how images are created. I believe however it will cause a fully copy of the image for new instances (like using a dir pool), but with the benefit that afterwards you can create cheap snapshots.

But this is why I suggested zfs.clone_copy=rebase as it sounded like it would fit your needs.

Have you tried it?

Ahh, so zfs.clone_copy=rebase so will make the copy of the container the same as if I created the container manually, rather than bundling all in a new image?

lxc storage set default zfs.clone_copy rebase
Error: Invalid value for a boolean "rebase"

Yes. It will copy the differences between the image and the source instance into the new volume.

You must be running an older version.

I am using the stable version 4.0.5

Right so that is an LTS (Long Term Support) release and doesn’t get the newest features, so this will be in LXD 5 or you can switch to the latest/stable channel (But keep in mind you won’t be able to switch back as it will alter your database with latest schema).


Alternatively, if your not dead set on ZFS, have you looked at LVM?
In thin pool mode (the default for LXD), it supports efficient snapshots, and restoring from arbitrary snapshots, as well as copying from instances or instance snapshots, without preventing future restores or deletions.

I am interested in the best setup for LXD on a production server. I did not look at LVM because the documentation recommends ZFS and btrfs.