Error: Storage pool "lxdpool" unavailable on this server after update to LXD 5.6

LXD
5

So LXD 5.6 introduced this change:

Which initialises any missing datasets and sets up the pool’s top-level policy on mount (pool start up).

It sounds like because your pool didn’t have the matching policy of legacy (which disables the unnecessary auto mount of the top level dataset) that ZFS tried to unmount it and this seemed to causes issues (perhaps because you have running instances on that pool).

It may also be due to this snap/zfs issue:

In that sometimes after a snap refresh the mount tables inside the snap namespace can get out of sync with the host and this can confuse the ZFS tool run inside the snap to think something is/isn’t mounted when it infact is.

In these situations I would suggest rebooting the host to allow the snap mount table to be brought into sync.

I have tested various scenarios on mount the ZFS datasets in different orders, and it does appear you can get ZFS mounts confused if you, say, start a container, and then mount the zfs pool’s top level dataset and then try and stop the container and the container’s path no longer is available in the host’s mount namespace as its been over mounted.

So in general I would say you have done the correct thing by bringing all of your ZFS pool datasets into legacy mode (so there is no mounting except that done explicitly by LXD).