I have an lxd host whose SSD failed and which I’ve now replaced. The system partition is non-zfs, but the lxd storage is on zfs (dataset “zfs/lxd”). I had an up-to-date replica of the zfs dataset on another host, so I’ve copied it back onto the rebuilt host (using syncoid) and also an up-to-date copy of lxd init --dump
The machine was originally running Ubuntu 18.04 + lxd snap 4.18. The rebuilt machine is running Ubuntu 20.04, also with lxd snap 4.18.
My problem is how to get lxd to recognise this existing storage dataset.
If I run “lxd init --preseed <lxd.dump” (which was an “lxd --dump” taken previously), it tells me it can’t use the storage because it’s not empty.
Fair enough, I edited the dump file to set storage_pools: []
:
...
storage_pools: []
#- config:
# source: zfs/lxd
# zfs.pool_name: zfs/lxd
# description: ""
# name: default
# driver: zfs
...
lxd init --preseed
is happy with that.
Now I need to get the storage back. However, I can’t find a way to make it work.
root@nuc2:~# lxd recover
This LXD server currently has the following storage pools:
Would you like to recover another storage pool? (yes/no) [default=no]: yes
Name of the storage pool: default
Name of the storage backend (ceph, btrfs, cephfs, dir, lvm, zfs): zfs
Source of the storage pool (block device, volume group, dataset, path, ... as applicable): zfs/lxd
Additional storage pool configuration property (KEY=VALUE, empty when done):
Would you like to recover another storage pool? (yes/no) [default=no]:
The recovery process will be scanning the following storage pools:
- NEW: "default" (backend="zfs", source="zfs/lxd")
Would you like to continue with scanning for lost volumes? (yes/no) [default=yes]:
Scanning for unknown volumes...
Error: Failed validation request: Failed mounting pool "default": Cannot mount pool as "zfs.pool_name" is not specified
OK, let me set that attribute:
root@nuc2:~# lxd recover
This LXD server currently has the following storage pools:
Would you like to recover another storage pool? (yes/no) [default=no]: yes
Name of the storage pool: default
Name of the storage backend (ceph, btrfs, cephfs, dir, lvm, zfs): zfs
Source of the storage pool (block device, volume group, dataset, path, ... as applicable): zfs/lxd
Additional storage pool configuration property (KEY=VALUE, empty when done): zfs.pool_name=zfs/lxd
Additional storage pool configuration property (KEY=VALUE, empty when done):
Would you like to recover another storage pool? (yes/no) [default=no]:
The recovery process will be scanning the following storage pools:
- NEW: "default" (backend="zfs", source="zfs/lxd")
Would you like to continue with scanning for lost volumes? (yes/no) [default=yes]:
Scanning for unknown volumes...
No unknown volumes found. Nothing to do.
Now, the dataset does exist, but is mounted at the default location:
root@nuc2:~# zfs list -r zfs/lxd
NAME USED AVAIL REFER MOUNTPOINT
zfs/lxd 66.0G 149G 96K /zfs/lxd
zfs/lxd/containers 66.0G 149G 112K /zfs/lxd/containers
zfs/lxd/containers/apt-cacher 9.12G 149G 5.20G /zfs/lxd/containers/apt-cacher
zfs/lxd/containers/cache2 5.50G 149G 2.93G /zfs/lxd/containers/cache2
... etc
I remounted it - this gave some errors but appears to have worked:
root@nuc2:~# zfs set mountpoint=/var/snap/lxd/common/lxd/storage-pools/default zfs/lxd
cannot mount '/var/snap/lxd/common/lxd/storage-pools/default': directory is not empty
property may be set but unable to remount filesystem
root@nuc2:~# ls /var/snap/lxd/common/lxd/storage-pools/default/containers
apt-cacher cache2 ...etc
root@nuc2:~# zfs list -r zfs/lxd
NAME USED AVAIL REFER MOUNTPOINT
zfs/lxd 66.0G 149G 96K /var/snap/lxd/common/lxd/storage-pools/default
zfs/lxd/containers 66.0G 149G 112K /var/snap/lxd/common/lxd/storage-pools/default/containers
zfs/lxd/containers/apt-cacher 9.12G 149G 5.20G /var/snap/lxd/common/lxd/storage-pools/default/containers/apt-cacher
zfs/lxd/containers/cache2 5.50G 149G 2.93G /var/snap/lxd/common/lxd/storage-pools/default/containers/cache2
...
Still no effect though:
root@nuc2:~# lxd recover
This LXD server currently has the following storage pools:
Would you like to recover another storage pool? (yes/no) [default=no]: yes
Name of the storage pool: default
Name of the storage backend (ceph, btrfs, cephfs, dir, lvm, zfs): zfs
Source of the storage pool (block device, volume group, dataset, path, ... as applicable): zfs/lxd
Additional storage pool configuration property (KEY=VALUE, empty when done): zfs.pool_name=zfs/lxd
Additional storage pool configuration property (KEY=VALUE, empty when done):
Would you like to recover another storage pool? (yes/no) [default=no]: no
The recovery process will be scanning the following storage pools:
- NEW: "default" (backend="zfs", source="zfs/lxd")
Would you like to continue with scanning for lost volumes? (yes/no) [default=yes]:
Scanning for unknown volumes...
No unknown volumes found. Nothing to do.
I’m now rather stuck. I think I am forced to discard the entire dataset, use “lxd init”, and then re-replicated the contents of the dataset into the target area and then try “lxd recover”. But for future reference I’d really like to understand how lxd recover is supposed to be used, and whether it can be used with a pre-existing dataset created by lxd.
Thanks,
Brian.