I had to re-install the server that my LXD installation was on. Before the re-install I had several LXD containers stored on the BTRFS parition /srv/lxd, which is a separate LVM LV formatted as BTRFS:
/dev/mapper/MainVG-lxd on /srv/lxd type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)
After the re-install, which involved wiping / and /boot but no other partitions, I’ve run LXD init:
Would you like to use LXD clustering? (yes/no) [default=no]:
Do you want to configure a new storage pool? (yes/no) [default=yes]: no
Would you like to connect to a MAAS server? (yes/no) [default=no]:
Would you like to create a new local network bridge? (yes/no) [default=yes]: no
Would you like to configure LXD to use an existing bridge or host interface? (yes/no) [default=no]:
Would you like LXD to be available over the network? (yes/no) [default=no]:
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:
And now I’m attempting to ‘reconnect’ the existing LXD storage pool but get this error:
$ lxc storage create lxd-partition btrfs source=/srv/lxd
Error: Requested btrfs subvolume exists but is not empty
$ ls /srv/lxd/
containers custom images virtual-machines-snapshots
containers-snapshots custom-snapshots virtual-machines
You need to follow the disaster recovery instructions in the backup part of the documentation.
You’ll have to bind-mount your /srv/lxd over /var/lib/lxd/storage-pools/NAME or /var/snap/lxd/common/lxd/storage-pools/NAME (depending on whether using deb or snap), then use lxd import NAME for each of your containers. This will automatically re-add the storage pool for you and re-create as much of the database as possible from the serialized backup data that’s part of the instance volumes.
No luck, I just get “The instance “xx” does not seem to exist on any storage pool”:
sudo mkdir /var/snap/lxd/common/lxd/storage-pools/lxd-partition
sudo mount -o bind /srv/lxd /var/snap/lxd/common/lxd/storage-pools/lxd-partition
# should see containers, custom, images etc.
ls /var/snap/lxd/common/lxd/storage-pools/lxd-partition
containers custom images virtual-machines-snapshots
containers-snapshots custom-snapshots virtual-machines
# list containers to import
sudo ls /srv/lxd/containers/
cntrdtop emby-bionic focaltest sncedge ttrss vpncntr
sudo lxd import emby-bionic
Error: The instance "emby-bionic" does not seem to exist on any storage pool
sudo nsenter --mount=/run/snapd/ns/lxd.mnt -- mount -o bind /var/lib/snapd/hostfs/srv/lxd /var/snap/lxd/common/lxd/storage-pools/lxd-partition
And then attempt that lxd import again.
The reason for this is that LXD when run as a snap is operating in a mount namespace different than the host’s so the mount you did manually would only have worked had LXD not been started yet.
The full procedure was (where my storage pool is named lxd-partition and it’s located at /srv/lxd):
sudo mkdir /var/snap/lxd/common/lxd/storage-pools/lxd-partition
sudo nsenter --mount=/run/snapd/ns/lxd.mnt -- mount -o bind /var/lib/snapd/hostfs/srv/lxd /var/snap/lxd/common/lxd/storage-pools/lxd-partition
# list containers to import
sudo ls /srv/lxd/containers/
# for each container
sudo lxd import containername
As a suggestion, could you add this as a working example to the documentation at https://lxd.readthedocs.io/en/latest/backup/? I can see it’s partly documented, but there are important parts missing like using nsenter for Snap-installed LXD. And TBH it’s very non-intuitive without an example
