Storage pool not mounted?

We had a power loss and since then I can’t start some of our lxd containers. Actually all of them which are in a storage pool that is not on the root partition.

In /var/lib/lxd/containers the links to the containers (/var/lib/lxd/storage-pools/lxc1/containerXY) that are not on the root partition are missing.
Also a ‘lxc storage list’ shows all storage pools where they are supposed to be - storage lxc1 on /mnt/lxc1.

I guess somehow I need to mount the storage from /mnt/lxc1 to /var/lib/lxd/storage-pools/lxc1

How can I do that?

My first thought was that lxd started during the reboot before the partitions /mnt/lxd1 got mounted so I restarted lxd now that /mnt/lxd1 was there for sure. Didn’t help.

Two machines were affected by the power loss and both show the same symptoms.

Running lxd version 2.18
using ‘dir’ driver

I managed to fix this issue but I’d like to hear the opinion of someone familiar with the system if the solution is ok and maybe why this could happen.

I created a link from /mnt/lxc4/container to /var/lib/lxd/storage-pools/lxc4/container and everything was fine again.

Any idea how those links (if they were links initially) can get lost by a power loss.

hey It’s normally recommended to use a device for lxd that way you don’t have to worry about mounting, if you are using a directory and using an external device you’ll have to configure /etc/fstab.

It’s much easier to just create a zfs pool and use the pool with lxd. Then you don’t have to worry about mounting stuff.

The devices are mounted to /mnt/lxc1, /mnt/lxc2, /mnt/lxc3 etc. where I created the storage pools. Or do you mean I should create the storage pool with directly on the device like lxc storage create pool1 dir source=/dev/sdb1?

When I created the storage pool on these locations /mnt/lxc1…4 everything was working as expected. After the power loss the connection between /var/lib/lxd/storage-pools/lxc4/container and /mnt/lxc1…4/container got lost. As I said I just created a symlink between those two and everything is working again. Not sure if there was a symlink initially. For me it seems to be a bug if the connection between /var/lib/lxd/storage-pools/lxc4/container and /mnt/lxc1…4/container gets lost by a power loss on two separate machines. Also it makes me doubt that there was just a symlink initially.

We tried to use ZFS but that turned out to be unusable because of bad performance and issues with docker.

I see, If you use zfs you can add an existing zfs pool or let lxd create you a new pool by telling lxd which device you want to use so yes you can pass in /dev/sda for example into lxd init and it will set up a pool for you

or if you have an existing pool you just pass it the name of the pool (choose not to setup a new zfs pool and it will let you enter the name of an existing pool)

if you are not using zfs and want to use dir, then basically just setup /etc/fstab to setup your mounts /mnt/lxc1 /mnt/lxc2 /mnt/lxc3 etc…