Rebooted a server and since then the lxd container is not starting. It is giving an error saying: Error: Storage pool "vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7" unavailable on this server
Need help on this as I am not able to access this container and the data inside. I don’t want to loose this data. Could you please help on this?
time="2023-04-24T13:08:58+02:00" level=warning msg=" - Couldn't find the CGroup blkio.weight, disk priority will be ignored"
time="2023-04-24T13:08:58+02:00" level=warning msg=" - Couldn't find the CGroup memory swap accounting, swap limits will be ignored"
time="2023-04-24T13:09:12+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:09:23+02:00" level=warning msg="Failed to initialize fanotify, falling back on inotify" err="Failed to initialize fanotify: invalid argument"
time="2023-04-24T13:09:23+02:00" level=error msg="Error getting disk usage" err="Storage pool is unavailable on this server" instance=qblxc-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7 instanceType=container project=default
time="2023-04-24T13:10:25+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:10:46+02:00" level=error msg="Error getting disk usage" err="Storage pool is unavailable on this server" instance=qblxc-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7 instanceType=container project=default
time="2023-04-24T13:11:39+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:12:52+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:14:06+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:15:19+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:16:33+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:17:47+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:19:00+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:20:14+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:21:27+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:22:41+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
time="2023-04-24T13:23:54+02:00" level=error msg="Failed mounting storage pool" err="Failed to mount \"/dev/loop27\" on \"/var/snap/lxd/common/lxd/storage-pools/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7\" using \"btrfs\": invalid argument" pool=vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7
Previously losetup -a was not showing this loop device. So I ran this command: losetup -L -f --show /var/snap/lxd/common/lxd/disks/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7.img
I’m not experienced enough to say whether getting the entire container out of the storage pool and into another container is workable, but providing the file system in very messed up, we should be able to get specific data, e.g. MySQL data and logs, or a user’s home directory.
I created a BTRFS loopback storage pool device and a container on it, and was able to mount the IMG and drill down into the file system mount point to get the test file created in the container.
mount -vt btrfs -o loop /var/snap/lxd/common/lxd/disks/{yourdisk}.img /{mount-point}
e.g.
mount -vt btrfs -o loop /var/snap/lxd/common/lxd/disks/btrfsloop1.img /mnt
cp /mnt/containers/btrfstest1/rootfs/root/btrfstest1.txt /home/someuser/restore/
It would be interesting to see whether the above mount will work in your scenario.
Got this error: mount: /mnt/backup/restore: wrong fs type, bad option, bad superblock on /dev/loop27, missing codepage or helper program, or other error.
This is the response of the command you provided: /var/snap/lxd/common/lxd/disks/vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7.img: BTRFS Filesystem label "vol-ce9e2a76-8b0c-437b-81d4-4c4f47f147e7", sectorsize 4096, nodesize 16384, leafsize 16384, UUID=bc75b9c2-25be-4180-a00a-1a4afec0715b, 406211244032/1201999998976 bytes used, 1 devices
Hi. If you haven’t already, I recommend that you find other forums relating to BTRFS which increase your chances of someone knowing more about the file system and help you recover its’ contents.