Phantom snapshot

LXD
1

Hello,

I am trying to remove a snapshot that isn’t there.

root@vps01:/var/snap/lxd/common/lxd/storage-pools/storage1/containers-snapshots# lxc delete cent8-desktop/deploy1
root@vps01:~# lxc info centos-desk
Name: centos-desk
Location: none
Remote: unix://
Architecture: x86_64
Created: 2019/09/21 13:29 UTC
Status: Stopped
Type: container
Profiles: default
Snapshots:
  02-05-20-22-33 (taken at 2020/02/05 21:33 UTC) (stateless)

When i try to remove the snapshot i get the following error:

root@vps01:~# lxc delete centos-desk/02-05-20-22-33
Error: lstat /var/snap/lxd/common/lxd/storage-pools/storage1/containers-snapshots/centos-desk/02-05-20-22-33/: no such file or directory

how can i remove a snapshot that isn’t there?

2

What storage backend is that?

3

Most of the time, the answer is to just create what LXD is looking for, then it can delete it, but the actual answer is somewhat dependent on the backend, so best to know that first.

In the past LXD was extremely accommodating to such situations, effectively never complaining if something it wanted to delete was gone already, but this also had the opposite effect of hiding some bugs or having systems keep operating when their storage was badly corrupted. Our newly rewritten storage layer expects what’s on disk to match what LXD knows should exist, if that’s not the case, we now error out rather than move on.

4

That’s stating clearly the reliability goals of LXD, People building production solutions on LXD should so be deeply aware that they have the burden of adding resilience to it.

5

Hi,

I use btrfs as storage backend. I tried recreating the snapshot, but i get the following error message:

root@vps01:~# lxc snapshot centos-desk 02-05-20-22-33
Error: Add snapshot info to the database: This instance_snapshot already exists
6

Ok, since you’re on btrfs, you’d want something like this:

nsenter --mount=/run/snapd/ns/lxd.mnt -- /snap/lxd/current/bin/btrfs subvol create /var/snap/lxd/common/lxd/storage-pools/storage1/containers-snapshots/centos-desk/02-05-20-22-33

If that succeeds, then you should be able to delete that snapshot.

7

Hi Stephane,

I get the following error when running:

ERROR: can’t access ‘/var/snap/lxd/common/lxd/storage-pools/storage1/containers-snapshots/centos-desk’: No such file or directory

8

Can you check:

nsenter --mount=/run/snapd/ns/lxd.mnt -- ls -lh /var/snap/lxd/common/lxd/storage-pools/storage1/containers-snapshots

If the container directory is missing, then run:

nsenter --mount=/run/snapd/ns/lxd.mnt -- mkdir /var/snap/lxd/common/lxd/storage-pools/storage1/containers-snapshots/centos-desk

At which point the original command (btrfs subvol create) should be happy.