Deleted Container Trying To Delete Snapshot :( Can I Get It Back?

1

I wanted to delete a snapshot so I typed:-

lxc delete container snapshot

instead of

lxc delete container/snaphot

It replied “not found”. Then to my horror I discovered that it deleted the container! The “not found” was I guess lxd trying to find a container called snapshot which it also wanted to delete but it could not find it.

I wish it would’ve asked if I was sure or any other confirmation :cry:

Is there a possibility to undelete it?

2

While I don’t know a direct answer, I would recommend to not use the storage pool the deleted container was in, until you get an answer.

Because:
As far as I know, in general (computer principle) information will not be deleted (completely) until it is overridden by other data (this depends of course on the delete method used).
Because of this computer forensics can still recover data, that was “deleted”.

But I don’t know if this principle also applies to LXD.

Was important data stored in the container?
(you of course don’t have to/should not give details, if it is sensitive/private data)

Edit:
To avoid that in the future, you can use this flag:
-i, --interactive Require user confirmation

3

About two hours of work :unamused:

So if I type -i at the end then get prompted? I will now do that every time I have to delete a snapshot…

4

Very unfortunate.
Did you use any volumes or disk devices, where (some of) the data could still be?

Yep.

What type of storage was the container in (zfs, btrfs etc.)?
It seems that with btrfs (and maybe also lvm and ceph) you might have better chances than with zfs.
Hint for btrfs: Restore - btrfs Wiki
But I have a feeling that the situation in lxd might be different :thinking:, but don’t count on me, I don’t have enough knowledge about this.

Update: Of course no snapshots are available, if you delete the container.
And sry if that might seem uninformed, but I never use snapshots:
Are any of the snapshots still available? (guess not because you deleted the container…)

5

lxc alias add delete "delete -i" can be used to always prompt

3 Likes
6

Thanks for your help.

I did my work again, so I wasted one hour. It’s in the past now and I have learned my lesson!

The thing is, no matter how many snapshots you have, once you delete the container, all of them are gone!

1 Like
7

Thanks for that!

8

@stgraber
Just in case other users read this.

There is no method (or no easy method) to restore a container (without backup)?

9

It depends on your storage backend and what kind of undeleting that may support.
But in general, that’s going to be quite hard indeed.

I’ve had limited success with undeletion tools on ext4 before though.
If you somehow do manage to undelete things, then you can use lxd import to re-import the container into the LXD database.

1 Like
10

I wonder if we could make the lxc delete command check whether all of the requested instances exist before deleting any of them, that way this would avoid deleting anything if specifying a non-existent instance thinking it was a snapshot name?

1 Like
11

In terms of usability, there is already the following,

$ lxc launch ubuntu: mycontainer
Creating mycontainer
Starting mycontainer
$ lxc delete mycontainer
Error: The instance is currently running, stop it first or pass --force
$

It looks natural to me to have something like

$ lxc launch ubuntu: mycontainer
Creating mycontainer
Starting mycontainer
$ lxc snapshot mycontainer mysnapshot0
$ lxc delete mycontainer
Error: The instance is currently running, stop it first or pass --force
$ lxc stop mycontainer
$ lxc delete mycontainer
Error: The instance has one or more snapshots, delete them first or pass --force

You may also use gettext plural forms to specify the number of snapshots to the user, i.e.

$ lxc delete mycontainer
Error: The instance has 1 snapshot, delete them first or pass --force
12

Yes that would be safer, although I was thinking of a way to improve the situation whilst avoiding changing the current behaviour (which users will likely be expecting, especially through script automation etc).

If we run the basic existence validation on deletion before running any actual delete processes are tried, then you’d still get the same response, albeit it without deleting the container and its snapshots first.

13

Some more ideas:

  1. Only activate default prompt when a user deletes more than one container.

  2. Seperate the command for deleting snapshots, like:
    lxc delete -sn container1:snapshot2

  3. Ask the user in lxd init, whether he wants to activate user confirmation.

  4. Add stgrabers proposed alias:
    lxc alias add delete "delete -i"
    to the documentation for “best practice”.

14

Note that there is a security.protection.delete key which can be set on containers to prevent any deletion without the key first being unset.

3 Likes