I have recovered from database corruption, now can't remove storage because its not empty

LXD
1

issue this container (benchmark-19) doesn’t exist how do i remove along with image etc? There is no data in this pool, it had to be created as loop back file to start LXD now i am trying to tidy up.

$ lxc storage delete LXD
$ lxc storage list
±-----±------------±-------±-------±--------+
| NAME | DESCRIPTION | DRIVER | SOURCE | USED BY |
±-----±------------±-------±-------±--------+
| LXD | | zfs | LXD | 5 |
±-----±------------±-------±-------±--------+
| lxd | | zfs | lxd | 45 |
±-----±------------±-------±-------±--------+

lxc storage show LXD
config:
source: LXD
volatile.initial_source: /dev/sdb
zfs.pool_name: LXD
description: “”
name: LXD
driver: zfs
used_by:

  • /1.0/containers/benchmark-19
  • /1.0/images/f75468c572cc50eca7f76391182e6fdaf58431f84c3d35a2c92e83814e701698
  • /1.0/profiles/cloud-init
  • /1.0/storage-pools/LXD/volumes/custom/backups
  • /1.0/storage-pools/LXD/volumes/custom/images
    status: Created
    locations:
  • none

$lxd sql global “select * from storage_pools;”
±—±-----±-------±------------±------+
| id | name | driver | description | state |
±—±-----±-------±------------±------+
| 1 | LXD | zfs | | 1 |
| 3 | lxd | zfs | | 1 |
±—±-----±-------±------------±------+

$ lxd sql global “select * from storage_pools_config;”
±—±----------------±--------±------------------------±---------+
| id | storage_pool_id | node_id | key | value |
±—±----------------±--------±------------------------±---------+
| 3 | 1 | 1 | volatile.initial_source | /dev/sdb |
| 4 | 1 | 1 | zfs.pool_name | LXD |
| 5 | 1 | 1 | source | LXD |
| 9 | 3 | 1 | source | lxd |
| 10 | 3 | 1 | volatile.initial_source | lxd |
| 11 | 3 | 1 | zfs.pool_name | lxd |
±—±----------------±--------±------------------------±---------+

Can i do:
$ lxd sql global “delete from storage_pools_config where storage_pool_id =1;”
and then remove the zpool manually.
$ zpool destroy LXD

2

You can do that, but the pool might still be referenced in some profiles, since the use the pool name, not ID. You might want to check that too.

3

Good idea. checked.

So i issued $ lxd sql global “delete from storage_pools_config where storage_pool_id =1;”

now $ lxd sql global “select * from storage_pools_config;”
±—±----------------±--------±------------------------±------+
| id | storage_pool_id | node_id | key | value |
±—±----------------±--------±------------------------±------+
| 9 | 3 | 1 | source | lxd |
| 10 | 3 | 1 | volatile.initial_source | lxd |
| 11 | 3 | 1 | zfs.pool_name | lxd |
±—±----------------±--------±------------------------±------+
good and shows correct but
$ lxc storage list still shows the dependency that i want to remove, especially as this stops LXD starting if its not there.
±-----±------------±-------±-------±--------+
| NAME | DESCRIPTION | DRIVER | SOURCE | USED BY |
±-----±------------±-------±-------±--------+
| LXD | | zfs | | 4 |
±-----±------------±-------±-------±--------+
| lxd | | zfs | lxd | 46 |
±-----±------------±-------±-------±--------+
I found the 4 references above in the table storage_volumes with the same storage_pool_id so i deleted those too with
$ lxd sql global “delete from storage_volumes where storage_pool_id=1;”
then i could issue
$ lxc storage delete LXD
Error: Failed to delete the ZFS pool: Failed to run: zpool destroy -f LXD: cannot destroy ‘LXD’: pool is busy
so i re-put the data in that i deleted
$ lxd sql global “Insert into storage_pools_config values (3,1,1,‘volatile.initial_source’, ‘/dev/sdb’);”
$ lxd sql global “Insert into storage_pools_config values (4,1,1,’ zfs.pool_name’, ‘LXD’);”
$ lxd sql global “Insert into storage_pools_config values (5,1,1,‘source’, ‘LXD’);”
Turns out that this didn’t fix it either.

So I $ lxc storage edit LXD
remove the 4 lines of dependencies that don’t exist.
then tried $ lxc storage delete LXD and this still gave the same error, (the pool is busy) , even changing the volatile.initial_source value to the path of the .img file did not work.

Finally removing the zpool manually worked (zpool destroy -f LXD), and then run l xc storage delete LXD worked and tried everything up.