After upgrading debian 9 to 10, lxd do not start anymore
Here is the error:
EROR[07-19|20:53:49] Failed to start the daemon: Load storage pool for container “vm647606” in project “default”: No such object
Error: Load storage pool for container “vm647606” in project “default”: No such object
Give us some information about the storage pool. Is it ZFS, btrfs, dir or something else?
My wild guess is that the storage pool is using ZFS and somehow your new kernel does not have ZFS support.
It is likely that with the old kernel, you have installed ZFS using, for example, the zfs-dkms package, but on the new kernel this package has not been setup. If that is the case, then configure properly ZFS on the new kernel (maybe sudo apt install zfs-dkms), then restart your computer.
# lxd -d -v
INFO[07-20|07:40:32] LXD 3.8 is starting in normal mode path=/var/lib/lxd
INFO[07-20|07:40:32] Kernel uid/gid map:
INFO[07-20|07:40:32] - u 0 0 4294967295
INFO[07-20|07:40:32] - g 0 0 4294967295
INFO[07-20|07:40:32] Configured LXD uid/gid map:
INFO[07-20|07:40:32] - u 0 1000000 65536
INFO[07-20|07:40:32] - g 0 1000000 65536
WARN[07-20|07:40:32] AppArmor support has been disabled because of lack of kernel support
INFO[07-20|07:40:32] Kernel features:
INFO[07-20|07:40:32] - netnsid-based network retrieval: no
INFO[07-20|07:40:32] - uevent injection: no
INFO[07-20|07:40:32] - unprivileged file capabilities: no
INFO[07-20|07:40:32] Initializing local database
DBUG[07-20|07:40:32] Initializing database gateway
DBUG[07-20|07:40:32] Start database node id=1 address=
DBUG[07-20|07:40:32] Raft: Initial configuration (index=1): [{Suffrage:Voter ID:1 Address:0}]
DBUG[07-20|07:40:32] Raft: Node at 0 [Leader] entering Leader state
DBUG[07-20|07:40:32] Dqlite: starting event loop
DBUG[07-20|07:40:32] Dqlite: accepting connections
INFO[07-20|07:40:32] Starting /dev/lxd handler:
INFO[07-20|07:40:32] - binding devlxd socket socket=/var/lib/lxd/devlxd/sock
INFO[07-20|07:40:32] REST API daemon:
INFO[07-20|07:40:32] - binding Unix socket socket=/var/lib/lxd/unix.socket
INFO[07-20|07:40:32] - binding TCP socket socket=[::]:8448
INFO[07-20|07:40:32] Initializing global database
DBUG[07-20|07:40:32] Dqlite: handling new connection (fd=19)
DBUG[07-20|07:40:32] Dqlite: connected address=0 attempt=0
INFO[07-20|07:40:32] Initializing storage pools
DBUG[07-20|07:40:32] Initializing and checking storage pool "default"
DBUG[07-20|07:40:32] Checking ZFS storage pool "default"
INFO[07-20|07:40:32] Applying patch: storage_api_permissions
DBUG[07-20|07:40:32] Mounting ZFS storage volume "references" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "references" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm613997" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm613997" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm615814" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm615814" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm618252" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm618252" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm619684" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm619684" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm620349" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm620349" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm620362" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm620362" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm621372" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm621372" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm622677" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm622677" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm622846" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm622846" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm623965" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm623965" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm626732" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm626732" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm628679" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm628679" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm628947" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm628947" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm629927" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm629927" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm630487" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm630487" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm631081" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm631081" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm631671" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm631671" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm633999" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm633999" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm635132" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm635132" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm635976" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm635976" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm637440" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm637440" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm637956" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm637956" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm640262" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm640262" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm640342" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm640342" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm640477" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm640477" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm640709" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm640709" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm641369" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm641369" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm641376" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm641376" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm641407" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm641407" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm641983" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm641983" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm642484" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm642484" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm642997" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm642997" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm643333" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm643333" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm643699" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm643699" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm643962" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm643962" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm644034" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm644034" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm644345" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm644345" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm644540" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm644540" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm644856" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm644856" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm644872" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm644872" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm645017" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm645017" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm645134" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm645134" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm645638" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm645638" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm645760" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm645760" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm645936" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm645936" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm646091" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm646091" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm646109" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm646109" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm646260" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm646260" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm646533" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm646533" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm646708" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm646708" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume "vm647546" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume "vm647546" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume for container "vm613997" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume for container "vm613997" on storage pool "default"
DBUG[07-20|07:40:32] Unmounting ZFS storage volume for container "vm613997" on storage pool "default"
DBUG[07-20|07:40:32] Unmounted ZFS storage volume for container "vm613997" on storage pool "default"
DBUG[07-20|07:40:32] Mounting ZFS storage volume for container "vm615814" on storage pool "default"
DBUG[07-20|07:40:32] Mounted ZFS storage volume for container "vm615814" on storage pool "default"
DBUG[07-20|07:40:32] Unmounting ZFS storage volume for container "vm615814" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm615814" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm618252" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm618252" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm618252" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm618252" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm619684" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm619684" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm619684" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm619684" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm620349" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm620349" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm620349" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm620349" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm620362" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm620362" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm620362" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm620362" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm621372" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm621372" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm621372" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm621372" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm622677" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm622677" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm622677" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm622677" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm622846" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm622846" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm622846" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm622846" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm623965" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm623965" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm623965" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm623965" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm626732" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm626732" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm626732" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm626732" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm628679" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm628679" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm628679" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm628679" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm628947" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm628947" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm628947" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm628947" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm629927" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm629927" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm629927" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm629927" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm630487" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm630487" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm630487" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm630487" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm631081" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm631081" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm631081" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm631081" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm631671" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm631671" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm631671" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm631671" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm633999" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm633999" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm633999" on storage pool "default"
DBUG[07-20|07:40:33] Unmounted ZFS storage volume for container "vm633999" on storage pool "default"
DBUG[07-20|07:40:33] Mounting ZFS storage volume for container "vm635132" on storage pool "default"
DBUG[07-20|07:40:33] Mounted ZFS storage volume for container "vm635132" on storage pool "default"
DBUG[07-20|07:40:33] Unmounting ZFS storage volume for container "vm635132" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm635132" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm635976" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm635976" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm635976" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm635976" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm637440" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm637440" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm637440" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm637440" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm637956" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm637956" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm637956" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm637956" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm640262" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm640262" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm640262" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm640262" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm640342" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm640342" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm640342" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm640342" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm640477" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm640477" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm640477" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm640477" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm640709" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm640709" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm640709" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm640709" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm641369" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm641369" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm641369" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm641369" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm641376" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm641376" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm641376" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm641376" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm641407" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm641407" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm641407" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm641407" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm641983" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm641983" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm641983" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm641983" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm642484" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm642484" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm642484" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm642484" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm642997" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm642997" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm642997" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm642997" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm643333" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm643333" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm643333" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm643333" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm643699" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm643699" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm643699" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm643699" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm643962" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm643962" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm643962" on storage pool "default"
DBUG[07-20|07:40:34] Unmounted ZFS storage volume for container "vm643962" on storage pool "default"
DBUG[07-20|07:40:34] Mounting ZFS storage volume for container "vm644034" on storage pool "default"
DBUG[07-20|07:40:34] Mounted ZFS storage volume for container "vm644034" on storage pool "default"
DBUG[07-20|07:40:34] Unmounting ZFS storage volume for container "vm644034" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm644034" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm644345" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm644345" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm644345" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm644345" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm644540" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm644540" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm644540" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm644540" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm644856" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm644856" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm644856" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm644856" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm644872" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm644872" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm644872" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm644872" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm645017" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm645017" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm645017" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm645017" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm645134" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm645134" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm645134" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm645134" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm645638" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm645638" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm645638" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm645638" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm645760" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm645760" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm645760" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm645760" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm645936" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm645936" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm645936" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm645936" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm646091" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm646091" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm646091" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm646091" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm646109" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm646109" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm646109" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm646109" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm646260" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm646260" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm646260" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm646260" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm646533" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm646533" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm646533" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm646533" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm646708" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm646708" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm646708" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm646708" on storage pool "default"
DBUG[07-20|07:40:35] Mounting ZFS storage volume for container "vm647546" on storage pool "default"
DBUG[07-20|07:40:35] Mounted ZFS storage volume for container "vm647546" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume for container "vm647546" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume for container "vm647546" on storage pool "default"
DBUG[07-20|07:40:35] Database error: &errors.errorString{s:"No such object"}
DBUG[07-20|07:40:35] Unmounting ZFS storage volume "vm647546" on storage pool "default"
DBUG[07-20|07:40:35] Unmounted ZFS storage volume "vm647546" on storage pool "default"
DBUG[07-20|07:40:35] Unmounting ZFS storage volume "vm646708" on storage pool "default"
WARN[07-20|07:40:46] Failed to unmount ZFS filesystem via zfs unmount: umount: /var/lib/lxd/storage-pools/default/custom/vm646708: target is busy.
cannot unmount '/var/lib/lxd/storage-pools/default/custom/vm646708': umount failed
. Trying lazy umount (MNT_DETACH)...
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm646708" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm646533" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm646533" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm646260" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm646260" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm646109" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm646109" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm646091" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm646091" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm645936" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm645936" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm645760" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm645760" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm645638" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm645638" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm645134" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm645134" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm645017" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm645017" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm644856" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm644856" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm644540" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm644540" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm644345" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm644345" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm644034" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm644034" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm643962" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm643962" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm643699" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm643699" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm642997" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm642997" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm642484" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm642484" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm641983" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm641983" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm641407" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm641407" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm641376" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm641376" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm641369" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm641369" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm640477" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm640477" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm640342" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm640342" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm640262" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm640262" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm637956" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm637956" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm635976" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm635976" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm631081" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm631081" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm628947" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm628947" on storage pool "default"
DBUG[07-20|07:40:46] Unmounting ZFS storage volume "vm626732" on storage pool "default"
DBUG[07-20|07:40:46] Unmounted ZFS storage volume "vm626732" on storage pool "default"
EROR[07-20|07:40:46] Failed to start the daemon: Load storage pool for container "vm647606" in project "default": No such object
INFO[07-20|07:40:46] Starting shutdown sequence
INFO[07-20|07:40:46] Stopping REST API handler:
INFO[07-20|07:40:46] - closing socket socket=[::]:8448
INFO[07-20|07:40:46] - closing socket socket=/var/lib/lxd/unix.socket
INFO[07-20|07:40:46] Stopping /dev/lxd handler:
INFO[07-20|07:40:46] - closing socket socket=/var/lib/lxd/devlxd/sock
INFO[07-20|07:40:46] Closing the database
DBUG[07-20|07:40:46] Dqlite: closing client
DBUG[07-20|07:40:46] Stop database gateway
DBUG[07-20|07:40:46] Stop raft instance
DBUG[07-20|07:40:46] Dqlite: stopping event loop
DBUG[07-20|07:40:46] Dqlite: event loop stopped
INFO[07-20|07:40:46] Unmounting temporary filesystems
INFO[07-20|07:40:46] Done unmounting temporary filesystems
INFO[07-20|07:40:46] Saving simplestreams cache
INFO[07-20|07:40:46] Saved simplestreams cache
Error: Load storage pool for container "vm647606" in project "default": No such object
Thanks, well received but sadly haven’t had time to load it and poke at it today.
You mentioned that the ability to delete affected containers may be helpful to you.
This could be done (preferably after a backup), by creating /var/lib/lxd/database/patch.global.sql, containing:
DELETE FROM containers WHERE name='name-of-container';
DELETE FROM storage_volumes WHERE name='name-of-container' AND type=0;
This does assume that you don’t have multiple projects with conflicting container names (or you’ll need to tweak the WHERE clause to account for that).
It will delete the container and all related entries from the database, but it will not remove any snapshot for the container, so you may end up with an inconsistent database if you don’t also manually remove those snapshots and matching storage volume for the snapshots.
I’ll make some time to poke at your database tomorrow to see what’s going on with it, hopefully with that, we can take a more targeted approach and just fix whatever needs fixing. There are a number of data migration patches that you need to go through during that startup which all require the database and on-disk state to line up. Most likely you have a database record for something which no longer exists on disk or the reverse case where you have extra bits on disk with no match DB record (would actually line up better with the error).