Error: ZFS storage pool "default" could not be imported: cannot import 'default': I/O error


I have a challenging issue to fix on an old LXD server…
One of my friend has an LXD instance, unreachable since few months…

Bad time for me, I needed to boot the server on rescue mode, I changed the root password before to understand why his website was down… / was full ! Anyway.

root@vps-02:/var/log# cat /etc/lsb-release 
root@vps-02:/var/log# dpkg -l |grep lxd
ii  lxd                                 2.21-0ubuntu3~16.04.1                      amd64        Container hypervisor based on LXC - daemon
ii  lxd-client                          2.21-0ubuntu3~16.04.1                      amd64        Container hypervisor based on LXC - client
root@vps-02:/var/lib/lxd# lxd --debug --group lxd
INFO[02-15|23:09:55] LXD 2.21 is starting in normal mode      path=/var/lib/lxd
INFO[02-15|23:09:55] Kernel uid/gid map: 
INFO[02-15|23:09:55]  - u 0 0 4294967295 
INFO[02-15|23:09:55]  - g 0 0 4294967295 
INFO[02-15|23:09:55] Configured LXD uid/gid map: 
INFO[02-15|23:09:55]  - u 0 100000 65536 
INFO[02-15|23:09:55]  - g 0 100000 65536 
WARN[02-15|23:09:55] CGroup memory swap accounting is disabled, swap limits will be ignored. 
DBUG[02-15|23:09:55] Initializing and checking storage pool "data". 
DBUG[02-15|23:09:55] Initializing a DIR driver. 
DBUG[02-15|23:09:55] Checking DIR storage pool "data". 
DBUG[02-15|23:09:55] Initializing and checking storage pool "default". 
DBUG[02-15|23:09:55] Initializing a ZFS driver. 
DBUG[02-15|23:09:55] Checking ZFS storage pool "default". 
DBUG[02-15|23:09:55] ZFS storage pool "default" does not exist. Trying to import it. 
INFO[02-15|23:09:56] Unmounting temporary filesystems 
INFO[02-15|23:09:56] Done unmounting temporary filesystems 
INFO[02-15|23:09:56] Closing the database 
INFO[02-15|23:09:56] Saving simplestreams cache 
INFO[02-15|23:09:56] Saved simplestreams cache 
error: ZFS storage pool "default" could not be imported: cannot import 'default': I/O error
        Destroy and re-create the pool from
        a backup source.

One volume /dev/sda3 seems to contain something

root@vps-02:/var/log# fdisk -l
Disque /dev/sda : 60 GiB, 64424509440 octets, 125829120 secteurs
Unités : secteur de 1 × 512 = 512 octets
Taille de secteur (logique / physique) : 512 octets / 512 octets
taille d'E/S (minimale / optimale) : 512 octets / 512 octets
Type d'étiquette de disque : dos
Identifiant de disque : 0x445df72d

Périphérique Amorçage   Début       Fin  Secteurs Taille Id Type
/dev/sda1    *            2048  19531775  19529728   9,3G 83 Linux
/dev/sda2             19533822  23437311   3903490   1,9G  5 Étendue
/dev/sda3             23437312 125829119 102391808  48,8G 83 Linux
root@vps-02:/var/log# zpool list
no pools available

I have a dump of lxd.db (maybe too long to copy/paste it right now)

root@vps-02:/var/lib/lxd# sqlite3 /var/lib/lxd/lxd.db .dump

I have found on the DB some references of /dev/sda3

CREATE TABLE storage_pools_config (
    storage_pool_id INTEGER NOT NULL,
    key VARCHAR(255) NOT NULL,
    value TEXT,
    UNIQUE (storage_pool_id, key),
    FOREIGN KEY (storage_pool_id) REFERENCES storage_pools (id) ON DELETE CASCADE
INSERT INTO "storage_pools_config" VALUES(5,2,'source','default');
INSERT INTO "storage_pools_config" VALUES(6,2,'volatile.initial_source','/dev/sda3');
INSERT INTO "storage_pools_config" VALUES(7,2,'zfs.pool_name','default');
INSERT INTO "storage_pools_config" VALUES(8,3,'source','/var/lib/lxd/storage-pools/data');

default zfs pool seems to exist but I don’t know what to do exactly to fix it.

root@vps-02:/var/lib/lxd# ls
containers  devices  devlxd  disks  images  lxd.db  networks  security  server.crt  server.key  shmounts  snapshots  storage-pools  unix.socket
root@vps-02:/var/lib/lxd# ls storage-pools/*/*
mfr-doc  mfr-forums  mfr-piwik  mfr-wiki  mfr-www  ssl
mfr-doc-01  mfr-doc-02  mfr-etcd-02  mfr-forums-01  mfr-mariadb-02  mfr-mem-02  mfr-piwik-01  mfr-rev-02  mfr-rsync-02  mfr-wiki-01  mfr-www-01
mfr-wiki-01  mfr-www-01

I saw few times on internet this kind of issue but before doing wrong operations, I prefer to ask to the best LXD community of the world ^^

Thank you for your help

Hello guys :slight_smile:

Any ideas ?