Lxc import runs out of space even when disk has free space

I’m trying to import an existing container image and am getting an error that the disk is out of space when it has plenty of free space. The image in question is a zstd compressed image that is 470GB compressed and about 600GB uncompressed. I have a 4TB SSD mounted on /var/snap/lxd. My storage pool is another 4TB SSD setup as ZFS. This is ubuntu server 22.04.1 running on a physical machine with 64GB of RAM. Is there some other directory that needs space for the lxc image import process?

josh@lxdtest2:~$ lxc image import mnt/tmp/102-metadata.tar.gz mnt/tmp/102.img.zstd --alias image1
Error: Failed syncing image between nodes: Failed remote image download: Failed transferring image "5e2e8260d2488ca2da1a500725e68251724c81a55e920b531984020ddd64011f" from "lxdtest2.lan:8443": write /var/snap/lxd/common/lxd/images/lxd_build_338937687/lxd_tar_91185809: no space left on device
josh@lxdtest2:~$ df -h
Filesystem                         Size  Used Avail Use% Mounted on
tmpfs                              6.3G  1.6M  6.3G   1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv   98G   14G   80G  15% /
tmpfs                               32G     0   32G   0% /dev/shm
tmpfs                              5.0M     0  5.0M   0% /run/lock
/dev/nvme0n1p2                     2.0G  246M  1.6G  14% /boot
/dev/nvme0n1p1                     1.1G  6.1M  1.1G   1% /boot/efi
tmpfs                              6.3G  4.0K  6.3G   1% /run/user/1000
proxmox@nas00.lan:/mnt/main/pve    9.3T  2.5T  6.8T  27% /home/josh/mnt
/dev/sdb                           3.6T  470G  3.0T  14% /var/snap/lxd
tmpfs                              1.0M     0  1.0M   0% /var/snap/lxd/common/ns
josh@lxdtest2:~$ ls -lh mnt/tmp/102.img.zstd
-rwxrwx--- 1 1001 root 470G Feb  2 13:45 mnt/tmp/102.img.zstd
josh@lxdtest2:~$ sudo df -h /var/snap/lxd/common/lxd/images/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb        3.6T  470G  3.0T  14% /var/snap/lxd
josh@lxdtest2:~$ zpool list
NAME      SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
default  3.62T  5.38G  3.62T        -         -     0%     0%  1.00x    ONLINE  -

I was able to resolve this by mounting the SSD on /var

1 Like

Alternatively you could have used a custom volume for the temporary backup scratch space.

See https://linuxcontainers.org/lxd/docs/master/howto/storage_volumes/#use-the-volume-for-backups-or-images

1 Like