"lxc publish" fails due to "no space left on device" despite lots of space available


(Amaury) #1

Hello,

Trying to publish a snapshot fails on my server with the following error message
Error: Failed to copy file content: write /var/lib/lxd/images/lxd_build_398875765/lxd_build_tar_386476143: no space left on device

Both Zpool and / have plenty of space.

  • 80Gb of LXD zfs pool is available
  • 40+Gb is available on root fs

Container size is arounf 10Gb so it is difficult to understand where the filesystem issue arises.
Is there a way to specify temporary and target directories for images?
Is there a way to understand precisely how “lxc publish” works so that filesystem issues can be dealt with?

Best,
Amaury


(Stéphane Graber) #2

So as you see, LXD will create the tarball in /var/lib/lxd/images, then compress it for you.

The out of disk space most commonly happens when your container contains sparse files as those will get expanded when turned into a tarball and so can make your container massively bigger. The compression will then cut that back down, but you do need the temporary space.

du can let you figure out the tarball size by looking at apparent size in those cases:

stgraber@castiana:~$ sudo du -sch test/
1.0K	test1/
1.0K	total
stgraber@castiana:~$ sudo du -sch --apparent-size test/
9.4G	test1/
9.4G	total

If that was a container, it’d show up as 1KB disk usage but then turn into a 9.4GB tarball.


(Amaury) #3

Thanks Stéphane,
Indeed the “real” consumption of disk space of the dump was 10x the size of the container.
Is there any plan to be able to specify an output path to the “lxc publish” command?
Best,
Amaury


(Bruce) #4

Likewise, it is possible to specific an output path when using lxc publish?


(Dragan Espenschied) #5

If you are using ZFS, it makes sense to create a compressed file system and have the mountpoint be /var/lib/lxd/images. During the “publish” process, disk space will be consumed, after the gzip process has concluded, it’ll be ok again.

If you are not using ZFS, you can symlink /var/lib/lxd/images to a place on the file system where enough space is available.