LXD export and root file system size

If I have relatively small / filesystem, but large /opt partition and I want to export a container to /opt, and said container is larger than free space on / then lxc export fails:

Error: Create backup: Backup storage: Failed to run: tar -cf /var/snap/lxd/common/lxd/backups/main/backup0 --xattrs -C /var/snap/lxd/common/lxd/backups/lxd_backup_583972121 --transform s,^./,backup/, .: tar: /var/snap/lxd/common/lxd/backups/main/backup0:   Wrote only 2048 of 10240 bytes
tar: Error is not recoverable: exiting now

Then the machine is left in broken state: disk is full, lxc list is unresponsive, etc. I have to go to /var/snap/lxd/common/lxd/backups and erase everything there, then I have to reboot the machine, snap restart lxd does not help.

I tried to avoid this problem symlinking backup directory, making /opt/backups and pointing /var/snap/lxd/common/lxd/backups to /opt/backups. Does not work:

Error: Create backup: Backup storage: stat /var/snap/lxd/common/lxd/backups: no such file or directory

So, it seems that I can’t avoid dumping container to root filesystem first. Is it possible to change this behavior to export containers even if root filesystem is smaller than largest container? Leaving a lot of rarely used space in root fs does not make sense when SSDs are involved… what about an option --backup-tmpdir or something like that?

The symlink won’t work because of the way the snap environment works.
A bind-mount would work though.

So make sure that /var/snap/lxd/common/lxd/backups is a directory, then use mount --bind /opt/backups /var/snap/lxd/common/lxd/backups.

We have an open issue upstream to make it configurable where images and backups are stored to avoid such issues.

Yep, thanks, this works. What about free space check before export?

We can’t look for free space ahead of time as it’s impossible to tell how big a backup will be ahead of time. Some containers will be using 2GB on ZFS but will take 6GB or more in backup. Some containers on dir backend can shrink significantly, …

What we need though is to ensure that LXD cleans things up on failure to generate the backup so that you don’t have to go cleaning things up and reboot at least. @monstermunchkin

I sent a fix for the error codepath cleanup, it’s been merged upstream.

Hey @stgraber,

Could you supply the link to the issue here?

Cheers

1 Like