Issues with export ('io: read/write on closed pipe')

Dear colleagues,

I am having issues with reliably exporting LXC containers to tarballs.

LXD/LXC version: 4.0.6
Installed via: snap
Host OS: Ubuntu 20.04
File system: ZFS

The error is always of this form:

ubuntu@lxc-host-r2:~/export$ lxc stop container
ubuntu@lxc-host-r2:~/export$ lxc export container container.tar
Error: Create backup: Backup create: Error adding "/var/snap/lxd/common/lxd/storage-pools/zfspool/backup.501085870/rootfs/home/ubuntu/random-file" as "backup/container/rootfs/home/ubuntu/random-file" to tarball: Failed to copy file content "/var/snap/lxd/common/lxd/storage-pools/zfspool/backup.501085870/rootfs/home/ubuntu/random-file": io: read/write on closed pipe

The random-file here is a stand-in name for a file in the container’s file system. Exporting fails at a different file each time. Looking inside the containers, there is really nothing special about the files. They are all owned by a regular system user (“ubuntu” typically), are not corrupted, or otherwise conspicuous.

There is plenty of disk space available at the destination directory (also ZFS).

Any ideas what could be the source of the error?

You could be running out of disk space on the root partition where LXD runs. This is used as an intermediate holding area to generate the tarball before its streamed to the target file on the client.

You could try creating a custom volume on one of your storage pools and then specifying that LXD use that for the intermediate holding area.

See storage.backups_volume at Server settings | LXD

See also How can I control the file system path used by lxc import commands to avoid failures? - #2 by tomp

Thank you for this hint Thomas!

Do I need to limit the size of the storage volume, as in the example you gave?

lxc storage volume create <pool> <volume name> size=nGB
                                               ^^^^^^^^

Or can this be left for ZFS to figure out? Is this in general about ZFS pools or LXD pools?

It depends what storage pool you’re using for your backup volume (note it doesn’t need to be the same as the one your exporting the instance from) and whether it always requires a volume size (or if the default volume size of 10GB is sufficient for those that do).

By default ZFS, BTRFS and DIR storage pools don’t enforce a volume size limit, whereas LVM and CEPH always require one (because each volume is blocked storage backed).

You could create a dir storage pool at a custom location to use for backups, e.g.

lxc storage create backup dir source=/some/path/with/plenty/of/space
lxc storage create backup backupvol
lxc config set storage.backups_volume=backup/backupvol

Perfect, Thomas! :clinking_glasses: I created a backup volume on ZFS (with transparent compression enabled) and everything worked fine. Happy with my 13 GB tarball :slight_smile:

1 Like

As I didn’t exactly find the expected answer here, but it seems to come high on search engines, I’ll add my 1c here.

I wanted to create a “backups” volume on my existing large storage default, so I finally came with:

lxc storage volume create default backups

followed by the standard

lxc config set storage.backups_volume=default/backups

and voilà, I could export my container to another network/server