Lxc export throws (Backup Storage) error - Unable to generate compressed file

I am trying to export a container to compressed tar file using following command,

lxc export nextcloud nextcloud_optimized.tar.xz --container-only --optimized-storage

and end up getting following error after running for about an hour,

Error: Create backup: Backup storage: open /var/snap/lxd/common/lxd/backups/nextcloud/backup0: no such file or directory

Note that on checking the location mentioned above, there exists a file backup0.compressed under /var/snap/lxd/common/lxd/backups/nextcloud

Scarping through github issues, came across this particular issue but that seems to have been resolved.

It’d be great if someone can guide me in resolving this issue and/or point in right direction.

Thanks :slight_smile:

P.S - I am using Client/Server version 3.11, ZFS for storage and container in question is ~80 GB.

Edit 1:
Ran the same command again and this time it returned a different error, see below,
Error: Create backup: Backup storage: remove /var/snap/lxd/common/lxd/backups/nextcloud/backup0

I was also running lxc monitor --pretty --type=logging --loglevel=debug which had an entry in there pointing to removal of container backup before the export operation failed.

Based on this I believe lxc export is performing a cleanup when it encounters missing backup0 file and fails right away. Also this failure leaves a dangling file backup0.compressed in /var/snap/lxd/common/lxd/backups/nextcloud directory.

If I try to reproduce your command I’ll only get a file “nextcloud_optimized.tar.xz” in the pwd. The backup-directory stays empty because you only generated an export file.

lxc export --help

Examples:
lxc export u1 backup0.tar.gz
Download a backup tarball of the u1 container.

Flags:
–container-only Whether or not to only backup the container (without snapshots)
–optimized-storage Use storage driver optimized format (can only be restored on a similar pool)

“can only be restored on a similar pool”, I guess that might be the troublemaker. What means similar? Lousy documented as usual.

I understand that part and it works as expected but the problem isn’t the actual generated file but the fact that running lxc export fails with following error,

Error: Create backup: Backup storage: open /var/snap/lxd/common/lxd/backups/nextcloud/backup0: no such file or directory

After the export operation fails, if I navigate to /var/snap/lxd/common/lxd/backups/nextcloud path then I do see backup0 file in there.

So my question,

  1. Why does it say no file or directory exists when there is a file in there?
  2. How do I go about resolving/debugging this issue?

first of all: is your container mounted by zfs ? If it is check whether a file in /backup exists.
Next I would check the permissions.

ad point two: step by step.
ad point one: honestly, I got no idea. Ask these guys from LXD who love cryptic error-messages.

try to solve it yourself:

  • copy that file (if exists) on a save place (i.e. /root/)
  • next delete the file in …/backup
  • check permissions (are you current user allowed to export …) ?
  • retry your export command

Does the error still exist ?

I’d suggest to do a check with a test container with minimal size (Alpine are very smallish) if it works that’s a 5 mn way to see if you have such a problem.

I have other containers which are far smaller in size ~2.5 GB (vs ~80 GB) and I have no problem exporting them.

The size of the container CAN’T be the trouble source. If a command works it must work for all sizes as well. If the size would be the trouble maker, the error code would be an other one. Believe me.
Check this (can only be restored on a similar pool) carefully again, please.

Noted. I am running the export operation again, without the --optimized-storage flag. Will report the findings but based on the documentation, I’d imagine --optimized-storage flag provides for better compression (based on storage pool used) and thus be restored only on similar pool (zfs in this case).

Why should “similar pool”-meaning point to zfs ? In my understanding is zfs an other layer. And if we exporting a container to a *.tar-file, what has this to do with zfs? What is a tar-ball made of ? Magic? Do we talking about containers? Ok then …
Again as long as similar is not explained definitively its meaning can belong to everything. Don’t trust the wizard of oz. :slight_smile:
Be brave and focus on things beyond the curtain.

In this case I’d suggest to check available disk space on system partition as well as available memory during the export to ensure that there is no resource problem.

In general a good idea. But in that case can’t be the trouble source. If partition(space) is not enough or other things underlying you’ll get an error from the underneath system because it’s escalated up from OS. Not from LXD except THEY catched this error and tossed it into their own error-handler. The same if something struggles with memory. In any case.
That’s the point LXD-makers come into the game. Ask them directly. And ask why they don’t document those important ‘features’ like orphans. Maybe you’ll get an answer.

unless there is some problem in the bubbling of low level errors toward the user interface or even the logs. Have seen it happen with other programs.

That is exactly what I was talking about. Hence LXD muddled on that issues.

Quick update,

I ran export without the optimized-storage flag and ran into the same error. I also noticed while running export,

  • Swap slowly increased continuously (end up eating 1.05 GB out of 2 GB).
  • RAM consumption also increased from average of 6 GB to 10 GB (total is 16 GB) but never went past that mark.
  • Rsync operation ran successfully and dumped container in /var/snap/lxd/common/lxd/backups directory with name lxd_backup_457021993.
  • A backup0 file is generated followed by backup0.compressed which is when the export operation fails with error as mentioned above.

Edit 1: Here’s the complete command lxc export nextcloud nextcloud_optimized.tar.xz --container-only

1 Like

Can you please post the exact command you made ?

Updated my comment above with command.

Thank you very much indeed.

No, I cannot confirm your experience. Can you just accomplish the lxc export without ANY flags and parameters ?

What LXD | LXC version you are using?
Please do a “df -h” at your command line and post it, thanks.

Tried but I get the same error.

3.11

Bump. Anyone?

Well, I’m still here :slight_smile:
someone asked you to post df -h but you did not; I have hinted myself that the available space on system partition mattered but you did not reply on this point.
So, I’m not yet sure that you have at least 200 Gb free on system partition before an export.