Incremental backup of running containers

LXD
1

I am trying to set up incremental backups of containers.
In the latest lxc that I am using (v.3.13), it is possible to do:

lxc copy test remote:test --refresh

which supposedly should do incremental copy of my test container. Unfortunately, it does not seem to work when the container is running, as I get:

Error: Failed container creation: Cannot refresh a running container

I am trying the following workaround. I first copy the container locally:

lxc copy test test-copy

My storage pool is btrfs, so the local copy is shallow (fast).
Then I copy the resulting (not running) container to the remote:

lxc copy test-copy remote:test --refresh

My question: should this work?

I tried it on my machine (with the image ubuntu:18.04), and I get in the last step errors:

Error: Failed container creation:
 - https://xxx.xx.xx.xxx:8443: Error transferring container data: exit status 1
 - https://10.xx.xx.x:8443: Error transferring container data: Unable to connect to: 10.xx.xx.x:8443
 - https://[fd42:****:****:****::*]:8443: Error transferring container data: Unable to connect to: [fd42:****:****:****::*]:8443

In syslog I have the following error message from btrfs:

host lxd.daemon[1649]: t=2019-05-29T22:56:06+0200 lvl=eror msg="Problem with btrfs send: ERROR: subvolum /var/snap/lxd/common/lxd/storage-pools/default/containers-snapshots/test-copy/backup is not read-only\n"

This happens only when my container (test) has a snapshot (backup). When there are no snapshots, the refresh copy works fine. I have tried to reproduce this situation on a test lxd instance, which seem to use btrfs as well, but there it worked fine.

I am wondering if I experience some bug (with btrfs) or this is because I am not suppose to copy a volume used by a running container?

2

Looks like the problem is not only with running containers. When I stop the container before making the snapshot, the same error takes place:

lxc launch ubuntu:18.04 test
lxc stop test
lxc list # ensure test is STOPPED
lxc snapshot test backup
lxc copy test test-copy
lxc copy test-copy remote:test --refresh

After about 10 seconds pause the same error message as above appears and the same error message is logged in syslog.

I can reproduce the same problem on 3 different hosts with ubuntu 18.04 (fresh install), btrfs storage pool, and snap version of lxd (v. 3.13).

Interestingly, copying of the original container works fine:

lxc copy test remote:test --refresh
3

I have opened a ticket for this problem: