Incus copy container on btrfs with subvols created inside container

Incus
1

I am probably doing something unordinary:
I create a privileged Alpine container, installed docker in it and made /var/lib/docker/volumes a btrfs subvolume.
I created as well a docker container using a volume, which I changed to a btrfs subvolume too (so created the subvol and rsynced all data from the original docker volume) .
I do all this to use it in a cluster setup, so I can sync the cluster data using btrfs snapshots. So far so good and all runs fine.
Now when I want to copy the incus container to another host to form the cluster, I get:

incus copy --refresh <remote_host>:rtr1 local: -v
Error: Failed instance creation: Error transferring instance data: Failed migration on target: Failed creating instance on target: Failed to run: btrfs receive -e /var/lib/incus/storage-pools/ssd/containers/migration.724019367: exit status 1 (At subvol router ERROR: creating subvolume router failed: File exists)

The docker volume (btrfs subvolume) name is router in this case
Exporting the container works fine, but get the same error on importing. Both incus hosts run their storage on btrfs. Both incus versions are on 6.7

As workaround I create a new incus container on the second host and just import all data/docker images…after that the cluster runs fine…just weird that copying the container (to be lazy ;-)) doesn’t seem to work…

2

We do have some logic to handle nested subvolumes during transfer, but that’s very very finicky logic :slight_smile:

Running Docker on btrfs in the container may have caused an interesting btrfs tree to develop where we fail to detect the proper order of those subvols, leading to a child making it prior to the parent which could cause this kind of error. Another potential source of issue would be any btrfs subvol marked as ro as those then need special handling to transfer them as rw, then alter them. Depending on when the bit is flipped, it could then impact child subvols too.

3

Hi Stephane,

Thanks for the feedback. So I can better rsync the container then, convert the needed paths back into a btrfs subvol and do a container recovery if I want to copy it. Not sure if that saves time in the end though :wink: