Copying containers between hosts with unlike storage pools

Ran into an interesting issue today. I have two Ubuntu 17.04 servers - each running LXD 2.12.
Server 1 storage pool is normal directory and server 2’s storage pool is btrfs.

I did a “lxc copy server1:alpine1 server2:alpine1” and was give the message:

error: Migration failed on target host: Failed to run: btrfs subvolume create /var/lib/lxd/storage-pools/default/containers/alpine1: ERROR: not a btrfs filesystem: /var/lib/lxd/storage-pools/default/containers

I understand that the storage pools don’t match, thus LXC can’t do a “btrfs send snapshot” to the other side. However, how can I get the container copied to the other server? Do I have to create an image and launch that image on server-2? Seems to me LXC should catch this type of error and fall back to normal “dir” to “dir” copying.

Thoughts?

LXD will automatically fallback to the appropriate migration method which in this case will mean rsync. However, if you send from a directory storage pool to a btrfs storage pool LXD will still need to create an empty btrfs subvolume on the receiving end (or “sink” as we like to say internally) and then rsync the contents from the source into the subvolume. The issue you’re describing sounds like an issue with the btrfs storage pool on the target remote (“sink”). If you can reproduce this it would be good if you could open an issue on Github and provide all relevant information about the source and the target. For migration failures the most important resource for us to look at are the logs from the source and target that can be found in /var/log/lxd. Also, we’d need information about the storage pools the containers are on (lxc storage list on the host and target, lxc storage show and lxc storage show ).

Thanks for the reply. My initial statement was wrong; the source server is btrfs and the destination server has both a directory and btrfs storage pool (dir is the default one).

I think the root issue is the destination has 2 storage pools; both of which are identified as btrfs. However, the default storage pool is actually a directory pool (not btrfs).

root@LXD-Testing-003:/var/lib/lxd/storage-pools/default/containers# lxc storage list --verbose
±--------±-------±-------------------------------------±--------+
| NAME | DRIVER | SOURCE | USED BY |
±--------±-------±-------------------------------------±--------+
| btrfs | btrfs | fb5c03f9-7dc3-42d3-b5f7-c5f1a730b2fd | 0 |
±--------±-------±-------------------------------------±--------+
| default | btrfs | 91cbb470-ca96-4d2d-8653-8fae12acea62 | 1 |
±--------±-------±-------------------------------------±--------+

The error message said “ERROR: not a btrfs filesystem: /var/lib/lxd/storage-pools/default/containers”. And, that particular directory is a normal directory.

Any way to figure out why LXD thinks the default volume is btrfs instead of normal dir?

Hm, that is indeed a bit weird. Was the target server updated from a non-storage-api LXD instance (anything < 2.9) to a storage-api LXD instance (anything > 2.9)? Can you please open a Github issue for this so we can track this there. Here it’s a bit more difficult. I’m certain we can figure this out. :slight_smile:

Thanks @brauner. I have opened a new ticket to track the issue (Storage volumes report differently than underlying filesystem #3271)

As the ticket says, I am not sure exactly how I got into this state. And, none of the “lxc volume” command tells me exactly why it thinks the directory is btrfs. Happy to provide any/all details you need to track this down. In the end, it could be user error, but there must be a way to fix it?

1 Like

Certainly does look like the default pool got mis-detected as being btrfs somehow, anyway, we’ll keep track of this on Github and can provide you with what should be a simple one-line SQL statement to fix the type of your pool.