Copying containers from LXD 3.6 (snap) to 3.0.1 (non-snap) fails with rsync protocol incompatibility?

Hi there,

I was hoping someone could help me. Note: Of possible extreme relevance here is that the source LXD host is running LXD in a snap - from what I can tell the container storage itself is inside the snap …

I recently had an issue where I had to send my laptop away for a warranty service. Prior to sending it away I decided to copy my containers to another laptop so I could continue onsite work for a client.

The source laptop was running Ubuntu 18.04, LXC latest version from the Ubuntu repos - version 3.0.1 it seems. Initially the destination laptop was running LXC version 2.x from the Ubuntu 16.04 repository, and I was unable to copy the containers to it due to feature flag incompatibility or somesuch, so in the end I ended up doing a “snap install lxd” on theUbuntu 16.04 machine which gave me LXD version 3.6. At this time I was able to then copy the containers to the destination and start them. I have been working for them fine for the last couple of weeks.

To summarise:

  • Source: Ubuntu 18.04, LXD 3.0.1
  • Destination: Ubuntu 16.04, LXD 3.6 (installed via snap)

The containers were copied by setting up a LXC remote and using: lxc copy source:containername. During this time I got a progress bar.

Now, my laptop has come back and I need to reverse the process. Currently I have the remotes set up and I can list the containers on the source (the source is now the Ubuntu 16.04 machine running LXD 3.6 via snap) and execute commands on the containers via lxc exec.

When I try the copy though, I get no progress and no data is transferred:

bob@destination:~$ lxd copy source:container
[no output, I hit Ctrl-C hours later]

I have tried “pushing” the container from the source to the destination also:
bob@source:~$ lxd copy container destination:
[again, no output…]

An empty container directory is created on the destination, but it never receives any data. I need to use “lxc delete container” to reattempt the copy.

I did find a few lines of potentially relevant information in journalctl on the destination - recall the destination machine now is an Ubuntu 18.04 running LXD 3.0.1 (not 3.6 via snap). These lines are:

Oct 18 23:52:58 dev5 lxd[1663]: lvl=warn msg="Unable to update backup.yaml at this time." name=container t=2018-10-18T23:52:58+1300
Oct 18 23:52:59 dev5 lxd[1663]: lvl=eror msg="Rsync receive failed: /var/lib/lxd/containers/container/: exit status 2: " t=2018-10-18T23:52:59+1300
Oct 18 23:52:59 dev5 lxd[1663]: err="exit status 2" lvl=eror msg="Error during migration sink" t=2018-10-18T23:52:59+1300
Oct 18 23:52:59 dev5 lxd[1663]: lvl=warn msg="Unable to update backup.yaml at this time." name=container t=2018-10-18T23:52:59+1300

The lines of particular interest is:
Rsync receive failed: /var/lib/lxd/containers/container/: exit status 2

Exit status 2 of rsync (according to the man page of rsync) is “Protocol incompatibility”.

I am able to rsync files between the two machines without problems.

Also possibly relevant:

bob@destination:~$ lxc remote list
+-----------------+------------------------------------------+---------------+-----------+--------+--------+
|      NAME       |                   URL                    |   PROTOCOL    | AUTH TYPE | PUBLIC | STATIC |
+-----------------+------------------------------------------+---------------+-----------+--------+--------+
| local (default) | unix://                                  | lxd           | tls       | NO     | YES    |
+-----------------+------------------------------------------+---------------+-----------+--------+--------+
| source          | https://10.23.1.116:8443                 | lxd           | tls       | NO     | NO     |
+-----------------+------------------------------------------+---------------+-----------+--------+--------+

I look forward to your comments.

Cheers,

  • Bob -

Yes, you need to copy to a LXD version which either always passes the xattr flag to rsync OR supports negotiating the rsync options. 3.0.1 doesn’t do either of those, 3.0.2 I believe should.

Bang on. Thank you for your quick response. I’ve installed the snap version on both ends and the containers are now transferring.

With the non-snap LXC I used to access the files in the containers via /var/lib/lxc/containers/* - sure - it was not the ideal way to access files but it was convenient. Is there a similar method for LXC via snap, as I see the container file system is held somewhere other than /var/lib/lxc.

I believe the correct way is to mount a directory from the host into the guest, but in lieu of that where can I find the files?

Cheers,

  • Bob -

/var/snap/lxd/common/mntns/var/snap/lxd/common/lxd/storage-pools is where you’ll find the data, that’s a rather long path to reach the data and whenever possible we obviously recommend you use lxc file commands rather than go behind LXD’s back and poke at the filesystem directly.

1 Like