Migrate my LXC containers from snap to apt

LXD
1

Hello,

I installed LXD via snap and I’m not happy about how snap is managing it. I saw that the version available on APT is much more stable and do not auto-refresh on its own.

I would like to know how could I migrate all my containers from the snap installation to the APT installation on the same host. Is the command lxc.migrate would do the trick? Also, my LXC version via snap is 3.17 and the APT version is 3.0.0 if i’m not wrong, is it an issue? I was thinking of doing snap refresh lxc --channel=3.0.0/stable before migrating if it is sufficient.

2

LXD cannot be downgraded, so you can’t refresh to 3.0 before moving your containers.

Your best bet to do what you want is to first configure the snap version of LXD to listen on the network by setting core.https_address and core.trust_password.

Once that’s done, install the deb version of LXD (3.0.3), then add your snap version of LXD as a remote by making sure to always use /usr/bin/lxc (you’ll temporarily have two versions of the client utility, so using the full path is needed).

After that, you should be able to /usr/bin/lxc move the containers between the two servers.
Once the snap based server empty, you can remove the snap.

3

Thank you for your answer! I will try it with vagrant first.

4

There is something that I don’t understand: when trying to run apt install lxd it output: snap "lxd" is already installed, see 'snap help refresh'. How can I install the DEV version of LXD (3.0.3)? I tried also to install lxd from source as it is explain on the github repo (with go get repo and make etc) but I was stuck on this :

vagrant@n1:~/go$ ./bin/lxc move lxd.snap:migrate local:migrate
Error: Get http://unix.socket/1.0: dial unix /var/lib/lxd/unix.socket: connect: no such file or directory

Am I missing a point?

I used this article to add a remote.

5

I did manage to install the DEB version of LXD by running in order:

  • sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) universe"
  • wget http://archive.ubuntu.com/ubuntu/pool/main/l/lxd/lxd-client_3.0.3-0ubuntu1~18.04.1_amd64.deb
  • sudo dpkg -i lxd-client_3.0.3-0ubuntu1~18.04.1_amd64.deb
    *wget http://archive.ubuntu.com/ubuntu/pool/main/l/lxd/lxd_3.0.3-0ubuntu1~18.04.1_amd64.deb
  • sudo dpkg -i lxd_3.0.3-0ubuntu1~18.04.1_amd64.deb

Now if I do sudo /usr/bin/lxc ls I don’t see any containers which is normal I guess. If I switch the default remote to lxd.snap one then I can see my containers again.

But the problem now is:

vagrant@n1:~$ sudo /usr/bin/lxc move lxd.snap:migrate local:migrate
Error: Unable to perform container live migration. CRIU isn't installed on the source server

I tried the command sudo snap set lxd criu.enable=true but it doesn’t change anything. Do you know what could be the reason of that?

I can see it there but still doesn’t work:

vagrant@n1:~$ sudo cat /var/snap/lxd/common/config
# This file is auto-generated, do NOT manually edit
daemon_debug=false
daemon_group=lxd
ceph_builtin=false
openvswitch_builtin=false
criu_enable=true
shiftfs_enable=auto
6

That’s simply complaining because your container is running, you’ll want them to be stopped before the move.

7

@stgraber Thank you I didn’t think about doing that. I managed to move my containers on vagrant so I will do it on my real cluster soon.

I had 2 little issues:

  1. when trying to move a container there was this output:
sudo /usr/bin/lxc move lxd.snap:migrate local: 
Error: Failed container creation: Unknown configuration key: volatile.idmap.current

I found a comment from you “The volatile key is probably fine as you couldn’t have used shiftfs (as it requires additional configuration) and only new containers would have been affected even then, so unsetting it (volatile.idmap.current) should be fine.” on this Github issue.
I deleted the volatile.idmap.XxX lines via sudo /usr/bin/lxc edit config snap.lxd:migrate and the migration did succeed after. Is it an issue to erase those lines in my case?

  1. I was not able to create a ZFS pool on the same disk (/dev/sdc) that the one used by the LXD from Snap. Do you know if it is possible to migrate a ZFS pool from the LXD-Snap to the LXD-deb? I was thinking of using a disk for the transition between the snap and deb.

I intend to write a little article to detail the migration procedure from snap to apt.

8
  1. Should be fine in your case too. If you had manually enabled shiftfs, you’d remember it :slight_smile:
  2. So that may be slightly tricky because I believe you need a LXD version newer than 3.0 to support cross-pool container copy/move which you’d need in order to do your temporary storage approach. Another solution would be to use a remote system to temporarily hold your containers, then move them to the source system once you’ve moved LXD to the deb.

An alternative to 2) would be to setup the LXD deb with a loop based zfs pool, then once the containers have been moved over, you can manually reconfigure zfs to use a partition instead of the file. That will require the partition to be larger than the size of the file backed pool and will need you to manually drive a zpool replace to switch from file to partition.

9

Regarding this complaint: you can configure when and how often snaps are refreshed. See this page for more information: https://snapcraft.io/docs/keeping-snaps-up-to-date

Maybe this solves your main annoyance about the LXD snap package.

10

Hello @janxb, I already saw the link you mentionned and tried several stuff mentioned in it like setting a timer or “freezing” the auto-update by using the --channel flag but:

  • If one on your teammate use snap refresh lxd by mistake it will change the version,
  • I don’t see the utility of setting a timer if you choose to track a particular version,
  • snap is using its own environment and runtime which is quiet opaque in my opinion:
    • When I wanted to upgrade my LXD (snap) version, it is very often stuck in the state Copy snap "lxd" data and I have to abort the change and systemctl restart snapd to get rid of it.
  • May be not related to snap but I am not able to delete several containers because its dataset is busy even if it is not (no mountpoint). After digging, it seems to be related to ioctl with this log saying ioctl(3, _IOC(0, 0x5a, 0x18, 0), 0x7ffe4ad22b50) = -1 EBUSY (Device or resource busy) and I don’t know what to do…