I want to rebuild an LXD host by reinstalling the OS, and keep the containers as they are.
The reason for reinstalling is that I want to repartition the root filesystem (which is an SSD VPS), so that I can create a new ZFS filesystem on local SSD storage, which is now part of the root filesystem.
The containers are in a separate detachable zfs filesystem which I will not touch. I will detach it from the system while I reinstall the OS, and then reattach it.
Other than repartitioning, the OS will be the same, Ubuntu 18.04 + snap lxd.
All my container configuration is done via profiles.
What is the best way to do this? Can I somehow copy and restore all of /snap? Or export and restore the lxd snap? Or just its data? By the way, can i put /snap on a zfs filesystem, which I can backup and restore at any time?
This system has about a handful of small containers, but I am also interested in a solution that scales to many containers.
I’m thinking of trying something like this:
Detach the zfs storage pool
Reinstall the OS
Create a temporary zfs storage pool at the same location as before
lxd init with the same parameters as before, including the network address
Create a dummy container for each old container, with the same guest OS as before.
Stop all containers
Detach the zfs storage pool filesystem and reattach the original one in its place
Recreate the LXD profiles and apply them to the containers.
Restart the containers
This will not copy the containers, although it creates temporary dummy containers.
I have tried a variation of this where instead of swapping the whole zfs storage pool, I replaced a container root filesystem, using “zfs receive” which I had previously “zfs send” from a container at another system.
You can put /var/snap on a ZFS dataset, that’s fine. /snap isn’t very useful to keep around as that’s mostly a bunch of mountpoints of the read-only snap contents, the data is what you really care about and that’s in /var/snap.
Same steps as above, except instead of backup/restore /var/snap/lxd, use this:
snap save lxd
copy the snapshot file from /var/lib/snapd/snapshots to the new machine (create the directory first, as it may not exist)
snap restore {id}
In both cases, when I tried “lxc list” after restore, I got the error: Error: Get http://unix.socket/1.0: dial unix /var/snap/lxd/common/lxd/unix.socket: connect: no such file or directory
@stgraber hope you are doing well. My mistake was to use the lxd/lxc version that comes with ubuntu 18.04
ii lxd 3.0.3-0ubuntu1~18.04.1 amd64 Container hypervisor based on LXC - daemon
ii lxd-client 3.0.3-0ubuntu1~18.04.1 amd64 Container hypervisor based on LXC - client
Instead of using the snap one:
$ sudo snap find lxd
Name Version Publisher Notes Summary
lxd 3.16 canonical✓ - System container manager and API
lxd-demo-server 0+git.6d54658 stgraber - Online software demo sessions using LXD
nova ocata james-page - OpenStack Compute Service (nova)
satellite 0.1.2 alanzanattadev - Advanced scalable Open source intelligence platform
nova-hypervisor ocata james-page - OpenStack Compute Service - KVM Hypervisor (nova)
Hello, @gpatel-fr I will proceed and test the migration lxd.migrate from one lxd server (ubuntu 18.04) lxd 3.0 to another (ubuntu 18.10) using lxd 3.16.
Err, I misunderstood you; lxd.migrate is an in-place migration. From a LXD server to another, you can just copy the containers (lxc copy, see doc for setting up a LXD instance as a server, lxc remote add )