SO I have 5 servers in my cluster, all running 3.0.3 deb, How do I move them to the latest snap.
Can I use my broken 5th cluster server and reinstall as SNAP LXD and then attach ZFS pool.
Then I would need to connect other servers and move those containers over.
All without losing Data.
Otherwise, if every time you have to upgrade or reboot you to start from scratch then the whole cluster thing makes no sense.
lxd.migrate can be used to migrate from deb to snap, including in clustered environments, the upgrade is done in-place and needs to be done on all clustered nodes before the cluster will come back online (due to schema/api version differences between 3.0.x and 3.13).
This is too risky. If one server can be a pain to upgrade. Upgrading 5 at a time is just not possible in a production environment. Sounds like a great way to get fired.
I don’t understand why you can take servers in and out clusters easier, like moving containers. Then you can move all the containers from one to another and eventually do all of them with out any downtime.
You can do something like that, you could move containers from one node to your others, then use lxc cluster remove to get that node out of the cluster.
Reinstall the node with the snap, setup a new cluster on it, then move the containers of one of your nodes over to it using normal LXD migration. Then remove and wipe that node, install it as snap and join it to the new cluster, repeat until all nodes are done.
The wipe part there is because there’s not much point in running lxd.migrate on an empty node, it’s simpler to just remove LXD, reboot to be sure there’s not leftover network/storage bits left in the kernel and then install the snap.
Joining nodes in a cluster must be completely empty, so you can’t just take one node out with its containers, upgrade it and re-cluster it on the new cluster, that’s because database merges are rather impractical to do and likely to cause more problems than it would solve.