I can’t stand around and do nothing. So was able to create a new SNAP LXD on one of the servers that is working concurrent with the broken LXD, and I am able to mount via zfs my non-running containers, so I have access to them.
My new Snap LXD is not a cluster and not using ZFS, the new snap is working fine, can create containers, exec bash, etc… The new LXD has a neat Import container command. Unfortunately, the old 3.0.3 doesn’t. I tried creating a tar file manually but when I try to import it says /snap/bin/lxc import Q-PODCAST.tar.gz
Importing container: 100% (818.77MB/s)Error: Backup is missing index.yaml
IF I could import my containers that would solve half the problem, how can I create this missing index.yaml.
Also if I start lxd init in none cluster can I later upgrade to cluster or should I do it from the beginning? I don’t remember if when you from uncluster to cluster if it erases your data.
You’re trying to import a container tarball as a backup tarball, this will not work as your hand generated tarball does not contain the index files and extra metadata that a real backup would.
So I have my cluster working well again minus Zfs fragmentation. But I still need to move them to SNAP to keep them with current softwares right. What is best way to move cluster over. I don’t think Migrate is a good idea. I rather move container at a time if possible.
What is best practices here?
Indeed, lxd.migrate for clusters is still pretty new and due to clusters having to be consistent, may be a bit annoying as you’d need to run it on all of the nodes within a one hour window so that all nodes get onto the new version. lxd.migrate will effectively hold until that’s the case.
Note that lxd.migrate needs to shutdown containers as part of the migration and that containers will be offline until the cluster is back online (all nodes upgraded to snap).
In your case, if you have a bit of spare hardware, it’s likely easier to either setup a new cluster and move your container over to it with lxc move or at least setup one system that can temporarily run your containers while you wipe your old systems and setup a cluster with the snap, then move containers back from that standalone system onto your new cluster.
It may not be possible in short terms I have more than a few containers and some are very large. May do them one by one as I create new ones for new clients on new servers.
I had to get another server for drive capacity and zfs fragmentation issues first. The two latest servers are performing well in the cluster but is cluster list show, they show database NO. What does that mean? How do I get them to yes.
That would normally mean that they do not replicate the database, which is a bit surprising given that your cluster is working. Maybe it’s just the database state being wrong, or maybe there’s an actual replication issue.
Can you show:
lxd sql global “SELECT * FROM nodes;” (on any of the machines)
lxd sql local “SELECT * FROM raft_nodes;” (this one on both machines)