I’m thinking of moving the current /var/lib/lxd to a zfs dataset liblxd so that I can restore the full lxd server elsewhere. If everything, containers and /var/lib/lxd is on zfs, then it seems true that I should be able to send and receive the snapshots and have everything working at destination as an exact copy of the src. Do people agree that this should work? (I will install lxd at destination so that init script, log dir etc is created).
how would you go about doing the initial migration on a running server? Steps I’m considering:
create a liblxd ds and have it mounted at its default location /zpool/liblxd
stop lxd
copy all the data from /var/lib/lxd to /zpool/liblxd
change the mountpoint of the liblxd dataset to be /var/lib/lxd
Yeah, that should work. Having a ZFS dataset backing /var/lib/lxd itself and then a different path from the same zpool used for container storage through a LXD storage pool would be fine.
On a clean system, that’d look like:
systemctl stop lxd lxd.socket
rm -Rf /var/lib/lxd
zfs create tank/lxd-daemon -o /var/lib/lxd
systemctl start lxd
lxc storage create default zfs source=tank/lxd
lxc profile device add default root disk pool=default path=/
Assuming a zpool called “tank”, this will then have the LXD lib files stored in a “lxd-daemon” dataset while all the containers and images will be stored under a LXD managed dataset called “lxd”.
I do kind of the same. I have 2 servers with ZFS and one of them is the backup server. On both I run LXD on their own, so I use the backup server also as test server. Since the backup server and the main server are running LXD from var/lib/lxd (which are offcourse different datasets on the backup server, I use bind mounts for the LXD dataset to mount as /var/lib/lxd.
So in case my main server doesn’t feel well, I stop LXD on the backup server and rebind mount the main server LXD dataset to /var/lib/lxd and then start LXD again. Seems to work
Stephan, but what about a running system? I’m not starting from scratch. I tried on a test host and the result was a mess if I mv’ed out /var/lib/lxd first to .orig and then rsync’ed stuff back with -x . All the mounts for the containers + shmounts and devlxd were still pointing to the old path:
administrator@lxd-test:~$ mount | grep orig
tmpfs on /var/lib/lxd.orig/shmounts type tmpfs (rw,relatime,size=100k,mode=711)
tmpfs on /var/lib/lxd.orig/devlxd type tmpfs (rw,relatime,size=100k,mode=755)
data/lxd/containers/spike-test1 on /var/lib/lxd.orig/storage-pools/default/containers/spike-test1 type z
fs (rw,relatime,xattr,noacl)
I manually unmounted the container, but wasn’t sure about unmounting those other ones too. Eventually I just rebooted the test host and when it came back everything looked ok.
Any idea how to do this cleanly on a production host I can’t reboot?
