Like the OP, I multiboot into different Linux OSes. This is useful for many reasons including redundancy, ease of repair and the ability to try out developmental versions on bare metal. In my case, I purposefully want the same LXD config in all of them, but it’s hard to find implementation guides online. This thread is the closest one I’ve found.
I finally solved the puzzle and am reviving this thread for posterity if nothing else.
The OP almost had the solution, but was tripped up by the use of soft symlinks. The key is in the second post in this thread by simos. If each installed OS can be fooled into thinking that the /var/snap/lxd directory is native to itself, then LXD will launch without a hitch sharing the same ZFS pool.
The solution is to use bind mounts rather than symlinks. Let’s say we have the original OS on partition 1 called “Obiwan”. The newer OS is on partition 2 called “Luke”:
- In preparation, let’s rename Luke’s LXD configuration. This also moves the old LXD out of the way. From Obiwan, mount Luke. Then:
sudo mv path/to/luke/var/snaps/lxd path/to/luke/var/snaps/lxd_bak
- In Luke we need to mount Obiwan at launch. So, in fstab, add the following:
/dev/disk/by-uuid/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /mnt/obiwan ext4 errors=remount-ro 0 1
- Then add the bind mount:
/mnt/obiwan/var/snap/lxd /var/snap/lxd none bind 0 0
That’s it. It’s that simple. Reboot and the bind mount will bind the LXD directory in Obiwan to Luke. The LXD containers will automagically load and all containers that were available in Obiwan are now available in Luke. If containers are created or modified in either OS, they will automagically show up in the other.
I don’t know why symlinks don’t work. I’m guessing that it may be due to soft symlinks not being actual directories, whereas a bind mount acts like a hard link, which is enough to fool snap and LXD into thinking that they are dealing with a natively resident directory.
The only fly in the ointment is ZFS. If you created your zpool first, outside of LXD like I do, then you will always need to go back to Obiwan to maintain/adjust the zpool, since the pool is associated with Obiwan’s ID. Of course, you can export it from Obiwan, then import it with Luke, but I haven’t experimented with that because I don’t want to mess up the good thing that I have going right now.
A word of caution though: this setup does make Luke more fragile. If the mount points change or break, fstab will choke and Luke’s bootup process will grind to a halt. fstab is called very early and the system won’t know what to do with broken mounts.