Firstly, thank you to all involved in the development of lxd. It’s a fantastic tool.
We’re currently running lxd via the deb package in ubuntu 18.04. When we upgrade to 20.04, lxd will only be available as a snap. We’ve encountered problems when migrating to the snap version of lxd. Below are the problems and our workarounds. Perhaps this will be useful for others preparing to upgrade.
Only one layer of nesting
We are using lxd to simulate a computer network. Previously we had four levels of nesting. Nested lxd made it easy to segment the network and delegate administration to different team members.
The snap version of lxd only supports lxd in lxd. The deb version allows multiple levels of nesting. It took a long time to discover that this was a regression and not a mistake caused by us. The only notice of this we could find is buried as a comment on a github issue. I think this should be acknowledged more prominently, especially as 20.04 comes closer.
We failed to find a workaround to regain nested containers. We eventually settled on reworking our architecture to a single layer of nesting, with multiple network bridges and containers all at the same level in the hierarchy. This works, but the previous architecture was so much easier to manage.
lxc exec container bash mysteriously dying
Snap refresh will kill active
lxc exec connections. The default snap behaviour is to attempt a refresh multiple times a day. If there is an lxd update, active
lxc exec connections die. This is obviously frustrating when you’re in the middle of working on something inside a container.
There is no way to disable snap refresh. You can delay it for 60 days (640kb ought to be enough for anyone) with
snap set system refresh.hold=2038-01-01T00:00:00+00:00, and then manually invoke
snap refresh at a convenient time.
Alternatively, memory holing
/etc/hosts will also work to disable unexpected refreshes.