On one of my server, upgrade of lxd throught snap wen’t wrong (upgrade hang).
I had to abort it.
Since, lxc command is no more available (/snap/bin directory is empty)
snap info lxd
installed: 4.15 (20948) 71MB désactivé
(désactivé = disabled)
Hopefully, all my containers are still running but I can’t interact with them.
What can/must I do to re-install lxd/lxc ?
I have to reboot the server because of process in D state (probably the root cause of snap hang), what precaution must I take ?
(fix lxd snap, backup, export of lxd configuration ?)
What’s stuck in
D state? That’s usually a hint of something pretty wrong with the system. Anything bad looking in
You can look at
snap changes and
snap change ID to get more detailed information on what’s going on.
Yes I know , I ordered another server but I would try to fix this one (if possible) and I think a reboot can arrange things.
Process in D state are mostly sync process and process of a broken LXD container
(first problem occur in this container, we had to force stop it, then after we can’t make start again, problem don’t start, so we rebuild it, then 2/3 days after snap hang)
dmesg is full of :
[57764516.769242] audit: type=1400 audit(1627328341.109:122088): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxd-vivi2_</var/snap/lxd/common/lxd>" name="/run/systemd/unit-root/boot/" pid=7901 comm="(ionclean)" flags="ro, remount, noatime, bind"
Snap change :
Run health check of "lxd" snap
Consider re-refresh of "lxd"
Yeah, there’s really nothing you can do to unstick a process that’s in D (I/O wait) state wait on the kernel, forcefully rebooting the machine is probably your best bet.
Yep, but I must re-install lxd before reboot otherwise my container will not start on boot ?
What would you do ?
Try snap install lxd ?
What can I, must I backup before reboot ?
LXD is installed, it just seems stuck halfway through a refresh. I don’t think you’ll be able to get it out of that state when your kernel is stuck and won’t do I/O for you.
Your best bet is to force a reboot and then once the system is back up, you’ll see if snapd just completes the refresh or if you need to play with
snap enable lxd or force a new refresh with
snap cancel ID and
snap refresh lxd.
Keep in mind that LXD is still installed and that under no circumstances should you try to remove it to reinstall it, a
snap remove will make you lose data.
A year after, I finally resolve this issue (in the time I have recreated all my containers, restore all my service on another server).
I reboot the server, LXD container don’t start, I had to run those command:
sudo systemctl restart snapd.service
for i in :lxd-support :network :network-bind :system-observe ; do sudo snap connect lxd$i ; done
And everything start again.