Lxd service fails to start after restore from backup

1

I had to restore my root drive from a backup and now lxd service wont startup

lxd.service - LXD - main daemon
Loaded: loaded (/lib/systemd/system/lxd.service; indirect; vendor preset: enabled)
Active: inactive (dead) since Tue 2018-09-18 12:38:07 BST; 3s ago
Docs: man:lxd(1)
Process: 6125 ExecStartPost=/usr/bin/lxd waitready --timeout=600 (code=exited, status=0/SUCCESS)
Process: 6121 ExecStartPost=/usr/lib/lxd/profile-config (code=exited, status=0/SUCCESS)
Process: 6119 ExecStart=/usr/bin/lxd --group lxd --logfile=/var/log/lxd/lxd.log (code=exited, status=0/SUCCESS)
Process: 6087 ExecStartPre=/usr/lib/x86_64-linux-gnu/lxc/lxc-apparmor-load (code=exited, status=0/SUCCESS)
Main PID: 6119 (code=exited, status=0/SUCCESS)
Tasks: 2
Memory: 8.6M
CPU: 119ms
CGroup: /system.slice/lxd.service
├─5501 /usr/lib/x86_64-linux-gnu/lxc/lxc-monitord /var/lib/lxd/containers 6
└─5503 [lxc monitor] /var/lib/lxd/containers turnOSv2-5

Sep 18 12:38:07 dellsAthena systemd[1]: Starting LXD - main daemon…
Sep 18 12:38:07 dellsAthena systemd[1]: Started LXD - main daemon.
Sep 18 12:38:07 dellsAthena systemd[1]: Stopping LXD - main daemon…
Sep 18 12:38:07 dellsAthena systemd[1]: Stopped LXD - main daemon.
Sep 18 12:38:07 dellsAthena systemd[1]: Dependency failed for LXD - main daemon.
Sep 18 12:38:07 dellsAthena systemd[1]: lxd.service: Job lxd.service/start failed with result ‘dependency’.

If i run sudo lxd --debug --group lxd manually then it works

Linux dellsAthena 4.15.0-34-generic #37~16.04.1-Ubuntu SMP Tue Aug 28 10:44:06 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

NAME=“Ubuntu”
VERSION=“16.04.5 LTS (Xenial Xerus)”
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME=“Ubuntu 16.04.5 LTS”
VERSION_ID=“16.04”
HOME_URL=“http://www.ubuntu.com/”
SUPPORT_URL=“http://help.ubuntu.com/”
BUG_REPORT_URL=“http://bugs.launchpad.net/ubuntu/”
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

lxd version 2.0.11 previously installed with apt

dell@dellsAthena:lxd $ ls -al
total 88
drwxr-xr-x 9 lxd nogroup 4096 Sep 18 12:38 .
drwxr-xr-x 89 root root 4096 Sep 18 10:51 …
drwx–x--x 5 root root 4096 Aug 14 13:07 containers
drwx–x--x 5 root root 4096 Aug 14 13:07 devices
drwxr-xr-x 2 root root 40 Sep 18 12:38 devlxd
drwx------ 2 root root 4096 Aug 28 08:46 images
-rw-r–r-- 1 root root 51200 Sep 18 12:38 lxd.db
drwx------ 4 root root 4096 Feb 9 2018 security
-rw-r–r-- 1 root root 1903 Feb 9 2018 server.crt
-rw------- 1 root root 3243 Feb 9 2018 server.key
drwx–x--x 3 root root 60 Sep 18 12:38 shmounts
drwx------ 3 root root 4096 Aug 23 10:06 snapshots
srw-rw---- 1 root lxd 0 Sep 18 12:38 unix.socket

lxd-bridge.service - LXD - network bridge
Loaded: loaded (/lib/systemd/system/lxd-bridge.service; static; vendor preset: enabled)
Active: active (exited) since Tue 2018-09-18 12:30:13 BST; 14min ago
Docs: man:lxd(1)
Process: 1592 ExecStart=/usr/lib/lxd/lxd-bridge.start (code=exited, status=0/SUCCESS)
Main PID: 1592 (code=exited, status=0/SUCCESS)
Tasks: 1
Memory: 2.8M
CPU: 31ms
CGroup: /system.slice/lxd-bridge.service
└─2058 dnsmasq -s lxd -S /lxd/ -u lxd --strict-order --bind-interfaces --pid-file=/run/lxd-bridge//dnsmasq.pid --dhcp-no-override --except-interface=lo --interfac

Sep 18 12:30:13 dellsAthena dnsmasq-dhcp[2058]: DHCP, sockets bound exclusively to interface lxdbr0
Sep 18 12:30:13 dellsAthena dnsmasq[2058]: using local addresses only for domain lxd
Sep 18 12:30:13 dellsAthena dnsmasq[2058]: no servers found in /etc/resolv.conf, will retry
Sep 18 12:30:13 dellsAthena dnsmasq[2058]: read /etc/hosts - 9 addresses
Sep 18 12:30:13 dellsAthena systemd[1]: Started LXD - network bridge.
Sep 18 12:30:15 dellsAthena dnsmasq[2058]: reading /etc/resolv.conf
Sep 18 12:30:15 dellsAthena dnsmasq[2058]: using local addresses only for domain lxd
Sep 18 12:30:15 dellsAthena dnsmasq[2058]: using nameserver 127.0.1.1#53
Sep 18 12:38:07 dellsAthena dnsmasq-dhcp[2058]: DHCPREQUEST(lxdbr0) 192.168.0.216 00:16:3e:14:52:14
Sep 18 12:38:07 dellsAthena dnsmasq-dhcp[2058]: DHCPACK(lxdbr0) 192.168.0.216 00:16:3e:14:52:14 turnOSv2-5

2 
Sep 18 12:38:07 dellsAthena systemd[1]: Dependency failed for LXD - main daemon.

So that’s a systemd related issue. Maybe journalctl will give you a bit more of hint as to what’s wrong in that regard.

Depending on how your restored your backup, you may need to run systemctl restart lxd.socket to have it rebind to the right path.

3

thanks for getting back to me, yer I previously tried systemctl restart lxd.socket and it didn’t fix the problem. Maybe journalctl as you recommend will give more details about what the missing dependency is.

4

Anything in systemctl --failed?

5

UNIT LOAD ACTIVE SUB DESCRIPTION
● lxd-containers.service loaded failed failed LXD - container startup/shutdown
● lxd.socket loaded failed failed LXD - unix socket

LOAD = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB = The low-level unit activation state, values depend on unit type.

2 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use ‘systemctl list-unit-files’.

6

if i look at one of the logs for one of my containers,which tries to startup on reboot it has the following in the log

lxc 20180924074851.704 WARN lxc_commands - commands.c:lxc_cmd_rsp_recv:177 - Command get_cgroup failed to receive response: Connection reset by peer

7

My Lxd directory is bind mounted with fstab, which was previously working fine before i did the restore

8

Sorry for the delay, have been travelling for work lately, making keep up with forum posts harder than usual.

So your systemd does look a bit unhappy. I’d first try:

  • systemctl start lxd.socket

If that doesn’t fail miserably, then try:

  • systemctl start lxd-containers.service

If that still works, then there’s a good chance that this will too:

  • systemctl start lxd.service

It’s unclear as to why lxd.socket would fail to start though, it pretty much only depends on /var/lib/lxd being a thing and systemd should in general understand your /etc/fstab enough to wait for that (hopefully).

9

Thanks for this. In the end i have to re-install my root partition as more and more problems ensued