When I am run This command, it shows the below error.
Error: Get “http://unix.socket/1.0”: dial unix /var/snap/lxd/common/lxd/unix.socket: connect: connection refused.
Snap version:
snap 2.51.7
snapd 2.51.7
series 16
ubuntu 20.04
kernel 5.4.0-88-generic
LXD version:
4.19
This means your LXD server isn’t running.
Please can you check the contents of /var/snap/lxd/common/lxd/logs/lxd.log for errors indicating the issue.
t=2021-10-05T06:13:41+0000 lvl=info msg=“Starting shutdown sequence” signal=interrupt
t=2021-10-05T06:13:41+0000 lvl=info msg=“Closing the database”
t=2021-10-05T06:13:41+0000 lvl=info msg=“Stop database gateway”
t=2021-10-05T06:13:41+0000 lvl=info msg=“Stopping REST API handler:”
t=2021-10-05T06:13:41+0000 lvl=info msg=" - closing socket" socket=[::]:8443
t=2021-10-05T06:13:41+0000 lvl=info msg=" - closing socket" socket=/var/snap/lxd/comm on/lxd/unix.socket
t=2021-10-05T06:13:41+0000 lvl=info msg=“Stopping /dev/lxd handler:”
t=2021-10-05T06:13:41+0000 lvl=info msg=" - closing socket" socket=/var/snap/lxd/comm on/lxd/devlxd/sock
t=2021-10-05T06:13:41+0000 lvl=info msg=“Not unmounting temporary filesystems (instan ces are still running)”
t=2021-10-05T06:13:41+0000 lvl=info msg=“Daemon stopped”
Yep that shows it was asked to shutdown and did so cleanly.
What do you see in journalctl when trying to run lxc ls for example, it should be trying to start up.
root@lxd:~# journalctl
– Logs begin at Thu 2021-02-25 10:53:10 GMT, end at Wed 2021-10-06 09:17:01 BST. –
Feb 25 10:53:10 ubuntu kernel: microcode: microcode updated early to revision 0xb000038, date = 2019-06-18
Feb 25 10:53:10 ubuntu kernel: Linux version 5.4.0-66-generic (buildd@lgw01-amd64-039) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #74-Ubuntu SMP Wed Jan 27 >
Feb 25 10:53:10 ubuntu kernel: Command line: BOOT_IMAGE=/vmlinuz-5.4.0-66-generic root=UUID=b20cf795-441c-4ec9-a59e-b5fc59722cda ro maybe-ubiquity
Feb 25 10:53:10 ubuntu kernel: KERNEL supported cpus:
Feb 25 10:53:10 ubuntu kernel: Intel GenuineIntel
Feb 25 10:53:10 ubuntu kernel: AMD AuthenticAMD
Feb 25 10:53:10 ubuntu kernel: Hygon HygonGenuine
Feb 25 10:53:10 ubuntu kernel: Centaur CentaurHauls
Feb 25 10:53:10 ubuntu kernel: zhaoxin Shanghai
Feb 25 10:53:10 ubuntu kernel: x86/fpu: Supporting XSAVE feature 0x001: ‘x87 floating point registers’
Feb 25 10:53:10 ubuntu kernel: x86/fpu: Supporting XSAVE feature 0x002: ‘SSE registers’
Feb 25 10:53:10 ubuntu kernel: x86/fpu: Supporting XSAVE feature 0x004: ‘AVX registers’
Feb 25 10:53:10 ubuntu kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
Feb 25 10:53:10 ubuntu kernel: x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using ‘standard’ format.
Feb 25 10:53:10 ubuntu kernel: BIOS-provided physical RAM map:
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x0000000000000000-0x0000000000099fff] usable
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x000000000009a000-0x000000000009ffff] reserved
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x0000000000100000-0x0000000078e86fff] usable
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x0000000078e87000-0x000000007969dfff] reserved
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x000000007969e000-0x00000000796f7fff] ACPI data
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x00000000796f8000-0x0000000079d2bfff] ACPI NVS
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x0000000079d2c000-0x000000008fffffff] reserved
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed44fff] reserved
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
Feb 25 10:53:10 ubuntu kernel: BIOS-e820: [mem 0x0000000100000000-0x000000307fffffff] usable
Feb 25 10:53:10 ubuntu kernel: NX (Execute Disable) protection: active
Feb 25 10:53:10 ubuntu kernel: SMBIOS 3.0 present.
Feb 25 10:53:10 ubuntu kernel: DMI: Supermicro PIO-628U-TR4T±ST031/X10DRU-i+, BIOS 3.1c 05/10/2019
Feb 25 10:53:10 ubuntu kernel: tsc: Fast TSC calibration using PIT
Feb 25 10:53:10 ubuntu kernel: tsc: Detected 2099.987 MHz processor
Feb 25 10:53:10 ubuntu kernel: e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
Feb 25 10:53:10 ubuntu kernel: e820: remove [mem 0x000a0000-0x000fffff] usable
Feb 25 10:53:10 ubuntu kernel: last_pfn = 0x3080000 max_arch_pfn = 0x400000000
Feb 25 10:53:10 ubuntu kernel: MTRR default type: write-back
Feb 25 10:53:10 ubuntu kernel: MTRR fixed ranges enabled:
Feb 25 10:53:10 ubuntu kernel: 00000-9FFFF write-back
Feb 25 10:53:10 ubuntu kernel: A0000-BFFFF uncachable
Feb 25 10:53:10 ubuntu kernel: C0000-FFFFF write-protect
Feb 25 10:53:10 ubuntu kernel: MTRR variable ranges enabled:
Feb 25 10:53:10 ubuntu kernel: 0 base 000080000000 mask 3FFF80000000 uncachable
Feb 25 10:53:10 ubuntu kernel: 1 base 380000000000 mask 3F8000000000 uncachable
Feb 25 10:53:10 ubuntu kernel: 2 base 0000C5400000 mask 3FFFFFC00000 write-through
Feb 25 10:53:10 ubuntu kernel: 3 base 0000C5800000 mask 3FFFFF800000 write-through
Feb 25 10:53:10 ubuntu kernel: 4 base 0000C5F00000 mask 3FFFFFF00000 uncachable
That only shows the first few boot messages.
You need to look at the last few hundred lines that occur after running lxc ls.
So try sudo journalctl -r -n 200 -b -g lxd
root@lxd:~# sudo journalctl -r -n 200 -b -g lxd
– Logs begin at Thu 2021-02-25 10:53:10 GMT, end at Wed 2021-10-06 09:37:15 BST. –
Oct 06 09:37:15 lxd sudo[1814243]: root : TTY=pts/0 ; PWD=/root ; USER=root ; COMMAND=/usr/bin/journalctl -r -n 200 -b -g lxd
Oct 06 09:27:15 lxd systemd[4244]: snap.lxd.lxc.af456d6a-6dae-4c45-b316-c211db64ba61.scope: Succeeded.
Oct 06 09:27:15 lxd systemd[4244]: Started snap.lxd.lxc.af456d6a-6dae-4c45-b316-c211db64ba61.scope.
Oct 06 07:46:36 lxd systemd[4244]: snap.lxd.lxc.206ff15f-0223-4e9d-9ae6-5bcf673fadf6.scope: Succeeded.
Oct 06 07:46:36 lxd systemd[4244]: Started snap.lxd.lxc.206ff15f-0223-4e9d-9ae6-5bcf673fadf6.scope.
Oct 06 07:43:01 lxd systemd[4244]: snap.lxd.lxc.f4b3b6b3-172e-40a9-8a8b-2e3a895b8043.scope: Succeeded.
Oct 06 07:43:01 lxd systemd[4244]: Started snap.lxd.lxc.f4b3b6b3-172e-40a9-8a8b-2e3a895b8043.scope.
Oct 06 07:39:14 lxd systemd[4244]: snap.lxd.lxd.7826daea-024b-4e55-84ef-ff22acbe1bf8.scope: Succeeded.
Oct 06 07:39:14 lxd systemd[4244]: Started snap.lxd.lxd.7826daea-024b-4e55-84ef-ff22acbe1bf8.scope.
Oct 06 07:33:34 lxd systemd[4244]: snap.lxd.lxc.0c9bf70b-e15b-4411-85cf-acc397143642.scope: Succeeded.
Oct 06 07:33:34 lxd systemd[4244]: Started snap.lxd.lxc.0c9bf70b-e15b-4411-85cf-acc397143642.scope.
Oct 06 07:27:51 lxd systemd[4244]: snap.lxd.lxc.bbcb7a6e-31f7-4138-b65e-e5a878421b96.scope: Succeeded.
Oct 06 07:27:51 lxd systemd[4244]: Started snap.lxd.lxc.bbcb7a6e-31f7-4138-b65e-e5a878421b96.scope.
Oct 06 07:27:42 lxd systemd[4244]: snap.lxd.lxc.a94fc3eb-3d47-4e89-907f-ee127fe9f637.scope: Succeeded.
Oct 06 07:27:42 lxd systemd[4244]: Started snap.lxd.lxc.a94fc3eb-3d47-4e89-907f-ee127fe9f637.scope.
Oct 06 07:17:16 lxd systemd[4244]: snap.lxd.lxc.337b8e15-acee-4f13-bfd0-663eca6d40c1.scope: Succeeded.
Oct 06 07:17:15 lxd systemd[4244]: Started snap.lxd.lxc.337b8e15-acee-4f13-bfd0-663eca6d40c1.scope.
Oct 06 07:17:15 lxd systemd[4244]: snap.lxd.lxc.dcfcb05b-ed9c-44da-9adb-c3053223c00c.scope: Succeeded.
Oct 06 07:17:15 lxd systemd[4244]: Started snap.lxd.lxc.dcfcb05b-ed9c-44da-9adb-c3053223c00c.scope.
Oct 06 07:16:56 lxd systemd[4244]: snap.lxd.lxc.79fa9f7b-9e9d-4606-9bdc-a4d96a3cb296.scope: Succeeded.
Oct 06 07:16:55 lxd systemd[4244]: Started snap.lxd.lxc.79fa9f7b-9e9d-4606-9bdc-a4d96a3cb296.scope.
Oct 06 07:16:14 lxd systemd[4244]: snap.lxd.lxc.9309e8a2-e82e-45ec-ba08-2a1a9656c59f.scope: Succeeded.
Oct 06 07:16:14 lxd systemd[4244]: Started snap.lxd.lxc.9309e8a2-e82e-45ec-ba08-2a1a9656c59f.scope.
Oct 06 07:05:30 lxd snapd[2967]: storehelpers.go:551: cannot refresh: snap has no updates available: “core18”, “core20”, “lxd”, “snapd”
Oct 06 04:19:18 lxd kernel: audit: type=1400 audit(1633490358.540:2580): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-TA-cNT>
Oct 06 04:19:18 lxd audit[4048528]: AVC apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-TA-cNTLDNSMSP03_</var/snap/lxd/common/l>
Oct 06 02:20:31 lxd snapd[2967]: storehelpers.go:551: cannot refresh: snap has no updates available: “core18”, “core20”, “lxd”, “snapd”
Oct 06 01:52:09 lxd kernel: audit: type=1400 audit(1633481529.614:2579): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-cNTLDN>
Oct 06 01:52:09 lxd audit[3352963]: AVC apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-cNTLDNLSSQLU001_</var/snap/lxd/common/l>
Oct 06 01:00:01 lxd kernel: audit: type=1400 audit(1633478401.000:2578): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-cNTLDN>
Oct 06 01:00:01 lxd audit[3104162]: AVC apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-cNTLDNLSSQLP001_</var/snap/lxd/common/l>
Oct 06 00:46:48 lxd kernel: audit: type=1400 audit(1633477608.938:2577): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-cNTLDN>
Oct 06 00:46:48 lxd audit[3043725]: AVC apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-cNTLDNLSSQLP001_</var/snap/lxd/common/l>
Oct 06 00:46:48 lxd kernel: audit: type=1400 audit(1633477608.910:2576): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-cNTLDN>
Oct 06 00:46:48 lxd audit[3043716]: AVC apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-cNTLDNLSSQLP001_</var/snap/lxd/common/l>
Oct 06 00:36:03 lxd audit[2993437]: AVC apparmor=“STATUS” operation=“profile_replace” info=“same as current profile, skipping” label="lxd-cNTLDNLSSQLP001_</var/snap>
Oct 06 00:36:03 lxd audit[2993438]: AVC apparmor=“STATUS” operation=“profile_replace” info=“same as current profile, skipping” label="lxd-cNTLDNLSSQLP001_</var/snap>
Oct 06 00:36:03 lxd audit[2993444]: AVC apparmor=“STATUS” operation=“profile_replace” info=“same as current profile, skipping” label="lxd-cNTLDNLSSQLP001_</var/snap>
Oct 06 00:36:03 lxd audit[2993441]: AVC apparmor=“STATUS” operation=“profile_replace” info=“same as current profile, skipping” label="lxd-cNTLDNLSSQLP001_</var/snap>
Oct 06 00:36:03 lxd audit[2993443]: AVC apparmor=“STATUS” operation=“profile_replace” info=“same as current profile, skipping” label="lxd-cNTLDNLSSQLP001_</var/snap>
Oct 06 00:36:03 lxd audit[2993439]: AVC apparmor=“STATUS” operation=“profile_replace” info=“same as current profile, skipping” label="lxd-cNTLDNLSSQLP001_</var/snap>
Oct 06 00:36:03 lxd audit[2993433]: AVC apparmor=“STATUS” operation=“profile_replace” info=“same as current profile, skipping” label="lxd-cNTLDNLSSQLP001_</var/snap>
Oct 06 00:36:03 lxd audit[2993442]: AVC apparmor=“STATUS” operation=“profile_replace” info=“same as current profile, skipping” label="lxd-cNTLDNLSSQLP001_</var/snap>
Oct 06 00:36:03 lxd audit[2993435]: AVC apparmor=“STATUS” operation=“profile_replace” info=“same as current profile, skipping” label="lxd-cNTLDNLSSQLP001_</var/snap>
Oct 06 00:36:03 lxd audit[2993440]: AVC apparmor=“STATUS” operation=“profile_replace” info=“same as current profile, skipping” label="lxd-cNTLDNLSSQLP001_</var/snap>
Oct 06 02:20:31 lxd snapd[2967]: storehelpers.go:551: cannot refresh: snap has no updates available: “core18”, “core20”, “lxd”, “snapd”
looks suspicious.
Please can you show output of sudo snap info lxd and ps aux | grep lxd.
root@lxd:~# sudo snap info lxd and ps aux | grep lxd
name: lxd
store-url: https://snapcraft.io/lxd
contact: Issues · lxc/incus · GitHub
Supported configuration options for the snap (snap set lxd
[default=lxd]
- daemon.preseed: Pass a YAML configuration to lxd init on initial
/var/snap/lxd/common/global-conf/ (config.yml and servercerts)
No, they were separate commands (they were separately quoted):
sudo snap info lxd
and
ps aux | grep lx
root@lxd:~# sudo snap info lxd
name: lxd
summary: LXD - container and VM manager
publisher: Canonical✓
store-url: https://snapcraft.io/lxd
contact: Issues · lxc/incus · GitHub
license: unset
description: |
LXD is a system container and virtual machine manager.
It offers a simple CLI and REST API to manage local or remote instances,
uses an image based workflow and support for a variety of advanced features.
Images are available for all Ubuntu releases and architectures as well
as for a wide number of other Linux distributions. Existing
integrations with many deployment and operation tools, makes it work
just like a public cloud, except everything is under your control.
LXD containers are lightweight, secure by default and a great
alternative to virtual machines when running Linux on Linux.
LXD virtual machines are modern and secure, using UEFI and secure-boot
by default and a great choice when a different kernel or operating
system is needed.
With clustering, up to 50 LXD servers can be easily joined and managed
together with the same tools and APIs and without needing any external
dependencies.
Supported configuration options for the snap (snap set lxd [=…]):
- ceph.builtin: Use snap-specific Ceph configuration [default=false]
- ceph.external: Use the system's ceph tools (ignores ceph.builtin) [default=false]
- criu.enable: Enable experimental live-migration support [default=false]
- daemon.debug: Increase logging to debug level [default=false]
- daemon.group: Set group of users that can interact with LXD [default=lxd]
- daemon.preseed: Pass a YAML configuration to `lxd init` on initial start
- daemon.syslog: Send LXD log events to syslog [default=false]
- lvm.external: Use the system's LVM tools [default=false]
- lxcfs.pidfd: Start per-container process tracking [default=false]
- lxcfs.loadavg: Start tracking per-container load average [default=false]
- lxcfs.cfs: Consider CPU shares for CPU usage [default=false]
- openvswitch.builtin: Run a snap-specific OVS daemon [default=false]
- shiftfs.enable: Enable shiftfs support [default=auto]
For system-wide configuration of the CLI, place your configuration in
/var/snap/lxd/common/global-conf/ (config.yml and servercerts)
commands:
root@lxd:~# ps aux | grep lx
root 3812 1.1 0.0 3068436 12016 ? Sl Oct03 51:17 lxcfs /var/snap/lxd/common/var/lib/lxcfs -p /var/snap/lxd/common/lxcfs.pid
lxd 3984 0.0 0.0 7204 3584 ? Ss Oct03 0:01 dnsmasq --keep-in-foreground --strict-order --bind-interfaces --except-interface=lo --pid-file= --no-ping --interface=lxdbr0 --dhcp-rapid-commit --quiet-dhcp --quiet-dhcp6 --quiet-ra --listen-address=10.84.149.1 --dhcp-no-override --dhcp-authoritative --dhcp-leasefile=/var/snap/lxd/common/lxd/networks/lxdbr0/dnsmasq.leases --dhcp-hostsfile=/var/snap/lxd/common/lxd/networks/lxdbr0/dnsmasq.hosts --dhcp-range 10.84.149.2,10.84.149.254,1h --listen-address=fd42:18b9:a901:b901::1 --enable-ra --dhcp-range ::,constructor:lxdbr0,ra-stateless,ra-names -s lxd --interface-name _gateway.lxd,lxdbr0 -S /lxd/ --conf-file=/var/snap/lxd/common/lxd/networks/lxdbr0/dnsmasq.raw -u lxd -g lxd
root 4738 0.0 0.0 2163956 19148 ? Ss Oct03 0:00 [lxc monitor] /var/snap/lxd/common/lxd/containers cNTLDNLSAPPP001
root 9643 0.0 0.0 2163444 19228 ? Ss Oct03 0:00 [lxc monitor] /var/snap/lxd/common/lxd/containers cNTLDNLSAPPU001
root 11640 0.0 0.0 2237688 18276 ? Ss Oct03 0:00 [lxc monitor] /var/snap/lxd/common/lxd/containers cNTLDNLSSQLP001
root 15754 0.0 0.0 2311420 19428 ? Ss Oct03 0:00 [lxc monitor] /var/snap/lxd/common/lxd/containers cNTLDNLSSQLU001
root 16172 0.0 0.0 3640 1152 ? Ss Oct03 0:00 snapfuse /var/lib/snapd/snaps/lxd_21029.snap /snap/lxd/21029 -o ro,nodev,allow_other,suid
root 16177 0.0 0.0 3508 164 ? Ss Oct03 0:00 snapfuse /var/lib/snapd/snaps/lxd_20326.snap /snap/lxd/20326 -o ro,nodev,allow_other,suid
1000000 21831 0.0 0.0 3508 164 ? Ss Oct03 0:00 snapfuse /var/lib/snapd/snaps/lxd_21029.snap /snap/lxd/21029 -o ro,nodev,allow_other,suid
1000000 21833 0.0 0.0 3708 1224 ? Ss Oct03 0:00 snapfuse /var/lib/snapd/snaps/lxd_21545.snap /snap/lxd/21545 -o ro,nodev,allow_other,suid
root 22354 0.0 0.0 2163956 18932 ? Ss Oct03 0:00 [lxc monitor] /var/snap/lxd/common/lxd/containers p-promt-cNTLDNSMSP02
1000000 23404 0.0 0.0 3508 164 ? Ss Oct03 0:00 snapfuse /var/lib/snapd/snaps/lxd_21029.snap /snap/lxd/21029 -o ro,nodev,allow_other,suid
1000000 23407 0.0 0.0 3956 1404 ? Ss Oct03 0:01 snapfuse /var/lib/snapd/snaps/lxd_21545.snap /snap/lxd/21545 -o ro,nodev,allow_other,suid
root 719003 0.0 0.0 2237432 18972 ? Ss Oct03 0:00 [lxc monitor] /var/snap/lxd/common/lxd/containers TA-cNTLDNSMSP03
1000000 719511 0.0 0.0 3508 1100 ? Ss Oct03 0:15 snapfuse /var/lib/snapd/snaps/lxd_21029.snap /snap/lxd/21029 -o ro,nodev,allow_other,suid
1000000 719526 0.0 0.0 3956 1532 ? Ss Oct03 0:15 snapfuse /var/lib/snapd/snaps/lxd_21545.snap /snap/lxd/21545 -o ro,nodev,allow_other,suid
root 1862671 0.0 0.0 6432 2540 pts/0 R+ 09:43 0:00 grep --color=auto lx
Hrm, I’m not sure.
Any ideas @stgraber
Those AppArmor denials look problematic:
Oct 06 00:46:48 lxd audit[3043716]: AVC apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile="lxd-cNTLDNLSSQLP001_</var/snap/lxd/common/l>
I have another issue with another server
root@lxd1:~# lxc list
Error: Get “http://unix.socket/1.0”: EOF
Can you follow the same diagnostic steps as above, as the error you’ve posted here just tells us the same thing, that LXD isn’t running, but not why.
root@lxd1:~# sudo journalctl -r -n 200 -b -g lxd
– Logs begin at Wed 2019-05-29 05:15:35 +06, end at Wed 2021-10-06 14:55:49 +06. –
Oct 06 14:55:49 lxd1 sudo[1808579]: root : TTY=pts/1 ; PWD=/root ; USER=root ; COMMAND=/bin/journalctl -r -n 200 -b -g lxd
Oct 06 14:55:48 lxd1 lxd.daemon[1808406]: => Starting LXD
Oct 06 14:55:48 lxd1 systemd[1]: Started Service for snap application lxd.daemon.
Oct 06 14:55:48 lxd1 systemd[1]: Stopped Service for snap application lxd.daemon.
Oct 06 14:55:48 lxd1 systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 22930.
Oct 06 14:55:48 lxd1 systemd[1]: snap.lxd.daemon.service: Failed with result ‘exit-code’.
Oct 06 14:55:48 lxd1 systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
Oct 06 14:55:48 lxd1 lxd.daemon[1808064]: => LXD failed to start
Oct 06 14:55:46 lxd1 lxd.daemon[1808064]: => Starting LXD
Oct 06 14:55:45 lxd1 systemd[1]: Started Service for snap application lxd.daemon.
Oct 06 14:55:45 lxd1 systemd[1]: Stopped Service for snap application lxd.daemon.
Oct 06 14:55:45 lxd1 systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 22929.
Oct 06 14:55:45 lxd1 systemd[1]: snap.lxd.daemon.service: Failed with result ‘exit-code’.
Oct 06 14:55:45 lxd1 systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
Oct 06 14:55:45 lxd1 lxd.daemon[1807763]: => LXD failed to start
Oct 06 14:55:43 lxd1 lxd.daemon[1807763]: => Starting LXD
Oct 06 14:55:42 lxd1 systemd[1]: Started Service for snap application lxd.daemon.
Oct 06 14:55:42 lxd1 systemd[1]: Stopped Service for snap application lxd.daemon.
Oct 06 14:55:42 lxd1 systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 22928.
Oct 06 14:55:42 lxd1 systemd[1]: snap.lxd.daemon.service: Failed with result ‘exit-code’.
Oct 06 14:55:42 lxd1 systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
Oct 06 14:55:42 lxd1 lxd.daemon[1807552]: => LXD failed to start
Oct 06 14:55:40 lxd1 lxd.daemon[1807552]: => Starting LXD
Oct 06 14:55:39 lxd1 systemd[1]: Started Service for snap application lxd.daemon.
Oct 06 14:55:39 lxd1 systemd[1]: Stopped Service for snap application lxd.daemon.
Oct 06 14:55:39 lxd1 systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 22927.
Oct 06 14:55:39 lxd1 systemd[1]: snap.lxd.daemon.service: Failed with result ‘exit-code’.
Oct 06 14:55:39 lxd1 systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
Oct 06 14:55:39 lxd1 lxd.daemon[1807235]: => LXD failed to start
Oct 06 14:55:37 lxd1 lxd.daemon[1807235]: => Starting LXD
Oct 06 14:55:36 lxd1 systemd[1]: Started Service for snap application lxd.daemon.
Oct 06 14:55:36 lxd1 systemd[1]: Stopped Service for snap application lxd.daemon.
Oct 06 14:55:36 lxd1 systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 22926.
Oct 06 14:55:36 lxd1 systemd[1]: snap.lxd.daemon.service: Failed with result ‘exit-code’.
Oct 06 14:55:36 lxd1 systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
Oct 06 14:55:36 lxd1 lxd.daemon[1807031]: => LXD failed to start
Oct 06 14:55:34 lxd1 lxd.daemon[1807031]: => Starting LXD
Oct 06 14:55:33 lxd1 systemd[1]: Started Service for snap application lxd.daemon.
Oct 06 14:55:33 lxd1 systemd[1]: Stopped Service for snap application lxd.daemon.
Oct 06 14:55:33 lxd1 systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 22925.
Oct 06 14:55:33 lxd1 systemd[1]: snap.lxd.daemon.service: Failed with result ‘exit-code’.
Oct 06 14:55:33 lxd1 systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
Oct 06 14:55:33 lxd1 lxd.daemon[1806660]: => LXD failed to start
Oct 06 14:55:31 lxd1 lxd.daemon[1806660]: => Starting LXD
Oct 06 14:55:30 lxd1 systemd[1]: Started Service for snap application lxd.daemon.
root@lxd1:~# sudo snap info lxd
name: lxd
summary: LXD - container and VM manager
publisher: Canonical✓
store-url: Install lxd on Linux | Snap Store
contact: https://github.com/lxc/lxd/issues
license: unset
description: |
LXD is a system container and virtual machine manager.
It offers a simple CLI and REST API to manage local or remote instances,
uses an image based workflow and support for a variety of advanced features.
Images are available for all Ubuntu releases and architectures as well
as for a wide number of other Linux distributions. Existing
integrations with many deployment and operation tools, makes it work
just like a public cloud, except everything is under your control.
LXD containers are lightweight, secure by default and a great
alternative to virtual machines when running Linux on Linux.
LXD virtual machines are modern and secure, using UEFI and secure-boot
by default and a great choice when a different kernel or operating
system is needed.
With clustering, up to 50 LXD servers can be easily joined and managed
together with the same tools and APIs and without needing any external
dependencies.
Supported configuration options for the snap (snap set lxd [=…]):
- ceph.builtin: Use snap-specific Ceph configuration [default=false]
- ceph.external: Use the system's ceph tools (ignores ceph.builtin) [default=false]
- criu.enable: Enable experimental live-migration support [default=false]
- daemon.debug: Increase logging to debug level [default=false]
- daemon.group: Set group of users that can interact with LXD [default=lxd]
- daemon.preseed: Pass a YAML configuration to `lxd init` on initial start
- daemon.syslog: Send LXD log events to syslog [default=false]
- lvm.external: Use the system's LVM tools [default=false]
- lxcfs.pidfd: Start per-container process tracking [default=false]
- lxcfs.loadavg: Start tracking per-container load average [default=false]
- lxcfs.cfs: Consider CPU shares for CPU usage [default=false]
- openvswitch.builtin: Run a snap-specific OVS daemon [default=false]
- shiftfs.enable: Enable shiftfs support [default=auto]
For system-wide configuration of the CLI, place your configuration in
/var/snap/lxd/common/global-conf/ (config.yml and servercerts)
commands:
root@lxd1:~# ps aux | grep lx
root 4245 0.0 0.0 298992 2484 ? Ssl Sep06 0:27 /usr/bin/lxcfs /var/lib/lxcfs
root 4885 0.0 0.0 743604 4788 ? Sl Sep06 17:34 lxcfs /var/snap/lxd/common/var/lib/lxcfs -p /var/snap/lxd/common/lxcfs.pid
root 1813685 10.0 0.0 2624 1760 ? Ss 14:56 0:00 /bin/sh /snap/lxd/21624/commands/daemon.start
root 1813823 42.0 0.0 2091196 85760 ? Sl 14:56 0:00 lxd --logfile /var/snap/lxd/common/lxd/logs/lxd.log --group lxd
root 1813824 15.0 0.0 1794792 41092 ? Sl 14:56 0:00 lxd waitready
root 1813825 0.0 0.0 2624 128 ? S 14:56 0:00 /bin/sh /snap/lxd/21624/commands/daemon.start
root 1813865 0.0 0.0 6436 740 pts/1 S+ 14:56 0:00 grep --color=auto lx
root 2262451 0.0 0.0 1718528 17568 ? Ss Oct03 0:03 [lxc monitor] /var/snap/lxd/common/lxd/containers grafana-prod
root 2264438 0.0 0.0 1646460 17872 ? Ss Oct03 0:02 [lxc monitor] /var/snap/lxd/common/lxd/containers grafana-uat
Are your four servers in a cluster?
What does /var/snap/lxd/common/lxd/logs/lxd.log container on the 2nd server?
NO, two servers are in same location and down server is in running server remote list. Here is the running server log
root@lxd:/mnt/L-NFS/Image# tail -f /var/snap/lxd/common/lxd/logs/lxd.log
t=2021-10-06T13:13:42+0600 lvl=info msg=“Done pruning expired instance backups”
t=2021-10-06T13:13:42+0600 lvl=info msg=“Done updating images”
t=2021-10-06T14:13:42+0600 lvl=info msg=“Pruning expired instance backups”
t=2021-10-06T14:13:42+0600 lvl=info msg=“Updating images”
t=2021-10-06T14:13:42+0600 lvl=info msg=“Done pruning expired instance backups”
t=2021-10-06T14:13:42+0600 lvl=info msg=“Done updating images”
t=2021-10-06T15:13:42+0600 lvl=info msg=“Updating images”
t=2021-10-06T15:13:42+0600 lvl=info msg=“Pruning expired instance backups”
t=2021-10-06T15:13:42+0600 lvl=info msg=“Done updating images”
t=2021-10-06T15:13:42+0600 lvl=info msg=“Done pruning expired instance backups”
Sorry, it’s a bit of a mess going through all the partial information above.
On all your machines, please run:
root@lxd:~# snap refresh lxd
snap “lxd” has no updates available
root@lxd:~# snap list lxd
Name Version Rev Tracking Publisher Notes
lxd 4.19 21624 latest/stable canonical✓ -
root@lxd:~# lxc info
Error: Get “http://unix.socket/1.0”: dial unix /var/snap/lxd/common/lxd/unix.socket: connect: connection refused
root@lxd:~# journalctl -u snap.lxd.daemon -n 300
– Logs begin at Thu 2021-02-25 10:53:10 GMT, end at Thu 2021-10-07 04:35:03 BST. –
Oct 05 07:13:33 lxd lxd.daemon[3812]: Running destructor lxcfs_exit
Oct 05 07:13:33 lxd lxd.daemon[3812]: Running constructor lxcfs_init to reload liblxcfs
Oct 05 07:13:33 lxd lxd.daemon[3812]: mount namespace: 5
Oct 05 07:13:33 lxd lxd.daemon[3812]: hierarchies:
Oct 05 07:13:33 lxd lxd.daemon[3812]: 0: fd: 6:
Oct 05 07:13:33 lxd lxd.daemon[3812]: 1: fd: 7: name=systemd
Oct 05 07:13:33 lxd lxd.daemon[3812]: 2: fd: 8: hugetlb
Oct 05 07:13:33 lxd lxd.daemon[3812]: 3: fd: 9: net_cls,net_prio
Oct 05 07:13:33 lxd lxd.daemon[3812]: 4: fd: 10: devices
Oct 05 07:13:33 lxd lxd.daemon[3812]: 5: fd: 11: rdma
Oct 05 07:13:33 lxd lxd.daemon[3812]: 6: fd: 12: memory
Oct 05 07:13:33 lxd lxd.daemon[3812]: 7: fd: 13: pids
Oct 05 07:13:33 lxd lxd.daemon[3812]: 8: fd: 14: perf_event
Oct 05 07:13:33 lxd lxd.daemon[3812]: 9: fd: 15: cpu,cpuacct
Oct 05 07:13:33 lxd lxd.daemon[3812]: 10: fd: 16: blkio
Oct 05 07:13:33 lxd lxd.daemon[3812]: 11: fd: 17: freezer
Oct 05 07:13:33 lxd lxd.daemon[3812]: 12: fd: 19: cpuset
Oct 05 07:13:33 lxd lxd.daemon[3812]: Kernel supports pidfds
Oct 05 07:13:33 lxd lxd.daemon[3812]: Kernel does not support swap accounting
Oct 05 07:13:33 lxd lxd.daemon[3812]: api_extensions:
Oct 05 07:13:33 lxd lxd.daemon[3812]: - cgroups
Oct 05 07:13:33 lxd lxd.daemon[3812]: - sys_cpu_online
Oct 05 07:13:33 lxd lxd.daemon[3812]: - proc_cpuinfo
Oct 05 07:13:33 lxd lxd.daemon[3812]: - proc_diskstats
Oct 05 07:13:33 lxd lxd.daemon[3812]: - proc_loadavg
Oct 05 07:13:33 lxd lxd.daemon[3812]: - proc_meminfo
Oct 05 07:13:33 lxd lxd.daemon[3812]: - proc_stat
Oct 05 07:13:33 lxd lxd.daemon[3812]: - proc_swaps
Oct 05 07:13:33 lxd lxd.daemon[3812]: - proc_uptime
Oct 05 07:13:33 lxd lxd.daemon[3812]: - shared_pidns
Oct 05 07:13:33 lxd lxd.daemon[3812]: - cpuview_daemon
Oct 05 07:13:33 lxd lxd.daemon[3812]: - loadavg_daemon
Oct 05 07:13:33 lxd lxd.daemon[3812]: - pidfds
Oct 05 07:13:33 lxd lxd.daemon[3812]: Reloaded LXCFS
Oct 05 07:13:34 lxd lxd.daemon[518199]: t=2021-10-05T06:13:34+0000 lvl=eror msg=“Failed to start the daemon” err="Failed initializing storage pool "my-lvm": V>
Oct 05 07:13:34 lxd lxd.daemon[518199]: Error: Failed initializing storage pool “my-lvm”: Volume group my-lvm not found
Oct 05 07:13:34 lxd lxd.daemon[518039]: => LXD failed to start
Oct 05 07:13:34 lxd systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
Oct 05 07:13:34 lxd systemd[1]: snap.lxd.daemon.service: Failed with result ‘exit-code’.
Oct 05 07:13:34 lxd systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 856.
Oct 05 07:13:34 lxd systemd[1]: Stopped Service for snap application lxd.daemon.
Oct 05 07:13:34 lxd systemd[1]: Started Service for snap application lxd.daemon.
Oct 05 07:13:34 lxd lxd.daemon[518569]: => Preparing the system (21624)
Oct 05 07:13:34 lxd lxd.daemon[518569]: ==> Loading snap configuration
Oct 05 07:13:34 lxd lxd.daemon[518569]: ==> Setting up mntns symlink (mnt:[4026533936])