My 4 server cluster stop working.. What should I do to restart it

After running for a year now without an issue it has stopped working. I could no longer do lxc list.
I reboot 3 out of the 4 servers, this usually got it working. One server is still serving web pages and I don’t want to risk this by rebooting it. All the servers are running lxd fine, just I get Error: Get “http://unix.socket/1.0”: EOF when I do lxc list. Some 2 show lxd version 5.2 , 2 other show : 5.3
Any ideas on how to get this party going…

Thanks is advance

What does /var/snap/lxd/common/lxd/logs/lxd.log on the 3 problem servers show?

Q1 says… time=“2022-07-15T20:24:39-04:00” level=warning msg=“Wait for other cluster nodes to upgrade their versions, cluster not started yet”
Q2 says… time=“2022-07-15T20:25:51-04:00” level=warning msg=“Cluster member info not found” address=“84.17.40.59:8443”
time=“2022-07-15T20:25:51-04:00” level=debug msg=“Starting heartbeat round” local=“84.17.40.19:8443” mode=normal
time=“2022-07-15T20:25:51-04:00” level=debug msg=“Heartbeat updating local raft members” members="[{{1 84.17.40.18:8443 voter} Q1} {{2 84.17.40.20:8443 spare} Q3} {{3 84.17.40.19:8443 voter} Q2} {{4 84.17.40.21:8443 voter} Q4} {{5 84.17.40.59:8443 spare} }]"
time=“2022-07-15T20:25:51-04:00” level=error msg=“Unaccounted raft node(s) not found in ‘nodes’ table for heartbeat: {NodeInfo:{ID:5 Address:84.17.40.59:8443 Role:spare} Name:}”
time=“2022-07-15T20:25:51-04:00” level=debug msg=“Matched trusted cert” fingerprint=25d36dc2d614d50d9b4f477363c7f35ad7b0d06fee1b327d65c15ae41999f3be subject=“CN=root@q4,O=linuxcontainers.org
time=“2022-07-15T20:25:51-04:00” level=info msg=“Dqlite proxy started” local=“84.17.40.19:8443” name=dqlite remote=“84.17.40.21:54388”
time=“2022-07-15T20:25:51-04:00” level=debug msg=“Error Response\n\t{\n\t\t"type”: “error”,\n\t\t"status": “”,\n\t\t"status_code": 0,\n\t\t"operation": “”,\n\t\t"error_code": 503,\n\t\t"error": “LXD daemon setup in progress”,\n\t\t"metadata": null\n\t}" http_code=503
time=“2022-07-15T20:25:51-04:00” level=info msg=“Dqlite proxy stopped” local=“84.17.40.19:36908” name=raft remote=“84.17.40.21:8443”
time=“2022-07-15T20:25:51-04:00” level=info msg=“Dqlite proxy stopped” local=“84.17.40.19:8443” name=dqlite remote=“84.17.40.21:54388”
time=“2022-07-15T20:25:51-04:00” level=info msg=“Dqlite proxy stopped” local=“84.17.40.19:8443” name=dqlite remote=“84.17.40.21:54386”
time=“2022-07-15T20:25:52-04:00” level=debug msg=“Matched trusted cert” fingerprint=25d36dc2d614d50d9b4f477363c7f35ad7b0d06fee1b327d65c15ae41999f3be subject=“CN=root@q4,O=linuxcontainers.org
time=“2022-07-15T20:25:52-04:00” level=info msg=“Dqlite proxy started” local=“84.17.40.19:8443” name=dqlite remote=“84.17.40.21:54394”
time=“2022-07-15T20:25:53-04:00” level=info msg=“Dqlite connected outbound” local=“84.17.40.19:36914” name=raft remote=“84.17.40.21:8443”
time=“2022-07-15T20:25:53-04:00” level=info msg=“Dqlite proxy started” local=“84.17.40.19:36914” name=raft remote=“84.17.40.21:8443”
time=“2022-07-15T20:25:53-04:00” level=debug msg=“Matched trusted cert” fingerprint=25d36dc2d614d50d9b4f477363c7f35ad7b0d06fee1b327d65c15ae41999f3be subject=“CN=root@q4,O=linuxcontainers.org
time=“2022-07-15T20:25:53-04:00” level=info msg=“Dqlite proxy started” local=“84.17.40.19:8443” name=dqlite remote=“84.17.40.21:54396
Q3 says… time=“2022-07-14T22:32:55-04:00” level=debug msg=“Initializing database gateway”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Starting database node” id=2 local=“84.17.40.20:8443” role=spare
time=“2022-07-14T22:32:55-04:00” level=info msg=“Starting cluster handler:”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Starting /dev/lxd handler:”
time=“2022-07-14T22:32:55-04:00” level=info msg=” - binding devlxd socket" socket=/var/snap/lxd/common/lxd/devlxd/sock
time=“2022-07-14T22:32:55-04:00” level=info msg=“REST API daemon:”
time=“2022-07-14T22:32:55-04:00” level=info msg=" - binding Unix socket" inherited=true socket=/var/snap/lxd/common/lxd/unix.socket
time=“2022-07-14T22:32:55-04:00” level=info msg=" - binding TCP socket" socket=“84.17.40.20:8443”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Initializing global database”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Connecting to global database”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Dqlite connected outbound” local=“84.17.40.20:36178” name=dqlite remote=“84.17.40.18:8443”
time=“2022-07-14T22:32:55-04:00” level=debug msg=“Dqlite: attempt 1: server 84.17.40.18:8443: connect to reported leader 84.17.40.19:8443”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Dqlite connected outbound” local=“84.17.40.20:60032” name=dqlite remote=“84.17.40.19:8443”
time=“2022-07-14T22:32:55-04:00” level=debug msg=“Dqlite: attempt 1: server 84.17.40.18:8443: connected”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Connected to global database”
time=“2022-07-14T22:32:55-04:00” level=debug msg=“Database error” err=“this node’s version is behind, please upgrade”
time=“2022-07-14T22:32:55-04:00” level=error msg=“Failed to start the daemon” err=“Failed to initialize global database: failed to ensure schema: this node’s version is behind, please upgrade”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Starting shutdown sequence” signal=interrupt
time=“2022-07-14T22:32:55-04:00” level=info msg=“Handing over cluster member role” address=“84.17.40.20:8443”
time=“2022-07-14T22:32:55-04:00” level=debug msg=“Connecting to a remote LXD over HTTPS”
time=“2022-07-14T22:32:55-04:00” level=debug msg=“Sending request to LXD” etag= method=POST url=“https://84.17.40.19:8443/internal/cluster/handover
time=“2022-07-14T22:32:55-04:00” level=debug msg="\n\t{\n\t\t"address": “84.17.40.20:8443”\n\t}"
time=“2022-07-14T22:32:55-04:00” level=warning msg=“Could not handover member’s responsibilities” err=“LXD daemon setup in progress”
time=“2022-07-14T22:32:55-04:00” level=debug msg=“Cancel ongoing or future gRPC connection attempts”
time=“2022-07-14T22:32:55-04:00” level=debug msg=“Cancel ongoing or future gRPC connection attempts”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Stop database gateway”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Stopping REST API handler:”
time=“2022-07-14T22:32:55-04:00” level=info msg=" - closing socket" socket=“84.17.40.20:8443”
time=“2022-07-14T22:32:55-04:00” level=info msg=" - closing socket" socket=/var/snap/lxd/common/lxd/unix.socket
time=“2022-07-14T22:32:55-04:00” level=info msg=“Stopping /dev/lxd handler:”
time=“2022-07-14T22:32:55-04:00” level=info msg=" - closing socket" socket=/var/snap/lxd/common/lxd/devlxd/sock
time=“2022-07-14T22:32:55-04:00” level=info msg=“Not unmounting temporary filesystems (instances are still running)”
time=“2022-07-14T22:32:55-04:00” level=info msg=“Daemon stopped”
Q4 says… time=“2022-07-15T20:25:45-04:00” level=warning msg=" - Couldn’t find the CGroup memory swap accounting, swap limits will be ignored"
time=“2022-07-15T20:25:46-04:00” level=error msg=“Failed to start the daemon” err=“Failed to initialize global database: failed to ensure schema: this node’s version is behind, please upgrade”
time=“2022-07-15T20:25:47-04:00” level=warning msg=“Could not handover member’s responsibilities” err=“LXD daemon setup in progress”
time=“2022-07-15T20:25:48-04:00” level=warning msg=“Dqlite proxy failed” err=“first: local → remote: read unix @->@0c0dc: read: connection reset by peer” local=“84.17.40.21:8443” name=dqlite remote=“84.17.40.19:36902”

So two are complaining about the upgrade status of others… Going to try to upgrade them somehow.

Q1 says
sqlite3 /var/snap/lxd/common/lxd/database/global/db.bin “SELECT * FROM nodes;”
1|Q1||84.17.40.18:8443|60|312|2022-07-04 10:01:32.462403214-04:00|0|2|
2|Q3||84.17.40.20:8443|60|313|2022-07-04 10:06:32.382257763-04:00|0|2|
3|Q2||84.17.40.19:8443|60|315|2022-07-04 10:06:31.618628733-04:00|0|2|
4|Q4||84.17.40.21:8443|60|313|2022-07-04 10:06:34.475008997-04:00|0|2|
Q2 says
sqlite3 /var/snap/lxd/common/lxd/database/global/db.bin “SELECT * FROM nodes;”
1|Q1||84.17.40.18:8443|60|312|2022-07-04 10:01:32.462403214-04:00|0|2|
2|Q3||84.17.40.20:8443|60|313|2022-07-04 10:06:32.382257763-04:00|0|2|
3|Q2||84.17.40.19:8443|60|315|2022-07-04 10:06:31.618628733-04:00|0|2|
4|Q4||84.17.40.21:8443|60|313|2022-07-04 10:06:34.475008997-04:00|0|2|
Q3 Says
1|Q1||84.17.40.18:8443|60|306|2022-05-28 11:00:49.07030894-04:00|0|2|
2|Q3||84.17.40.20:8443|60|306|2022-05-28 11:00:47.088539154-04:00|0|2|
3|Q2||84.17.40.19:8443|60|306|2022-05-28 11:00:39.099070585-04:00|0|2|
4|Q4||84.17.40.21:8443|60|306|2022-05-28 11:00:48.794241424-04:00|0|2|
Q4 says
1|Q1||84.17.40.18:8443|60|315|2022-07-15 20:40:45.916359296-04:00|0|2|
2|Q3||84.17.40.20:8443|60|313|2022-07-04 10:08:02.294602249-04:00|0|2|
3|Q2||84.17.40.19:8443|60|315|2022-07-15 20:40:41.736339108-04:00|0|2|
4|Q4||84.17.40.21:8443|60|313|2022-07-15 20:38:02.471201464-04:00|0|2|

Can you show ‘snap info lxd’ on each server please

Q1snap info lxd
name: lxd
summary: LXD - container and VM manager
publisher: Canonical✓
store-url: Install lxd on Linux | Snap Store
contact: Issues · lxc/lxd · GitHub
license: unset
description: |
LXD is a system container and virtual machine manager.

It offers a simple CLI and REST API to manage local or remote instances,
uses an image based workflow and support for a variety of advanced features.

Images are available for all Ubuntu releases and architectures as well
as for a wide number of other Linux distributions. Existing
integrations with many deployment and operation tools, makes it work
just like a public cloud, except everything is under your control.

LXD containers are lightweight, secure by default and a great
alternative to virtual machines when running Linux on Linux.

LXD virtual machines are modern and secure, using UEFI and secure-boot
by default and a great choice when a different kernel or operating
system is needed.

With clustering, up to 50 LXD servers can be easily joined and managed
together with the same tools and APIs and without needing any external
dependencies.

Supported configuration options for the snap (snap set lxd [=…]):

- ceph.builtin: Use snap-specific Ceph configuration [default=false]
- ceph.external: Use the system's ceph tools (ignores ceph.builtin) [default=false]
- criu.enable: Enable experimental live-migration support [default=false]
- daemon.debug: Increase logging to debug level [default=false]
- daemon.group: Set group of users that have full control over LXD [default=lxd]
- daemon.user.group: Set group of users that have restricted LXD access [default=lxd]
- daemon.preseed: Pass a YAML configuration to `lxd init` on initial start
- daemon.syslog: Send LXD log events to syslog [default=false]
- daemon.verbose: Increase logging to verbose level [default=false]
- lvm.external: Use the system's LVM tools [default=false]
- lxcfs.pidfd: Start per-container process tracking [default=false]
- lxcfs.loadavg: Start tracking per-container load average [default=false]
- lxcfs.cfs: Consider CPU shares for CPU usage [default=false]
- openvswitch.builtin: Run a snap-specific OVS daemon [default=false]
- openvswitch.external: Use the system's OVS tools (ignores openvswitch.builtin) [default=false]
- ovn.builtin: Use snap-specific OVN configuration [default=false]
- shiftfs.enable: Enable shiftfs support [default=auto]

For system-wide configuration of the CLI, place your configuration in
/var/snap/lxd/common/global-conf/ (config.yml and servercerts)
commands:

  • lxd.benchmark
  • lxd.buginfo
  • lxd.check-kernel
  • lxd.lxc
  • lxd.lxc-to-lxd
  • lxd
  • lxd.migrate
    services:
    lxd.activate: oneshot, enabled, inactive
    lxd.daemon: simple, enabled, active
    lxd.user-daemon: simple, enabled, inactive
    snap-id: J60k4JY0HppjwOjW8dZdYc8obXKxujRu
    tracking: latest/stable
    refresh-date: 12 days ago, at 10:06 EDT
    channels:
    latest/stable: 5.3-91e042b 2022-07-06 (23270) 107MB -
    latest/candidate: 5.3-b403e7f 2022-07-05 (23282) 107MB -
    latest/beta: ↑
    latest/edge: git-df84330 2022-07-15 (23321) 107MB -
    5.3/stable: 5.3-91e042b 2022-07-06 (23270) 107MB -
    5.3/candidate: 5.3-b403e7f 2022-07-05 (23282) 107MB -
    5.3/beta: ↑
    5.3/edge: ↑
    5.2/stable: 5.2-79c3c3b 2022-06-01 (23155) 106MB -
    5.2/candidate: 5.2-79c3c3b 2022-05-27 (23155) 106MB -
    5.2/beta: ↑
    5.2/edge: ↑
    5.1/stable: 5.1-1f6f485 2022-05-12 (23037) 84MB -
    5.1/candidate: 5.1-1c377db 2022-05-17 (23101) 84MB -
    5.1/beta: ↑
    5.1/edge: ↑
    5.0/stable: 5.0.0-b0287c1 2022-04-20 (22923) 83MB -
    5.0/candidate: 5.0.0-b0287c1 2022-04-19 (22923) 83MB -
    5.0/beta: ↑
    5.0/edge: ↑
    4.24/stable: 4.24-c92c0b2 2022-04-01 (22754) 82MB -
    4.24/candidate: 4.24-c92c0b2 2022-04-01 (22754) 82MB -
    4.24/beta: ↑
    4.24/edge: ↑
    4.0/stable: 4.0.9-8e2046b 2022-03-26 (22753) 71MB -
    4.0/candidate: 4.0.9-8e2046b 2022-03-24 (22753) 71MB -
    4.0/beta: ↑
    4.0/edge: git-407205d 2022-03-31 (22797) 73MB -
    3.0/stable: 3.0.4 2019-10-10 (11348) 55MB -
    3.0/candidate: 3.0.4 2019-10-10 (11348) 55MB -
    3.0/beta: ↑
    3.0/edge: git-81b81b9 2019-10-10 (11362) 55MB -
    installed: 5.3-924be6a (23243) 107MB in-cohort

Q2
snap info lxd
name: lxd
summary: LXD - container and VM manager
publisher: Canonical✓
store-url: Install lxd on Linux | Snap Store
contact: Issues · lxc/lxd · GitHub
license: unset
description: |
LXD is a system container and virtual machine manager.

It offers a simple CLI and REST API to manage local or remote instances,
uses an image based workflow and support for a variety of advanced features.

Images are available for all Ubuntu releases and architectures as well
as for a wide number of other Linux distributions. Existing
integrations with many deployment and operation tools, makes it work
just like a public cloud, except everything is under your control.

LXD containers are lightweight, secure by default and a great
alternative to virtual machines when running Linux on Linux.

LXD virtual machines are modern and secure, using UEFI and secure-boot
by default and a great choice when a different kernel or operating
system is needed.

With clustering, up to 50 LXD servers can be easily joined and managed
together with the same tools and APIs and without needing any external
dependencies.

Supported configuration options for the snap (snap set lxd [=…]):

- ceph.builtin: Use snap-specific Ceph configuration [default=false]
- ceph.external: Use the system's ceph tools (ignores ceph.builtin) [default=false]
- criu.enable: Enable experimental live-migration support [default=false]
- daemon.debug: Increase logging to debug level [default=false]
- daemon.group: Set group of users that have full control over LXD [default=lxd]
- daemon.user.group: Set group of users that have restricted LXD access [default=lxd]
- daemon.preseed: Pass a YAML configuration to `lxd init` on initial start
- daemon.syslog: Send LXD log events to syslog [default=false]
- daemon.verbose: Increase logging to verbose level [default=false]
- lvm.external: Use the system's LVM tools [default=false]
- lxcfs.pidfd: Start per-container process tracking [default=false]
- lxcfs.loadavg: Start tracking per-container load average [default=false]
- lxcfs.cfs: Consider CPU shares for CPU usage [default=false]
- openvswitch.builtin: Run a snap-specific OVS daemon [default=false]
- openvswitch.external: Use the system's OVS tools (ignores openvswitch.builtin) [default=false]
- ovn.builtin: Use snap-specific OVN configuration [default=false]
- shiftfs.enable: Enable shiftfs support [default=auto]

For system-wide configuration of the CLI, place your configuration in
/var/snap/lxd/common/global-conf/ (config.yml and servercerts)
commands:

  • lxd.benchmark
  • lxd.buginfo
  • lxd.check-kernel
  • lxd.lxc
  • lxd.lxc-to-lxd
  • lxd
  • lxd.migrate
    services:
    lxd.activate: oneshot, enabled, inactive
    lxd.daemon: simple, enabled, active
    lxd.user-daemon: simple, enabled, inactive
    snap-id: J60k4JY0HppjwOjW8dZdYc8obXKxujRu
    tracking: latest/stable
    refresh-date: 18 days ago, at 20:35 EDT
    channels:
    latest/stable: 5.3-91e042b 2022-07-06 (23270) 107MB -
    latest/candidate: 5.3-b403e7f 2022-07-05 (23282) 107MB -
    latest/beta: ↑
    latest/edge: git-df84330 2022-07-15 (23321) 107MB -
    5.3/stable: 5.3-91e042b 2022-07-06 (23270) 107MB -
    5.3/candidate: 5.3-b403e7f 2022-07-05 (23282) 107MB -
    5.3/beta: ↑
    5.3/edge: ↑
    5.2/stable: 5.2-79c3c3b 2022-06-01 (23155) 106MB -
    5.2/candidate: 5.2-79c3c3b 2022-05-27 (23155) 106MB -
    5.2/beta: ↑
    5.2/edge: ↑
    5.1/stable: 5.1-1f6f485 2022-05-12 (23037) 84MB -
    5.1/candidate: 5.1-1c377db 2022-05-17 (23101) 84MB -
    5.1/beta: ↑
    5.1/edge: ↑
    5.0/stable: 5.0.0-b0287c1 2022-04-20 (22923) 83MB -
    5.0/candidate: 5.0.0-b0287c1 2022-04-19 (22923) 83MB -
    5.0/beta: ↑
    5.0/edge: ↑
    4.24/stable: 4.24-c92c0b2 2022-04-01 (22754) 82MB -
    4.24/candidate: 4.24-c92c0b2 2022-04-01 (22754) 82MB -
    4.24/beta: ↑
    4.24/edge: ↑
    4.0/stable: 4.0.9-8e2046b 2022-03-26 (22753) 71MB -
    4.0/candidate: 4.0.9-8e2046b 2022-03-24 (22753) 71MB -
    4.0/beta: ↑
    4.0/edge: git-407205d 2022-03-31 (22797) 73MB -
    3.0/stable: 3.0.4 2019-10-10 (11348) 55MB -
    3.0/candidate: 3.0.4 2019-10-10 (11348) 55MB -
    3.0/beta: ↑
    3.0/edge: git-81b81b9 2019-10-10 (11362) 55MB -
    installed: 5.3-924be6a (23243) 107MB in-cohort

Q3
snap info lxd
name: lxd
summary: LXD - container and VM manager
publisher: Canonical✓
store-url: Install lxd on Linux | Snap Store
contact: Issues · lxc/lxd · GitHub
license: unset
description: |
LXD is a system container and virtual machine manager.

It offers a simple CLI and REST API to manage local or remote instances,
uses an image based workflow and support for a variety of advanced features.

Images are available for all Ubuntu releases and architectures as well
as for a wide number of other Linux distributions. Existing
integrations with many deployment and operation tools, makes it work
just like a public cloud, except everything is under your control.

LXD containers are lightweight, secure by default and a great
alternative to virtual machines when running Linux on Linux.

LXD virtual machines are modern and secure, using UEFI and secure-boot
by default and a great choice when a different kernel or operating
system is needed.

With clustering, up to 50 LXD servers can be easily joined and managed
together with the same tools and APIs and without needing any external
dependencies.

Supported configuration options for the snap (snap set lxd [=…]):

- ceph.builtin: Use snap-specific Ceph configuration [default=false]
- ceph.external: Use the system's ceph tools (ignores ceph.builtin) [default=false]
- criu.enable: Enable experimental live-migration support [default=false]
- daemon.debug: Increase logging to debug level [default=false]
- daemon.group: Set group of users that have full control over LXD [default=lxd]
- daemon.user.group: Set group of users that have restricted LXD access [default=lxd]
- daemon.preseed: Pass a YAML configuration to `lxd init` on initial start
- daemon.syslog: Send LXD log events to syslog [default=false]
- lvm.external: Use the system's LVM tools [default=false]
- lxcfs.pidfd: Start per-container process tracking [default=false]
- lxcfs.loadavg: Start tracking per-container load average [default=false]
- lxcfs.cfs: Consider CPU shares for CPU usage [default=false]
- openvswitch.builtin: Run a snap-specific OVS daemon [default=false]
- openvswitch.external: Use the system's OVS tools (ignores openvswitch.builtin) [default=false]
- ovn.builtin: Use snap-specific OVN configuration [default=false]
- shiftfs.enable: Enable shiftfs support [default=auto]

For system-wide configuration of the CLI, place your configuration in
/var/snap/lxd/common/global-conf/ (config.yml and servercerts)
commands:

  • lxd.benchmark
  • lxd.buginfo
  • lxd.check-kernel
  • lxd.lxc
  • lxd.lxc-to-lxd
  • lxd
  • lxd.migrate
    services:
    lxd.activate: oneshot, enabled, inactive
    lxd.daemon: simple, enabled, inactive
    lxd.user-daemon: simple, enabled, inactive
    snap-id: J60k4JY0HppjwOjW8dZdYc8obXKxujRu
    tracking: latest/stable
    refresh-date: 21 days ago, at 11:01 EDT
    channels:
    latest/stable: 5.3-91e042b 2022-07-06 (23270) 107MB -
    latest/candidate: 5.3-b403e7f 2022-07-05 (23282) 107MB -
    latest/beta: ↑
    latest/edge: git-df84330 2022-07-15 (23321) 107MB -
    5.3/stable: 5.3-91e042b 2022-07-06 (23270) 107MB -
    5.3/candidate: 5.3-b403e7f 2022-07-05 (23282) 107MB -
    5.3/beta: ↑
    5.3/edge: ↑
    5.2/stable: 5.2-79c3c3b 2022-06-01 (23155) 106MB -
    5.2/candidate: 5.2-79c3c3b 2022-05-27 (23155) 106MB -
    5.2/beta: ↑
    5.2/edge: ↑
    5.1/stable: 5.1-1f6f485 2022-05-12 (23037) 84MB -
    5.1/candidate: 5.1-1c377db 2022-05-17 (23101) 84MB -
    5.1/beta: ↑
    5.1/edge: ↑
    5.0/stable: 5.0.0-b0287c1 2022-04-20 (22923) 83MB -
    5.0/candidate: 5.0.0-b0287c1 2022-04-19 (22923) 83MB -
    5.0/beta: ↑
    5.0/edge: ↑
    4.24/stable: 4.24-c92c0b2 2022-04-01 (22754) 82MB -
    4.24/candidate: 4.24-c92c0b2 2022-04-01 (22754) 82MB -
    4.24/beta: ↑
    4.24/edge: ↑
    4.0/stable: 4.0.9-8e2046b 2022-03-26 (22753) 71MB -
    4.0/candidate: 4.0.9-8e2046b 2022-03-24 (22753) 71MB -
    4.0/beta: ↑
    4.0/edge: git-407205d 2022-03-31 (22797) 73MB -
    3.0/stable: 3.0.4 2019-10-10 (11348) 55MB -
    3.0/candidate: 3.0.4 2019-10-10 (11348) 55MB -
    3.0/beta: ↑
    3.0/edge: git-81b81b9 2019-10-10 (11362) 55MB -
    installed: 5.2-79c3c3b (23155) 106MB in-cohort

Q4
snap info lxd
name: lxd
summary: LXD - container and VM manager
publisher: Canonical✓
store-url: Install lxd on Linux | Snap Store
contact: Issues · lxc/lxd · GitHub
license: unset
description: |
LXD is a system container and virtual machine manager.

It offers a simple CLI and REST API to manage local or remote instances,
uses an image based workflow and support for a variety of advanced features.

Images are available for all Ubuntu releases and architectures as well
as for a wide number of other Linux distributions. Existing
integrations with many deployment and operation tools, makes it work
just like a public cloud, except everything is under your control.

LXD containers are lightweight, secure by default and a great
alternative to virtual machines when running Linux on Linux.

LXD virtual machines are modern and secure, using UEFI and secure-boot
by default and a great choice when a different kernel or operating
system is needed.

With clustering, up to 50 LXD servers can be easily joined and managed
together with the same tools and APIs and without needing any external
dependencies.

Supported configuration options for the snap (snap set lxd [=…]):

- ceph.builtin: Use snap-specific Ceph configuration [default=false]
- ceph.external: Use the system's ceph tools (ignores ceph.builtin) [default=false]
- criu.enable: Enable experimental live-migration support [default=false]
- daemon.debug: Increase logging to debug level [default=false]
- daemon.group: Set group of users that have full control over LXD [default=lxd]
- daemon.user.group: Set group of users that have restricted LXD access [default=lxd]
- daemon.preseed: Pass a YAML configuration to `lxd init` on initial start
- daemon.syslog: Send LXD log events to syslog [default=false]
- lvm.external: Use the system's LVM tools [default=false]
- lxcfs.pidfd: Start per-container process tracking [default=false]
- lxcfs.loadavg: Start tracking per-container load average [default=false]
- lxcfs.cfs: Consider CPU shares for CPU usage [default=false]
- openvswitch.builtin: Run a snap-specific OVS daemon [default=false]
- openvswitch.external: Use the system's OVS tools (ignores openvswitch.builtin) [default=false]
- ovn.builtin: Use snap-specific OVN configuration [default=false]
- shiftfs.enable: Enable shiftfs support [default=auto]

For system-wide configuration of the CLI, place your configuration in
/var/snap/lxd/common/global-conf/ (config.yml and servercerts)
commands:

  • lxd.benchmark
  • lxd.buginfo
  • lxd.check-kernel
  • lxd.lxc
  • lxd.lxc-to-lxd
  • lxd
  • lxd.migrate
    services:
    lxd.activate: oneshot, enabled, inactive
    lxd.daemon: simple, enabled, inactive
    lxd.user-daemon: simple, enabled, inactive
    snap-id: J60k4JY0HppjwOjW8dZdYc8obXKxujRu
    tracking: latest/stable
    refresh-date: 21 days ago, at 11:03 EDT
    channels:
    latest/stable: 5.3-91e042b 2022-07-06 (23270) 107MB -
    latest/candidate: 5.3-b403e7f 2022-07-05 (23282) 107MB -
    latest/beta: ↑
    latest/edge: git-df84330 2022-07-15 (23321) 107MB -
    5.3/stable: 5.3-91e042b 2022-07-06 (23270) 107MB -
    5.3/candidate: 5.3-b403e7f 2022-07-05 (23282) 107MB -
    5.3/beta: ↑
    5.3/edge: ↑
    5.2/stable: 5.2-79c3c3b 2022-06-01 (23155) 106MB -
    5.2/candidate: 5.2-79c3c3b 2022-05-27 (23155) 106MB -
    5.2/beta: ↑
    5.2/edge: ↑
    5.1/stable: 5.1-1f6f485 2022-05-12 (23037) 84MB -
    5.1/candidate: 5.1-1c377db 2022-05-17 (23101) 84MB -
    5.1/beta: ↑
    5.1/edge: ↑
    5.0/stable: 5.0.0-b0287c1 2022-04-20 (22923) 83MB -
    5.0/candidate: 5.0.0-b0287c1 2022-04-19 (22923) 83MB -
    5.0/beta: ↑
    5.0/edge: ↑
    4.24/stable: 4.24-c92c0b2 2022-04-01 (22754) 82MB -
    4.24/candidate: 4.24-c92c0b2 2022-04-01 (22754) 82MB -
    4.24/beta: ↑
    4.24/edge: ↑
    4.0/stable: 4.0.9-8e2046b 2022-03-26 (22753) 71MB -
    4.0/candidate: 4.0.9-8e2046b 2022-03-24 (22753) 71MB -
    4.0/beta: ↑
    4.0/edge: git-407205d 2022-03-31 (22797) 73MB -
    3.0/stable: 3.0.4 2019-10-10 (11348) 55MB -
    3.0/candidate: 3.0.4 2019-10-10 (11348) 55MB -
    3.0/beta: ↑
    3.0/edge: git-81b81b9 2019-10-10 (11362) 55MB -
    installed: 5.2-79c3c3b (23155) 106MB in-cohort

Have you run snap refresh lxd on q3 and q4 as they are behind.

snap refresh lxd on all of them seem to fix the problem. Thanks

1 Like