Upgrading a different cluster to .5, one machine wont upgrade, others stuck

Pardon the whole sordid story below, but figured best way to explain. The Other three upgrade fine, well actually still stuck waiting for this one.

root@Q3:/home/ic2000# apt upgrade
Reading package lists… Done
Building dependency tree
Reading state information… Done
Calculating upgrade… Done
Get more security updates through Ubuntu Pro with ‘esm-apps’ enabled:
libavformat58 python2.7-dev libavfilter7 mc libmagickcore-6.q16-6-extra sntp
imagemagick libswresample3 libzmq5 python2.7-minimal libmagickwand-6.q16-6
libpython2.7 python2.7 libpython2.7-dev mc-data libpostproc55 ntp
imagemagick-6.q16 libavcodec58 libavutil56 libswscale5 libopenexr24
libmysofa1 libmagickcore-6.q16-6 libpython2.7-minimal libpython2.7-stdlib
imagemagick-6-common
Learn more about Ubuntu Pro at Ubuntu Pro | Ubuntu
The following NEW packages will be installed:
incus-base nftables
The following packages will be upgraded:
incus incus-client libnss-myhostname libnss-systemd libpam-systemd libsystemd0 libudev1 systemd systemd-sysv udev
10 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 91.3 MB of archives.
After this operation, 21.0 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 Index of /ubuntu focal-updates/main amd64 libnss-systemd amd64 245.4-4ubuntu3.23 [96.2 kB]
Get:2 Index of /ubuntu focal-updates/main amd64 udev amd64 245.4-4ubuntu3.23 [1,366 kB]
Get:3 Index of /incus/stable/ focal/main amd64 incus amd64 0.5-202401260134-ubuntu20.04 [25.0 MB]
Get:4 Index of /ubuntu focal-updates/main amd64 libudev1 amd64 245.4-4ubuntu3.23 [75.6 kB]
Get:5 Index of /ubuntu focal-updates/main amd64 systemd-sysv amd64 245.4-4ubuntu3.23 [10.3 kB]
Get:6 Index of /ubuntu focal-updates/main amd64 libpam-systemd amd64 245.4-4ubuntu3.23 [186 kB]
Get:7 Index of /ubuntu focal-updates/main amd64 systemd amd64 245.4-4ubuntu3.23 [3,811 kB]
Get:8 Index of /ubuntu focal-updates/main amd64 libsystemd0 amd64 245.4-4ubuntu3.23 [268 kB]
Get:9 Index of /ubuntu focal/universe amd64 nftables amd64 0.9.3-2 [51.2 kB]
Get:10 Index of /ubuntu focal-updates/main amd64 libnss-myhostname amd64 245.4-4ubuntu3.23 [41.4 kB]
Get:11 Index of /incus/stable/ focal/main amd64 incus-client amd64 0.5-202401260134-ubuntu20.04 [4,342 kB]
Get:12 Index of /incus/stable/ focal/main amd64 incus-base amd64 0.5-202401260134-ubuntu20.04 [56.1 MB]
Fetched 91.3 MB in 5s (19.3 MB/s)
(Reading database … 196559 files and directories currently installed.)
Preparing to unpack …/libnss-systemd_245.4-4ubuntu3.23_amd64.deb …
Unpacking libnss-systemd:amd64 (245.4-4ubuntu3.23) over (245.4-4ubuntu3.22) …
Preparing to unpack …/udev_245.4-4ubuntu3.23_amd64.deb …
Unpacking udev (245.4-4ubuntu3.23) over (245.4-4ubuntu3.22) …
Preparing to unpack …/libudev1_245.4-4ubuntu3.23_amd64.deb …
Unpacking libudev1:amd64 (245.4-4ubuntu3.23) over (245.4-4ubuntu3.22) …
Setting up libudev1:amd64 (245.4-4ubuntu3.23) …
(Reading database … 196559 files and directories currently installed.)
Preparing to unpack …/systemd-sysv_245.4-4ubuntu3.23_amd64.deb …
Unpacking systemd-sysv (245.4-4ubuntu3.23) over (245.4-4ubuntu3.22) …
Preparing to unpack …/libpam-systemd_245.4-4ubuntu3.23_amd64.deb …
Unpacking libpam-systemd:amd64 (245.4-4ubuntu3.23) over (245.4-4ubuntu3.22) …
Preparing to unpack …/systemd_245.4-4ubuntu3.23_amd64.deb …
Unpacking systemd (245.4-4ubuntu3.23) over (245.4-4ubuntu3.22) …
Preparing to unpack …/libsystemd0_245.4-4ubuntu3.23_amd64.deb …
Unpacking libsystemd0:amd64 (245.4-4ubuntu3.23) over (245.4-4ubuntu3.22) …
Setting up libsystemd0:amd64 (245.4-4ubuntu3.23) …
Selecting previously unselected package nftables.
(Reading database … 196559 files and directories currently installed.)
Preparing to unpack …/nftables_0.9.3-2_amd64.deb …
Unpacking nftables (0.9.3-2) …
Preparing to unpack …/incus_0.5-202401260134-ubuntu20.04_amd64.deb …
Unpacking incus (0.5-202401260134-ubuntu20.04) over (0.4-202312232117-ubuntu20.04) …
dpkg: warning: unable to delete old directory ‘/var/log/incus’: Directory not empty
dpkg: warning: unable to delete old directory ‘/var/lib/incus’: Directory not empty
Preparing to unpack …/incus-client_0.5-202401260134-ubuntu20.04_amd64.deb …
Unpacking incus-client (0.5-202401260134-ubuntu20.04) over (0.4-202312232117-ubuntu20.04) …
Selecting previously unselected package incus-base.
Preparing to unpack …/incus-base_0.5-202401260134-ubuntu20.04_amd64.deb …
Unpacking incus-base (0.5-202401260134-ubuntu20.04) …
Preparing to unpack …/libnss-myhostname_245.4-4ubuntu3.23_amd64.deb …
Unpacking libnss-myhostname:amd64 (245.4-4ubuntu3.23) over (245.4-4ubuntu3.22) …
Setting up nftables (0.9.3-2) …
Setting up systemd (245.4-4ubuntu3.23) …
Setting up udev (245.4-4ubuntu3.23) …
update-initramfs: deferring update (trigger activated)
Setting up libnss-myhostname:amd64 (245.4-4ubuntu3.23) …
Setting up incus-client (0.5-202401260134-ubuntu20.04) …
Setting up systemd-sysv (245.4-4ubuntu3.23) …
Setting up incus-base (0.5-202401260134-ubuntu20.04) …
Job failed. See “journalctl -xe” for details.
dpkg: error processing package incus-base (–configure):
installed incus-base package post-installation script subprocess returned error exit status 1
Setting up libnss-systemd:amd64 (245.4-4ubuntu3.23) …
dpkg: dependency problems prevent configuration of incus:
incus depends on incus-base (= 0.5-202401260134-ubuntu20.04); however:
Package incus-base is not configured yet.

dpkg: error processing package incus (–configure):
dependency problems - leaving unconfigured
Setting up libpam-systemd:amd64 (245.4-4ubuntu3.23) …
No apport report written because the error message indicates its a followup error from a previous failure.
Processing triggers for man-db (2.
9.1-1) …
Processing triggers for ureadahead (0.100.0-21) …
Processing triggers for dbus (1.12.16-2ubuntu2.3) …
Processing triggers for libc-bin (2.31-0ubuntu9.14) …
Processing triggers for initramfs-tools (0.136ubuntu6.7) …
update-initramfs: Generating /boot/initrd.img-5.4.0-170-generic
Errors were encountered while processing:
incus-base
incus
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@Q3:/home/ic2000# apt upgrade
Reading package lists… Done
Building dependency tree
Reading state information… Done
Calculating upgrade… Done
Get more security updates through Ubuntu Pro with ‘esm-apps’ enabled:
libavformat58 python2.7-dev libavfilter7 mc libmagickcore-6.q16-6-extra sntp
imagemagick libswresample3 libzmq5 python2.7-minimal libmagickwand-6.q16-6
libpython2.7 python2.7 libpython2.7-dev mc-data libpostproc55 ntp
imagemagick-6.q16 libavcodec58 libavutil56 libswscale5 libopenexr24
libmysofa1 libmagickcore-6.q16-6 libpython2.7-minimal libpython2.7-stdlib
imagemagick-6-common
Learn more about Ubuntu Pro at Ubuntu Pro | Ubuntu
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
2 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up incus-base (0.5-202401260134-ubuntu20.04) …
Job failed. See “journalctl -xe” for details.
dpkg: error processing package incus-base (–configure):
installed incus-base package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of incus:
incus depends on incus-base (= 0.5-202401260134-ubuntu20.04); however:
Package incus-base is not configured yet.

dpkg: error processing package incus (–configure):
dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
Processing triggers for libc-bin (
2.31-0ubuntu9.14) …
Errors were encountered while processing:
incus-base
incus
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@Q3:/home/ic2000#
root@Q3:/home/ic2000#
root@Q3:/home/ic2000#
root@Q3:/home/ic2000#
root@Q3:/home/ic2000#
root@Q3:/home/ic2000# apt update
Hit:1 Index of /ubuntu focal InRelease
Hit:2 Index of /ubuntu focal-updates InRelease
Hit:3 Index of /ubuntu focal-backports InRelease
Hit:4 Index of /ubuntu focal-security InRelease
Hit:5 Index of /incus/stable/ focal InRelease
Reading package lists… Done
Building dependency tree
Reading state information… Done
All packages are up to date.
root@Q3:/home/ic2000# apt upgrade
Reading package lists… Done
Building dependency tree
Reading state information… Done
Calculating upgrade… Done
Get more security updates through Ubuntu Pro with ‘esm-apps’ enabled:
libavformat58 python2.7-dev libavfilter7 mc libmagickcore-6.q16-6-extra sntp
imagemagick libswresample3 libzmq5 python2.7-minimal libmagickwand-6.q16-6
libpython2.7 python2.7 libpython2.7-dev mc-data libpostproc55 ntp
imagemagick-6.q16 libavcodec58 libavutil56 libswscale5 libopenexr24
libmysofa1 libmagickcore-6.q16-6 libpython2.7-minimal libpython2.7-stdlib
imagemagick-6-common
Learn more about Ubuntu Pro at Ubuntu Pro | Ubuntu
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
2 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up incus-base (0.5-202401260134-ubuntu20.04) …
Job failed. See “journalctl -xe” for details.
dpkg: error processing package incus-base (–configure):
installed incus-base package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of incus:
incus depends on incus-base (= 0.5-202401260134-ubuntu20.04); however:
Package incus-base is not configured yet.

dpkg: error processing package incus (–configure):
dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
Processing triggers for libc-bin (
2.31-0ubuntu9.14) …
Errors were encountered while processing:
incus-base
incus
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@Q3:/home/ic2000# sudo dpkg --configure -a
Setting up incus-base (0.5-202401260134-ubuntu20.04) …
Job failed. See “journalctl -xe” for details.
dpkg: error processing package incus-base (–configure):
installed incus-base package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of incus:
incus depends on incus-base (= 0.5-202401260134-ubuntu20.04); however:
Package incus-base is not configured yet.

dpkg: error processing package incus (–configure):
dependency problems - leaving unconfigured
Processing triggers for libc-bin (2.31-0ubuntu9.14) …
Errors were encountered while processing:
incus-base
incus
root@Q3:/home/ic2000# sudo dpkg --remove --force-remove-reinstreq incus-base
dpkg: dependency problems prevent removal of incus-base:
incus depends on incus-base (= 0.5-202401260134-ubuntu20.04).

dpkg: error processing package incus-base (–remove):
dependency problems - not removing
Errors were encountered while processing:
incus-base
root@Q3:/home/ic2000# sudo dpkg --remove --force-remove-reinstreq incus
(Reading database … 196631 files and directories currently installed.)
Removing incus (0.5-202401260134-ubuntu20.04) …
root@Q3:/home/ic2000# sudo apt-get clean
root@Q3:/home/ic2000# sudo apt-get update
Hit:1 Index of /ubuntu focal InRelease
Hit:2 Index of /ubuntu focal-updates InRelease
Hit:3 Index of /ubuntu focal-backports InRelease
Hit:4 Index of /ubuntu focal-security InRelease
Hit:5 Index of /incus/stable/ focal InRelease
Reading package lists… Done
root@Q3:/home/ic2000# sudo apt-get install incus-base incus
Reading package lists… Done
Building dependency tree
Reading state information… Done
incus-base is already the newest version (0.5-202401260134-ubuntu20.04).
incus-base set to manually installed.
The following NEW packages will be installed:
incus
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
Need to get 25.0 MB of archives.
After this operation, 68.3 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 Index of /incus/stable/ focal/main amd64 incus amd64 0.5-202401260134-ubuntu20.04 [25.0 MB]
Fetched 25.0 MB in 2s (16.2 MB/s)
Selecting previously unselected package incus.
(Reading database … 196573 files and directories currently installed.)
Preparing to unpack …/incus_0.5-202401260134-ubuntu20.04_amd64.deb …
Unpacking incus (0.5-202401260134-ubuntu20.04) …
Setting up incus-base (0.5-202401260134-ubuntu20.04) …
Job failed. See “journalctl -xe” for details.
dpkg: error processing package incus-base (–configure):
installed incus-base package post-installation script subprocess returned error exit status 1
No apport report written because the error message indicates its a followup error from a previous failure.
dpkg: dependency problems prevent configuration of incus:
incus depends on incus-base (= 0.5-202401260134-ubuntu20.04); however:
Package incus-base is not configured yet.

dpkg: error processing package incus (–configure):
dependency problems - leaving unconfigured
Processing triggers for libc-bin (2.31-0ubuntu9.14) …
Errors were encountered while processing:
incus-base
incus
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@Q3:/home/ic2000# journalctl -xe
Jan 28 04:28:24 Q3 sshd[222211]: Invalid user user from 23.94.143.132 port 55210
Jan 28 04:28:24 Q3 sshd[222211]: pam_unix(sshd:auth): check pass; user unknown
Jan 28 04:28:24 Q3 sshd[222211]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=23.94.143.132
Jan 28 04:28:25 Q3 incusd[7335]: time=“2024-01-28T04:28:25-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:28:25 Q3 incusd[7335]: time=“2024-01-28T04:28:25-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:28:25 Q3 incusd[7335]: time=“2024-01-28T04:28:25-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:28:25 Q3 sshd[222213]: Invalid user sftpuser from 221.214.2.98 port 2462
Jan 28 04:28:25 Q3 sshd[222213]: pam_unix(sshd:auth): check pass; user unknown
Jan 28 04:28:25 Q3 sshd[222213]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=221.214.2.98
Jan 28 04:28:25 Q3 smbd[222210]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:28:26 Q3 smbd[222209]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:28:26 Q3 sshd[222211]: Failed password for invalid user user from 23.94.143.132 port 55210 ssh2
Jan 28 04:28:27 Q3 sshd[222213]: Failed password for invalid user sftpuser from 221.214.2.98 port 2462 ssh2
Jan 28 04:28:27 Q3 sshd[222211]: Received disconnect from 23.94.143.132 port 55210:11: Bye Bye [preauth]
Jan 28 04:28:27 Q3 sshd[222211]: Disconnected from invalid user user 23.94.143.132 port 55210 [preauth]
Jan 28 04:28:27 Q3 sshd[222213]: Received disconnect from 221.214.2.98 port 2462:11: Bye Bye [preauth]
Jan 28 04:28:27 Q3 sshd[222213]: Disconnected from invalid user sftpuser 221.214.2.98 port 2462 [preauth]
Jan 28 04:28:30 Q3 incusd[7335]: time=“2024-01-28T04:28:30-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 04:28:32 Q3 sshd[222217]: Invalid user postgres from 189.4.10.114 port 33474
Jan 28 04:28:32 Q3 sshd[222217]: pam_unix(sshd:auth): check pass; user unknown
Jan 28 04:28:32 Q3 sshd[222217]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=189.4.10.114
Jan 28 04:28:32 Q3 smbd[222219]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:28:33 Q3 smbd[222222]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:28:33 Q3 sshd[222217]: Failed password for invalid user postgres from 189.4.10.114 port 33474 ssh2
Jan 28 04:28:34 Q3 sshd[222220]: Invalid user admin from 196.0.120.211 port 54906
Jan 28 04:28:34 Q3 sshd[222220]: pam_unix(sshd:auth): check pass; user unknown
Jan 28 04:28:34 Q3 sshd[222220]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=196.0.120.211
Jan 28 04:28:34 Q3 sshd[222217]: Received disconnect from 189.4.10.114 port 33474:11: Bye Bye [preauth]
Jan 28 04:28:34 Q3 sshd[222217]: Disconnected from invalid user postgres 189.4.10.114 port 33474 [preauth]
Jan 28 04:28:35 Q3 smbd[222224]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:28:35 Q3 incusd[7335]: time=“2024-01-28T04:28:35-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:28:35 Q3 incusd[7335]: time=“2024-01-28T04:28:35-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:28:35 Q3 incusd[7335]: time=“2024-01-28T04:28:35-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:28:36 Q3 sshd[222220]: Failed password for invalid user admin from 196.0.120.211 port 54906 ssh2
Jan 28 04:28:36 Q3 sshd[222225]: error: kex_exchange_identification: Connection closed by remote host
Jan 28 04:21:45 Q3 sshd[220686]: pam_unix(sshd:auth): check pass; user unknown
Jan 28 04:21:45 Q3 sshd[220686]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=123.207.59.181
Jan 28 04:21:45 Q3 smbd[220692]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:21:47 Q3 incusd[7335]: time=“2024-01-28T04:21:47-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:21:47 Q3 incusd[7335]: time=“2024-01-28T04:21:47-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:21:47 Q3 incusd[7335]: time=“2024-01-28T04:21:47-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:21:47 Q3 sshd[220694]: Invalid user test from 221.214.2.98 port 2443
Jan 28 04:21:47 Q3 sshd[220694]: pam_unix(sshd:auth): check pass; user unknown
Jan 28 04:21:47 Q3 sshd[220694]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=221.214.2.98
Jan 28 04:21:47 Q3 sshd[220686]: Failed password for invalid user botuser from 123.207.59.181 port 48114 ssh2
Jan 28 04:21:47 Q3 smbd[220696]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:21:48 Q3 smbd[220688]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:21:49 Q3 smbd[220693]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:21:50 Q3 sshd[220694]: Failed password for invalid user test from 221.214.2.98 port 2443 ssh2
Jan 28 04:21:50 Q3 sshd[220686]: Received disconnect from 123.207.59.181 port 48114:11: Bye Bye [preauth]
Jan 28 04:21:50 Q3 sshd[220686]: Disconnected from invalid user botuser 123.207.59.181 port 48114 [preauth]
Jan 28 04:21:50 Q3 smbd[220698]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:21:52 Q3 sshd[220694]: Received disconnect from 221.214.2.98 port 2443:11: Bye Bye [preauth]
Jan 28 04:21:52 Q3 sshd[220694]: Disconnected from invalid user test 221.214.2.98 port 2443 [preauth]
Jan 28 04:21:53 Q3 incusd[7335]: time=“2024-01-28T04:21:53-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 04:21:55 Q3 smbd[220699]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:21:58 Q3 incusd[7335]: time=“2024-01-28T04:21:58-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:21:58 Q3 incusd[7335]: time=“2024-01-28T04:21:58-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:21:58 Q3 incusd[7335]: time=“2024-01-28T04:21:58-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:22:05 Q3 incusd[7335]: time=“2024-01-28T04:22:05-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 04:22:08 Q3 smbd[220705]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:22:09 Q3 sshd[220706]: Invalid user server from 221.214.2.98 port 2444
Jan 28 04:22:09 Q3 sshd[220706]: pam_unix(sshd:auth): check pass; user unknown
Jan 28 04:22:09 Q3 sshd[220706]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=221.214.2.98
Jan 28 04:22:10 Q3 incusd[7335]: time=“2024-01-28T04:22:10-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:22:10 Q3 incusd[7335]: time=“2024-01-28T04:22:10-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:22:10 Q3 incusd[7335]: time=“2024-01-28T04:22:10-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 04:22:10 Q3 incusd[7335]: time=“2024-01-28T04:22:10-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 04:22:10 Q3 smbd[220704]: pam_unix(samba:session): session closed for user nobody
Jan 28 04:22:11 Q3 sshd[220706]: Failed password for invalid user server from 221.214.2.98 port 2444 ssh2

root@Q3:/home/ic2000# mc
X11 connection rejected because of wrong authentication.

root@Q3:/home/ic2000# apt upgrade
Reading package lists… Done
Building dependency tree
Reading state information… Done
Calculating upgrade… Done
Get more security updates through Ubuntu Pro with ‘esm-apps’ enabled:
libavformat58 python2.7-dev libavfilter7 mc libmagickcore-6.q16-6-extra sntp
imagemagick libswresample3 libzmq5 python2.7-minimal libmagickwand-6.q16-6
libpython2.7 python2.7 libpython2.7-dev mc-data libpostproc55 ntp
imagemagick-6.q16 libavcodec58 libavutil56 libswscale5 libopenexr24
libmysofa1 libmagickcore-6.q16-6 libpython2.7-minimal libpython2.7-stdlib
imagemagick-6-common
Learn more about Ubuntu Pro at Ubuntu Pro | Ubuntu
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
2 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up incus-base (0.5-202401260134-ubuntu20.04) …
Job failed. See “journalctl -xe” for details.
dpkg: error processing package incus-base (–configure):
installed incus-base package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of incus:
incus depends on incus-base (= 0.5-202401260134-ubuntu20.04); however:
Package incus-base is not configured yet.

dpkg: error processing package incus (–configure):
dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
Processing triggers for libc-bin (
2.31-0ubuntu9.14) …
Errors were encountered while processing:
incus-base
incus
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@Q3:/home/ic2000# sudo apt-get install incus-base incus
Reading package lists… Done
Building dependency tree
Reading state information… Done
incus is already the newest version (0.5-202401260134-ubuntu20.04).
incus-base is already the newest version (0.5-202401260134-ubuntu20.04).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
2 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up incus-base (0.5-202401260134-ubuntu20.04) …
Job failed. See “journalctl -xe” for details.
dpkg: error processing package incus-base (–configure):
installed incus-base package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of incus:
incus depends on incus-base (= 0.5-202401260134-ubuntu20.04); however:
Package incus-base is not configured yet.

dpkg: error processing package incus (–configure):
dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
Processing triggers for libc-bin (2.31-0ubuntu9.14) …
Errors were encountered while processing:
incus-base
incus
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@Q3:/home/ic2000# sudo dpkg --configure -a
dpkg: dependency problems prevent configuration of incus:
incus depends on incus-base (= 0.5-202401260134-ubuntu20.04); however:
Package incus-base is not configured yet.

dpkg: error processing package incus (–configure):
dependency problems - leaving unconfigured
Errors were encountered while processing:
incus
root@Q3:/home/ic2000#

Please show journalctl -u incus -n 300 on the system where the upgrade is failing.

Jan 28 19:11:02 Q3 incusd[7335]: time=“2024-01-28T19:11:02-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:11:07 Q3 incusd[7335]: time=“2024-01-28T19:11:07-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:11:12 Q3 incusd[7335]: time=“2024-01-28T19:11:12-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:12 Q3 incusd[7335]: time=“2024-01-28T19:11:12-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:12 Q3 incusd[7335]: time=“2024-01-28T19:11:12-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:15 Q3 incusd[7335]: time=“2024-01-28T19:11:15-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:11:20 Q3 incusd[7335]: time=“2024-01-28T19:11:20-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:20 Q3 incusd[7335]: time=“2024-01-28T19:11:20-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:20 Q3 incusd[7335]: time=“2024-01-28T19:11:20-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:28 Q3 incusd[7335]: time=“2024-01-28T19:11:28-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:11:33 Q3 incusd[7335]: time=“2024-01-28T19:11:33-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:33 Q3 incusd[7335]: time=“2024-01-28T19:11:33-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:33 Q3 incusd[7335]: time=“2024-01-28T19:11:33-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:33 Q3 incusd[7335]: time=“2024-01-28T19:11:33-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:11:38 Q3 incusd[7335]: time=“2024-01-28T19:11:38-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:11:43 Q3 incusd[7335]: time=“2024-01-28T19:11:43-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:43 Q3 incusd[7335]: time=“2024-01-28T19:11:43-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:43 Q3 incusd[7335]: time=“2024-01-28T19:11:43-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:43 Q3 incusd[7335]: time=“2024-01-28T19:11:43-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:11:49 Q3 incusd[7335]: time=“2024-01-28T19:11:49-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:11:54 Q3 incusd[7335]: time=“2024-01-28T19:11:54-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:54 Q3 incusd[7335]: time=“2024-01-28T19:11:54-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:54 Q3 incusd[7335]: time=“2024-01-28T19:11:54-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:11:54 Q3 incusd[7335]: time=“2024-01-28T19:11:54-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:11:59 Q3 incusd[7335]: time=“2024-01-28T19:11:59-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:12:04 Q3 incusd[7335]: time=“2024-01-28T19:12:04-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:04 Q3 incusd[7335]: time=“2024-01-28T19:12:04-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:04 Q3 incusd[7335]: time=“2024-01-28T19:12:04-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:10 Q3 incusd[7335]: time=“2024-01-28T19:12:10-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:12:15 Q3 incusd[7335]: time=“2024-01-28T19:12:15-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:15 Q3 incusd[7335]: time=“2024-01-28T19:12:15-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:15 Q3 incusd[7335]: time=“2024-01-28T19:12:15-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:15 Q3 incusd[7335]: time=“2024-01-28T19:12:15-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:12:17 Q3 incusd[7335]: time=“2024-01-28T19:12:17-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:12:22 Q3 incusd[7335]: time=“2024-01-28T19:12:22-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:22 Q3 incusd[7335]: time=“2024-01-28T19:12:22-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:22 Q3 incusd[7335]: time=“2024-01-28T19:12:22-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:26 Q3 incusd[7335]: time=“2024-01-28T19:12:26-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:12:31 Q3 incusd[7335]: time=“2024-01-28T19:12:31-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:31 Q3 incusd[7335]: time=“2024-01-28T19:12:31-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:31 Q3 incusd[7335]: time=“2024-01-28T19:12:31-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:40 Q3 incusd[7335]: time=“2024-01-28T19:12:40-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:12:45 Q3 incusd[7335]: time=“2024-01-28T19:12:45-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:45 Q3 incusd[7335]: time=“2024-01-28T19:12:45-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:45 Q3 incusd[7335]: time=“2024-01-28T19:12:45-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:51 Q3 incusd[7335]: time=“2024-01-28T19:12:51-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:12:56 Q3 incusd[7335]: time=“2024-01-28T19:12:56-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:56 Q3 incusd[7335]: time=“2024-01-28T19:12:56-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:56 Q3 incusd[7335]: time=“2024-01-28T19:12:56-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:12:56 Q3 incusd[7335]: time=“2024-01-28T19:12:56-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:13:02 Q3 incusd[7335]: time=“2024-01-28T19:13:02-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:13:07 Q3 incusd[7335]: time=“2024-01-28T19:13:07-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:07 Q3 incusd[7335]: time=“2024-01-28T19:13:07-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:07 Q3 incusd[7335]: time=“2024-01-28T19:13:07-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:07 Q3 incusd[7335]: time=“2024-01-28T19:13:07-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:13:14 Q3 incusd[7335]: time=“2024-01-28T19:13:14-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:13:19 Q3 incusd[7335]: time=“2024-01-28T19:13:19-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:19 Q3 incusd[7335]: time=“2024-01-28T19:13:19-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:19 Q3 incusd[7335]: time=“2024-01-28T19:13:19-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:19 Q3 incusd[7335]: time=“2024-01-28T19:13:19-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:13:24 Q3 incusd[7335]: time=“2024-01-28T19:13:24-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:13:29 Q3 incusd[7335]: time=“2024-01-28T19:13:29-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:29 Q3 incusd[7335]: time=“2024-01-28T19:13:29-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:29 Q3 incusd[7335]: time=“2024-01-28T19:13:29-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:29 Q3 incusd[7335]: time=“2024-01-28T19:13:29-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:13:34 Q3 incusd[7335]: time=“2024-01-28T19:13:34-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:13:39 Q3 incusd[7335]: time=“2024-01-28T19:13:39-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:39 Q3 incusd[7335]: time=“2024-01-28T19:13:39-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:39 Q3 incusd[7335]: time=“2024-01-28T19:13:39-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:45 Q3 incusd[7335]: time=“2024-01-28T19:13:45-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:13:50 Q3 incusd[7335]: time=“2024-01-28T19:13:50-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:50 Q3 incusd[7335]: time=“2024-01-28T19:13:50-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:50 Q3 incusd[7335]: time=“2024-01-28T19:13:50-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:13:50 Q3 incusd[7335]: time=“2024-01-28T19:13:50-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:13:56 Q3 incusd[7335]: time=“2024-01-28T19:13:56-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:14:01 Q3 incusd[7335]: time=“2024-01-28T19:14:01-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:01 Q3 incusd[7335]: time=“2024-01-28T19:14:01-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:01 Q3 incusd[7335]: time=“2024-01-28T19:14:01-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:01 Q3 incusd[7335]: time=“2024-01-28T19:14:01-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:14:08 Q3 incusd[7335]: time=“2024-01-28T19:14:08-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:14:08 Q3 incusd[7335]: time=“2024-01-28T19:14:08-05:00” level=warning msg=“Failed adding member event listener client” err="unexp>
Jan 28 19:14:11 Q3 incusd[7335]: time=“2024-01-28T19:14:11-05:00” level=warning msg=“Failed adding member event listener client” err="unexp>
Jan 28 19:14:13 Q3 incusd[7335]: time=“2024-01-28T19:14:13-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:13 Q3 incusd[7335]: time=“2024-01-28T19:14:13-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:14:19 Q3 incusd[7335]: time=“2024-01-28T19:14:19-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:14:24 Q3 incusd[7335]: time=“2024-01-28T19:14:24-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:24 Q3 incusd[7335]: time=“2024-01-28T19:14:24-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:24 Q3 incusd[7335]: time=“2024-01-28T19:14:24-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:24 Q3 incusd[7335]: time=“2024-01-28T19:14:24-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:14:31 Q3 incusd[7335]: time=“2024-01-28T19:14:31-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:14:36 Q3 incusd[7335]: time=“2024-01-28T19:14:36-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:36 Q3 incusd[7335]: time=“2024-01-28T19:14:36-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:36 Q3 incusd[7335]: time=“2024-01-28T19:14:36-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:36 Q3 incusd[7335]: time=“2024-01-28T19:14:36-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:14:40 Q3 incusd[7335]: time=“2024-01-28T19:14:40-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:14:45 Q3 incusd[7335]: time=“2024-01-28T19:14:45-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:45 Q3 incusd[7335]: time=“2024-01-28T19:14:45-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:45 Q3 incusd[7335]: time=“2024-01-28T19:14:45-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:51 Q3 incusd[7335]: time=“2024-01-28T19:14:51-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:14:56 Q3 incusd[7335]: time=“2024-01-28T19:14:56-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:56 Q3 incusd[7335]: time=“2024-01-28T19:14:56-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:56 Q3 incusd[7335]: time=“2024-01-28T19:14:56-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:14:56 Q3 incusd[7335]: time=“2024-01-28T19:14:56-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:15:02 Q3 incusd[7335]: time=“2024-01-28T19:15:02-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:15:07 Q3 incusd[7335]: time=“2024-01-28T19:15:07-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:07 Q3 incusd[7335]: time=“2024-01-28T19:15:07-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:07 Q3 incusd[7335]: time=“2024-01-28T19:15:07-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:07 Q3 incusd[7335]: time=“2024-01-28T19:15:07-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:15:14 Q3 incusd[7335]: time=“2024-01-28T19:15:14-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:15:19 Q3 incusd[7335]: time=“2024-01-28T19:15:19-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:19 Q3 incusd[7335]: time=“2024-01-28T19:15:19-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:19 Q3 incusd[7335]: time=“2024-01-28T19:15:19-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:19 Q3 incusd[7335]: time=“2024-01-28T19:15:19-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:15:26 Q3 incusd[7335]: time=“2024-01-28T19:15:26-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:15:30 Q3 incusd[7335]: time=“2024-01-28T19:15:30-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:30 Q3 incusd[7335]: time=“2024-01-28T19:15:30-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:30 Q3 incusd[7335]: time=“2024-01-28T19:15:30-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:30 Q3 incusd[7335]: time=“2024-01-28T19:15:30-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:15:37 Q3 incusd[7335]: time=“2024-01-28T19:15:37-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:15:42 Q3 incusd[7335]: time=“2024-01-28T19:15:42-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:42 Q3 incusd[7335]: time=“2024-01-28T19:15:42-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:42 Q3 incusd[7335]: time=“2024-01-28T19:15:42-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:42 Q3 incusd[7335]: time=“2024-01-28T19:15:42-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:15:49 Q3 incusd[7335]: time=“2024-01-28T19:15:49-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:15:54 Q3 incusd[7335]: time=“2024-01-28T19:15:54-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:54 Q3 incusd[7335]: time=“2024-01-28T19:15:54-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:54 Q3 incusd[7335]: time=“2024-01-28T19:15:54-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:15:54 Q3 incusd[7335]: time=“2024-01-28T19:15:54-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:15:59 Q3 incusd[7335]: time=“2024-01-28T19:15:59-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:16:04 Q3 incusd[7335]: time=“2024-01-28T19:16:04-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:04 Q3 incusd[7335]: time=“2024-01-28T19:16:04-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:04 Q3 incusd[7335]: time=“2024-01-28T19:16:04-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:04 Q3 incusd[7335]: time=“2024-01-28T19:16:04-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:16:08 Q3 incusd[7335]: time=“2024-01-28T19:16:08-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:16:13 Q3 incusd[7335]: time=“2024-01-28T19:16:13-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:13 Q3 incusd[7335]: time=“2024-01-28T19:16:13-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:13 Q3 incusd[7335]: time=“2024-01-28T19:16:13-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:19 Q3 incusd[7335]: time=“2024-01-28T19:16:19-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:16:24 Q3 incusd[7335]: time=“2024-01-28T19:16:24-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:24 Q3 incusd[7335]: time=“2024-01-28T19:16:24-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:24 Q3 incusd[7335]: time=“2024-01-28T19:16:24-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:24 Q3 incusd[7335]: time=“2024-01-28T19:16:24-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:16:30 Q3 incusd[7335]: time=“2024-01-28T19:16:30-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:16:35 Q3 incusd[7335]: time=“2024-01-28T19:16:35-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:35 Q3 incusd[7335]: time=“2024-01-28T19:16:35-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:35 Q3 incusd[7335]: time=“2024-01-28T19:16:35-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:35 Q3 incusd[7335]: time=“2024-01-28T19:16:35-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:16:40 Q3 incusd[7335]: time=“2024-01-28T19:16:40-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:16:45 Q3 incusd[7335]: time=“2024-01-28T19:16:45-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:45 Q3 incusd[7335]: time=“2024-01-28T19:16:45-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:45 Q3 incusd[7335]: time=“2024-01-28T19:16:45-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:52 Q3 incusd[7335]: time=“2024-01-28T19:16:52-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:16:57 Q3 incusd[7335]: time=“2024-01-28T19:16:57-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:57 Q3 incusd[7335]: time=“2024-01-28T19:16:57-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:57 Q3 incusd[7335]: time=“2024-01-28T19:16:57-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:16:57 Q3 incusd[7335]: time=“2024-01-28T19:16:57-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:16:59 Q3 incusd[7335]: time=“2024-01-28T19:16:59-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:17:04 Q3 incusd[7335]: time=“2024-01-28T19:17:04-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:04 Q3 incusd[7335]: time=“2024-01-28T19:17:04-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:04 Q3 incusd[7335]: time=“2024-01-28T19:17:04-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:13 Q3 incusd[7335]: time=“2024-01-28T19:17:13-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:17:18 Q3 incusd[7335]: time=“2024-01-28T19:17:18-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:18 Q3 incusd[7335]: time=“2024-01-28T19:17:18-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:18 Q3 incusd[7335]: time=“2024-01-28T19:17:18-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:18 Q3 incusd[7335]: time=“2024-01-28T19:17:18-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:17:24 Q3 incusd[7335]: time=“2024-01-28T19:17:24-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:17:29 Q3 incusd[7335]: time=“2024-01-28T19:17:29-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:29 Q3 incusd[7335]: time=“2024-01-28T19:17:29-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:29 Q3 incusd[7335]: time=“2024-01-28T19:17:29-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:29 Q3 incusd[7335]: time=“2024-01-28T19:17:29-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:17:36 Q3 incusd[7335]: time=“2024-01-28T19:17:36-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:17:41 Q3 incusd[7335]: time=“2024-01-28T19:17:41-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:41 Q3 incusd[7335]: time=“2024-01-28T19:17:41-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:41 Q3 incusd[7335]: time=“2024-01-28T19:17:41-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:41 Q3 incusd[7335]: time=“2024-01-28T19:17:41-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:17:46 Q3 incusd[7335]: time=“2024-01-28T19:17:46-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:17:51 Q3 incusd[7335]: time=“2024-01-28T19:17:51-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:51 Q3 incusd[7335]: time=“2024-01-28T19:17:51-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:51 Q3 incusd[7335]: time=“2024-01-28T19:17:51-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:17:51 Q3 incusd[7335]: time=“2024-01-28T19:17:51-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:17:55 Q3 incusd[7335]: time=“2024-01-28T19:17:55-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:18:00 Q3 incusd[7335]: time=“2024-01-28T19:18:00-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:00 Q3 incusd[7335]: time=“2024-01-28T19:18:00-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:00 Q3 incusd[7335]: time=“2024-01-28T19:18:00-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:07 Q3 incusd[7335]: time=“2024-01-28T19:18:07-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:18:12 Q3 incusd[7335]: time=“2024-01-28T19:18:12-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:12 Q3 incusd[7335]: time=“2024-01-28T19:18:12-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:12 Q3 incusd[7335]: time=“2024-01-28T19:18:12-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:12 Q3 incusd[7335]: time=“2024-01-28T19:18:12-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:18:16 Q3 incusd[7335]: time=“2024-01-28T19:18:16-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:18:21 Q3 incusd[7335]: time=“2024-01-28T19:18:21-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:21 Q3 incusd[7335]: time=“2024-01-28T19:18:21-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:21 Q3 incusd[7335]: time=“2024-01-28T19:18:21-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:27 Q3 incusd[7335]: time=“2024-01-28T19:18:27-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:18:32 Q3 incusd[7335]: time=“2024-01-28T19:18:32-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:32 Q3 incusd[7335]: time=“2024-01-28T19:18:32-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:32 Q3 incusd[7335]: time=“2024-01-28T19:18:32-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:37 Q3 incusd[7335]: time=“2024-01-28T19:18:37-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:18:42 Q3 incusd[7335]: time=“2024-01-28T19:18:42-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:42 Q3 incusd[7335]: time=“2024-01-28T19:18:42-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:42 Q3 incusd[7335]: time=“2024-01-28T19:18:42-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:44 Q3 incusd[7335]: time=“2024-01-28T19:18:44-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:18:49 Q3 incusd[7335]: time=“2024-01-28T19:18:49-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:49 Q3 incusd[7335]: time=“2024-01-28T19:18:49-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:49 Q3 incusd[7335]: time=“2024-01-28T19:18:49-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:18:58 Q3 incusd[7335]: time=“2024-01-28T19:18:58-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:19:03 Q3 incusd[7335]: time=“2024-01-28T19:19:03-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:03 Q3 incusd[7335]: time=“2024-01-28T19:19:03-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:03 Q3 incusd[7335]: time=“2024-01-28T19:19:03-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:03 Q3 incusd[7335]: time=“2024-01-28T19:19:03-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:19:08 Q3 incusd[7335]: time=“2024-01-28T19:19:08-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:19:13 Q3 incusd[7335]: time=“2024-01-28T19:19:13-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:13 Q3 incusd[7335]: time=“2024-01-28T19:19:13-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:13 Q3 incusd[7335]: time=“2024-01-28T19:19:13-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:19 Q3 incusd[7335]: time=“2024-01-28T19:19:19-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:19:24 Q3 incusd[7335]: time=“2024-01-28T19:19:24-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:24 Q3 incusd[7335]: time=“2024-01-28T19:19:24-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:24 Q3 incusd[7335]: time=“2024-01-28T19:19:24-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:24 Q3 incusd[7335]: time=“2024-01-28T19:19:24-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:19:31 Q3 incusd[7335]: time=“2024-01-28T19:19:31-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:19:35 Q3 incusd[7335]: time=“2024-01-28T19:19:35-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:35 Q3 incusd[7335]: time=“2024-01-28T19:19:35-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:35 Q3 incusd[7335]: time=“2024-01-28T19:19:35-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:35 Q3 incusd[7335]: time=“2024-01-28T19:19:35-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:19:42 Q3 incusd[7335]: time=“2024-01-28T19:19:42-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:19:47 Q3 incusd[7335]: time=“2024-01-28T19:19:47-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:47 Q3 incusd[7335]: time=“2024-01-28T19:19:47-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:47 Q3 incusd[7335]: time=“2024-01-28T19:19:47-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:47 Q3 incusd[7335]: time=“2024-01-28T19:19:47-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:19:52 Q3 incusd[7335]: time=“2024-01-28T19:19:52-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:19:57 Q3 incusd[7335]: time=“2024-01-28T19:19:57-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:57 Q3 incusd[7335]: time=“2024-01-28T19:19:57-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:19:57 Q3 incusd[7335]: time=“2024-01-28T19:19:57-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:03 Q3 incusd[7335]: time=“2024-01-28T19:20:03-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:20:08 Q3 incusd[7335]: time=“2024-01-28T19:20:08-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:08 Q3 incusd[7335]: time=“2024-01-28T19:20:08-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:08 Q3 incusd[7335]: time=“2024-01-28T19:20:08-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:08 Q3 incusd[7335]: time=“2024-01-28T19:20:08-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:20:13 Q3 incusd[7335]: time=“2024-01-28T19:20:13-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:20:17 Q3 incusd[7335]: time=“2024-01-28T19:20:17-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:17 Q3 incusd[7335]: time=“2024-01-28T19:20:17-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:17 Q3 incusd[7335]: time=“2024-01-28T19:20:17-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:25 Q3 incusd[7335]: time=“2024-01-28T19:20:25-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:20:30 Q3 incusd[7335]: time=“2024-01-28T19:20:30-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:30 Q3 incusd[7335]: time=“2024-01-28T19:20:30-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:30 Q3 incusd[7335]: time=“2024-01-28T19:20:30-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:30 Q3 incusd[7335]: time=“2024-01-28T19:20:30-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:20:36 Q3 incusd[7335]: time=“2024-01-28T19:20:36-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:20:41 Q3 incusd[7335]: time=“2024-01-28T19:20:41-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:41 Q3 incusd[7335]: time=“2024-01-28T19:20:41-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:41 Q3 incusd[7335]: time=“2024-01-28T19:20:41-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:41 Q3 incusd[7335]: time=“2024-01-28T19:20:41-05:00” level=warning msg="Heartbeat round duration greater than heartbeat interv>
Jan 28 19:20:43 Q3 incusd[7335]: time=“2024-01-28T19:20:43-05:00” level=warning msg=“Could not rebalance cluster member roles” err="Daemon >
Jan 28 19:20:48 Q3 incusd[7335]: time=“2024-01-28T19:20:48-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:48 Q3 incusd[7335]: time=“2024-01-28T19:20:48-05:00” level=warning msg=“Failed adding member event listener client” err="read >
Jan 28 19:20:48 Q3 incusd[7335]: time=“2024-01-28T19:20:48-05:00” level=warning msg=“Failed adding member event listener client” err="read >
lines 267-301/301 (END)

Sounds like some connectivity issues between the servers which is preventing the database from coming online.

Can you run incus admin sql global "SELECT * FROM nodes" on one of the servers that upgraded correctly?

They are all the same

incus admin sql global “SELECT * FROM nodes”
±—±-----±------------±-----------------±-------±---------------±------------------------------------±------±-----±------------------+
| id | name | description | address | schema | api_extensions | heartbeat | state | arch | failure_domain_id |
±—±-----±------------±-----------------±-------±---------------±------------------------------------±------±-----±------------------+
| 1 | Q4 | | 84.17.40.21:8443 | 70 | 372 | 2024-01-29T08:07:53.447475535-05:00 | 0 | 2 | |
| 2 | Q3 | | 84.17.40.20:8443 | 70 | 367 | 2024-01-29T08:07:47.807478347-05:00 | 0 | 2 | |
| 3 | Q2 | | 84.17.40.19:8443 | 70 | 373 | 2024-01-29T08:07:54.680868046-05:00 | 0 | 2 | |
| 4 | Q1 | | 84.17.40.18:8443 | 70 | 373 | 2024-01-29T08:07:53.642159268-05:00 | 0 | 2 | |
±—±-----±------------±-----------------±-------±---------------±------------------------------------±------±-----±------------------+

Q3 seems to be the issue, but may be it is the others.

Can you show dpkg -l incus on all systems?
It definitely looks like Q3 is running an older version here.

I reran Apt update Apt Upgrade to see what would happen

Yeah, so Q3 is definitely mad about something.

On Q3, can you run:

  • pkill -9 incusd
  • systemctl daemon-reload
  • systemctl restart incus

That did it, thank you!