My 4 node LXD cluster runs on raspberry pi 4 8gig, OS ubuntu server 21.10.
Each node is daily backed up using dd command to my NAS.
After I restored the dd backup image to node 4 I notice its part of the cluster again but encounter error with any LXC/LXD command in node 4.
lxc cluster list from node 1/2/3 gives below output
sysadm@lxcnprd01:~$ lxc cluster list
±----------±------------------------------±-----------------±-------------±---------------±------------±-------±------------------+
| NAME | URL | ROLES | ARCHITECTURE | FAILURE DOMAIN | DESCRIPTION | STATE | MESSAGE |
±----------±------------------------------±-----------------±-------------±---------------±------------±-------±------------------+
| lxcnprd01 | https://lxcnprd01.don.io:8443 | database-standby | aarch64 | default | | ONLINE | Fully operational |
±----------±------------------------------±-----------------±-------------±---------------±------------±-------±------------------+
| lxcnprd02 | https://lxcnprd02.don.io:8443 | database | aarch64 | default | | ONLINE | Fully operational |
±----------±------------------------------±-----------------±-------------±---------------±------------±-------±------------------+
| lxcnprd03 | https://lxcnprd03.don.io:8443 | database-leader | aarch64 | default | | ONLINE | Fully operational |
| | | database | | | | | |
±----------±------------------------------±-----------------±-------------±---------------±------------±-------±------------------+
| lxcnprd04 | https://lxcnprd04.don.io:8443 | database | aarch64 | default | | ONLINE | Fully operational |
±----------±------------------------------±-----------------±-------------±---------------±------------±-------±------------------+
But from node 4 I get below output
sysadm@lxcnprd04:~$ lxc cluster list
internal error, please report: running “lxd.lxc” failed: transient scope could not be started, job /org/freedesktop/systemd1/job/73 finished with result failed
sysadm@lxcnprd04:~$ lxc ls
internal error, please report: running “lxd.lxc” failed: transient scope could not be started, job /org/freedesktop/systemd1/job/77 finished with result failed
sysadm@lxcnprd04:~$
I can still able to --target containers to node 4 from other nodes without any issue.
For any LXC command I see below in syslog
Jan 28 13:46:42 lxcnprd04 systemd[3051]: snap.lxd.lxc.420f9333-099b-44cb-875f-c2c7c05bad85.scope: Failed to add PIDs to scope’s control group: Permission denied
Jan 28 13:46:42 lxcnprd04 systemd[3051]: snap.lxd.lxc.420f9333-099b-44cb-875f-c2c7c05bad85.scope: Failed with result ‘resources’.
Jan 28 13:46:42 lxcnprd04 systemd[3051]: Failed to start snap.lxd.lxc.420f9333-099b-44cb-875f-c2c7c05bad85.scope.
Note: I did ‘sudo fsck.ext4 -f /dev/sda2’ to fix filesystem inconsistencies after restore node4.
Help please!