Lxd recover problem

On one of my Ubuntu 22.04 desktop systems I have two 2TB NVME M.2 SSD’s on the Asus x570 motherboard.

NVME1 was EXT4 and was where Ubuntu 22.04 was installed, including my Home directory.

NVME2 is BTRFS and where I configured LXD’s “storage” location.

I recently had to re-install Ubuntu 22.04 on NVME1.

After installation of Ubuntu 22.04 I installed SNAP LXD and configured /dev/nvme2 as the LXD BTRFS storage.

I then used:

$ sudo lxd recover

To try to recover all my previous LXD Containers and VMs.

The following is the result which looks partially successful!

$ lxc ls
±-------------------±-------------- ±-------±-------±-----------------------------±--------------------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
±-------------------±---------------±-------±-------±-----------------------------±--------------------+
| edgevpn-gui | STOPPED | | | CONTAINER | 0 |
±-------------------±---------------±-------±-------±-----------------------------±--------------------+
| ubuntu-1 | STOPPED | | | CONTAINER | 0 |
±-------------------±---------------±-------±-------±-----------------------------±--------------------+
| ubuntu-base | STOPPED | | | CONTAINER | 0 |
±-------------------±---------------±------±--------±----------------------------±--------------------+
| ubuntu-vm | STOPPED | | | VIRTUAL-MACHINE | 0 |
±-------------------±---------------±------±--------±----------------------------±--------------------+
| win11 | STOPPED | | | VIRTUAL-MACHINE | 0 |
±-------------------±---------------±------±--------±----------------------------±--------------------+
| win11-clone | STOPPED | | | VIRTUAL-MACHINE | 0 |
±-------------------±---------------±------±--------±----------------------------±--------------------+
| wirespider1 | STOPPED | | | CONTAINER | 0 |
±-------------------±---------------±------±--------±----------------------------±--------------------+

So I tried to Start one of the LXD Containers (ubuntu-base) and it failed with the LOG mentioning "newuidmap binary is missing"

$ lxc start ubuntu-base
Error: Failed to run: /snap/lxd/current/bin/lxd forkstart ubuntu-base /var/snap/lxd/common/lxd/containers /var/snap/lxd/common/lxd/logs/ubuntu-base/lxc.conf: exit status 1
Try lxc info --show-log ubuntu-base for more info

$ lxc info --show-log ubuntu-base
Name: ubuntu-base
Status: STOPPED
Type: container
Architecture: x86_64
Created: 2023/04/24 18:56 EDT
Last Used: 2023/06/28 20:27 EDT

Log:

lxc ubuntu-base 20230629002724.938 WARN conf - …/src/src/lxc/conf.c:lxc_map_ids:3621 - newuidmap binary is missing
lxc ubuntu-base 20230629002724.938 WARN conf - …/src/src/lxc/conf.c:lxc_map_ids:3627 - newgidmap binary is missing
lxc ubuntu-base 20230629002724.939 WARN conf - …/src/src/lxc/conf.c:lxc_map_ids:3621 - newuidmap binary is missing
lxc ubuntu-base 20230629002724.939 WARN conf - …/src/src/lxc/conf.c:lxc_map_ids:3627 - newgidmap binary is missing
lxc ubuntu-base 20230629002725.865 ERROR cgfsng - …/src/src/lxc/cgroups/cgfsng.c:cgfsng_mount:2139 - No such file or directory - Failed to create cgroup at_mnt 24()
lxc ubuntu-base 20230629002725.865 ERROR conf - …/src/src/lxc/conf.c:lxc_mount_auto_mounts:852 - No such file or directory - Failed to mount “/sys/fs/cgroup”
lxc ubuntu-base 20230629002725.865 ERROR conf - …/src/src/lxc/conf.c:lxc_setup:4433 - Failed to setup remaining automatic mounts
lxc ubuntu-base 20230629002725.865 ERROR start - …/src/src/lxc/start.c:do_start:1272 - Failed to setup container “ubuntu-base”
lxc ubuntu-base 20230629002725.865 ERROR sync - …/src/src/lxc/sync.c:sync_wait:34 - An error occurred in another process (expected sequence number 4)
lxc ubuntu-base 20230629002725.896 WARN network - …/src/src/lxc/network.c:lxc_delete_network_priv:3631 - Failed to rename interface with index 0 from “eth0” to its initial name “vethe8c77677”
lxc ubuntu-base 20230629002725.897 ERROR lxccontainer - …/src/src/lxc/lxccontainer.c:wait_on_daemonized_start:878 - Received container state “ABORTING” instead of “RUNNING”
lxc ubuntu-base 20230629002725.897 ERROR start - …/src/src/lxc/start.c:__lxc_start:2107 - Failed to spawn container “ubuntu-base”
lxc ubuntu-base 20230629002725.897 WARN start - …/src/src/lxc/start.c:lxc_abort:1036 - No such process - Failed to send SIGKILL via pidfd 19 for process 4020
lxc 20230629002725.253 ERROR af_unix - …/src/src/lxc/af_unix.c:lxc_abstract_unix_recv_fds_iov:218 - Connection reset by peer - Failed to receive response
lxc 20230629002725.253 ERROR commands - …/src/src/lxc/commands.c:lxc_cmd_rsp_recv_fds:128 - Failed to receive file descriptors for command “get_init_pid”

Attempts to Start any of the LXD Containers gives the same similar “errors”

Any suggestions how to fix this problem.

The error is:

Sounds like:

Tom

Your email reminded me that I had forgotten 1 step when I reinstalled my Ubuntu 22.04 on NVME1.

I had again installed Mullvad VPN !!

Mullvad hijacks NET_CLS on CGROUP1.

Since I had previously posted the FIX for that some time ago on the LinuxContainers Forum I just followed my own steps to redirect Mullvad’s CGROUP1 mount to a directory on /OPT
Reran LXD INIT and the did LXD RECOVER and pointed to my previous LXD Storage volume on NVME2.

Added the 1 line to the MullVad SystemD file and reloaded SystemD.

Rebooted and and everything is back Working !

God I am such a dope for not remembering my own solution :sunglasses: to that Mullvad cgroup1 problem.

Thanks so much for the kick in the head.

Brian

1 Like