Can't start or delete container

Hi,

1 container that has been created is unable to start neither can it be deleted. I can start and delete newer containers so it seems to be 1 container only.

When I try to start I get:

lxc BDB-TEST-LAB 20210504142305.858 WARN cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1129 - File exists - Failed to create directory “/sys/fs/cgroup/cpuset//lxc.monitor.BDB-TEST-LAB”
lxc BDB-TEST-LAB 20210504142305.859 WARN cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1129 - File exists - Failed to create directory “/sys/fs/cgroup/cpuset//lxc.payload.BDB-TEST-LAB”
lxc BDB-TEST-LAB 20210504142305.866 WARN cgfsng - cgroups/cgfsng.c:fchowmodat:1550 - No such file or directory - Failed to fchownat(17, memory.oom.group, 1000000000, 0, AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW )
lxc BDB-TEST-LAB 20210504142305.939 ERROR dir - storage/dir.c:dir_mount:152 - No such file or directory - Failed to mount “/var/snap/lxd/common/lxd/containers/BDB-TEST-LAB/rootfs” on “/var/snap/lxd/common/lxc/”
lxc BDB-TEST-LAB 20210504142305.939 ERROR conf - conf.c:lxc_mount_rootfs:1272 - Failed to mount rootfs “/var/snap/lxd/common/lxd/containers/BDB-TEST-LAB/rootfs” onto “/var/snap/lxd/common/lxc/” with options “(null)”
lxc BDB-TEST-LAB 20210504142305.939 ERROR conf - conf.c:lxc_setup_rootfs_prepare_root:3155 - Failed to setup rootfs for
lxc BDB-TEST-LAB 20210504142305.939 ERROR conf - conf.c:lxc_setup:3291 - Failed to setup rootfs
lxc BDB-TEST-LAB 20210504142305.939 ERROR start - start.c:do_start:1218 - Failed to setup container “BDB-TEST-LAB”
lxc BDB-TEST-LAB 20210504142305.939 ERROR sync - sync.c:__sync_wait:36 - An error occurred in another process (expected sequence number 5)
lxc BDB-TEST-LAB 20210504142305.942 WARN network - network.c:lxc_delete_network_priv:3185 - Failed to rename interface with index 0 from “eth0” to its initial name “vethb7936e83”
lxc BDB-TEST-LAB 20210504142305.942 ERROR lxccontainer - lxccontainer.c:wait_on_daemonized_start:860 - Received container state “ABORTING” instead of “RUNNING”
lxc BDB-TEST-LAB 20210504142305.942 ERROR start - start.c:__lxc_start:1999 - Failed to spawn container “BDB-TEST-LAB”
lxc BDB-TEST-LAB 20210504142305.942 WARN start - start.c:lxc_abort:1013 - No such process - Failed to send SIGKILL via pidfd 30 for process 1427529
lxc 20210504142306.132 WARN commands - commands.c:lxc_cmd_rsp_recv:126 - Connection reset by peer - Failed to receive response for command “get_state”

When I try to delete I get:

Error: Error deleting storage volume: Failed setting subvolume writable “/var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-TEST-LAB”: Failed to run: btrfs property set -ts /var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-TEST-LAB ro false: ERROR: failed to get flags for /var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-TEST-LAB: Invalid argument

Why is this and how can I delete the container? (no need to recover in this case)

thanks

What’s in ls -lh /var/snap/lxd/common/mntns/var/snap/lxd/common/lxd/storage-pools/default/containers/?

d–x------ 1 root root 0 May 4 14:57 BDB-TEST-LAB

Ok, are you supposed to have any other container on the system?

The error suggests that the container’s path isn’t a brfs subvolume.
If all you want is to get rid of it, rm -rf /var/snap/lxd/common/mntns/var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-TEST-LAB should do the trick

I have 47 containers.

I used your command but the container still shows up in the lxc list. How can I remove it?
also, any idea how this could have happened?

Hi,

The rm -rf command remove the container from that directory but it is still visible everywhere else…
How can I completely remove it?

thnx

lxc delete should work fine once the directory is gone.

error is the same as before:

$ lxc delete BDB-TEST-LAB

Error: Error deleting storage volume: Failed setting subvolume writable “/var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-TEST-LAB”: Failed to run: btrfs property set -ts /var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-TEST-LAB ro false: ERROR: failed to get flags for /var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-TEST-LAB: Invalid argument

Hmm, that’s odd, I’d have expected it to move on if it doesn’t exist…

Anyway, you can do nsenter --mount=/run/snapd/ns/lxd.mnt /snap/lxd/current/bin/btrfs subvol create /var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-TEST-LAB and then do the delete again, that should clear it.

doesn’t seem to work either.

$ sudo nsenter --mount=/run/snapd/ns/lxd.mnt /snap/lxd/current/bin/btrfs subvol create /var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-TEST-LAB

ERROR: cannot access ‘/var/snap/lxd/common/lxd/storage-pools/default/containers’: No such file or directory

@lxd:/var/snap/lxd/common/lxd/containers$ sudo ls -ls
total 192
4 lrwxrwxrwx 1 root root 70 Apr 26 13:14 BDB-EMAC-LAB -> /var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-EMAC-LAB
4 lrwxrwxrwx 1 root root 77 Nov 23 14:17 BDB-LAB-VRF-Leaking -> /var/snap/lxd/common/lxd/storage-pools/default/containers/BDB-LAB-VRF-Leaking
4 d–x------ 2 root root 4096 May 4 14:57 BDB-TEST-LAB

don’t know why but the problem container doesn’t have a link but is a dir here…

You can delete that directory, that’s fine.

After deleting this dir I was able to do lxc delete.
Thanks for your help!