Debian11: dreaded No space left on device - Failed to clone a new set of namespaces

LXD
1

I just spent an hour trying to troubleshoot this issue and feel I’ve exhausted all threads on this forum, along with other results on Google.

I started by following this guide: https://ubuntu.com/blog/lxd-on-debian-using-snapd

Though it’s slightly outdated, it seems insanely easy. The basics are covered - /var isn’t short on storage, in fact, no partition is short on storage. This led me to believe there was a problem with the storage backend – I tried btrfs, then lvm, then dir for the hell of it - all resulting in the same thing. Obviously it has to be something else, but search results were being exhausted.

Name: horage
Status: STOPPED
Type: container
Architecture: x86_64
Created: 2022/01/22 20:41 CET
Last Used: 2022/01/22 20:42 CET

Log:

lxc horage 20220122194200.453 WARN     cgfsng - cgroups/cgfsng.c:__cgroup_tree_create:735 - File exists - Creating the final cgroup 10(lxc.payload.horage) failed
lxc horage 20220122194200.453 WARN     cgfsng - cgroups/cgfsng.c:cgroup_tree_create:795 - File exists - Failed to create payload cgroup 10(lxc.payload.horage)
lxc horage 20220122194200.453 WARN     cgfsng - cgroups/cgfsng.c:__cgroup_tree_create:735 - File exists - Creating the final cgroup 10(lxc.payload.horage-1) failed
lxc horage 20220122194200.453 WARN     cgfsng - cgroups/cgfsng.c:cgroup_tree_create:795 - File exists - Failed to create payload cgroup 10(lxc.payload.horage-1)
lxc horage 20220122194200.453 WARN     cgfsng - cgroups/cgfsng.c:__cgroup_tree_create:735 - File exists - Creating the final cgroup 10(lxc.payload.horage-2) failed
lxc horage 20220122194200.453 WARN     cgfsng - cgroups/cgfsng.c:cgroup_tree_create:795 - File exists - Failed to create payload cgroup 10(lxc.payload.horage-2)
lxc horage 20220122194200.453 WARN     cgfsng - cgroups/cgfsng.c:__cgroup_tree_create:735 - File exists - Creating the final cgroup 10(lxc.payload.horage-3) failed
lxc horage 20220122194200.453 WARN     cgfsng - cgroups/cgfsng.c:cgroup_tree_create:795 - File exists - Failed to create payload cgroup 10(lxc.payload.horage-3)
lxc horage 20220122194200.453 ERROR    start - start.c:lxc_spawn:1734 - No space left on device - Failed to clone a new set of namespaces
lxc horage 20220122194200.453 ERROR    start - start.c:__lxc_start:2074 - Failed to spawn container "horage"
lxc horage 20220122194200.453 ERROR    lxccontainer - lxccontainer.c:wait_on_daemonized_start:867 - Received container state "ABORTING" instead of "RUNNING"
lxc horage 20220122194205.459 ERROR    conf - conf.c:userns_exec_1:5038 - Failed to clone process in new user namespace
lxc horage 20220122194205.459 WARN     cgfsng - cgroups/cgfsng.c:cgfsng_payload_destroy:569 - No space left on device - Failed to destroy cgroups
lxc 20220122194205.485 ERROR    af_unix - af_unix.c:lxc_abstract_unix_recv_fds_iov:218 - Connection reset by peer - Failed to receive response
lxc 20220122194205.485 ERROR    commands - commands.c:lxc_cmd_rsp_recv_fds:127 - Failed to     receive file descriptors
2

further…

└─[20:44]# df -h
Filesystem                   Size  Used Avail Use% Mounted on
udev                         7.7G     0  7.7G   0% /dev
tmpfs                        1.6G  1.3M  1.6G   1% /run
/dev/nvme0n1p3                47G  5.4G   42G  12% /
tmpfs                        7.7G     0  7.7G   0% /dev/shm
tmpfs                        5.0M     0  5.0M   0% /run/lock
/dev/nvme0n1p2               948M  124M  824M  14% /boot
/dev/nvme0n1p1                93M  3.8M   89M   5% /boot/efi
/dev/loop0                    44M   44M     0 100% /snap/snapd/14295
/dev/loop1                    62M   62M     0 100% /snap/core20/1270

latest init options:

└─[17:55]# lxd init
Would you like to use LXD clustering? (yes/no) [default=no]:
Do you want to configure a new storage pool? (yes/no) [default=yes]:
Name of the new storage pool [default=default]:
Name of the storage backend to use (dir, lvm, ceph, btrfs) [default=btrfs]: lvm
Create a new LVM pool? (yes/no) [default=yes]:
Would you like to use an existing empty block device (e.g. a disk or partition)? (yes/no) [default=no]:
Size in GB of the new loop device (1GB minimum) [default=9GB]:
Would you like to connect to a MAAS server? (yes/no) [default=no]:
Would you like to create a new local network bridge? (yes/no) [default=yes]:
What should the new bridge be called? [default=lxdbr0]:
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
Would you like the LXD server to be available over the network? (yes/no) [default=no]:
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:

Further:

lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536

…added to /etc/lxc/default.conf

3

SOLVED: user.max_user_namespaces = 0 in sysctl.conf was causing this issue.