Container folder duplication in /sys/fs/cgroup/unified/lxc/

LXD
1

Hi,
We are using LXD 3.0.3 on Ubuntu 18.04 in many production systems.
In a few systems, I see the ‘duplicated’ folders in sysfs for some containers. In the example below, container STSG is present with the folders STSG and STSG-1 and WilRo with the folders WilRo and WilRo-1:

# ll /sys/fs/cgroup/unified//lxc
total 0
drwxr-xr-x 35 root root   0 Mar  5 02:01 ./
dr-xr-xr-x  6 root root   0 Apr 17  2022 ../
drwxrwxr-x  2 root 100000 0 Mar  5 02:01 B436/
...........................
drwxrwxr-x  2 root 100000 0 Feb 21 17:59 STSG/
drwxrwxr-x  2 root 100000 0 Feb 23 01:06 STSG-1/
..................
drwxrwxr-x  2 root 100000 0 Mar  4 02:02 TTA/
drwxrwxr-x  2 root 100000 0 Mar  4 01:07 Wag/
drwxrwxr-x  2 root 100000 0 Feb  8 01:04 WilRo/
drwxrwxr-x  2 root 100000 0 Feb 21 18:00 WilRo-1/

Right now it is not possible to stop the STSG container:

# lxc exec stop STSG
Error: not found
# lxc exec stop STSG-1
Error: not found

No error messages were generated either in the lxd.log, in STSG/lxc.log, or in the journal.
I was more lucky with the second container WilRo. I was able to stop it but the folder /sys/fs/cgroup/unified/lxc/WilRo was not deleted. Only WilRo-1 disappeared.
Then I started the WilRo container again, it was started successfully. But the following error messages were generated in the WilRo.log:

lxc WilRo 20230305091611.984 ERROR    cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1219 - File exists - Failed to create directory "/sys/fs/cgroup/unified//lxc/WilRo"
lxc WilRo 20230305091611.984 ERROR    cgfsng - cgroups/cgfsng.c:create_path_for_hierarchy:1243 - Failed to create cgroup "/sys/fs/cgroup/unified//lxc/WilRo"
lxc WilRo 20230305091611.984 ERROR    cgfsng - cgroups/cgfsng.c:cgfsng_payload_create:1321 - Failed to create cgroup "/sys/fs/cgroup/unified//lxc/WilRo"
lxc WilRo 20230305091612.300 WARN     conf - conf.c:lxc_setup_devpts:1616 - Invalid argument - Failed to unmount old devpts instance

And the /sys/fs/cgroup/unified/lxc/WilRo-1 was created again.

Any idea why the problem happens and how to avoid it? For me, it looks like an LXC issue.

Thank you for your time,
Leonid

2

Hi Leonid,

Couldn’t you show us cat /proc/1/cgroup from the problematic machine?

This -NNN suffix after cgroup name is just an index of the cgroup hierarchy. For unified cgroup hierarchy you should see only one cgroup per container, AFAIU. Let’s try to understand what’s happening here.

Btw, would you mind to update to a newer versions of Ubuntu/LXD? 18.04/3 looks too old.

Regards,
Alex

3 
# cat /proc/1/cgroup 
12:hugetlb:/
11:rdma:/
10:memory:/
9:cpuset:/
8:perf_event:/
7:net_cls,net_prio:/
6:pids:/
5:freezer:/
4:devices:/
3:cpu,cpuacct:/
2:blkio:/
1:name=systemd:/init.scope
0::/init.scope