Following the upgrade to 7.1, I’m unable to successfully start various OCI containers that had previously been working fine under 7.0.
What appears to be happening on incus start vw is that the container repeatedly tries to create all the cgroups it needs, but appears to be failing to create the final one (10)
Log (lxc.log):
lxc vw 20260530110922.536 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-1) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-1)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-2) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-2)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-3) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-3)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-4) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-4)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-5) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-5)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-6) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-6)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-7) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-7)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-8) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-8)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-9) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-9)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-10) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-10)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-11) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-11)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-12) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-12)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-13) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-13)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-14) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-14)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-15) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-15)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-16) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-16)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-17) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-17)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-18) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-18)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-19) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-19)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-20) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-20)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-21) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-21)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-22) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-22)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-23) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-23)
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-24) failed
lxc vw 20260530110922.537 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-24)
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-25) failed
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-25)
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-26) failed
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-26)
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-27) failed
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-27)
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-28) failed
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-28)
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-29) failed
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-29)
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-30) failed
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-30)
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgroup_tree_create:507 - File exists - Creating the final cgroup 10(lxc.monitor.vw-31) failed
lxc vw 20260530110922.538 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgroup_tree_create:550 - File exists - Failed to create monitor cgroup 10(lxc.monitor.vw-31)
lxc vw 20260530110922.594 ERROR utils - ../src/lxc/utils.c:safe_mount:1332 - Invalid argument - Failed to mount "none" onto "/opt/incus/lib/lxc/rootfs/run"
lxc vw 20260530110923.998 WARN cgfsng - ../src/lxc/cgroups/cgfsng.c:cgfsng_monitor_destroy:665 - Device or resource busy - Failed to destroy 10(lxc.monitor.vw-32)
lxc 20260530110923.998 ERROR af_unix - ../src/lxc/af_unix.c:lxc_abstract_unix_recv_fds_iov:218 - Connection reset by peer - Failed to receive response
lxc 20260530110923.998 ERROR commands - ../src/lxc/commands.c:lxc_cmd_rsp_recv_fds:128 - Failed to receive file descriptors for command "get_state"
This leaves a pile of redundant cgroup hierarchies in /sys/fs/cgroup which are not cleaned up by incus on service restart.
The OCI container reverts to stopped status.
After checking cgroup.procs was empty in each of the lxc.monitor.vw* cgroups created, I cleared the duplicated cgroups manually to see whether this would rectify the problem, however the same problem occurs on container start.
As an aside, a non-OCI container appears to start with no problems.
Any advice on how to rectify this problem and restart my OCI containers?