I have a server running Ubuntu-Server 20.04 with 8 cores. Cores 0-1 (system) are dedicated to os processes and 2-7 (user) are reserved (shielded) for user processes via cset. I would like to provision LXC containers via LXD and have them execute only on the “user” cpuset. With docker, that can be done by passing the option --cgroup-parent=/user (https://docs.docker.com/engine/reference/commandline/dockerd/#miscellaneous-options). What would be the LXD equivalent of that? I tried setting raw.lxc=lxc.cgroup.dir=/user in the configuration of my container but that doesn’t work: when I run:
inside the container only cpus 0 and 1 show up, which means that the container is running under the system cgroup - and indeed, under /sys/fs/cgroup/cpuset/system/user I do see the monitor and payload of my container. So I tried manually setting the cpuset.cpus of the container group to 2-7 with raw.lxc=lxc.cgroup.cpuset.cpus=2-7 but that of course didn’t work either: the container would just fail to start. The error message in the log is:
> Name: test-container > Location: none > Remote: unix:// > Architecture: x86_64 > Created: 2020/12/14 16:26 UTC > Status: Stopped > Type: container > Profiles: default > > Log: > > lxc test-container 20201214192010.525 WARN cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1152 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset/system/lxc.monitor.test-container" > lxc test-container 20201214192010.526 WARN cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1152 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset/system/lxc.payload.test-container" > lxc test-container 20201214192010.581 ERROR cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2873 - Permission denied - Failed to set "cpuset.cpus" to "2-7" > lxc test-container 20201214192010.581 ERROR start - start.c:lxc_spawn:1741 - Failed to setup cgroup limits for container "test-container" > lxc test-container 20201214192010.581 ERROR lxccontainer - lxccontainer.c:wait_on_daemonized_start:860 - Received container state "ABORTING" instead of "RUNNING" > lxc test-container 20201214192010.583 ERROR start - start.c:__lxc_start:1999 - Failed to spawn container "test-container" > lxc test-container 20201214192010.583 WARN start - start.c:lxc_abort:1013 - No such process - Failed to send SIGKILL via pidfd 31 for process 168790 > lxc 20201214192010.925 WARN commands - commands.c:lxc_cmd_rsp_recv:126 - Connection reset by peer - Failed to receive response for command "get_state"
I suspect this has to do with the fact that the lxd daemon is running under system.
What is the best way to achieve that? The only relevant threads which I found are: What is the best way to use numactl or taskset and chrt in lxd which cpus are isolated from the host
How to allocate cores which are in isolcpus list of host
where the only solution seems to be to manually assign the cpus to the containers.
OS: Ubuntu Server 20.04
LXD version: 4.8