according to the documentation you can use a combination of
lxc config set $container limits.cpu $number
and
lxc config set $container limits.cpu.allowance 10ms/100ms
I would expect this combination to limit the amount of cores $container can use and to allow every core in this container to peak to 10% of the core’s power.
But using limits.cpu.allowance limits the $container always to 1 core. No matter if you use it in combination with limits.cpu or not. And no matter if the server has more cores available.
As soon as i unset limits.cpu.allowance eighter all cpu cores of the server are accessible or, if i used limits.cpu the defined amount of cpu cores.
Is this behaviour correct ? If so, how to limit the $container to X cores and at the same time assigning a specific maximum cpu time to it ?
Setting limits.cpu.allowance doesn’t cause LXD to setup the cpuset configuration, so you aren’t pineed to any specific core at that stage. Your 10ms/100ms means you get a total of 10% of one core worth of CPU time.
limits.cpu lets you pin your container to some number of cores.
So if you have:
limits.cpu: 8
limits.cpu.allowance: 10ms/100ms
It expands in the kernel as your container being locked to 8 fixed CPU cores (determined by LXD) AND a total CPU consumption of 10% of one core.
In theory what you want may be closer to:
limits.cpu: 8
limits.cpu.allowance: 80ms/100ms
Which means you still get pinned to 8 cores but now get to use up to 80% of a core worth’s of CPU time, or assuming tasks spread nicely, an average of 10% per core you have access to.
Ah, pretty sure you’re not loosing them, it’s just that lxcfs doesn’t really have a way to render a combination of cpuset and CFS quotas both applied at the same time and it seems to go with the CFS route leading to the single CPU appearing in cpuinfo.
Try running:
grep Cpus_allowed_list /proc/self/status
To see what the actual pinning is like inside the container.
But the problem is, that the application checks via /proc/cpuinfo the number / sort of available cores.
Also applications like top get only one core.
That results in a situation, where the application will only start 1 thread, instead of 3 threads on 3 cpu cores. So even mathematically the cpu power is there, because of the limitation to render that combination, the application can`t make use of it.
Any idea from your side how this can be overcome ?
This would need extra logic in lxcfs to detect this case and try to do something smarter, likely rendering the right set of CPUs and alter their frequency to match the average cpu time you’d expect to get or something (not sure what the CFS code in lxcfs does exactly).
