test with 4 cpus (after lxc launch ubuntu:lts test I also ran lxc config set test limits.cpu 4)
test2 with 8 cpus (same, but with 8 cpus)
test3 with 8 cpus (same, but with 8 cpus)
I then ran lxc exec <container> bash for each of these containers.
On test, I just monited online CPU cores from lscpu command. Got On-line CPU(s) list: 0,5,7,10 all the time.
For test2 and test3 I not only monitored online CPU cores, but also ran stress --cpu 8. The on-line core list for test2 and test3 was same, and remained same once the load was applied (On-line CPU(s) list: 1-4,6,8,9,11), which means htop on host showed less than 30% usage on 4 threads.
So I would like to ask, how to automatically spread the load between unused cores? Here is written that LXD supports dynamic load balancing when using core count instead of specific CPUs - how can I enable that?
LXD’s dynamic pinning just means that you can do limits.cpu=8 and get 8 vcpu rather than have to specify which 8 threads you want to be pinned to. Every time an instance starts or stops, LXD will go through all instances and update their pinning, trying to have as many instances per physical CPU thread.
This doesn’t take load into account at all and actively updating the pinning based on load would actually create a lot of load just by itself (forcing tasks and memory to be moved).
If load-balancing with a maximum CPU limit is what you’re looking for, you should unset limits.cpu (give access to all CPUs) and set limits.cpu.allowance=800% which will allow up to 8 full threads worth of CPU time when the system is under load, when the system isn’t loaded, the limit can be exceeded.
For a hard limit, you can use limits.cpu.allowance=80ms/10ms which should result in a hard limit of up to 8 full threads worth of time regardless of system load.
Worth noting that with those two, a top output won’t be super useful as burning all CPU in a container may not result in exactly 8 threads running at 100% but potentially 16 threads running at 50%.
I still think it would be nice feature to have especially for VPS hostings - run some check every ten seconds and in worst case scenario burn some resources to distribute the load evenly (Giving someone access to all the cores isn’t in this case optional).
It’s very naive, but it should be good enough in practice. It’s also inefficient, since it uses system() for interaction with lxd (it uses lxc list --format json and then lxc config set <instance> limits.cpu <list> for each container), and each of these calls takes more than 50 ms on i7-9xxx.
BTW, is /sys/fs/cgroup/cpuset/lxc.payload.<container_name>/cpuset.cpu correct path for changing assigned cpus? Thanks
