Get container with highest cpu and disk usage

1

Hi everyone,

maybe it’s basic stuff, but I can’t figure it out.

I wrote a small go script that gets the state of the container. State object is large, but there is what I need:

{
  ...
  "disk": {
    "root": {
      "usage": 211594752
    }
  },
  ...
  "cpu": {
    "usage": 109046765729
  }
}

The disk usage is in bytes. Right? So the container uses 211 MB

The cpu usage goes barely up if the container does nothing. I tested with stress tool for 10 seconds and the number raised very quick. Couldn’t find info what that number stands for.

Let’s say I have 3 container and I want to find out, which of them uses in the moment the most cpu and disk. For disk I just pick the highest number and done.

But for cpu?

Thanks for the help

2

CPU usage is number of seconds of CPU usage.

3

Ahh, it’s nanoseconds :slight_smile: Thanks Stéphane

Checked it with stress:

  • 1 core and 10 seconds -> number increased by 10 seconds
  • 2 cores and 10 seconds -> number increased by 20 seconds

But the number always goes up. How can I find out which container used the most cpu in the last seconds/minutes?

*edit: I could compare cpu usage now and (now - 1 second). If the usage got 1 sec higher -> 1 cpu used. Does that make sense?

4

You cant

5

Can you say what I can?

6

What are you trying todo, container scaling?

7

Lets say I have a virtual machine and in it 2-3 contaiener. Now the vm has high disk/cpu usage. For my thesis I want to look for container that mostly uses disk/cpu and migrate it to other vm which has less disk/cpu usage.

8

I would speculate its quite difficult to do what your trying todo based on CPU time

You can enable load averages per container using LXCFS but the load averages arent exported via LXD, so you will have to customise LXD or write a program to extract the information

9

I found a solution for my case:

Installed netdata on all VMs. Netdata provide an API

  • Disk usage of the vm: “<vm_ip>:19999/api/v1/data?chart=disk_space._&after=-1”
  • CPU usage of the vm: “<vm_ip>:19999/api/v1/data?chart=system.cpu&after=-1”
  • RAM usage of the vm: “<vm_ip>:19999/api/v1/data?chart=system.ram&after=-1”

And for each container I can do this:

  • CPU usage of the container: “<vm_ip>:19999/api/v1/data?chart=cgroup_lxc.payload_<CONTAINER_NAME>.cpu_per_core&after=-1”
  • RAM usage of the container: chart=cgroup_lxc.payload_<CONTAINER_NAME>.mem_usage_limit

Maybe it helps somebody

10

Dont all the containers report the same cpu load?

11

I don’t know if load and usage are the same, but here is what I checked:

  • VM has 2 CPUs
  • -> Container1: limit 1 CPU and allowance 10%
  • -> Container2: no limits -> 2 CPUs

Now I run in both container stress -c 2 -t 10

  • Container1 uses ~10% of cpu0
  • Container2 uses ~90% of cpu0 and ~100% of cpu1