Containers and CPU utilization

Hello to everyone . Is there a way that I can check for each container that i have create , cpu utilization and all this stuff from the command line ? Something like the top command maybe ?

Short answer not really

Long answer you can get there or there abouts but its work (hopefully someone has some nifty script)

Isn’t there something like top command ? I found lxc-top but it shows nothing despite i have create a container. In fact i create the container through lxd , so i guess it is normal that lxc-top gives me zero output

Yeah if you wanna see the process, go on the host install htop, run htop press f5 (tree) then f3 (search) yourContainerName it will show you all the processes

LXD has lxc info mycontainer, which gives you some information.

Each Linux container is essentially a process tree that is separated from the rest of the host’s processes through cgroups. What you need, is software that understands cgroups and groups together the load of each process tree (a.k.a. Linux container).
See, for example, ctop.

Alternatively, there are tools like netdata that you install on the host and can give you instant insight into the containers.

Thanks a lot both of you for your answers.

Hi Thodoris,
here is a shell script, that you maybe can extend to your needs:

#!/bin/bash

lxc_list=$(lxc list --format=json | jq '.[] | "\(.name) \(.state.memory.usage) \(.state.processes) \(.state.cpu.usage) \(.state.network.eth0.counters.bytes_received) \(.state.network.eth0.counters.bytes_sent)"')
format="%15s %16s %12s %18s %12s %12s\n"
printf "$format" name memusage procs cpu recv sent
while IFS=\  read name memusage processes cpuusage bytes_received bytes_sent; do
     name=$(echo "$name"|cut -c 2-)                     # remove the first character (a quote)
     bytes_sent=$(echo "$bytes_sent"|rev|cut -c 2-|rev) # remove the last character (a quote)
     printf "$format" $name $(echo "$memusage"|numfmt --to=si --suffix=B) \
                      $processes \
                      $cpuusage \
                      $(echo "$bytes_received"|numfmt --to=si --suffix=B) \
                      $(echo "$bytes_sent"|numfmt --to=si --suffix=B)
done <<< "$lxc_list"

It creates something like:

           name         memusage        procs                cpu         recv         sent
  lxd-dashboard            299MB          120       129182767000         50KB         14KB
      nextcloud            116MB          255      3027363446000        777MB        115MB
    rocket-chat            377MB          120       109302610000         44KB         15KB
       ubuntu22            267MB          122       137490014000         57KB         14KB
      wordpress            335MB          120       122135800000         54KB         14KB

Btw, @Stephane (maybe knows that): What unit has cpu.usage? How to show some more meaningful? (like a percentage of the host-cpu i.e. - any idea?)

1 Like

@tkasidakis
Have a look at systemd-cgtop e.g. systemd-cgtop --depth=1 -p
otherwise you can use lxc exec <container> -- [top | htop | btop | etc...]
GNU Parallel will also help you run things in … well parallel :slight_smile:.

LXD does have metrics exporter and a Grafana dashboard you can use for this, see:

I simplyfied this aproach to be put into an shell alias:

alias lxcstats="printf \"%-20s %-8s %-5s %-5s %-s\n\" name status mem disk cpu-time; lxc list status=running --format=json | \
  jq  -r '.[] | \"\(.name) \(.status) \(.state.memory.usage) \(.state.disk.root.usage) \(.state.cpu.usage)\"' | \
  numfmt --field=3,4 --to=iec|\
  printf '%-20s %-8s %-5s %-5s %(%-dd %-Hh %-Mm )T\n' \$(</dev/stdin)"

btw: state.cpu.usage are cpu miliseconds

1 Like

I am sharing this for future google searches. This should also work with the LXD fork of INCUS.

The --verbose is useful to see the commands being sent to containers you can remove it.

The -q is critical to prevent parallel from stripping the quotes

The backslashes can be removed they are useful for line breaks on a long single line command.

# Parallel be verbose, handle quotes, jobs use all cores
# If you have many containers consider limiting cores -j4
parallel --verbose -q -j+0  \ 
# The ::: is a Parallels parameter 
# The {} is replaced by the container name from the commands after :::
lxc exec {} -- bash -c "apt-get update && apt-get upgrade -y" ::: \ 
# List json data pipe into jq command, parse containers that are Ubuntu
# alter these values for alpine, Arch, etc. 
# currently running then trim to clean it up.
$(lxc ls -f json | jq '.[] | \
select(.type == "container") | \
select(.config."image.os" == "ubuntu") | \ 
select(.status == "Running") | .name' | tr -d '"')
# Results in:
# <container1>
# <container2>
# ...

Each container replaces {} and executes the commands.

Here it is on a single line:

parallel -q -j+0 lxc exec {} -- bash -c "apt-get update && apt-get upgrade -y" ::: $(lxc ls -f json | jq '.[] | select(.type == "container") | select(.config."image.os" == "ubuntu") | select(.status == "Running") | .name' | tr -d '"')

You can turn this into a shell script or use the pet command to load it into a collection of long commands for easy access. The pet command can also do snippets so if you wanted this to run for a specific host you can do so. Or if you want to change the inputs to jq query. i.e. prompt for these values before running the shell one-liner.

If you are curious about the Grok3 session. I was definitely impressed by how well it was able to assist me.