Idle container's memory usage

LXC
1

Hi,

I have a basic question about the memory consumption of lxc container, and would like to get some confirmation from this forum, can someone please help me understand it better? The question is this:

I notice the used memory reported by “free” (inside of container) is decreasing, as the number of the same container copies are increasing. All containers are copied from the same one, and all of them are idle. By just starting more containers in idle state, the used memory reported by free inside of the idle container is getting smaller (I would think it should remain the same since the container itself is idle). Here is an example

started one copy

ubuntu@guiapps:~$ free -h
total used free shared buff/cache available
Mem: 953Mi 102Mi 724Mi 0.0Ki 125Mi 850Mi
Swap: 953Mi 0.0Ki 953Mi

the above showing that the used memory is 102MB.

started another one, currently two are idle, type free in the above container, here is the result

$ free -h
total used free shared buff/cache available
Mem: 953Mi 70Mi 868Mi 0.0Ki 14Mi 883Mi
Swap: 953Mi 27Mi 925Mi

“used” field is showing 70MB, the only difference in the host, comparing with the previous “free” is another copy of this container is started.

I understand /proc/meminfo and free are correctly containerized already, so they will report the used memory correctly. But I am a bit unsure about the above behavior. I am wondering if this is because of the memory accounting - same resources are shared among the same containers, so that more copies will amortize the used memory for each? It would be great if some one can help me understand this better, or some pointers/tools/commands so that I can further look at this?

Thank you in advance!

2

Most likely some amount of shared in-kernel objects between the counters which is causing the shared stuff to be spread over multiple containers.

We don’t really pull that stuff directly ourselves, it’s just what the cgroups are reporting for that set of process.

3

Hi Stephane,

Thanks for your quick response! Thanks for the answer! I would think the same. I am wondering if there is a way to examine those kernel memory. Maybe this question is more for cgroup rather than lxc? :slight_smile:

4

actually, can you please let me know how this containerized free command get the stats from cgroup?

I see some inconsistency in used memory reporting through different ways:

lxc info gives this:

Memory (current): 58.36MiB

which is the same as

$ cat /sys/fs/cgroup/memory/lxc.payload./memory.usage_in_bytes
58863616

“free” gives this
$ free
total used free shared buff/cache available
Mem: 976560 47776 918428 60 10356 928784
Swap: 976560 49416 927144

so seems cgroup and “lxc info” is the same, but free give different value.

Looks like this is how free get used memory

  • used - Used memory. It is calculated as: used = total - free - buffers - cache

but not sure which one is more accurate.

5

Yeah, lxcfs is a bit fancier than what LXD does, instead of just pulling the memory usage it also looks at the memory.stat file or whatever it’s called to try and split out buffer/cache.

6
7

Thanks for pointing me to the code. I got you are saying, essentially the difference is the buffers and cache.