Running lxcfs inside docker container

connectrajeev · April 15, 2023, 1:22pm

Hello Everyone,

Well I am new to this forum and new to the lxcfs too and I am trying to use the lxcfs inside my docker-based container. I am expecting lxcfs to help provide me with correct information on the uptime and other container-specific information.

I am able to run the lxcfs (4.0.8) inside my container and I can see all values inside /var/lib/lxcfs/proc but when I invoke the uptime command, it shows me the uptime of my VM.

I am not able to specify the mount point of /var/lib/lxcfs/proc/uptime to /proc/uptime in my container manifest file it throws an error while starting the container. And the container hangs if I am performing the mount bind as follows:
mount -B /var/lib/lxcfs/proc/uptime /proc/uptime

Kindly suggest the right way to configure the lxcfs inside a docker container.

Thanks,
-Rajeev

amikhalitsyn · April 18, 2023, 1:06pm

Hi @connectrajeev

have you taken a look on https://github.com/lxc/lxcfs#using-with-docker ?

And yes, you don’t need to run lxcfs inside the container. You need to run lxcfs outside of the container and then use bindmounts.

connectrajeev · April 18, 2023, 2:45pm

Thanks, @amikhalitsyn
It is my bad, I was somehow under the impression that lxcfs should be run inside the container and I was trying that. But, now I have corrected it now and run the lxcfs on my host VM and mount the lxcfs mounted fuse inside my docker container.

After this, I see the uptime & free commands are returning the correct values but I noticed /proc/cpuinfo still shows all CPUs info available on my host VM. I have used --cpuset-cpus while creating my docker container and allotted 1,2 CPUs to my container, so I am expecting on the docker shell I should see only 2 CPU info in /proc/cpuinfo

Can you please tell me, why it is not working as expected, am I doing anything wrong?

Thanks,
-Rajeev

amikhalitsyn · April 18, 2023, 2:51pm

no, that’s correct behavior.

But if you really want to limit number of the CPU cores those are shown in /proc/cpuinfo you can run lxcfs with the --enable-cfs parameter. This should help.

connectrajeev · April 18, 2023, 3:09pm

Thanks for your response. Well, --enable-cfs is not helping, I still see the docker container is showing all CPU info in /proc/cpuinfo

amikhalitsyn · April 18, 2023, 3:15pm

in some cases it may not work:

if (cgroup_ops->can_use_cpuview(cgroup_ops) && opts && opts->use_cfs)

static bool cgfsng_can_use_cpuview(struct cgroup_ops *ops)
{
	struct hierarchy *cpu, *cpuacct;

	if (pure_unified_layout(ops))
		return true;

	cpu = ops->get_hierarchy(ops, "cpu");
	if (!cpu || is_unified_hierarchy(cpu))
		return false;

	cpuacct = ops->get_hierarchy(ops, "cpuacct");
	if (!cpuacct || is_unified_hierarchy(cpuacct))
		return false;

	return true;
}

In pure cgroup-v2 environment it will not work. That’s not a bug.

Can be useful to read:

github.com/lxc/lxcfs

lxcfs on /sys/devices/system/cpu

opened 02:29PM - 27 Jan 23 UTC

deemon87

# Required information LXD 5.0.2 # Issue description After update lxd 5.…0.1 -> 5.0.2 inside container (with cpu limit) there is mount point `lxcfs on /sys/devices/system/cpu/online type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)`, but in `/sys/devices/system/cpu` i see all cpu from hypervisor Before update was mount point `/sys/devices/system/cpu` and all cpu limits was correct. Now i have some problem with node_exporter (it show me all cpu not only limit), with clickhouse-client (`<jemalloc>: Number of CPUs detected is not deterministic. Per-CPU arena disabled.`)

github.com/lxc/lxcfs

when i use lxcfs-5.0.0/5.0.1/5.0.3 docker's cpu limit is not in effect

opened 03:30AM - 21 Feb 23 UTC

use-epoll

# Required information System: ubuntu20.04 Kernel: 5.4.0-139-generic Docke…r: 20.10.12 containerd: 1.5.9 (Controlled by docker) Installation steps: 1、cd ~/lxcfs-5.0.3 2、meson setup -Dinit-script=systemd --prefix=/usr build/ 3、meson install -C build/ 4、mkdir /var/lib/lxcfs/ 5、/usr/bin/lxcfs /var/lib/lxcfs docker run -it -rm --cpus=4 -m 4096m \ -v /var/lib/lxcfs/proc/cpuinfo:/proc/cpuinfo:rw \ -v /var/lib/lxcfs/proc/diskstats:/proc/diskstats:rw \ -v /var/lib/lxcfs/proc/meminfo:/proc/meminfo:rw \ -v /var/lib/lxcfs/proc/stat:/proc/stat:rw \ -v /var/lib/lxcfs/proc/swaps:/proc/swaps:rw \ -v /var/lib/lxcfs/proc/uptime:/proc/uptime:rw \ -v /var/lib/lxcfs/proc/slabinfo:/proc/slabinfo:rw \ -v /var/lib/lxcfs/sys/devices/system/cpu:/sys/devices/system/cpu:rw \ golang:1.19 /bin/bash golang:1.19 (Debian 11) # Issue description Question 1: When I use docker run in lxcfs-5.0.3 environment, the -m parameter limit does limit the memory space in the container, but when I use `-c 4` or `--cpus=4`, the container still shows the full cpu number of the host and does not limit it. PS: However, when I use `--cpuset-cpus="1,2,3,4"` parameter, the cpu is limited in the container. Question 2: I am currently using lxcfs-4.0.12 on my kubernetes(1.23.x) cluster (the current cpu and memory limits in k8s are normal), when I use python:3.9 image pyhton can correctly get the number of cpu I limit. However, when I use the golang:1.19 mirror, the `top` command correctly identifies my cpu limit, but golang's `runtime.Numcpu()` still gets the full number of cpus on the host PS: But I used docker `--cpuset-cpus="1,2,3,4"` in lxcfs-5.0.3, I used `runtime.Numcpu()` in golang:1.19 to also get the correct number of cpu after the limit

connectrajeev · April 20, 2023, 9:32am

Thanks for your response. I am using the cgroup-v1 environment. Can you please suggest, what can I do to fix the cpu & cpuacct related information on my containers? Is there a workaround/solution to fix it?

amikhalitsyn · April 24, 2023, 9:13am

sure, but please describe your configuration.

docker container configuration
LXCFS command line options
what exactly isn’t working properly for you