LXCFS does not work on ARM64 based Raspberry PI

LXCFS
1

It seems after setting resource limits on an ARM64 Raspberry PI 4 hosted containers, then running commands like:

htop
free
cat /proc/cpuinfo
cat /proc/meminfo

The hosts resources are shown instead of containers.

In comparison, same setup, same version on an x86_64 based LXD deployment shows things correctly.

*** I’m working on developing a HA Raspberry PI Cluster as a proof of concept ***

2

The rapsberry pi kernel is missing some of the cgroup controllers causing this issue.

1 Like
3

I see, are there any workarounds?

4

My Raspberry PI host OS is Ubuntu 20.04 btw as is the containers.

5

You’d need to switch to another kernel which has a more complete config or get the kernel you’re using to be rebuilt with what you need.

I don’t know if the generic Ubuntu kernel is bootable on raspberry pi, if it is, it would include those bits (though would likely miss some rpi bits), otherwise, you’ll need to file a bug against the rpi kernel to ask for those bits to be added.

Running lxd.check-kernel should show what’s missing.

1 Like
6

I’m using an ARM64 optimized Ubuntu 20.04 which was designed for Raspberry PI as far as I know.

7

Will check, thanks.

8 
/snap/lxd/19648/bin/lxc-checkconfig: 55: /snap/lxd/19648/bin/lxc-checkconfig: lxc-start: not found
LXC version 
Kernel configuration not found at /proc/config.gz; searching...
Kernel configuration found at /boot/config-5.4.0-1032-raspi
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
newuidmap is not installed
newgidmap is not installed
Network namespace: enabled

--- Control groups ---
Cgroups: enabled

Cgroup v1 mount points: 
/sys/fs/cgroup/systemd
/sys/fs/cgroup/cpu,cpuacct
/sys/fs/cgroup/pids
/sys/fs/cgroup/cpuset
/sys/fs/cgroup/perf_event
/sys/fs/cgroup/net_cls,net_prio
/sys/fs/cgroup/devices
/sys/fs/cgroup/freezer
/sys/fs/cgroup/rdma
/sys/fs/cgroup/blkio

Cgroup v2 mount points: 
/sys/fs/cgroup/unified

Cgroup v1 clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled

--- Misc ---
Veth pair device: enabled, not loaded
Macvlan: enabled, loaded
Vlan: enabled, not loaded
Bridges: enabled, loaded
Advanced netfilter: enabled, loaded
CONFIG_NF_NAT_IPV4: missing
CONFIG_NF_NAT_IPV6: missing
CONFIG_IP_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_IP6_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled, not loaded
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, not loaded
FUSE (for use with lxcfs): enabled, not loaded

--- Checkpoint/Restore ---
checkpoint restore: enabled
CONFIG_FHANDLE: enabled
CONFIG_EVENTFD: enabled
CONFIG_EPOLL: enabled
CONFIG_UNIX_DIAG: enabled
CONFIG_INET_DIAG: enabled
CONFIG_PACKET_DIAG: enabled
CONFIG_NETLINK_DIAG: enabled
File capabilities: 

Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /snap/lxd/19648/bin/lxc-checkconfig
9

The check actually looks broken or there’s other config causing the issue.

It shows the memory controller as being built into the kernel but it also shows that it’s not mounted, causing your issue.

1 Like
10

Hmm, corrective measures?

11

@stgraber,

Solved!

https://www.raspberrypi.org/forums/viewtopic.php?t=203128

The thread above found per your input, lead me to adjusting the file:

/boot/firmware/cmdline.txt

I added the following to the end of the existing configurations

cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1

After that I rebooted, and both CPU and Memory limits register in the container as intended via the commands originally noted above.

Thanks again @stgraber