I need to limit the network bandwidth available to each LXC container using cgroup’s net_cls.classid feature. Each LXC container would have its own classid
value in such a way that all packets from containers would be tagged with the classid
and afterwards classified in the correct host configured traffic class where the bandwidth limit applies.
To achieve this, I followed these steps:
-
Configure traffic control:
tc qdisc del dev eno54 root tc qdisc add dev eno54 root handle 10: htb tc class add dev eno54 parent 10: classid 10:1 htb rate 10mbit tc class add dev eno54 parent 10: classid 10:2 htb rate 50mbit tc filter add dev eno54 parent 10: protocol ip handle 1: cgroup
The device
eno54
is the physical network interface that connect the host with the network. It’s part of the bridge where container virtual network interfaces are added.brctl show br0 bridge name bridge id STP enabled interfaces br0 8000.00163ee2fda2 no eno54
-
Set the
classid
value in container config file.lxctest1 container config file has: lxc.cgroup.net_cls.classid = 0x00100001 lxctest2 container config file has: lxc.cgroup.net_cls.classid = 0x00100002
-
Start both containers. Check that classid is correct and that they belong to the bridge.
lxc-start -n lxctest1 lxc-start -n lxctest2 cat /sys/fs/cgroup/net_cls/lxc/lxctest1/net_cls.classid 1048577 cat /sys/fs/cgroup/net_cls/lxc/lxctest2/net_cls.classid 1048578 brctl show br0 bridge name bridge id STP enabled interfaces br0 8000.00163ee2fda2 no eno54 veth0-lxctest1 veth0-lxctest2
-
Start
iperf
in both containers.Expected behaviour: iperf running on container lxctest1 being limited to 10 Mbps and iperf running on lxctest2 container being limited to 50 Mbps.
What I got: both
iperf
running unconstrained at maximum speed. -
I took the
iperf
process running onlxctest1
container and checked that it was in the tasks of thecgroup
pstree -c -p 37108 lxc-start(37108)───systemd(37118)─┬─agetty(37167) ├─agetty(37168) ├─dbus-daemon(37157) ├─rsyslogd(37156)─┬─{rsyslogd}(37161) │ └─{rsyslogd}(37162) ├─sshd(37336)───sshd(41156)───bash(41167)───iperf3(41523) ├─systemd-journal(37131) └─systemd-logind(37153) cat /sys/fs/cgroup/net_cls/lxc/lxctest1/tasks 37118 37131 37153 37156 37157 37161 37162 37167 37168 37336 39618 41156 41167 41523 $ cat /proc/41523/cgroup 10:memory:/lxc/lxctest1 9:hugetlb:/lxc/lxctest1 8:perf_event:/lxc/lxctest1 7:cpuset:/lxc/lxctest1 6:devices:/lxc/lxctest1 5:net_cls,net_prio:/lxc/lxctest1 4:blkio:/lxc/lxctest1 3:cpu,cpuacct:/lxc/lxctest1 2:freezer:/lxc/lxctest1 1:name=systemd:/user.slice/user-0.slice/session-1288.scope/user.slice/user-0.slice/session-1288.scope
-
I don’t know how to check that packets going out the container are actually being tagged with the classid value, but the reality is that packets are not filtered acording this value on the host and are not going to the correct class, where bandwidth limit is applied.
-
I’m using Oracle Linux 7 and the standard lxc package delivered in this distribution. Versions:
uname -a Linux exapru-aa.dit.aeat 4.1.12-112.14.15.el7uek.x86_64 #2 SMP Thu Feb 8 09:58:19 PST 2018 x86_64 x86_64 x86_64 GNU/Linux cat /etc/oracle-release Oracle Linux Server release 7.4 yum info lxc Loaded plugins: ulninfo Installed Packages Name : lxc Arch : x86_64 Version : 1.1.5 Release : 2.0.9.el7 Size : 725 k Repo : installed From repo : ol7_latest Summary : Linux Containers userspace tools URL : http://linuxcontainers.org License : LGPLv2+ Description : Containers are insulated areas inside a system, which have their own namespace : for filesystem, network, PID, IPC, CPU and memory allocation and which can be : created using the Control Group and Namespace features included in the Linux : kernel. : : This package provides the lxc-* tools, which can be used to start a single : daemon in a container, or to boot an entire "containerized" system, and to : manage and debug your containers.
-
What is wrong here? Anything wrong with this LXC version? Anything wrong with the setup?