Limit network bandwidth to LXC container

I need to limit the network bandwidth available to each LXC container using cgroup’s net_cls.classid feature. Each LXC container would have its own classid value in such a way that all packets from containers would be tagged with the classid and afterwards classified in the correct host configured traffic class where the bandwidth limit applies.

To achieve this, I followed these steps:

  1. Configure traffic control:

    tc qdisc del dev eno54 root
    tc qdisc add dev eno54 root handle 10: htb
    tc class add dev eno54 parent 10: classid 10:1 htb rate 10mbit
    tc class add dev eno54 parent 10: classid 10:2 htb rate 50mbit
    tc filter add dev eno54 parent 10: protocol ip handle 1: cgroup
    

    The device eno54 is the physical network interface that connect the host with the network. It’s part of the bridge where container virtual network interfaces are added.

    brctl show br0
    bridge name     bridge id               STP enabled     interfaces
    br0             8000.00163ee2fda2       no              eno54
    
  2. Set the classid value in container config file.

    lxctest1 container config file has: lxc.cgroup.net_cls.classid = 0x00100001
    lxctest2 container config file has: lxc.cgroup.net_cls.classid = 0x00100002
    
  3. Start both containers. Check that classid is correct and that they belong to the bridge.

    lxc-start -n lxctest1
    lxc-start -n lxctest2
    
    cat /sys/fs/cgroup/net_cls/lxc/lxctest1/net_cls.classid
    1048577
    cat /sys/fs/cgroup/net_cls/lxc/lxctest2/net_cls.classid
    1048578
    
    brctl show br0
    bridge name     bridge id               STP enabled     interfaces
    br0             8000.00163ee2fda2       no              eno54
                                                         veth0-lxctest1
                                                         veth0-lxctest2
    
  4. Start iperf in both containers.

    Expected behaviour: iperf running on container lxctest1 being limited to 10 Mbps and iperf running on lxctest2 container being limited to 50 Mbps.

    What I got: both iperf running unconstrained at maximum speed.

  5. I took the iperf process running on lxctest1 container and checked that it was in the tasks of the cgroup

    pstree -c -p 37108
    lxc-start(37108)───systemd(37118)─┬─agetty(37167)
                                   ├─agetty(37168)
                                   ├─dbus-daemon(37157)
                                   ├─rsyslogd(37156)─┬─{rsyslogd}(37161)
                                   │                 └─{rsyslogd}(37162)
                                   ├─sshd(37336)───sshd(41156)───bash(41167)───iperf3(41523)
                                   ├─systemd-journal(37131)
                                   └─systemd-logind(37153)
    
    cat /sys/fs/cgroup/net_cls/lxc/lxctest1/tasks
    37118
    37131
    37153
    37156
    37157
    37161
    37162
    37167
    37168
    37336
    39618
    41156
    41167
    41523
    
    $ cat /proc/41523/cgroup
    10:memory:/lxc/lxctest1
    9:hugetlb:/lxc/lxctest1
    8:perf_event:/lxc/lxctest1
    7:cpuset:/lxc/lxctest1
    6:devices:/lxc/lxctest1
    5:net_cls,net_prio:/lxc/lxctest1
    4:blkio:/lxc/lxctest1
    3:cpu,cpuacct:/lxc/lxctest1
    2:freezer:/lxc/lxctest1
    1:name=systemd:/user.slice/user-0.slice/session-1288.scope/user.slice/user-0.slice/session-1288.scope
    
  6. I don’t know how to check that packets going out the container are actually being tagged with the classid value, but the reality is that packets are not filtered acording this value on the host and are not going to the correct class, where bandwidth limit is applied.

  7. I’m using Oracle Linux 7 and the standard lxc package delivered in this distribution. Versions:

    uname -a
    Linux exapru-aa.dit.aeat 4.1.12-112.14.15.el7uek.x86_64 #2 SMP Thu Feb 8 09:58:19 PST 2018 x86_64 x86_64 x86_64 GNU/Linux
    
    cat /etc/oracle-release
    Oracle Linux Server release 7.4
    
    yum info lxc
    Loaded plugins: ulninfo
    Installed Packages
    Name        : lxc
    Arch        : x86_64
    Version     : 1.1.5
    Release     : 2.0.9.el7
    Size        : 725 k
    Repo        : installed
    From repo   : ol7_latest
    Summary     : Linux Containers userspace tools
    URL         : http://linuxcontainers.org
    License     : LGPLv2+
    Description : Containers are insulated areas inside a system, which have their own namespace
             : for filesystem, network, PID, IPC, CPU and memory allocation and which can be
             : created using the Control Group and Namespace features included in the Linux
             : kernel.
             :
             : This package provides the lxc-* tools, which can be used to start a single
             : daemon in a container, or to boot an entire "containerized" system, and to
             : manage and debug your containers.
    
  8. What is wrong here? Anything wrong with this LXC version? Anything wrong with the setup?