Is there a way to set the mtu globally for lxcbr & veth- that is for existing and new containers?
/etc/lxc/default.conf just works for new containers but not existing ones and only covers the veth part not lxcbr0
Probably not much sense if veth mtu is different than lxcbr0
LXC is supposed to sync the MTU with the bridge by default, so if you get lxcbr0 to have the right mtu, all containers started afterwards should inherit it.
Looked into the man pages but failed to track down the part where lxc is setting its mtu for lxcbr0?
Tried different stuff in /etc/default/lxc-net but that is apparently only for dhcp/dnsmasq.
Looks like lxc-net doesnât support that. You could edit the lxc-net script to have it do that for you or setup a separate init script which runs after lxc-net and updates the MTU to the preferred value.
That comes at a bit of surprise. What is the point that the veth mtu can be set with lxc.net.[].mtu but not the mtu for lxcbr0?
Where is that lxc-net script located?
What would be the syntax?
What happens to the lxc-net script when the lxc package gets updated in the distoâs repo and installed on the host?
lxc-net is a script that runs effectively outside of LXC to run some standard Linux commands like iproute2 and dnsmasq to provide a working network bridge to those that donât already have one.
The script can usually be found at /usr/lib/x86_64-linux-gnu/lxc/lxc-net. If modified, a subsequent package update may indeed overwrite your changes, thatâs why a separate init script may be preferred here.
Alternatively you can also setup your own bridge and just have LXC use that instead.
The veth mtu being settable on its own is useful to those that use LXC with a veth pair that isnât bridged on the host, the same config key is also used when passing a physical interface into the container.
Sure everything can done here and there but the lack of support for setting the mtu for lxcbr0 is clearly not a point in favour in lxcâs usability.
Anyway, I tried modifying the lxc-net script but did not get anywhere. Changes made in this section
_ifup() {
}
mtu 9000 and lxc-net restart crashed the bridge
iface lxcbr0 mtu 9000 prevented lxc-net from restarting
iface ${LXC_BRIDGE} mtu 9000 and lxc-net restart crashed the bridge
For the first part of your comment, patches are welcome. Running jumbo frames on the lxcbr0 bridge hasnât been a thing our users have needed so far but making it possible through an additional variable should be pretty easy.
You most likely just want an extra line in _ifup() that looks like:
ip link set dev ${LXC_BRIDGE} mtu ${LXC_BRIDGE_MTU:-1500}
At which point a LXC_BRIDGE_MTU line in /etc/default/lxc-net should do what you want.
I would certainly want to contribute with patches if it was not for my illiteracy in coding⊠
Thanks for the syntax though. I probably will setup a bridge manually as I am sure by the time lxc gets an update I will have forgotten about the change in the lxc-net script and pulling me hair again, of what is left.
On second thought though I would reckon that a manual bridge requires setting iptables accordingly in the firewall and changes to dnsmasq.
I am afraid that is not working.
setting LXC_BRIDGE_MTU= in /etc/default/lxc-net to value of <=1500 is no issue but anything >1500 is crashing the bridge.
Changing the value ${LXC_BRIDGE_MTU:-1500} to -9000 does not work either.
Has the lxcbr0 mtu eventually a hard coded ceiling of 1500?
lxcbr0 is just a regular Linux bridge so thereâs no specific MTU limit on itself.
Whatâs the error youâre getting when trying to set a higher MTU?
LXC_BRIDGE_MTU="1501" in /etc/default/lxc-net
systemd[1]: Stopping LXC network bridge setupâŠ
kernel: [ 1238.778543] lxcbr0: port 1(vethU54JJ8) entered disabled state
systemd[1]: Stopped LXC network bridge setup.
systemd[1]: Starting LXC network bridge setupâŠ
lxc-net[27071]: iptables: Bad rule (does a matching rule exist in that chain?).
lxc-net[27071]: message repeated 5 times: [ iptables: Bad rule (does a matching rule exist in that chain?).]
lxc-net[27071]: iptables: No chain/target/match by that name.
lxc-net[27071]: iptables: No chain/target/match by that name.
lxc-net[27071]: /usr/lib/x86_64-linux-gnu/lxc/lxc-net: 163: /usr/lib/x86_64-linux-gnu/lxc/lxc-net: cannot create /proc/sys/net/ipv6/conf/lxcbr0/accept_dad: Directory nonexistent
kernel: [ 1238.876330] lxcbr0: port 1(vethU54JJ8) entered blocking state
kernel: [ 1238.876333] lxcbr0: port 1(vethU54JJ8) entered forwarding state
lxc-net[27071]: RTNETLINK answers: Invalid argument
lxc-net[27071]: Failed to setup lxc-net.
kernel: [ 1238.882630] lxcbr0: port 1(vethU54JJ8) entered disabled state
lxc-net[27071]: iptables: Bad rule (does a matching rule exist in that chain?).
lxc-net[27071]: message repeated 5 times: [ iptables: Bad rule (does a matching rule exist in that chain?).]
lxc-net[27071]: iptables: No chain/target/match by that name.
lxc-net[27071]: iptables: No chain/target/match by that name.
systemd[1]: lxc-net.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: lxc-net.service: Failed with result âexit-codeâ.
systemd[1]: Failed to start LXC network bridge setup.
Hmm, so turns out this is a kernel limit. A bridge with nothing connected to it yet can only have a MTU of 1500 or lower.
To get a higher MTU, you need to have something bridged into it which itself should have the expected MTU and at which point you can increase your bridgeâs MTU.
However bridging ANY device in which has a MTU lower than the bridgeâs current MTU will have the bridge adopt that lower MTU.
So the short version is, thereâs nothing we can do in the lxc-net script because the kernel just doesnât let you create a bridge with a MTU thatâs higher than 1500. The only way you can do it is by increasing the bridgeâs MTU to 9000 after the first container has started. Note that the bridge MTU will go back down to 1500 as soon as the last container is stoppedâŠ
Turned out to be rather trivial. As pointed out by @stgraber an empty bridge cannot exceed MTU of 1500 howver the bidge automatically inherits the smallest MTU integer of its enslaved devicesâŠ
Thus lxc.net.[i].mtu takes nicely care of it. After the lxc-container is up the bridge on the host has indeed inherited the containerâs veth MTU, in this case 9000. Et voila jumbo frames on the bridge.