I have a physical server with an external interface (MTU 1500), internal interface (MTU 1450), and lxdbr0 set to use MTU 1450.
In LXC there are 2 Ubuntu 22.04 guest VMs (single interface, MTU 1450) that receive service queries from the external network (MTU 1500), and get some data from back-end via internal network (MTU 1450). So far this has worked well.
$ lxc version Client version: 5.0.1 Server version: 5.0.1 $ snap list --all lxd Name Version Rev Tracking Publisher Notes lxd 5.0.0-b0287c1 22923 5.0/stable canonical✓ disabled lxd 5.0.1-9dcf35b 23541 5.0/stable canonical✓ - $ cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS" $ sudo lxc network show lxdbr0 config: bridge.mtu: "1450" ipv4.address: 192.168.1.1/24 ipv4.dhcp.ranges: 192.168.1.2-192.168.1.5 ipv4.nat: "true" ipv6.address: none description: "" name: lxdbr0 type: bridge used_by: - /1.0/instances/node1 - /1.0/instances/node2 - /1.0/profiles/default managed: true status: Created locations: - none
This problem started several days ago. I had
5.0.0-b0287c1 until recently, but it got updated together with OS packages so I can’t tell what specifically caused this change.
Now inbound service clients take a long time (2-6 seconds) to connect to service in LXC VMs. Once a client connects, everything works as fast as it used to, which makes me think of MTU and maybe DNS issues rather than bandwidth or packet loss.
I haven’t changed any firewall or LXC settings in recent days. The host’s network latency (inbound and outbound
ping) seems as unchanged (0.4ms to 18.104.22.168, < 40ms from my WAN client).
I’ve looked at various OS logs, NIC stats, MTU/MSS, but can’t find anything unusual. DNS queries resolve quickly both in LXC VMs and on the host.
I plan to use tcpdump to gather data from inbound connections from several symptomatic clients, but if anyone has an idea if it’d be worth to investigate LXC here, please let me know. So far I have no indications that anything is wrong with LXC and I’d rather focus on other areas. But if anyone has experienced increased latency in establishing of new connections to LXC VMs, please share if you think it’s related.
Is it worth changing external interface’s MTU from 1500 to 1450 (to be the same as the VMs and avoid fragmentation)? Current configuration (MTU 1500 on external NIC, 1450 on LXDBR) has worked well for me for several months, so I’m not sure I should mess with that. Does anyone have any experience or opinion with different vs. consistent MTU on host and LXC bridge? I don’t think that is related to my problem, but I’m curious about it in terms of best practices. The host is lightly loaded, so any benefits from less packet fragmentation would probably be negligible. But there may be other reasons why a consistent MTU value would be better.