Connection resets on container servers

LXD
1

Hi

I have a problem where network connections are being reset on my container servers. It’s not the only server to do this but it seems worse than the others. Is there some sort of limit to how many connections can come out of LXD containers or is there a linux setting I can look at to try and resolve this?

2

What’s your networking setup in LXD?

3

It’s using the host bridge

4

Ok, make sure to configure your host bridge to have a fixed MAC address, this will most likely fix your issue.

1 Like
5

Unfortunately I have done that already. :unamused:

6

That’s pretty odd, usually the issue is that bridges have their MAC address change to match the lowest MAC on the bridge, causing that kind of issue. Probably still worth keeping an eye on the MAC to make sure it’s not changing.

Other than that, maybe you’ve got some kind of MAC conflict going on, or something that’s tripping your switch somehow?

If connected to a managed switch, maybe look at the logs there for anything odd too.

7

Thanks, will do. I’ve finally managed to get a decent tcp dump going and the SYN packets that I’m looking at are getting to the other server but it doesn’t seem to be sending ACKs. This is all very wierd. Thanks for looking at this.

8

Can you get some long running ping/mtr traces going between the host and a container, and separately, between the host and an external system. This will then help to identify if there is connection interruption on the bridge or on the external network.

9

Will do

10

27000 packets and zero loss.

11

Can you describe your setup in a little more detail, such as:

  • How many containers are you running concurrently?
  • Are you using long-running TCP connections, and is it only those that are failing?
  • What applications are seeing the failed connections, are they inbound or outbound connections?
  • Is there anything in your system logs at the time it happens?
  • What sysctl settings have to changed if any?
12

Hi

Thanks for looking at this. I think it’s resolved. The target server was receiving the traffic and just not responding. We upgraded it and so far so good. Holding thumbs that you don’t need to think any more about this.

1 Like