For example: A single LXD HOST has Container C1 and C2 and C3.
Host IP: 10.0.1.100
C1 IP: 10.0.5.10 With Proxy from HOST to local Port 80
connect: tcp:10.0.5.10:80
listen: tcp:10.0.1.100:80
nat: "yes"
type: proxy
C2 IP: 10.0.5.20 With Proxy from HOST to local Port 80
connect: tcp:10.0.5.20:80
listen: tcp:10.0.1.100:90
nat: "no"
type: proxy
Now let’s perform some simple network tests from C3
C3 IP: 10.0.5.30
ping 10.0.1.100 (success)
telnet 10.0.1.100 80 (fail to C1)
telnet 10.0.1.100 90 (succes to C2)
The problem is that a Container (C3) cannot access proxied services of other container (exposed to host) if the proxy is using NAT.
Is this an inherit limitation of NAT? Can something be done that bypass this limitation? Thanks a bunch.
tomp
October 12, 2020, 12:50pm
2
Its quite normal when the br_netfilter kernel module isn’t loaded as iptables cannot track packets through the bridge.
Can you check in your LXD logs to see if you see the message mentioned here:
if ipFamily == "ipv4" {
connectAddr.Addr[i] = fmt.Sprintf("%s:%s", connectIP.String(), port)
} else if ipFamily == "ipv6" {
// IPv6 addresses need to be enclosed in square brackets.
connectAddr.Addr[i] = fmt.Sprintf("[%s]:%s", connectIP.String(), port)
}
}
err = d.checkBridgeNetfilterEnabled(ipFamily)
if err != nil {
logger.Warnf("Proxy bridge netfilter not enabled: %v. Instances using the bridge will not be able to connect to the proxy's listen IP", err)
} else {
if hostName == "" {
return fmt.Errorf("Proxy cannot find bridge port host_name to enable hairpin mode")
}
// br_netfilter is enabled, so we need to enable hairpin mode on instance's bridge port otherwise
// the instances on the bridge will not be able to connect to the proxy device's listn IP and the
// NAT rule added by the firewall below to allow instance <-> instance traffic will also not work.
_, err = shared.RunCommand("bridge", "link", "set", "dev", hostName, "hairpin", "on")
if err != nil {
If so, then you need to load the br_netfilter module using modprobe br_netfilter and reload LXD.
1 Like
@tomp You nailed it on the head. modprobe br_netfilter followed by lxd reload fixed the issue and the lxd.log contained exactly those error messages. Thanks!
1 Like