I’m looking for help in clarifying a few things related to LXD networking. I tried to get more insight (also about “bridges”) from web articles, but with limited success.
I noticed that LXD enables net.ipv4.ip_forward and adds the related iptables rules:
-A FORWARD -o lxcbr0 -j ACCEPT
-A FORWARD -i lxcbr0 -j ACCEPT
Could you explain why this is needed? It it because the host serves as a sort of router/middleman for the containers and forwards packets to them?
The ipv4.dhcp option in LXD seems to cause the following iptables rules to be added:
-A INPUT -i lxcbr0 -p udp --dport 67 -j ACCEPT
-A INPUT -i lxcbr0 -p udp --dport 53 -j ACCEPT
-A INPUT -i lxcbr0 -p tcp --dport 53 -j ACCEPT
-A OUTPUT -o lxcbr0 -p udp --sport 67 -j ACCEPT
-A OUTPUT -o lxcbr0 -p udp --sport 53 -j ACCEPT
-A OUTPUT -o lxcbr0 -p tcp --sport 53 -j ACCEPT
These are just to allow incoming/outgoing DHCP and DNS negotiation, in case the firewall drops by default, correct?
…is required for older buggy DHCP clients, presumably in containers. Can I assume that this rule would not be needed at all for modern containers (like recent Ubuntu, Alpine)?
The ipv4.firewall option serves as a master toggle for all of the above, correct?
Yes, exactly, the containers are on an isolated bridge and internet access in/out of that needs routing by the host. Those rules aren’t needed on systems that don’t have a firewall running, but they are for those systems where the default policy for forwarding isn’t accept. As a result, we just always inject those rules for consistency.
Correct
That’s correct, yes. I don’t know the state of most distros and which one had the patch applied. Last I checked, this patch wasn’t part of the official ISC dhcp client release, so anything using dhclient and that wasn’t distro patched would hit the issue unless this rule is present.
Yes, that key is used as a way to turn off our interactions with iptables/ip6tables for those that prefer to manage all that manually through separate firewalling scripts.
…be replaced with a -j SNAT --to-source x.x.x.x solution if the LXD host is behind a router? I assumed the source IP would just have to be set to the ip of the host’s ethernet interface (e.g. 192.168.10.2), but that doesn’t seem to work.
Maybe I’m misunderstanding what MASQUERADE actually does here, but reading online documentation didn’t bring me the insight I had hoped for.
masquerade is the same as nat overload in cisco speak, just hiding behind the egress interface IP.
I personally turn off nat and route to the container networks, run FRR with ospf on the container host and form adjacency with the core switch so it learns the routes of the various bridge networks via passive interfaces on the bridges.
natting works well if most of the connections are outbound only for the containers, but if you have a lot of inbound services then port forwarding with iptables becomes a bit of a chore. Also you could use haproxy inbound to the various containers so use as a tcp or http proxy.