@simos, just opened my eyes to the existence of the nat=true parameter on the proxy devices…
It was recommended in the context of having a better performance.
In another post, it is mentioned that non-nat proxy devices fork the connection.
And then there’s the additional work of the Proxy-protocol=true to pass the proper headers…
I use HA Proxy to route Inet HTTP traffic into the containers and LXD proxy devices to route the traffic from the host to the HAP container.
In the docs it says that 2 things are different if you use nat=true.
The container needs to have a fixed IP.
There is no need for proxy_protocol=true anymore.
My 2 questions are:
How much more performant would nat=true be when compared with non-nat proxy + proxy protocol=true?
Or at which levels of traffic would that start to be noticeable? or worth it?
(if it ain’t broken don’t fix it)
What are the consequences of using a fixed IP for the HAP container? (if any).
Any other reason to use one over the other in a medium traffic environment?
NAT mode uses iptables/nftables rather than a separate proxy process for each proxy device configured.
This has benefits and drawbacks. The benefits are increased performance (although it would be worth doing your own tests in your own environment to see the difference) because there isn’t a separate process proxying packets in userspace and this is done at the kernel level.
This also has the benefit that the original IP source address is passed through to the container, whereas using proxy mode the original source IP is lost (unless using proxy protocol to pass it through).
The drawbacks of NAT mode are that because it is done at the network level, it requires a true and predictable network connection between host and container in order to setup the firewall NAT rules. This is why we require a statically assigned IP in the LXD config, so that LXD knows what IP the container will be given and can setup the NAT rules to match. It can also be sure it won’t change on the next DHCP lease renew. The proxy mode device on the other hand actually forwards packets in userspace between the LXD host and the container’s network namespace, allowing for a port to be exposed to the container without the container actually being on the network (i.e one can forward packets to 127.0.0.1 inside the container).
The other drawback is that you can only do TCP->TCP or UDP->UDP forwarding and cannot do the more exotic combinations that the proxy mode device supports (like forwarding to a UNIX socket).
The only difference of using a fixed IP is that it won’t change over time. Normally LXD’s DHCP server tries to give out the same IP each time anyway based on MAC address, but if your leases are busy, it doesn’t guarantee it won’t change. By allocating a fixed DHCP lease address (using lxc config device set eth0 ipv4.address=n.n.n.n), this guarantees LXD’s DHCP server will always give out the same IP for that container’s MAC address.