So picking up where we left off from IRC…
The proxy by default opens a new connection and then ‘proxies’ the actual packets from the original inbound connection to your LXD host to the new connection into your container.
This is why you lose the original source address.
As @stgraber mentioned there are two ways around this whilst still using the proxy device:
- If the application supports the PROXY protocol header then you can enable this on the proxy device by setting
proxy_protocol=true, this will instruct the proxy device to add the special header to the connections containing the original source IP allowing the application to harvest the original source IP.
- If the application isn’t TCP or can’t support the PROXY protocol header then you can use the proxy’s NAT mode by setting
nat=true on the proxy device. This actually switches to using iptables/nftables to setup a destination NAT (DNAT) rule on the host that forwards the original packets to the container, which allows the original source address to be maintained.
However with NAT mode, unlike the proxy mode, the port forwarding is using actual networking between the host and the container, and so requires that the container is accessible using a static IP address from the perspective of the LXD host.
In the default LXD configuration (which you indicate you are using) LXD sets up a private bridge on the host with a random private subnet. The containers then use DHCP to request a temporary IP address in that subnet.
Whilst the address that the container is allocated may appear static, this is because the allocation algorithm used tries to always give the same IP to the same MAC address. However if there are a lot of containers running this isn’t guaranteed. The risk when using proxy device in NAT mode is that the IP address you are forwarding packets to isn’t actually the container you intended. This is why we require that when using the proxy in NAT mode you set a static IP.
To set a static IP for a container you do this:
lxc start <container>
lxc ls <container>
# Find the IPv4 address that has been allocated dynamically from the output
lxc stop <container>
lxc config device override <container> eth0 ipv4.address=<dynamic IP from above>
What this does is assign the dynamically assigned IP as a static DHCP lease in LXD. This means that LXD’s DHCP server will no longer allocate that IP to other containers, and will always allocate it to the container’s current MAC address. No changes are required inside the container, it still continues to use DHCP to configure its address, its just that its guaranteed to always be assigned the same one.
This then allows proxy NAT mode to be enabled.
To enable proxy NAT mode do this:
lxc config device set <container> <proxy device name> nat=true listen=tcp:<lxd.host.ip>:7777 connect=tcp:0.0.0.0:7777
lxc start <container>
Note, in order for NAT mode to work, you have to specify a real IP on the LXD host for listen, you can’t specify ‘0.0.0.0’, so please use the LXD host’s public IP (or IP you want the game service to appear on that is local to the LXD host).
Also, because the packets are being forwarded using ‘real’ networking, you cannot specify ‘127.0.0.1’ as the connect address, as that is just pointing to the LXD host’s local loopback interface. Instead specify ‘0.0.0.0’ which translates to the container’s statically allocated IP address we setup above.