The listen config currently creates hairpin rules nicely. But there is one typical scenario where the rules generated do not quite suffice. Consider the following scenario:
- One server with one public IP
- Container 1: a reverse proxy with a listen config for ports 80 and 443 on the public IP
- Containers 2 and 3: application servers that are normally accessed through the reverse proxy
When the application servers want to address each other in a location-agnostic manner, they should use the FQDN, which translates to the public IP. The current NAT rules translates the packets from the application container to the reverse proxy to the proxy local IP, but the response is not SNAT’ed or MASQUERADEd back, resulting in a timeout.
Affected examples include federated cloud applications such as Nextcloud as well as Nextcloud’s integration with Collabora Office. Basically this applies to any application that wants to talk over a REST HTTPS API with another one when it happens to be on the same server.
There are obviously a ton of workarounds possible (IPv6, split horizon DNS, application intelligence, manually added NAT rules), so this question is really about having something elegant that integrates well with LXD. That way, it would not require a lot of orchestration as the container landscape evolves.
Listen rules can use ports, but since SNAT rules cannot rely on ports, such rules cannot be created 1:1 with the listen rules. The only theoretical approach I’ve come up with so far is SNAT based on connection marking. Have there been any other approaches already explored in LXD? Are there any other solutions for LXD but implemented outside of LXD out there?