LXD 4.0.6.
I am trying to add a proxy for port 80 to a virtual machine, but I cant do it to a running instance (unlike containers), and then it says I can only do it with NAT mode, when I create the proxy with NAT mode, the VM wont start, so I have to assign a static IP address (probably because how NAT mode works) to get it to start.
It seems a little bit inflexible, going forward will this always be the case, i.e. is this a limitation with virtual machines?
Thanks.
Unlike containers, with VMs LXD cannot just switch into the namespace of the instance and setup a socket connection, because VMs run an entirely separate kernel from the LXD host.
This is what the proxy device relies upon for containers. It sets up a listening socket on the LXD host and then switches into the container’s namespace and creates a connection socket to the local loopback address (by default).
In this way LXD can proxy connections from the host to the container without actually utilising a network connection between the two.
This proxying of requests is also what causes the source address of the original request to be lost and when the connections appear inside the container the source address usually appears to be the local loop back address. This is somewhat mitigated by using the proxy_protocol mode if supported by the receiving application.
For scenarios where this isn’t possible, the proxy device also supports nat mode, where instead of actually proxying the connections, instead the LXD host sets up DNAT firewall rules to actually forward the request into the container. This allows the source address to be passed through to any application.
However this approach actually requires networking between the host and the container, and requires the container to be on a static address so that the DNAT rules can forward connections to the right place. It also requires the application to be listening on the static IP (rather than the loopback address).
VMs don’t have namespaces and so the original operation mode of the proxy device cannot be supported. Originally VMs didn’t support proxy devices at all. However limited support for proxy devices in nat mode were address, with the same requirements that the instance be given a static IP address.
Each device lists whether it is supported by a VM or not in the Device docs:
https://linuxcontainers.org/lxd/docs/master/instances#type-proxy
As for adding/changing the proxy device while the VM is on, we have just started adding support for hotplugging NICs, and it may well be possible to allow this also for proxy devices as beyond a static IP, there is no config required in the VM.