So I’ve been digging into LXD and it’s been a great ride so far. I’m really enjoying the flexibility of it and the ability to run containers and VMs if needed. I’ve been reading a ton of posts on this forum as I’ve been learning and finally made an account to join in. Enough preamble.
So I’ve been making containers to learn and trying different ways of connecting them to the network. I’ve created containers with a macvlans profile to get an IP address from DHCP, I’ve created containers that used dnsmasq but connected host ports to the containers, and I’ve made a NGINX container to reverse proxy into other containers. I feel comfortable executing any of these steps now.
The thing I’m failing to understand is when would you use these various methods of creating a connection. I can see using a macvlans profile if the container is running something like PiHole that benefits from having a dedicated IP address from the router, but in general should I be trying to avoid macvlans and try to reverse proxy connections? Or does that only work for web services, and I should be trying to port forward instead? I guess I’m asking what the best practice is for connecting these containers to a network.
Not a super technical question, and very noobish, but I appreciate any input or use case examples.
Macvlans are a simpler way of connecting an instance (container or VM) to an external network at layer 2 without having to go through the (relative) complexity of setting up a bridge, moving the external interface onto that bridge and then using that to connect containers to.
Macvlans likely have better raw performance than bridges, however bridges allow the instance to communicate with the LXD host’s IP, whereas macvlans do not allow this by design.
However both macvlans and external bridges provide the same core functionality of fully exposing an instance to an external network. This is great when running services that rely on broadcasts, such as DHCP, inside the instances (such as pi-hole as you mentioned).
However in some environments there is only a single IP address allocated to the host (think VPS providers) and they cannot allocate another IP to use for the container on the external network.
In that case, a solution of using the proxy device (https://linuxcontainers.org/lxd/docs/master/instances#type-proxy) will allow certain specific ports on the LXD host’s external IP to be forwarded into the container’s private IP (the IP that is allocated from the lxdbr0 default private bridge).
There is also some degree of security with reverse proxy, as the instances are not fully exposed onto the external network.
So it really comes down to what application and/or type of network you are running in.
Thank you for the reply. I have a follow up question about proxy devices. When would it be preferable to use a reverse proxy server like NGINX over connecting ports directly to the container with a proxy device? At home, I had several containers that had services that were wanting to use the same ports. I forget what they were, I think Jellyfin was one of them. Since they both wanted to communicate over the same port, when I tried to make proxy devices LXD gave an error because the ports were already in use. At that point is when I learned to use NGINX instead. Is that a correct approach to the situation, or did I do something incorrectly and LXD has a tool for this scenario that I haven’t learned yet?
The LXD proxy device is a simple TCP/UDP proxy, so as you found, you can only have one proxy device bound to an IP/Port combination on the host (which means it can only forward that port into a single container).
However some protocols that run above TCP (such as HTTP) support the ability to do virtual hosting of multiple web sites on a single IP/Port combination by inspecting the HTTP Host header and then reverse proxying to a different IP based on the site being requested. This is where applications like nginx can be helpful as they can operate at the application layer to more deeply inspect and route packets.
But nginx would have similar limitations when reverse proxying non-http applications, which don’t have the concept of virtual hosting, and in these cases it may be simpler to just use the LXD proxy device.