Hi there, thank you for all your work. I have a very nice and useful setup running with Bionic, Snap LXD, and a bunch of web services separated by containers using a single ip described by Simos Xenitellis here:
I use prerouting defined in /etc/ufw/before.rules to point to the HAProxy container that is using Letsencrypt. This setup gives me maximum isolation via containers, while also providing dense packaging. And I don’t have to do that via docker containers, but I still have the option to do so, because I can run docker containers inside of LXD containers. And I do. By the way, for some reason the Collabora Office Docker setup for my Nextcloud installations wasn’t stable, but luckily, Collabora is now providing package installs, so I switched.
Nonetheless, I had to add two additional public IP addresses (I only had to use one for roundabout ten separate sites in the last two years!!) for other services using the same ports.
Which method to use in this setup?
After checking around, I found out that there are currently several different methods:
- Turning the network interface of the host into a bridge. This didn’t work at all, while it should. There was another thread on here that discussed this. I will add the link, when I find it (somewhat like this: Make LXD use your server's public network interface) They also weren’t lucky. What happened to me was the networking stopped completely. I couldn’t make any connection.
Here: Lxd + Netplan + Static IP's in same subnet HOW-TO
I have the same problem sdurnov is describing. No network on host.
-
MACVlan setup. I didn’t try this, because it says that containers running on the host ip can’t talk to containers on the MACVlan IPs.
-
LXD Proxies. I wanted to try this, since it seems like the LXD project prefers this method, but didn’t find good documentation. I expect some quality content from Simos Xenitellis in the coming months (his work is awesome, btw.) Mainly I wanted to forward port ranges, instead of single ports and while I found confirmation, that it has been implemented via Github, I did not find documentation on how to set it up.
-
Simply adding Public IPs on the host’s network interface and using prerouting ipconfig rules, like I had before for my running HAProxy setup. I started out with this, but noticed that my containers on the first ip address from the host (it’s a /22 address, while the two additional IPs I got from my provider are /32, these are the instructions: netcup - IP-Adressen hinzufügen I strongly suspect it’s similar to how OVH does it) couldn’t reach the containers on the additional public IPs using the prerouting rules. I suppose there would be a way to make this work with some additional routing rules.
-
The method described above. Instead of using the method described by my provider and adding the IPs to the network interface, I can add the IPs to lxdbr0 and then add them inside the host. The only problem with this method for me is that “post-up” doesn’t work in Bionic, since it is on netplan. Thus, I change to static public IP and won’t use a private IPv4 at all, like you describe in this post. This is the reason I am putting up an answer to this particular post. Because it works fine with Netplan in Bionic.
Those are a lot of methods of routing IPs to containers. Do you have a preferred method? A best practice? Are all of those going to be supported in Focal? Will my setup with the line
still work like that in Focal?
Anyhow, thank you so much for all the useful stuff on here and all the great and useful software. :-
Edit:
In Bionic, you only need to edit /etc/netplan/50-cloud-init.yaml My, working, config looks like this, using your IPs. It looks like you can just add an IP to the DHCP4 setting and it will be added to the device.
network:
version: 2
ethernets:
eth0:
addresses: [94.130.21.165/32]
dhcp4: true
I couldn’t find a working Netplan config for a static setup without the private IP like the interfaces. I took the advice from @bodleytunes below and went with routed networking. I will be making a post over there, because my setup is a little different.