I have been using LXD on Ubuntu 16.04 having had dnsmasq allocating (static) IP addresses to the containers. The configuration was in /etc/default/lxd-dnsmasq (if I remember correctly at the moment) and lxd-bridge (/etc/default/lxd-bridge) was configured to read that file.
Now I wanted to create a new environment on Ubuntu 18.04. I realize that a lot has changed in Ubuntu Bionic since Xenial (e.g. netplan), but what happened to LXD and dnsmasq? Isn’t there such an easy way to allocate IP addresses anymore?
I have read about using profiles etc., but it appears to be much easier simply running the containers using DHCP and automagically giving them static IP addresses based on the container name (using dnsmasq).
If you have a managed network in LXD, then LXD is taking care of providing for you a dedicated dnsmasq service to provide DHCP (and DNS) to the containers.
I do not remember seeing a dnsmasq service that works with LXD but is installed on the host’s system as you describe.
In Ubuntu 18.04 you are not affected much by netplan; the networking is there and LXD is doing its own thing with its own dnsmasq service for the managed networking.
When you perform the default configuration with LXD, with the bridge and all (default name lxdbr0), then you get a managed network and you are provided with a dedicated dnsmasq service to support DHCP and DNS to the containers.
Everytime I create a container named container1 it automagically should get IP 10.0.0.1 when it fetches the IP using DHCP, container2 should get 10.0.0.2 and so on.
I know this is not as flexible as it should/could be, but I have an Apache reverse proxy in front of these containers and it needs to know the IP of an specific container… somehow.
This is at least how I have used LXD on Xenial so far.
My plan is to create a cluster consisting of two VM:s containing 2 containers each. The purpose of the cluster is to run a web site. Each VM will have an Apache reverse proxy in front of the containers. The containers should have “public” IP:s in order for the load balancer to be able to balance the load to the containers directly… or is there a better way to achieve this kind of cluster?
See Using static IPs with LXD on how to set a specific IP to a container on a managed network interface in LXD.
In practice, these dynamically provided IP address tend not to change. Obviously, there is no guarantee that LXD’s dnsmasq DHCP server will be giving out the same IP address so in any case it’s good to set specific IP addresses.
Having said that, it’s good that you talk more about your use-case.
I am not sure how well is Apache suited for load-balancing. You may want to try with HAProxy, and here’s a guide. Note that you can also put the load-balancer in a container.
Are your websites dynamic (requiring a DB)? Do you have a special requirement so that the DBs are synced between the two VMs?
I don’t want to build any load balancing stuff myself, I use our load balancer. This is roughly what I intend to do (I wonder how this base64-encoded image will look like in the ready post?):
In this solution the nodes should have several aliases (cnames). Apache can then based on them pick the right vhost. Or maybe there should be only one vhost per node and the right container is then selected based on some environment variable set on the load balancer? Anyway, that’s not LXD’s responsibility.
I’ll start playing around with lxd network and let’s see what happens.
I’m really confused. It has really become complicated to find answers.
The link you provided above didn’t work for me (or the link worked but the solution didn’t). The container always gets the DHCP address no matter what I do. Any thoughts?
Personally, I prefer to manage my own DHCP/DNS using the actual dnsmasq package. This requires that you configure the dhcp pool for your bridge and then create static dhcp entries for the containers you want,as per below example:
If you have set up your bridge to be managed by LXD, there will be a link under /etc/dnsmasq.d/ to /etc/dnsmasq.d-available/lxd, will actually specifically excludes that bridge from being bound by dnsmaq itself. If you want to follow my proposal, you should remove this symlink and restart dnsmasq.
Obviously the “LXD-internal” dnsmasq is operating, but I don’t know how to shut it down. The “distro-dnsmasq” cannot start because port 53 is already in use.
I don’t know what to do. It cannot possible be this difficult to manage containers in newer LXD versions. There MUST be an easy way to set a static IP to the container or preferable have the container in DHCP mode and define the DHCP-stuff outside LXD. By the way, the instructions in “Using static IPs with LXD” linked to above by @simos didn’t work either (=the container didn’t get the IP I wanted it to have). So… what is the problem? My fresh install of Bionic? Or am I myself the problem?
Note that you need to assign an available IP address to the container. The IP address 10.10.10.1 (ends with .1) is obviously not available because it’s the address for the bridge.
Also, regarding your question on how to get the LXD managed bridge not to provide DHCP services to the container, you can just set ipv4.dhcp to false (see https://lxd.readthedocs.io/en/latest/networks/).
Personally, I think it gets very messy (difficult to manage) when you do your own networking on the containers. I would suggest to look into what is provided by LXD and use those. There is lots of functionality in there.
OK, yeah, you mentioned the magic words. Thanks! The static IP obviously must be within the range that the IP and netmask of the bridge defines. I didn’t think of that. Now I can also add the NIC to the container and set the static IP.
You are of course also right that it’s probably easier to use the tools LXD provide than to do stuff in an unintended way.
This is very manual and when I start new containers I don’t want to do stuff manually. How could the lines you typed above (network attach and config device) be automated and applied when launching new containers? Would that be in profiles? How?
The lxc network attach command has a lxc network attach-profile counterpart.
You can create a separate profile like staticip and attach that setting to this profile.
I have not seen an easy way to specify the IP address during lxc launch (https://lxd.readthedocs.io/en/latest/containers/).
The other option is to do this as before, but not having to start/stop/start the container.
You can use lxc init (note: it’s lxc init, not lxd init) to initialize the container (created but not started yet).
Then, do the configuration of the static IPv4 address and finally start the container.
I think I stick to what you just explained (network attach and config device) and then just make an alias (i.e. script) the container launching. In my case that would be quite all right. I won’t be using the distro-dnsmasq of course.
