When manually creating a network on a LXD cluster, you indeed need to use --target NAME for the create operation, targeting each of the clustern nodes one by one, passing the key=value arguments that are node specific.
Once the node-specific stuff is done, you can run a non-targeted create which will then ensure that all nodes have their node-specific config and will make that network be usable in the cluster.
Will cross host networking work if i use lxd init to join an existing cluster? Or is there something extra I have to do?
I created a cluster and tried creating containers and pinging them doesnât seem to be working. I can ping containers on the same machine as expected.
I think there are some changes coming in 3.1.0 that will allow you to have cross-node networking work out of the box, using a network fan. @stgraber will be able to tell more exactly.
Right, lxd init in LXD 3.1 (and in fact in the current snap) will setup a Fan overlay bridge for you if you donât pick a physical bridge or NIC.
The Fan can either use ipip or vxlan for tunneling, LXD defaults to using vxlan.
The difference between it and regular vxlan is that itâs setup automatically for you. Each host gets its own /24 subnet with LXD doing normal DHCP/DNS on it and the host acting as the gateway. The Fan then effectively sets up route to all the Fan subnets of your other hosts.
That way, thereâs no need for some kind of HA gateway that does DNS/DHCP/gateway for all your hosts, yet all containers can still contact each other.
Yeah, thatâll make it to Ubuntu 18.04 when we release LXD 3.0.1, hopefully in the next week or so (+1 week delay for it actually making it to 18.04 due to Ubuntu update gating).
Yeah, so LXD makes a good guess as to what interface and subnet to use, but that doesnât always work.
If it doesnât, just answer ânoâ to the question during init and configure it by hand afterwards, this will let you override the underlay network which should fix the problem.
So I just managed to get zerotier to work as a fan underlay for lxd. Works beautifully. I think I learned a bunch about networking / linux / containers. Itâs all much clearer in my head.
I have one question thought when I use the hostname to ping another container on the same machine they can find each other, but when theyâre remote they canât seem to find each other via hostname i have to use the IP
Indeed! I will make a video showing exactly how to do it. I have a website that teaches people about devops and building web apps https://www.codemy.net in fact Iâm hosting the new version on this lxd cluster Iâm setting up.
Iâve also figured out how to route request based on domain name to the various lxd containers.