I have set up a three node cluster on a VPS for playing around. I want these to be on a LXD cluster. I am using ubuntu 18.04 with lxd 3.5. I have set up a VPN using wireguard. So I have a private network on wg0 and I can contact all of my nodes on this network.
I created a cluster, made my nodes join, and finally added a fan bridge (fanbr0) network which I have attached to containers as eth0. I can create containers which get correctly scheduled in all of the three nodes. I can exec each one of those and ping containers on any ones of the nodes.
However that’s where my luck ends. If I try to do any other networking I fail completely if the containers are on different nodes.
- I can’t use the lxd names (i.e. I can’t ping container-name.lxd which I understand should work since 3.4)
- I have tried setting up a HTTP server on one container in one host, and tried to run a HTTP GET on one container in another host. It just hangs.
- I can’t traceroute, but I can traceroute -T.
I thought I was having trouble with my firewall (ufw) so I enabled on all hosts incoming traffic through fanbr0. That didn’t help.
I feel I am close to success (being able to contact my containers on all nodes) but I am missing something (potentially simple). Can anyone give me a hint?