LXD cluster network question

LXD
1

I’ve created one network, and two containers attached to it in LXD cluster

user@node00:~/scripts$ lxc network create cls0
Network cls0 created
user@node00:~/scripts$ lxc network set cls0 ipv4.dhcp true
user@node00:~/scripts$ lxc launch -n cls0 ubuntu:17.10 cls0-node0
Creating cls0-node0
Starting cls0-node0
user@node00:~/scripts$ lxc launch -n cls0 ubuntu:17.10 cls0-node1
Creating cls0-node1
Starting cls0-node1
user@node00:~/scripts$ lxc ls
+------------+---------+---------------------+----------------------------------------------+------------+-----------+----------+
|    NAME    |  STATE  |        IPV4         |                     IPV6                     |    TYPE    | SNAPSHOTS | LOCATION |
+------------+---------+---------------------+----------------------------------------------+------------+-----------+----------+
| cls0-node0 | RUNNING | 10.109.4.127 (eth0) | fd42:6c81:86a:fe19:216:3eff:fe75:1f5c (eth0) | PERSISTENT | 0         | node00   |
+------------+---------+---------------------+----------------------------------------------+------------+-----------+----------+
| cls0-node1 | RUNNING | 10.109.4.214 (eth0) | fd42:6c81:86a:fe19:216:3eff:fed1:fe51 (eth0) | PERSISTENT | 0         | node02   |
+------------+---------+---------------------+----------------------------------------------+------------+-----------+----------+

However, container cls0-node0 can’t ping to cls0-node1. DNS is not working and also from cls0-node0

can’t ping to cls0-node1 with it’s IP address

How can I make containers in cluster, which are on different host nodes able to talk to each other

with their container names ?

and how can I attach host node to that network so host node can talk to containers in that network too?

What I want to achieve is, on top of LXD cluster composed of many nodes,

make separated logical clusters, which has own network.

And that network provides NAT to outside, containers attached to that network should be

able to reach other containers in the same network, but not containers in other networks

I’ve read documentation in here


but I couldn’t understand how network config works yet

1 Like
2

So as you’ve noticed, your configuration instructs LXD to setup a local bridge on each node and provide DHCP/DNS on it. This works as far as outgoing traffic is concerned but doesn’t get you cross-node networking or any kind of inbound networking.

LXD clustering right now very much assumes that you have an existing layer-2 infrastructure and that you’ll effectively just plug your containers into an existing physical network (VLAN) with a router provided DHCP/DNS for you. That’s the case which works best and if integrated with MAAS, LXD can even do IP management for you in this case.

If this is something you have, then create a new VLAN on your network, have your router provide DNS and DHCP on it, setup all your nodes to be attached to that VLAN and you can then connect your containers to that using something like:

lxc profile device remove default eth0
lxc profile device add default eth0 nic nictype=macvlan vlan=1010 parent=bond0 name=eth0

Which assumes that all your nodes have a bond0 device for the physical network and that you have a VLAN ID 1010 setup on it for your containers to use.

The go to alternative for this for setups without a dedicated network infrastructure would be to use a FAN bridge with LXD but we’re still working out some small problems with that, once we have those fixed, it will make it pretty simple to get a bunch of containers talking to each other without needing more complex network infrastructure.

1 Like
3

Thank you for your reply

But I have two questions

  1. Can’t I define a profile cluster wide, but configuration is different by all nodes in LXD cluster?

because some nodes have different NIC names

but I can’t edit profiles to be different by nodes

If its not possible, can’t I use macvlan for LXD network?

because network setting can vary by nodes

  1. For DNS part, can’t LXD network engine provide DNS functionality?

By making containers able to talk to each other through their container name as name service

but if it’s LXD assisted, it can plug DNS entry when container is being created and alter/remove one

when container address is changed or container is removed

or can I have this functionality with tools outside of LXD?

4

I’ve tried what you suggested
As expected, macvlan works but I’m not sure what to do with VLAN part
For switch side, (3Com baseline 2952 SFP switch) I’ve configured all ports to trunk mode.
But still vlan does not function. How should I setup my switch for container overlay network
with macvlan + VLAN setup?

Should I get inside and edit container’s network config
to recognize it’s network interface as VLAN tagged one?

5

I’ve figured it out

I just had to setup trunk option and also allow all vlan traffic to go through

all ports