LXD clustering - Fan networking

LXD
#1

Hi everyone,

I am creating LXC clustering between two nodes, when I try to bootstrap cluster on my first node, I got an error;
Invalid input: The auto-detected underlay (x.x.x.0/17) isn’t a /16 or /24, please specify manually.
All my nodes underlay network are in /17 range. Shall I use the /17 subnet range ?
I have few more questions ?
If I use the x.x.x.0/17 subnet then all the machines in that subnet by default will get 16k container IP’s ? So it means by defaults the machines in that particular subnet gets the capability to hold the n number of containers?

I have a scenario where I am using two VM in different subnet to configuration LXC clustering, I don’t want to specify the subnet range for the underlay network, I don’t want to enable this conf on other VM’s , for testing i just want to create fan network between these two VM’s ,
instead of Subnet Can I use the underlay server IP?

#2

If your underlay network is a /17 and your machines aren’t all in a subset of it that’s /24 or smaller, then you’ll need to set the underlay to /16.

This will still allow for 254 containers per system which likely will be enough.

The Fan networking does subnet mapping so that’s what LXD needs to provide to the kernel, there’s no point to point option for that.

#3

Thanks for your quick response @stgraber.
So, for instance if we have my vm’s to be in same subnet . Then i can use this subnet(x.x.x.x/17) for underlay network.

#4

Thank @stgraber,

I am so much confused right now with LXC networking, I understood it uses dnsmasq to create teh default networks. But in case it is totally different. I am trying to bootstrap a juju controller which in turns created a lxc brdige network for me, but the problem is that it useds one our private networkss to for the lxdbr0 brdige network? So it will result in ip conflict?
What is the default subnet lxc bridge ntwork will use Or is it make use of the VM subnet range?

#5

LXD managed bridges like lxdbr0 use a random unused /24 RFC1918 subnet in the 10.0.0.0/8 range. It attempts to ping the first and last address to see if used I believe.

But you can also provide whatever subnet you want to use instead, though I don’t know how juju may expose that.

#6

@Stgraber, yeah, I understood. So What will happen in the below scenario.

Incase, For example lxc make use of the random unused subnet of the lxc bridge network 199.133.x.x/24 and my host is in 192.138.x.x/27 subnet. I can ping my lxc containers from my host network and deploy my application in the containers with IP 192.139.x.1. And I expost port 80 using lxc config device , so that the traffic from to the host network is transferred to the container network. . Later on, someone from my our org create a subnet range 199.133.x.x/24 for theier application purpose as they haven’t aware of my container network. And I could see a VM with 192.139.x.1. is created and an web application is exposed on port 80 and this VM is reachable from my host server. So in that scenario, it will create a conflict right?
The lxc device will route traffic on my host to the destination containers but nslookup or curl on the 192.139.x.1:80 directs to which application ?

So what should I do in order to eliminate this ?

Shall I go to different class c subnet range, which is not in use.

And what will happen if we create such lxc bridge on more than 10 servers and it randomly used a subnet, range which is part of our internal network, this could cause an outage right ?

Thanks,
Ham

#7

@stgraber any guidance would be helpful.

#8

As long as you don’t connect the external NIC interface to the bridge (which isn’t done by default), then if your private lxdbr0 bridge does happen to conflict with a subnet on an upstream external network then the only impact will be that your instances will not be able to communicate outbound to services on that conflicting subnet, as it will believe it owns the subnet on the local LXD bridge and will send traffic requests there.

However because by default applies SNAT to traffic leaving the bridge via the external interfaces, your instances will not directly interfere with upstream addressing conflicts as they won’t appear to be coming from those conflicting addresses.

#9

Hey Thomas

Thanks for your reply. I understood by default it will create a conflict unless untill i connect it with my host NIC address. And also using proxy devices. I route my traffic on a particular port on the server to the container. This is not affect my VM application. What about the ping / nslookup? My container iP is 10.x.0.3 and I have another vm with ip 10.x.0.3 . if i do a ping/nslookup from my host having an ip address 10.x1.x.x , it resolves to which address to my container or to my server,?

#10

I don’t really understand what you mean by “conflict”, please can you give a concrete example of what you’re trying to achieve.

#11

Hi,

I meant what in case if in future if I add a subnet route default to all my VM’s with the subnet of the containers, it will create a conflict.

Can I create my containers with 192.168.x.x subnet ?

And also

#12

You can create a private managed bridge using which ever subnet you like.
However do you mean can you connect your LXD instances onto an existing network?