What is the purpose of the lxdbr0 bridge has a ip address? it is just a l2 network device

[chenchen (陈琛)]

I know that the bridge is a l2 network device.
so even if it is not assigned ip address for lxdbr0 bridge,
each container could still communicate each other by l2.
In the following documents，

Bridge network

it mentioned ,
“LXD also sets up a local dnsmasq process to provide DHCP, IPv6 route announcements and DNS services to the network. By default, it also performs NAT for the bridge.”
Compare dockerbr0 and lxdbr0, docker daemon and dnsmasq,
both of their default bridge has ip address, and random subnet.

I really can’t distinguish work on l2 network & work on l3 network.
In this way,Isn’t lxdbr0 working on the l3 network ?
Who can help me sort out the concept？
I always believe that NAT and bridge are two different network modes，
But I can’t understand the real purpose of NAT and bridge at the same time.
i’m a beginner of network, excuse me.

[tomp (Thomas Parrott)]

In LXD a “managed” bridge (like lxdbr0 or those created using lxc network create --type=bridge) provide the following services:

• A virtual private L2 bridge for connecting instances (and host) together without the requirement of a shared physical L2 network, using, by default, random private addresses.
• Local DHCP, IPv6 route announcements, and DNS for the instances connected to the bridge.
• Firewall SNAT rules on the LXD host to allow outbound connections from the instances to be NATed onto the LXD host’s external network.

Most of these features can be optionally changed or disabled if needed.
It is also possible to have LXD’s DHCP, IPv6 RA and DNS services be provided to an external network by specifying the bridge.external_interfaces option on the managed network.

See Bridge network - LXD documentation

One can also use an existing manually configured bridge (see Netplan) that is connected to an existing network which LXD instances can connect to (using lxc config device add <instance> eth0 nic nictype=bridged parent=<manual bridge>) in these cases LXD will not provide the DHCP, IPV6 RA, DNS or SNAT services.

See Instance configuration - LXD documentation

[chenchen (陈琛)]

Thank you for your answer.
please check if i really understand it.

so,

• a virtual private l2 bridge is just like a l2 vswitch.It also has a MAC address translation table.
• virtual private l2 bridge can be either separate a new virtual network or not to separate and just use the exsiting network.
• if i need my instances connect to exsiting network, i should use “unmanaged bridge”,
• if i need all my instances keep in a virtual subnet,and all of their connections onto the external network should share the ip address of lxd host,in this case i should use “managed bridge”.
• although virtual private l2 bridge(managed bridge) provides random private addresses, but instances still connect to each other by mac address.
• the random private addresses is used for connect to external network.
• the ability of DHCP、IPv6 RA and DNS is from dnsmasq，but not bridge itself.

am i right? Thank you for your quick response.

Another question：
I want to construct a vpc environment.it just like a router.
i need to link a lot of subnet to VPC,
and i can Set routing rules for VPC.
In this way, I can realize the isolation between instances.

i know i can use vyos or openwrt,but are there any concept in lxd?
I need a technology that can create routers in batches.

[tomp (Thomas Parrott)]

Yes and no.

A bridge is just an L2 switch, with MAC address table.
An LXD bridge network provides a virtual bridge interface (lxdbr0) for the instances to connect to.
By default LXD creates bridges that are “private” because they only have the local instances connected to them, they are not connected (bridged) to the external network, and thus the instances are not reachable on the external network.

A LXD bridge network also provides DHCP, IPv6 RA and DNS services to instances connected to the bridge. These services are also not available to the external network. These services are provided by a dnsmasq process that LXD starts, and configures it to only listen on the managed bridge interface (so as to try and not conflict with any other services running on the LXD host and to not expose the service to the wider network).

It also sets up SNAT firewall rules to allow the instances to make outbound requests to the external network so that they all “hide” behind the LXD host’s IP on the external network. The random private addresses are never seen on the external network.

The L2 managed bridge allows for communication either by IP or MAC address. Each instance gets their own private IP and they generate their own MAC address on the bridge.

However you can create an unmanaged bridge (nothing to do with LXD), which you then fully control and can choose to connect that bridge to an external network. You can then choose to connect instances to that unmanaged bridge (and by extension the external network). In this scenario your instances would depend on any DHCP, IPv6 RA and DNS services provided by the external network.

In such setups the instances will still generate their own MAC address for use on the external network.

[tomp (Thomas Parrott)]

@chenchen

Another question：
I want to construct a vpc environment.it just like a router.
i need to link a lot of subnet to VPC,
and i can Set routing rules for VPC.
In this way, I can realize the isolation between instances.

i know i can use vyos or openwrt,but are there any concept in lxd?
I need a technology that can create routers in batches.

You can setup multiple managed bridge networks using lxc network create <network> each one will get its own private random subnet, and the LXD host will act as a router between them.

Or you can look at using our OVN network integration, but that is quite a bit more complicated.

See

OVN network

https://discuss.linuxcontainers.org/tag/ovn

[chenchen (陈琛)]

What you said is very clear.
I believe now I have understood how lxdbr0 works.
thank you very much.

[chenchen (陈琛)]

• i have a preliminary understanding of ovn & ovs.
• i 'm building one private cloud based on lxd.

and i have another related question.
i have a ip address pool for external network(for example, 192.168.5.50~192.168.5.100)

• 1. ip addr of eth0 on lxdserver1 is 192.168.5.50
• 2. ip addr of eth0 on lxdserver2 is 192.168.5.51
• 3. ip addr of eth0 on lxdserver3 is 192.168.5.52
• 4. ip range 192.168.5.53-192.168.5.100 is for vswitch(I want my VPC to have an independent external IP address)

i have already knowed that lxdbr0 could be a uplink network,
so i can create some ovn networks and set their parent network to lxdbr0.
anyway, the traffic of any instance in these ovn networks will route to lxdbr0 by snat and dnat.
but these openVSwitch doesn’t have a external ip address.

It can be seen from the official user manual of LXD, It seems that LXD wants users to pay more attention to the private network level, Instead of configuring for routers. So in the lxd restapi, you can see that the security rule group is configured based on the network, but not based on a vpc.
I don’t know how to plan the use of external IP pools in LXD.

I know that the more routes, the greater the network communication overhead.
So when I create too many instance in my ovn network and which parent is lxdbr0,
The overall network performance maybe poor.

but now what i want to ask is ,

1. to make full use of external ip addr pool, how should I create an ovn network？
2. Try not to treat lxdbr0 as an uplink network, and the network performance will be better. Am I right?
3. I didn’t find the lxd rest API about network device , how can i list all network device but not network in lxd?
   (just like lxc vpc list, lxc ovs list),I will need to configure IPSec and GRE for VPC in the future
4. In the figure below, i think there is a vswitch in the background, and it has a nic in ovn network ovn1 and ip is 10.49.23.1, the vswitch also has a nic in bridge network lxdbr0 and ip maybe 10.241.203.240, this depends on the following configuration, am i right?
   ipv4.ovn.ranges=10.241.203.200-10.241.203.254
5. how can i get the ip of vswitch in external network？i have config the ipv4.ovn.ranges but I don’t know which one is occupied.
6. Can i create a virtual switch with an public external network IP address？(doesn’t treat the bridge network as a parent) Do you think it’s good to do this?

I ask a lot of questions at one go，
I’m really interested in learning LXD/LXC,
Looking forward to your reply.

[tomp (Thomas Parrott)]

That is a lot of question! And I’m not sure I understand all of them.

But first lets get the basic requirements clear:

My understanding is that you are asking how to get external IPs from the uplink range 192.168.5.53-192.168.5.100 into your instances?

What I don’t understand yet is why you are using OVN to do this, as opposed to something like routed NIC , or using ip{n}.routes on a bridged NIC? Or using a proxy device?

The lxdovn1 OVS bridge you see is for connecting the uplink network lxdbr0 to one or more OVN networks. This is because the only way to connect a virtual OVN router to the host system is via an OVS bridge. In the case where the uplink network isn’t itself an OVS bridge, LXD creates an intermediate OVS bridge and connects the uplink to it, and then connects the OVS bridge to the OVN virtual router’s external virtual port. If you had multiple OVN networks all using lxdbr0 as the uplink they would all be connected to the same lxdovn1 bridge.

Please can you start with explaining what you are trying to achieve and why, as that will give me the context I need to answer properly. Thanks

[chenchen (陈琛)]

Hi there,

The above figure is the basic network topology.
You can see that there are two types of routers in the figure.

• The router of type 1 is directly connected to the external network
• The router of type2 is for connecting the uplink network lxdbr0 to one or more OVN networks. in my case type2 router is lxdovn1.

Please check if my understanding is correct

• lxdovn1 is Both a bridge and a router ？
• lxd provide type2 router but no type 1 router ？or how to create a type 1 router ？ should i use ovs?

I want to do a function of managing multiple routers.
I’m going to provide my users with the ability to create a tunnel.
GRE or IPSec may be adopted for tunnel technology.
I plan to provide the functions of creating and deleting routers through the GUI interface.
I hope my users decide whether to mount more ovn networks for the router.
I found that the router seems to be created automatically in LXD.
As long as I create an ovn network and set its parent network to lxdbr0,
It seems that LXD will automatically create a virtual router, I don’t know if I’m right.

I have learned about how to create instances like instance1.
Can I create a type1 router based on LXD technology?
Will the router of type 1 perform better?

If you think there is something wrong with my topology drawing, you are very welcome to point out my mistakes and give me some suggestions.

[tomp (Thomas Parrott)]

All OVN networks in LXD provide a virtual router (which connects to the uplink network) and a virtual switch (which connects to the virtual router and the instances connect to it).

There aren’t router types in LXD OVN networks. Only different uplink networks used by the OVN networks.

If you want to create routes between OVN networks you can use the LXD built-in peer feature, see How to create peer routing relationships - LXD documentation

[chenchen (陈琛)]

Thank you for your answer.

[chenchen (陈琛)]

   I have 3 node lxd cluster(lxdserver1, lxdserver2, lxdserver3), when i execute lxc network list command on any node, I found that there was only one network named lxdbr0. Instances c1 , cont-TqZRpG96g3, cont-0cNcjdGx9O are connecting to lxdbr0. I can ping by ip from container c1 to container cont-TqZRpG96g3, but i can’t ping by ip from container c1 to container cont-0cNcjdGx9O.
   c1 and cont-TqZRpG96g3 is on the same host(lxdserver1),and cont-0cNcjdGx9O is on the host lxdserver2.
   I guess , each host has a bridge network called lxdbr0, but my three bridge networks happen to have the same address, this is why only one lxdbr0 is displayed by the lxc network list. Bridge Network is a host level network，and this is why the c1 couldn’t ping cont-0cNcjdGx9O, am i right?
   To prevent confusion when using lxdbr0，I’d better change the address of lxdbr0 of each server to be different, am i right?

[tomp (Thomas Parrott)]

Yes thats right. Although you won’t be able to change the lxdbr0 address on each cluster member to be different as LXD managed networks have the same address config on all cluster members.

During lxd init there would have likely been a question about using the fan overlay network, and if you had used that option a network called lxdfan0 would have been created using a fan overlay network.

Alternatively you can use a manually configured bridge network that is connected to a shared L2 physical network between all cluster members and then connect your instance NICs to that using the lxc config device add <instance> eth0 nic nictype=bridged parent=<manual bridge>.

Or you can use LXD’s OVN network type to create a OVN overlay networks between the cluster members.