Versions
- OS Version: Ubuntu 18.04 LTS
- LXD Version: 5.10. Routers are Linux containers running FRR.
- Kernel: 5.4.0-135-generic
- FRR Version: 8.4.2
Description
I tried to build a simple business scenario of VXLAN EVPN using LXD container, but I couldn’t learn the VTEP address of the other side after establishing the vxlan tunnel.
Routers are emulated by installing the FRR Routing Suite in the LXD container.
Topology diagram
The loopback address for leaf1 is 192.168.0.8/24，and the other two are similar.
Build process and configuration details
1. Create 5 containers, where leaf1, leaf2 and spine router install FRR.
2. Change the address in the container file /etc/netplan/50-cloud-init.yaml.
Leaf1(example):
network:
version: 2
ethernets:
eth0:
dhcp4: false
addresses: [2.2.2.253/24]
eth1:
dhcp4: false
addresses: [10.10.10.1/24]
- Use “lxc network create xxx ipv4.address=none ipv6.address=none” to create the bridges and connect them.
- Configure the protocols on the FRRs in the three routers of leaf1, leaf2, and spine, respectively.
spine:
router bgp 7677
bgp router-id 192.168.0.6
bgp bestpath as-path multipath-relax
neighbor fabric peer-group
neighbor fabric remote-as external
neighbor 192.168.0.8 peer-group fabric
neighbor 192.168.0.27 peer-group fabric
!
address-family l2vpn evpn
neighbor fabric activate
exit-address-family
!
leaf1:
router bgp 7675
bgp router-id 192.168.0.8
bgp bestpath as-path multipath-relax
neighbor fabric peer-group
neighbor fabric remote-as external
neighbor 192.168.0.6 peer-group fabric
!
address-family l2vpn evpn
neighbor fabric activate
advertise-all-vni
advertise-svi-ip
exit-address-family
leaf2:
router bgp 7676
bgp router-id 192.168.0.27
bgp bestpath as-path multipath-relax
neighbor fabric peer-group
neighbor fabric remote-as external
neighbor 192.168.0.6 peer-group fabric
!
address-family l2vpn evpn
neighbor fabric activate
advertise-all-vni
advertise-svi-ip
exit-address-family
!
Continue the configuration in the leaf1 and leaf2 container：
ip link add br10 type bridge
ip link add vxlan10 type vxlan id 10 local 192.168.0.8 dstport 4789 nolearning(on leaf2 the local address is 192.168.0.27)
ip link set br10 up
ip link set vxlan10 up
ip link set vxlan10 master br10
Results and phenomena
Leaf1:
leaf1# show bgp l2vpn evpn
BGP table version is 2, local router ID is 192.168.0.8
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 192.168.0.8:2
*> [2]:[0]:[48]:[0e:b0:e9:1e:0d:cb]:[128]:[fe80::c036:edff:fe7b:c5d3]
192.168.0.8 32768 i
ET:8 RT:7675:10
*> [3]:[0]:[32]:[192.168.0.8]
192.168.0.8 32768 i
ET:8 RT:7675:10
Displayed 2 out of 2 total prefixes
leaf1# show evpn vni
VNI Type VxLAN IF # MACs # ARPs # Remote VTEPs Tenant VRF
10 L2 vxlan10 1 1 0 default
leaf1# show bgp evpn vni
Advertise Gateway Macip: Disabled
Advertise SVI Macip: Enabled
Advertise All VNI flag: Enabled
BUM flooding: Head-end replication
VXLAN flooding: Enabled
Number of L2 VNIs: 1
Number of L3 VNIs: 0
Flags: * - Kernel
VNI Type RD Import RT Export RT Tenant VRF
* 10 L2 192.168.0.8:2 7675:10 7675:10 default
We can find that leaf1 does not learn the VTEP address of the other side, that is, there is no VXLAN tunnel established between leaf1 and leaf2.
I am not sure of the cause of the problem and have not found a good solution，so try to come here to find a possible answer.
- Is it a configuration error on my part?
- Using ip link in the LXD container does not change the configuration of the Linux kernel?
- Or do I need to configure tunnels and bridges in /etc/netplan/50-cloud-init.yaml instead of using “ip link”?
I would appreciate all suggestions and tips, thanks!