- OS Version: Ubuntu 18.04 LTS
- LXD Version: 5.10. Routers are Linux containers running FRR.
- Kernel: 5.4.0-135-generic
- FRR Version: 8.4.2
I tried to build a simple business scenario of VXLAN EVPN using LXD container, but I couldn’t learn the VTEP address of the other side after establishing the vxlan tunnel.
Routers are emulated by installing the FRR Routing Suite in the LXD container.
The loopback address for leaf1 is 192.168.0.8/24，and the other two are similar.
Build process and configuration details
1. Create 5 containers, where leaf1, leaf2 and spine router install FRR.
2. Change the address in the container file /etc/netplan/50-cloud-init.yaml.
network: version: 2 ethernets: eth0: dhcp4: false addresses: [220.127.116.11/24] eth1: dhcp4: false addresses: [10.10.10.1/24]
- Use “lxc network create xxx ipv4.address=none ipv6.address=none” to create the bridges and connect them.
- Configure the protocols on the FRRs in the three routers of leaf1, leaf2, and spine, respectively.
router bgp 7677 bgp router-id 192.168.0.6 bgp bestpath as-path multipath-relax neighbor fabric peer-group neighbor fabric remote-as external neighbor 192.168.0.8 peer-group fabric neighbor 192.168.0.27 peer-group fabric ! address-family l2vpn evpn neighbor fabric activate exit-address-family !
router bgp 7675 bgp router-id 192.168.0.8 bgp bestpath as-path multipath-relax neighbor fabric peer-group neighbor fabric remote-as external neighbor 192.168.0.6 peer-group fabric ! address-family l2vpn evpn neighbor fabric activate advertise-all-vni advertise-svi-ip exit-address-family
router bgp 7676 bgp router-id 192.168.0.27 bgp bestpath as-path multipath-relax neighbor fabric peer-group neighbor fabric remote-as external neighbor 192.168.0.6 peer-group fabric ! address-family l2vpn evpn neighbor fabric activate advertise-all-vni advertise-svi-ip exit-address-family !
Continue the configuration in the leaf1 and leaf2 container：
ip link add br10 type bridge ip link add vxlan10 type vxlan id 10 local 192.168.0.8 dstport 4789 nolearning(on leaf2 the local address is 192.168.0.27) ip link set br10 up ip link set vxlan10 up ip link set vxlan10 master br10
Results and phenomena
leaf1# show bgp l2vpn evpn BGP table version is 2, local router ID is 192.168.0.8 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete EVPN type-1 prefix: :[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id] EVPN type-2 prefix: :[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP] EVPN type-3 prefix: :[EthTag]:[IPlen]:[OrigIP] EVPN type-4 prefix: :[ESI]:[IPlen]:[OrigIP] EVPN type-5 prefix: :[EthTag]:[IPlen]:[IP] Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 192.168.0.8:2 *> :::[0e:b0:e9:1e:0d:cb]::[fe80::c036:edff:fe7b:c5d3] 192.168.0.8 32768 i ET:8 RT:7675:10 *> :::[192.168.0.8] 192.168.0.8 32768 i ET:8 RT:7675:10 Displayed 2 out of 2 total prefixes
leaf1# show evpn vni VNI Type VxLAN IF # MACs # ARPs # Remote VTEPs Tenant VRF 10 L2 vxlan10 1 1 0 default
leaf1# show bgp evpn vni Advertise Gateway Macip: Disabled Advertise SVI Macip: Enabled Advertise All VNI flag: Enabled BUM flooding: Head-end replication VXLAN flooding: Enabled Number of L2 VNIs: 1 Number of L3 VNIs: 0 Flags: * - Kernel VNI Type RD Import RT Export RT Tenant VRF * 10 L2 192.168.0.8:2 7675:10 7675:10 default
We can find that leaf1 does not learn the VTEP address of the other side, that is, there is no VXLAN tunnel established between leaf1 and leaf2.
I am not sure of the cause of the problem and have not found a good solution，so try to come here to find a possible answer.
- Is it a configuration error on my part?
- Using ip link in the LXD container does not change the configuration of the Linux kernel?
- Or do I need to configure tunnels and bridges in /etc/netplan/50-cloud-init.yaml instead of using “ip link”?
I would appreciate all suggestions and tips, thanks!