Hello
In the process of migrating an LXD setup to Incus, encountering a snag regarding Bridge networking (Bridge network - Incus documentation).
** Use case:
A local “tap0” interface exists and provides access to an overlay network.
A “vpnbr0” network bridge is declared in Incus to bridge the tap0 interface with container veth.
The setup works with LXD but fails with Incus.
** LXD
In the LXD setup, a “vpnbr0” is created to bridging a local “tap0” interface with container veth:
lxc network create vpnbr0 bridge.external_interfaces=tap0 ipv4.address=none ipv4.dhcp=false ipv6.address=none ipv6.dhcp=false
Each container receives an additional interface upon creation to have access to the bridge:
lxc config device add $CONTAINER eth1 nic name=eth1 nictype=bridged parent=vpnbr0
One can easily check that the setup works through:
> >$ ip a
> > (...)
> > tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master vpnbr0 state UNKNOWN group default qlen 1000
> > link/ether 3e:a5:01:7e:88:7d brd ff:ff:ff:ff:ff:ff
> > inet 192.168.0.15/24 brd 192.168.255.255 scope global tap0
> > valid_lft forever preferred_lft forever
> > inet6 fe80::xxx:1ff:xxx:887d/64 scope link proto kernel_ll
> > valid_lft forever preferred_lft forever
(note the “master vpnbr0” in the 1st line)
and
$ lxc network info vpnbr0
Name: vpnbr0
MAC address: 00:16:3e:6f:8e:12
MTU: 1500
State: up
Type: broadcast
Network usage:
Bytes received: 3.03MB
Bytes sent: 0B
Packets received: 88051
Packets sent: 0
Bridge:
ID: 8000.00163e6f8e12
STP: false
Forward delay: 1500
Default VLAN ID: 1
VLAN filtering: true
Upper devices: tap0, vethxxxxx, vethxxxxx....
** INCUS
When trying to do the exact same actions on Incus, the result differ however, and the bridge connection to tap0 seems broken:
incus network create vpnbr0 bridge.external_interfaces=tap0 ipv4.address=none ipv4.dhcp=false ipv6.address=none ipv6.dhcp=false
$ ip a
tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000 # <--- NO MENTION OF "master vpnbr0"
link/ether 36:67:89:88:db:54 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.15/24 brd 192.168.255.255 scope global tap0
valid_lft forever preferred_lft forever
inet6 fe80::3467:89ff:fe88:db54/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
shows no “master vpnbr0” unlike with LXC
$ incus network info vpnbr0
Name: vpnbr0
MAC address: 00:16:3e:2f:ee:9a
MTU: 1500
State: up
Type: broadcast
Network usage:
Bytes received: 650.87kB
Bytes sent: 0B
Packets received: 1999
Packets sent: 0
Bridge:
ID: 8000.00163e2fee9a
STP: false
Forward delay: 1500
Default VLAN ID: 1
VLAN filtering: true
Upper devices: vethxxxxx # <--- NO MENTION OF TAP0
As a consequence adding a new interface to each containers in Incus is useless as they do not have access to the overlay network.
Having looked at the Incus documentation I do not see any change from LXD to Incus, however the same actions fail silently for Incus, as evident above where the “tap0” interface is missing.
I made sure services are started in the same order as the creation of the “tap0” interface to soon may prevent it from being “claimed” by the bridge. The same setup works with the production LXD servers, so looking for any insight on why vpnbr0 fails to attach itself to tap0.
Thanks!
