Virtual interfaces on host are bouncing up and down

LXD
1

I am running LXD 4.0.7 on Gentoo Linux:

$ lxd --version
4.0.7
$ uname -a
Linux t470 5.4.109-gentoo #1 SMP Mon Jun 14 20:49:12 CEST 2021 x86_64 Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz GenuineIntel GNU/Linux

I am using a network bridge:

$ lxc network show lxdbr0
config:
  ipv4.address: 10.248.20.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:9c78:69e0:463f::1/64
  ipv6.nat: "true"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/instances/gentoo-PG-C01
- /1.0/instances/gentoo-WS-C01
- /1.0/instances/ubuntu-PG-C01
- /1.0/profiles/default
managed: true
status: Created
locations:
- none

I have three containers with these virtual interfaces:

12: vethe85a88ba@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether da:dc:5c:4c:24:bb brd ff:ff:ff:ff:ff:ff link-netnsid 0
14: vethb351eb70@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether 8e:fc:81:15:0f:fd brd ff:ff:ff:ff:ff:ff link-netnsid 1
18: vethb2d12909@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether aa:0e:9e:25:e7:36 brd ff:ff:ff:ff:ff:ff link-netnsid 2

My problem: Unless I have connected to a container via network once, these interfaces bounce up and down. In the best case, I catch the interface while it is up and can connect, in the worst case, connection attempts freeze and I have to use the console and reset the network inside the container.

I have two Gentoo containers and a Ubuntu container, and it is doing this in all of them. I get notifications on the host (which is a desktop machine) and they are so distracting that, if I don’t actually need the container right then, I will shut it down just to shut it up.

It seems like it is not attaching to the bridge. How can I fix this?

2

The first thing I would check is the syslog to see if another network manager on your system is messing with them.

1 Like
3

In dmesg output, I see this (when I grep for veth):

$ dmesg | grep veth
[   12.411000] IPv6: ADDRCONF(NETDEV_CHANGE): veth1ef8bb87: link becomes ready
[   12.416314] lxdbr0: port 1(veth1ef8bb87) entered blocking state
[   12.416315] lxdbr0: port 1(veth1ef8bb87) entered disabled state
[   12.416353] device veth1ef8bb87 entered promiscuous mode
[   12.416378] lxdbr0: port 1(veth1ef8bb87) entered blocking state
[   12.416379] lxdbr0: port 1(veth1ef8bb87) entered forwarding state
[   12.503632] physWgEBjB: renamed from veth402fc81c
[   12.510666] lxdbr0: port 1(veth1ef8bb87) entered disabled state
[   12.516630] lxdbr0: port 1(veth1ef8bb87) entered blocking state
[   12.516631] lxdbr0: port 1(veth1ef8bb87) entered forwarding state
[   12.565284] lxdbr0: port 2(veth8dba24d3) entered blocking state
[   12.565285] lxdbr0: port 2(veth8dba24d3) entered disabled state
[   12.565319] device veth8dba24d3 entered promiscuous mode
[   12.565327] lxdbr0: port 2(veth8dba24d3) entered blocking state
[   12.565328] lxdbr0: port 2(veth8dba24d3) entered forwarding state
[   12.661518] phys4zWzsP: renamed from veth957ea230
[   12.665523] lxdbr0: port 2(veth8dba24d3) entered disabled state
[   12.671616] lxdbr0: port 2(veth8dba24d3) entered blocking state
[   12.671617] lxdbr0: port 2(veth8dba24d3) entered forwarding state
[   12.728440] IPv6: ADDRCONF(NETDEV_CHANGE): veth21a05385: link becomes ready
[   12.729889] lxdbr0: port 3(veth21a05385) entered blocking state
[   12.729890] lxdbr0: port 3(veth21a05385) entered disabled state
[   12.729938] device veth21a05385 entered promiscuous mode
[   12.729948] lxdbr0: port 3(veth21a05385) entered blocking state
[   12.729949] lxdbr0: port 3(veth21a05385) entered forwarding state
[   12.843950] physCtv50C: renamed from veth1d5eb287
[   12.850112] lxdbr0: port 3(veth21a05385) entered disabled state
[   12.856636] lxdbr0: port 3(veth21a05385) entered blocking state
[   12.856638] lxdbr0: port 3(veth21a05385) entered forwarding state
[ 6395.157813] lxdbr0: port 3(veth21a05385) entered disabled state
[ 6395.160208] lxdbr0: port 3(veth21a05385) entered blocking state
[ 6395.160209] lxdbr0: port 3(veth21a05385) entered forwarding state
[ 6395.214272] device veth21a05385 left promiscuous mode
[ 6395.214304] lxdbr0: port 3(veth21a05385) entered disabled state
[ 6396.172280] lxdbr0: port 3(veth14d1779a) entered blocking state
[ 6396.172282] lxdbr0: port 3(veth14d1779a) entered disabled state
[ 6396.172370] device veth14d1779a entered promiscuous mode
[ 6396.172388] lxdbr0: port 3(veth14d1779a) entered blocking state
[ 6396.172389] lxdbr0: port 3(veth14d1779a) entered forwarding state
[ 6396.173654] lxdbr0: port 3(veth14d1779a) entered disabled state
[ 6396.176763] IPv6: ADDRCONF(NETDEV_CHANGE): veth17adeb10: link becomes ready
[ 6396.176789] IPv6: ADDRCONF(NETDEV_CHANGE): veth14d1779a: link becomes ready
[ 6396.176801] lxdbr0: port 3(veth14d1779a) entered blocking state
[ 6396.176802] lxdbr0: port 3(veth14d1779a) entered forwarding state
[ 6396.311245] physUrCfHZ: renamed from veth17adeb10

It’s not clear what is doing this. I wonder if it has something to do with udev/eudev?

4

I think this udev rule is supposed to set the attribuate NM_UNMANAGED=1 on veth interfaces:

# cat /lib/udev/rules.d/85-nm-unmanaged.rules
# Do not modify this file, it will get overwritten on updates.
# To override or extend the rules place a file in /etc/udev/rules.d

SUBSYSTEM!="net", GOTO="nm_unmanaged_end"
ACTION!="add|change", GOTO="nm_unmanaged_end"

# VirtualBox host networking. Out-of-tree driver that looks like an ordinary
# Ethernet. No parent device (lives in /virtual/), no support for ethtool
# to identify the driver, MAC address defaults to 08:00:27:, but can be
# changed. Interface name will have to do, it's always vboxnet*.
ENV{INTERFACE}=="vboxnet[0-9]*", ENV{NM_UNMANAGED}="1"

# VMWare host networking. Out-of-tree driver that looks like an ordinary
# Ethernet. No parent device (lives in /virtual/), no support for
# ethtool to identify the driver. They have their own MAC prefix that
# can not be changed.
ATTR{address}=="00:50:56:*", ENV{INTERFACE}=="vmnet[0-9]*", ENV{NM_UNMANAGED}="1"

# Parallels Workstation host networking. Out-of-tree driver that looks like
# an ordinary Ethernet. No parent device (lives in /virtual/),  no support for
# ethtool to identify the driver and the interface name is too generic.
# However, they have their own MAC prefix that can not be changed.
ATTR{address}=="00:1c:42:*", ENV{INTERFACE}=="vnic[0-9]*", ENV{NM_UNMANAGED}="1"

# Virtual Ethernet device pair. Often used to communicate with a peer interface
# in another net namespace and managed by libvirt, Docker or the like.
ENV{ID_NET_DRIVER}=="veth", ENV{NM_UNMANAGED}="1"

# USB gadget device. Unmanage by default, since whatever created it
# might want to set it up itself (e.g. activate an ipv4.method=shared
# connection).
ENV{DEVTYPE}=="gadget", ENV{NM_UNMANAGED}="1"

LABEL="nm_unmanaged_end"

But when I look at them using udevadm, I don’t see NM_UNMANAGED:

root@t470 2021-08-23 14:02:21 /etc/udev # udevadm info /sys/devices/virtual/net/veth14d1779a
calling: info
P: /devices/virtual/net/veth14d1779a
E: DEVPATH=/devices/virtual/net/veth14d1779a
E: ID_MM_CANDIDATE=1
E: IFINDEX=18
E: INTERFACE=veth14d1779a
E: SUBSYSTEM=net
E: USEC_INITIALIZED=6397407351

root@t470 2021-08-23 14:02:33 /etc/udev # udevadm info /sys/devices/virtual/net/veth1ef8bb87
calling: info
P: /devices/virtual/net/veth1ef8bb87
E: DEVPATH=/devices/virtual/net/veth1ef8bb87
E: ID_MM_CANDIDATE=1
E: IFINDEX=12
E: INTERFACE=veth1ef8bb87
E: SUBSYSTEM=net
E: USEC_INITIALIZED=12403312

root@t470 2021-08-23 14:02:51 /etc/udev # udevadm info /sys/devices/virtual/net/veth8dba24d3
calling: info
P: /devices/virtual/net/veth8dba24d3
E: DEVPATH=/devices/virtual/net/veth8dba24d3
E: ID_MM_CANDIDATE=1
E: IFINDEX=14
E: INTERFACE=veth8dba24d3
E: SUBSYSTEM=net
E: USEC_INITIALIZED=12558806
5

Okay, I created a custom rules file (/etc/udev/rules.d/85-nm-unmanaged.rules):

# Virtual Ethernet device pair. Often used to communicate with a peer interface
# in another net namespace and managed by libvirt, Docker or the like.
ENV{INTERFACE}=="veth*", ENV{NM_UNMANAGED}="1"

The interfaces are now no longer managed:

$ nmcli -o d
DEVICE        TYPE      STATE                   CONNECTION                  
enp0s31f6     ethernet  connected               Kabelgebundene Verbindung 1 
lxdbr0        bridge    connected (externally)  lxdbr0                      
wlp4s0        wifi      disconnected            --                          
dummy0        dummy     unmanaged               --                          
erspan0       erspan    unmanaged               --                          
veth576670f3  ethernet  unmanaged               --                          
veth68d924cb  ethernet  unmanaged               --                          
veth9671cbf0  ethernet  unmanaged               --                          
gre0          iptunnel  unmanaged               --                          
gretap0       iptunnel  unmanaged               --                          
sit0          iptunnel  unmanaged               --                          
tunl0         iptunnel  unmanaged               --                          
lo            loopback  unmanaged               --

and the interfaces are no longer bouncing up and down, either.

Thank you Tom!

1 Like
6

Everybody who still gets this error: This problem is caused by a funtion called userlandproxy. It has the task to ensure that networking part of docker works seamless between OS/Distro. This is in Linux currently not needed aka deprecated and not documented.

Just search for docker disable userlandproxy

1 Like