Systemd-networkd: not working in Debian Sid or Bullseye images

If I remember correctly, the issue was systemd-networkd in containers. On a normal host or a VM, it’ll work just fine.

Hmm. That is strange, because when using images:debian/buster or archlinux in container it works.

Hi, I think I’m stumbling over this same issue. For me too “Link File” and “Network File” are “n/a” in the networkctl output like @pulsarr has shown. And networkctl seems to get this from "/run/systemd/netif/links/. Which unfortunately doesn’t have NETWORK_FILE=… either. It seems to me that somehow bus_method_reload()->link_reconfigure()->…->link_reconfigure_internal() is never called in systemd-networkd? At least I never see the “Re-configuring with /etc/systemd/network/<iface>.network” message in my systemd logs, which I see on another, working, non-containered Debian Sid.

I’m trying with Debian Bullseye, systemd 247.3-6 in the container and a Raspberry Pi Linux kernel 5.9.2-v8+ here. LXC version is 4.19.

One more difference I noticed in a working, non-containered Debian Sid is that it had “Link File: /usr/lib/systemd/network/99-default.link” on my br0 automatically. And this 99-default.link comes from the udev package, which is not installed by default in an LXD Debian Bullseye container. However installing udev in the container with apt and rebooting the container did not seem to help for me.

I did a bit more testing. What works for me:

  • Debian Bullseye + systemd v246, v248 or v249 build from the source

What does not work for me:

  • Debian Bullseye + systemd v247 build from the source (couldn’t build v241 or v244)
  • Debian Sid with systemd v249 provided by the LXD Debian Sid systemd package

I was a bit surprised that systemd v249 works on Debian Bullseye and not on Sid for me.

Protocol of the commands I used for building and testing:

root$ lxc launch images:debian/sid debian-sid-systemd-test
root$ lxc exec debian-sid-systemd-test -- /bin/bash

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ networkctl reload
($ systemctl restart systemd-networkd)
$ reboot

root$ lxc exec debian-sid-systemd-test -- /bin/bash

root@debian-sid-systemd-test:~# networkctl -l status eth0
● 49: eth0                                                           
                     Link File: n/a
                  Network File: n/a
                          Type: ether
                         State: routable (pending)
                  Online state: unknown
                    HW Address: 00:16:3e:2c:96:cb
                           MTU: 1394 (min: 68, max: 65535)
                         QDisc: noqueue
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp
                       Address: 2a03:2260:200f:101:216:3eff:fe2c:96cb
                                fd01:fd59:0:1800:216:3eff:fe2c:96cb
                                fe80::216:3eff:fe2c:96cb
                       Gateway: fe80::ba:7aff:fedf:101
             Activation Policy: up
           Required For Online: yes

Oct 19 21:42:35 debian-sid-systemd-test systemd-networkd[74]: eth0: Link UP
Oct 19 21:42:35 debian-sid-systemd-test systemd-networkd[74]: eth0: Gained carrier
Oct 19 21:42:35 debian-sid-systemd-test systemd-networkd[74]: eth0: Gained IPv6LL
root@debian-sid-systemd-test:~#

=> NOT OK!

$ exit 
root$ lxc stop debian-sid-systemd-test

=====

root$ lxc launch images:debian/bullseye debian-bullseye-systemd-test
root$ lxc exec debian-bullseye-systemd-test -- /bin/bash

$ echo "deb-src http://deb.debian.org/debian bullseye main" > /etc/apt/sources.list.d/deb-src.list
$ apt-get update
$ apt-get install git python3-jinja2
$ apt-get build-dep systemd

$ git clone https://github.com/systemd/systemd.git
$ cd systemd
$ git checkout v241

$ meson build/ && ninja -C build

$ git checkout v249
$ meson setup --wipe build/
$ meson build/ && ninja -C build
$ exit

=====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v249
root$ lxc start debian-bullseye-systemd-test-v249
root$ lxc exec debian-bullseye-systemd-test-v249 -- /bin/bash

$ cd systemd
$ meson install -C build/
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v249 -- /bin/bash

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ networkctl reload
($ systemctl restart systemd-networkd)
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v249 -- /bin/bash

root@debian-bullseye-systemd-test-v249:~# networkctl -l status eth0
● 29: eth0                                                                     
                     Link File: /usr/lib/systemd/network/99-default.link
                  Network File: /etc/systemd/network/eth0.network
                          Type: ether
                         State: routable (configured)
                  Online state: online                                         
                        Driver: veth
                    HW Address: 00:16:3e:e1:7e:b4 (Xensource, Inc.)
                           MTU: 1394 (min: 68, max: 65535)
                         QDisc: noqueue
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp
                       Address: 10.204.32.155 (DHCP4 via 10.204.32.2)
                                2a03:2260:200f:101:216:3eff:fee1:7eb4
                                fd01:fd59:0:1800:216:3eff:fee1:7eb4
                                fe80::216:3eff:fee1:7eb4
                       Gateway: 10.204.32.2
                                fe80::ba:7aff:fedf:101
                           DNS: 10.204.32.1
                                fd01:fd59:0:1800::1:1
             Activation Policy: up
           Required For Online: yes
               DHCP4 Client ID: IAID:0x4dda998c/DUID
             DHCP6 Client IAID: 0x4dda998c
             DHCP6 Client DUID: DUID-EN/Vendor:0000ab11499382881370f8d70000

Oct 19 20:29:56 debian-bullseye-systemd-test-v249 systemd-networkd[85]: eth0: Link UP
Oct 19 20:29:56 debian-bullseye-systemd-test-v249 systemd-networkd[85]: eth0: Gained carrier
Oct 19 20:29:56 debian-bullseye-systemd-test-v249 systemd-networkd[85]: eth0: Gained IPv6LL
Oct 19 20:29:56 debian-bullseye-systemd-test-v249 systemd-networkd[85]: eth0: DHCPv4 address 10.204.32.155/20 via 10.204.32.2

=> OK!

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v249

=====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v247
root$ lxc start debian-bullseye-systemd-test-v247
root$ lxc exec debian-bullseye-systemd-test-v247 -- /bin/bash

$ cd systemd
$ git checkout v247
$ meson setup --wipe build/
$ meson build/ && ninja -C build
$ meson install -C build/
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v247 -- /bin/bash

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ systemctl start systemd-networkd
$ networkctl reload
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v247 -- /bin/bash

root@debian-bullseye-systemd-test-v247:~# networkctl -l status eth0
● 35: eth0                                                           
                     Link File: n/a                                  
                  Network File: n/a                                  
                          Type: ether                                
                         State: routable (pending)      
                    HW Address: 00:16:3e:c7:e7:6a (Xensource, Inc.)  
                           MTU: 1394 (min: 68, max: 65535)           
                         QDisc: noqueue                              
  IPv6 Address Generation Mode: eui64                                
          Queue Length (Tx/Rx): 1/1                                  
              Auto negotiation: no                                   
                         Speed: 10Gbps                               
                        Duplex: full                                 
                          Port: tp                                   
                       Address: 2a03:2260:200f:101:216:3eff:fec7:e76a
                                fd01:fd59:0:1800:216:3eff:fec7:e76a  
                                fe80::216:3eff:fec7:e76a             
                       Gateway: fe80::ba:7aff:fedf:101               

Oct 19 20:56:22 debian-bullseye-systemd-test-v247 systemd-networkd[79]: eth0: Gained IPv6LL
root@debian-bullseye-systemd-test-v247:~#

=> NOT OK!

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v247

=====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v241
root$ lxc start debian-bullseye-systemd-test-v241
root$ lxc exec debian-bullseye-systemd-test-v241 -- /bin/bash

$ cd systemd
$ git checkout v241
$ meson setup --wipe build/
$ meson build/ && ninja -C build
[...]
ninja: Entering directory `build'
[479/1743] Compiling C object src/network/libnetworkd-core.a.p/netdev_tunnel.c.o
../src/network/netdev/tunnel.c: In function ‘ipip_init’:
../src/network/netdev/tunnel.c:765:27: warning: implicit conversion from ‘enum <anonymous>’ to ‘FooOverUDPEncapType’ [-Wenum-conversion]
  765 |         t->fou_encap_type = FOU_ENCAP_DIRECT;
      |                           ^
[550/1743] Generating linux.c.o with a custom command
../src/boot/efi/linux.c:63:72: warning: ‘regparm’ attribute directive ignored [-Wattributes]
   63 | typedef VOID(*handover_f)(VOID *image, EFI_SYSTEM_TABLE *table, struct SetupHeader *setup) __attribute__((regparm(0)));
      |                                                                        ^~~~~~~~~~~
[552/1743] Generating shim.c.o with a custom command
../src/boot/efi/shim.c:21:9: warning: ‘sysv_abi’ attribute directive ignored [-Wattributes]
   21 |         EFI_STATUS __attribute__((sysv_abi)) (*shim_verify) (VOID *buffer, UINT32 size);
      |         ^~~~~~~~~~
../src/boot/efi/shim.c:25:9: warning: ‘sysv_abi’ attribute directive ignored [-Wattributes]
   25 |         EFI_STATUS __attribute__((sysv_abi)) (*generate_hash) (VOID *data, UINT32 datasize, VOID *context, UINT8 *sha256hash, UINT8 *sha1hash);
      |         ^~~~~~~~~~
../src/boot/efi/shim.c:27:9: warning: ‘sysv_abi’ attribute directive ignored [-Wattributes]
   27 |         EFI_STATUS __attribute__((sysv_abi)) (*read_header) (VOID *data, UINT32 datasize, VOID *context);
      |         ^~~~~~~~~~
[556/1743] Generating stub.so with a custom command
FAILED: src/boot/efi/stub.so 
/usr/bin/ld -o src/boot/efi/stub.so -T /usr/lib/elf_aarch64_efi.lds -shared -Bsymbolic -nostdlib -znocombreloc -L /usr/lib /usr/lib/crt0-efi-aarch64.o --defsym=EFI_SUBSYSTEM=0xa src/boot/efi/disk.c.o src/boot/efi/graphics.c.o src/boot/efi/measure.c.o src/boot/efi/pe.c.o src/boot/efi/util.c.o src/boot/efi/linux.c.o src/boot/efi/splash.c.o src/boot/efi/stub.c.o -lefi -lgnuefi /usr/lib/gcc/aarch64-linux-gnu/10/libgcc.a
/usr/bin/ld: src/boot/efi/graphics.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/pe.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/util.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/linux.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/splash.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/stub.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
[557/1743] Generating systemd_boot.so with a custom command
FAILED: src/boot/efi/systemd_boot.so 
/usr/bin/ld -o src/boot/efi/systemd_boot.so -T /usr/lib/elf_aarch64_efi.lds -shared -Bsymbolic -nostdlib -znocombreloc -L /usr/lib /usr/lib/crt0-efi-aarch64.o --defsym=EFI_SUBSYSTEM=0xa src/boot/efi/disk.c.o src/boot/efi/graphics.c.o src/boot/efi/measure.c.o src/boot/efi/pe.c.o src/boot/efi/util.c.o src/boot/efi/boot.c.o src/boot/efi/console.c.o src/boot/efi/shim.c.o -lefi -lgnuefi /usr/lib/gcc/aarch64-linux-gnu/10/libgcc.a
/usr/bin/ld: src/boot/efi/graphics.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/pe.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/util.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/boot.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/console.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/shim.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
[561/1743] Generating test-libsystemd-sym.c with a custom command (wrapped by meson to capture output)
ninja: build stopped: subcommand failed.
root@debian-bullseye-systemd-test-v241:~/systemd# echo $?
1
root@debian-bullseye-systemd-test-v241:~/systemd# 

=> build FAILS

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v241

====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v248
root$ lxc start debian-bullseye-systemd-test-v248
root$ lxc exec debian-bullseye-systemd-test-v248 -- /bin/bash

$ cd systemd
$ git checkout v248
$ meson setup --wipe build/
$ meson build/ && ninja -C build
$ meson install -C build/
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v248 -- /bin/bash

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ systemctl start systemd-networkd
$ networkctl reload
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v248 -- /bin/bash

root@debian-bullseye-systemd-test-v248:~# networkctl -l status eth0
● 45: eth0                                                                     
                     Link File: /usr/lib/systemd/network/99-default.link
                  Network File: /etc/systemd/network/eth0.network
                          Type: ether
                         State: routable (configured)
                        Driver: veth
                    HW Address: 00:16:3e:c9:2d:1d (Xensource, Inc.)
                           MTU: 1394 (min: 68, max: 65535)
                         QDisc: noqueue
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp
                       Address: 10.204.33.69 (DHCP4 via 10.204.32.2)
                                2a03:2260:200f:101:216:3eff:fec9:2d1d
                                fd01:fd59:0:1800:216:3eff:fec9:2d1d
                                fe80::216:3eff:fec9:2d1d
                       Gateway: 10.204.32.2
                                fe80::ba:7aff:fedf:101
                           DNS: 10.204.32.1
                                fd01:fd59:0:1800::1:1
             Activation Policy: up
               DHCP4 Client ID: IAID:0x511822df/DUID
             DHCP6 Client IAID: 0x511822df
             DHCP6 Client DUID: DUID-EN/Vendor:0000ab11499382881370f8d70000

Oct 19 21:28:45 debian-bullseye-systemd-test-v248 systemd-networkd[94]: eth0: Gained IPv6LL
Oct 19 21:28:45 debian-bullseye-systemd-test-v248 systemd-networkd[94]: eth0: DHCPv4 address 10.204.33.69/20 via 10.204.32.2
root@debian-bullseye-systemd-test-v248:~#

=> OK!

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v248

=====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v244
root$ lxc start debian-bullseye-systemd-test-v244
root$ lxc exec debian-bullseye-systemd-test-v244 -- /bin/bash

$ cd systemd
$ git checkout v244
$ meson setup --wipe build/
$ meson build/ && ninja -C build
[...]
ninja: Entering directory `build'
[504/1549] Compiling C object src/network/libnetworkd-core.a.p/netdev_tunnel.c.o
../src/network/netdev/tunnel.c: In function ‘ipip_sit_init’:
../src/network/netdev/tunnel.c:732:27: warning: implicit conversion from ‘enum <anonymous>’ to ‘FooOverUDPEncapType’ [-Wenum-conversion]
  732 |         t->fou_encap_type = FOU_ENCAP_DIRECT;
      |                           ^
../src/network/netdev/tunnel.c: In function ‘gre_erspan_init’:
../src/network/netdev/tunnel.c:774:27: warning: implicit conversion from ‘enum <anonymous>’ to ‘FooOverUDPEncapType’ [-Wenum-conversion]
  774 |         t->fou_encap_type = FOU_ENCAP_DIRECT;
      |                           ^
[593/1549] Generating stub.so with a custom command
FAILED: src/boot/efi/stub.so 
/usr/bin/ld -o src/boot/efi/stub.so -T /usr/lib/elf_aarch64_efi.lds -shared -Bsymbolic -nostdlib -znocombreloc -L /usr/lib /usr/lib/crt0-efi-aarch64.o --defsym=EFI_SUBSYSTEM=0xa src/boot/efi/disk.c.o src/boot/efi/graphics.c.o src/boot/efi/measure.c.o src/boot/efi/pe.c.o src/boot/efi/util.c.o src/boot/efi/linux.c.o src/boot/efi/splash.c.o src/boot/efi/stub.c.o -lefi -lgnuefi /usr/lib/gcc/aarch64-linux-gnu/10/libgcc.a
/usr/bin/ld: src/boot/efi/graphics.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
/usr/bin/ld: src/boot/efi/pe.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
/usr/bin/ld: src/boot/efi/util.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
/usr/bin/ld: src/boot/efi/linux.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
/usr/bin/ld: src/boot/efi/splash.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
/usr/bin/ld: src/boot/efi/stub.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
[598/1549] Generating test-libudev-sym.c with a custom command (wrapped by meson to capture output)
ninja: build stopped: subcommand failed.
root@debian-bullseye-systemd-test-v244:~/systemd# echo $?
1
root@debian-bullseye-systemd-test-v244:~/systemd# 

=> build FAILS

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v244

=====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v246
root$ lxc start debian-bullseye-systemd-test-v246
root$ lxc exec debian-bullseye-systemd-test-v246 -- /bin/bash

$ cd systemd
$ git checkout v246
$ meson setup --wipe build/
$ meson build/ && ninja -C build
$ meson install -C build/
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v246 -- /bin/bash

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ systemctl start systemd-networkd
$ networkctl reload
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v246 -- /bin/bash

root@debian-bullseye-systemd-test-v246:~# networkctl -l status eth0
● 57: eth0                                                                     
                     Link File: /usr/lib/systemd/network/99-default.link       
                  Network File: /etc/systemd/network/eth0.network              
                          Type: ether                                          
                         State: routable (configured)
                        Driver: veth                                           
                    HW Address: 00:16:3e:a6:e3:35 (Xensource, Inc.)            
                           MTU: 1394 (min: 68, max: 65535)                     
                         QDisc: noqueue                                        
  IPv6 Address Generation Mode: eui64                                          
          Queue Length (Tx/Rx): 1/1                                            
              Auto negotiation: no                                             
                         Speed: 10Gbps                                         
                        Duplex: full                                           
                          Port: tp                                             
                       Address: 10.204.32.252 (DHCP4 via 10.204.32.2)          
                                2a03:2260:200f:101:216:3eff:fea6:e335          
                                fe80::216:3eff:fea6:e335                       
                       Gateway: 10.204.32.2                                    
                                fe80::ba:7aff:fedf:101                         
                           DNS: 10.204.32.1                                    
                                fd01:fd59:0:1800::1:1                          
               DHCP4 Client ID: IAID:0x5c902b8f/DUID                           
             DHCP6 Client IAID: 0x5c902b8f                                     
             DHCP6 Client DUID: DUID-EN/Vendor:0000ab11499382881370f8d70000    

Oct 19 22:31:13 debian-bullseye-systemd-test-v246 systemd-networkd[73]: eth0: Gained IPv6LL
Oct 19 22:31:13 debian-bullseye-systemd-test-v246 systemd-networkd[73]: eth0: IPv6 successfully enabled
Oct 19 22:31:13 debian-bullseye-systemd-test-v246 systemd-networkd[73]: eth0: DHCPv4 address 10.204.32.252/20 via 10.204.32.2
root@debian-bullseye-systemd-test-v246:~#

=> OK!

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v246

=====

Edit:

With further bisect’ing I found:

So it seems to be related to the return code of some udevadm trigger call. More details here: https://github.com/systemd/systemd/pull/18559

Edit2:

Could solve the Debian Sid vs. Debian Bullseye + systemd v249 mystery:

The Debian Sid container works for me after installing the udev package (which is build from the systemd v249 source). Otherwise udevadm is missing.

Edit3:

Backporting/cherry-picking just 0e789e6d4 and ensuring that udev/udevadm is installed is sufficient on Debian Bullseye, tested this. Cherry-picking 0e789e6d4 onto v247 does not work cleanly, but fixing it up is straight forward.

1 Like

Just chiming in that this morning I discovered that I can’t get systemd-networkd to work in an Ubuntu 20.04 container, either. Of course the version of systemd on Ubuntu 20.04 is quite old: 245.4. By comparison, I’m running 249.4 on my Arch linux workstation.

I’ve reported this bug upstream at Debian and they are going to backport the mentioned fix to the systemd v247 Debian package.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=997006


@pgoetz v245 is one of the versions I haven’t tested. Can you check that you have the udev package and udevadm tool installed? Other than that I would expect that at least Ubuntu 20.10 (Grovvy, systemd v246) or Ubuntu 21.10 (Impish, systemd v248) work. Ubuntu 21.04 (Hirsute, systemd v247) is probably affected too though and probably needs the same backport of 0e789e6d4.

2 Likes
# dpkg -l | grep udev
ii  libudev1:amd64                 245.4-4ubuntu3.13                amd64        libudev shared library
ii  udev                           245.4-4ubuntu3.13                amd64        /dev/ and hotplug management daemon

No udevadm though. Is this a prerequisite?

/bin/udevadm should be part of the udev package. And yes, I think udevadm is necessary for systemd-networkd to work.

Yes, I noticed this when I listed the package contents. My interpretation of your previous comment was that they were separate.

Hello,

Sorry for my late reply, I just gave up. I’ll see when Debian will be decided.

Debian stable / bullseye has now received the necessary update+fix for (1) in systemd v247.3-7 and this package now works for me with LXD + systemd-networkd, too.

Summary again of how to switch to systemd-networkd in an LXD container:

root$ lxc launch images:debian/bullseye debian-bullseye-systemd-test
root$ lxc exec debian-bullseye-systemd-test -- /bin/bash

$ apt-get update
$ apt-get dist-upgrade
[ make sure you have systemd v247.3-7 or later ]
$ apt-get install udev

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ networkctl reload
($ systemctl restart systemd-networkd)
$ reboot

@monstermunchkin is this worth using in our images by default?

I don’t see why not.

I don’t know if it can help, but I deployed a container with lxc launch images:debian/11 last week, and I had the same problem : no IPv4 in container.

LXD 4.24-22170 on a host with Debian 11.

I just made what @T_X said on his last post, and it now works.

I saw that systemd v247.3-7 was already in the image, and udev too.

Edit : older container, launched a long time ago with lxc launch images:debian/10 had no problem with their network / DHCP.

@monstermunchkin any ideas?

There was a network issue with the image last week, but they have been fixed and work perfectly. I just launched a debian/11 container and it works just fine.

2 Likes

Hi everybody.

I made a new launch this morning, on another host and the issue is still here for me.

This host is a Debian 11 with LXD 4.24 rev 22710 (not in a cluster)

lxc image info images:debian/11

Fingerprint: 313492ff825913816ae8357588421203e0b969ca671d92e677c0601cc91c23a2
Size: 83.29MB
Architecture: x86_64
Type: container
Public: yes
Timestamps:
    Created: 2022/04/12 00:00 UTC
    Uploaded: 2022/04/12 00:00 UTC
    Expires: never
    Last used: never
Properties:
    variant: default
    os: Debian
    release: bullseye
    architecture: amd64
    serial: 20220412_05:24
    description: Debian bullseye amd64 (20220412_05:24)
    type: squashfs
Aliases:
    - debian/bullseye/default
    - debian/bullseye/default/amd64
    - debian/11/default
    - debian/11/default/amd64
    - debian/bullseye
    - debian/bullseye/amd64
    - debian/11
    - debian/11/amd64
Cached: no
Auto update: disabled
Profiles: []

lxc launch images:debian/11 test

lxc ls test
+------+---------+------+------+-----------+-----------+
| NAME |  STATE  | IPV4 | IPV6 |   TYPE    | SNAPSHOTS |
+------+---------+------+------+-----------+-----------+
| test | RUNNING |      |      | CONTAINER | 0         |
+------+---------+------+------+-----------+-----------+
lxc exec test -- ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
39: eth0@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:3f:63:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::216:3eff:fe3f:6321/64 scope link
       valid_lft forever preferred_lft forever
lxc config show test
architecture: x86_64
config:
  image.architecture: amd64
  image.description: Debian bullseye amd64 (20220412_05:24)
  image.os: Debian
  image.release: bullseye
  image.serial: "20220412_05:24"
  image.type: squashfs
  image.variant: default
  volatile.base_image: 313492ff825913816ae8357588421203e0b969ca671d92e677c0601cc91c23a2
  volatile.eno3.host_name: veth1335cbbb
  volatile.eno3.hwaddr: 00:16:3e:3f:63:21
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.power: RUNNING
  volatile.uuid: 726b38c9-0cf4-49cb-a8ec-a4679c88c3cc
devices: {}
ephemeral: false
profiles:
- default
stateful: false
description: ""
lxc profile show default
config: {}
description: Default LXD profile
devices:
  eno3:
    name: eth0
    network: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: default
used_by:
- /1.0/instances/test
[...]
lxc network show lxdbr0
config:
  ipv4.address: 10.167.139.1/24
  ipv4.nat: "true"
  ipv6.address: none
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/instances/test
- /1.0/profiles/default
managed: true
status: Created
locations:
- none
netstat -lpun | grep \:53
udp        0      0 10.167.139.1:53         0.0.0.0:*                           384833/dnsmasq
udp6       0      0 fe80::216:3eff:fe00::53 :::*                                384833/dnsmasq

Good to know : if i launch debian/10, network is up :slight_smile:

That’s odd. My host is Ubuntu 22.04, and I tried with both LXD 4.24 and LXD 5.0. The debian/11 image always gets an IPv4 address.

Could you check lxc exec test -- systemctl --failed? I suspect that systemd-networkd is failing for some reason. That would explain the missing IPv4 address.

lxc exec test -- systemctl --failed
  UNIT LOAD ACTIVE SUB DESCRIPTION
0 loaded units listed.