Systemd-networkd: not working in Debian Sid or Bullseye images

Hello,

I’m preparing the switch to Bullseye for my containers and I’m running into a network problem. I am not using the network management packages provided by Debian. Indeed, I think it is better to use systemd-networkd which is installed anyway.

Here is a part of the network configuration tasks via ansible:

---
# tasks file for common
- name: "systemd-networkd : déployer l'interface de loopback"
  become: yes
  notify:
    - "systemd-networkd : redémarrer le service"
  ansible.builtin.copy:
    owner: "root"
    group: "root"
    mode: u=rw,g=r,o=r
    dest: /etc/systemd/network/1-lo.network
    content: |
      [Match]
      Name=lo

      [Network]
      Address=127.0.0.1/8
      Address=::1/128

- name: "systemd-networkd : déployer la configuration des interfaces"
  become: yes
  notify:
    - "systemd-networkd : redémarrer le service"
  loop: "{{ instance.network.interfaces | dict2items }}"
  ansible.builtin.template:
    owner: "root"
    group: "root"
    mode: u=rw,g=r,o=r
    src: systemd-networkd.j2
    dest: /etc/systemd/network/{{ item.value.filename }}

- name: "systemd-networkd : désactiver le serveur DNS"
  become: yes
  notify:
    - "systemd-networkd : redémarrer le service"
  replace:
    path: /etc/systemd/resolved.conf
    regexp: "#DNSStubListener=yes"
    replace: DNSStubListener=no
  register: dns

- name: "systemd-networkd : démasquer les services"
  become: yes
  register: unmask
  vars:
    items:
      - "systemd-networkd.service"
      - "systemd-networkd.socket"
      - "systemd-networkd-wait-online.service"
      - "systemd-resolved.service"
  loop: "{{ items | flatten(levels=1) }}"
  ansible.builtin.systemd:
    masked: no
    name: "{{ item }}"

- name: "systemd-networkd : activer les services"
  become: yes
  register: enable
  vars:
    items:
      - "systemd-networkd.service"
      - "systemd-resolved.service"
  loop: "{{ items | flatten(levels=1) }}"
  ansible.builtin.systemd:
    enabled: yes
    name: "{{ item }}"

- name: "systemd-networkd : basculer de fournisseur réseau"
  become: yes
  when: unmask.changed or enable.changed
  shell: "systemctl stop networking && systemctl start systemd-networkd.service systemd-resolved.service"

- name: "systemd-networkd : désinstaller les paquets netbase"
  become: yes
  notify:
    - "systemd-networkd : supprimer les résidus de netbase"
    - "systemd-networkd : activer la gestion DNS par systemd-resolved"
  apt:
    state: absent
    name:
      - ifupdown
      - netbase
      - resolvconf

This gives the following file:

root@infra-ansible-test:~# cat /etc/systemd/network/2-eth0.network 
# Ansible managed

[Match]
Name=eth0

[Network]
DHCP=no
IPv6AcceptRA=no
LinkLocalAddressing=no

Address=192.168.50.214/24
Gateway=192.168.50.254
DNS=192.168.50.254

Address=fd00:ff50::d214/64
Gateway=fd00:ff50::d254
DNS=fd00:ff50::d254

That seems to be the problem:

pulsar@erebe:~$ lxc exec infra-ansible-test -- /bin/bash
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
root@infra-ansible-test:~# sudo systemctl status systemd-networkd
● systemd-networkd.service - Network Service
     Loaded: loaded (/lib/systemd/system/systemd-networkd.service; enabled; vendor preset: enabled)
    Drop-In: /run/systemd/system/service.d
             └─zzz-lxc-service.conf
     Active: active (running) since Thu 2021-07-01 11:34:53 UTC; 2h 49min ago
TriggeredBy: ● systemd-networkd.socket
       Docs: man:systemd-networkd.service(8)
   Main PID: 1256 (systemd-network)
     Status: "Processing requests..."
      Tasks: 1 (limit: 4596)
     Memory: 3.8M
        CPU: 229ms
     CGroup: /system.slice/systemd-networkd.service
             └─1256 /lib/systemd/systemd-networkd

Jul 01 11:34:53 infra-ansible-test systemd[1]: Starting Network Service...
Jul 01 11:34:53 infra-ansible-test systemd-networkd[1256]: Failed to increase receive buffer size for general netlink socket, ignoring: Operation not permitted
Jul 01 11:34:53 infra-ansible-test systemd-networkd[1256]: Failed to increase buffer size for device monitor, ignoring: Operation not permitted
Jul 01 11:34:53 infra-ansible-test systemd-networkd[1256]: Enumeration completed
Jul 01 11:34:53 infra-ansible-test systemd[1]: Started Network Service.
root@infra-ansible-test:~# sudo networkctl status
● State: off

Jul 01 11:34:53 infra-ansible-test systemd[1]: Starting Network Service...
Jul 01 11:34:53 infra-ansible-test systemd-networkd[1256]: Failed to increase receive buffer size for general netlink socket, ignoring: Operation not permitted
Jul 01 11:34:53 infra-ansible-test systemd-networkd[1256]: Failed to increase buffer size for device monitor, ignoring: Operation not permitted
Jul 01 11:34:53 infra-ansible-test systemd-networkd[1256]: Enumeration completed
Jul 01 11:34:53 infra-ansible-test systemd[1]: Started Network Service.
root@infra-ansible-test:~# sudo networkctl list
IDX LINK TYPE     OPERATIONAL SETUP
  1 lo   loopback carrier     pending
 44 eth0 ether    off         pending

2 links listed.
root@infra-ansible-test:~# sudo networkctl up eth0
root@infra-ansible-test:~# sudo networkctl list
IDX LINK TYPE     OPERATIONAL SETUP
  1 lo   loopback carrier     pending
 44 eth0 ether    degraded    pending

2 links listed.
root@infra-ansible-test:~# sudo networkctl status eth0
● 44: eth0                                                           
                     Link File: n/a
                  Network File: n/a
                          Type: ether
                         State: degraded (pending)
                    HW Address: 00:16:3e:41:43:f0
                           MTU: 1500 (min: 68, max: 65535)
                         QDisc: noqueue
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp
                       Address: fe80::216:3eff:fe41:43f0
                       Gateway: fe80::216:3eff:fe29:fea7

Jul 01 14:28:19 infra-ansible-test systemd-networkd[1256]: eth0: Link UP
Jul 01 14:28:19 infra-ansible-test systemd-networkd[1256]: eth0: Gained carrier
Jul 01 14:28:21 infra-ansible-test systemd-networkd[1256]: eth0: Gained IPv6LL
root@infra-ansible-test:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
44: eth0@if45: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:41:43:f0 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::216:3eff:fe41:43f0/64 scope link 
       valid_lft forever preferred_lft forever

It works in privileged mode, but for security reasons you will understand that this is not acceptable. It doesn’t work in nesting mode and in normal mode (neither nesting nor privileged).
Is this a known problem? Is there a solution or a workaround?

Thanks for reading and in advance for your help.

Which image are you using?

Hello,

The problem is the same with these images:

  • debian/bullseye
  • debian/sid

Does this work (just using a plain simple launch to check basic operation)?

lxc launch images:debian/bullseye c1

The image works after it is launched but because Debian does not use systemd-networkd by default. As soon as I switch the network to systemd-networkd, instead of netbase, I have no network. I would like to know if there is a solution to make systemd-networkd work under Bullseye and Sid since it works fine on Buster.

@monstermunchkin do you have any ideas here?

We use systemd-networkd in the Ubuntu images, so should work ok.

I’ve encountered the same problem. systemd-networkd was working in debian/buster images but in debian/bullseye images it does not work.

$cat /etc/systemd/network/10-outbound.network
[Match]
Name=outbound

[Network]
DHCP=ipv4


$ networkctl
35   outbound     ether    degraded    pending

$ networkctl renew outbound
Failed to renew dynamic configuration of interface outbound: Interface outbound is not managed by systemd-networkd

@pulsarr did you get it working?

@monstermunchkin do you know if systemd-networkd can be used in Debian Sid or Bullseye?

We disable it by default by masking the systemd-networkd unit files. You can unmask them again using

systemctl unmask systemd-networkd.service
systemctl unmask systemd-networkd.socket
systemctl unmask systemd-networkd-wait-online.service

Thanks, do you see them working OK in those images?

No, unfortunately not. I believe that’s the reason we disable systemd-networkd by default on debian.

I could not find anything against systemd-networkd in bullseye.
I did use this page SystemdNetworkd - Debian Wiki. But the last change is from the end of 2019.

I did unmask the services like @monstermunchkin wrote.

systemd-networkd debug log
Sep 24 13:51:13 debiantest systemd-networkd[198]: SELinux enabled state cached to: disabled
Sep 24 13:51:13 debiantest systemd-networkd[198]: Failed to increase receive buffer size for general netlink socket, ignoring: Operation not permitted
Sep 24 13:51:13 debiantest systemd-networkd[198]: sd-device-monitor: The udev service seems not to be active, disabling the monitor
Sep 24 13:51:13 debiantest systemd-networkd[198]: sd-device-monitor: No permission to stat PID1's netns, unable to determine if we are in host netns: Permission denied
Sep 24 13:51:13 debiantest systemd-networkd[198]: Failed to increase buffer size for device monitor, ignoring: Operation not permitted
Sep 24 13:51:13 debiantest systemd-networkd[198]: Bus bus-api-network: changing state UNSET → OPENING
Sep 24 13:51:13 debiantest systemd-networkd[198]: sd-bus: starting bus bus-api-network by connecting to /run/dbus/system_bus_socket...
Sep 24 13:51:13 debiantest systemd-networkd[198]: Bus bus-api-network: changing state OPENING → AUTHENTICATING
Sep 24 13:51:13 debiantest systemd-networkd[198]: Registering bus object implementation for path=/org/freedesktop/LogControl1 iface=org.freedesktop.LogControl1
Sep 24 13:51:13 debiantest systemd-networkd[198]: timestamp of '/etc/systemd/network' changed
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: New device has no master, continuing without
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: Flags change: +UP +LOWER_UP +RUNNING +MULTICAST +BROADCAST
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: Link 124 added
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: link pending udev initialization...
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: Saved original MTU: 1500
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: New device has no master, continuing without
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Failed to get driver, continuing without: Operation not supported
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Flags change: +LOOPBACK +UP +LOWER_UP +RUNNING
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Link 1 added
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: link pending udev initialization...
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Saved original MTU: 65536
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: Remembering foreign address: fe80::xx::xx/64 (valid forever)
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: link_check_ready(): link is in pending state.
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: Gained IPv6LL
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: link_check_ready(): link is in pending state.
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Remembering foreign address: ::1/128 (valid forever)
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: link_check_ready(): link is in pending state.
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Remembering foreign address: 127.0.0.1/8 (valid forever)
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: link_check_ready(): link is in pending state.
Sep 24 13:51:13 debiantest systemd-networkd[198]: rtnl: received non-static neighbor, ignoring.
Sep 24 13:51:13 debiantest systemd-networkd[198]: rtnl: received non-static neighbor, ignoring.
Sep 24 13:51:13 debiantest systemd-networkd[198]: rtnl: received non-static neighbor, ignoring.
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: Remembering route: dst: ff00::/8, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: local, proto: boot, type: unicast
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: Remembering route: dst: fe80::xx::xx/128, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: local, proto: kernel, type: local
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Remembering route: dst: ::1/128, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: local, proto: kernel, type: local
Sep 24 13:51:13 debiantest systemd-networkd[198]: outbound: Remembering route: dst: fe80::/64, src: n/a, gw: n/a, prefsrc: n/a, scope: global, table: main, proto: kernel, type: unicast
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Remembering route: dst: 127.255.255.255/32, src: n/a, gw: n/a, prefsrc: 127.0.0.1, scope: link, table: local, proto: kernel, type: broadcast
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Remembering route: dst: 127.0.0.1/32, src: n/a, gw: n/a, prefsrc: 127.0.0.1, scope: host, table: local, proto: kernel, type: local
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Remembering route: dst: 127.0.0.0/8, src: n/a, gw: n/a, prefsrc: 127.0.0.1, scope: host, table: local, proto: kernel, type: local
Sep 24 13:51:13 debiantest systemd-networkd[198]: lo: Remembering route: dst: 127.0.0.0/32, src: n/a, gw: n/a, prefsrc: 127.0.0.1, scope: link, table: local, proto: kernel, type: broadcast
Sep 24 13:51:13 debiantest systemd-networkd[198]: Remembering foreign routing policy rule: priority: 32766, ::/0 -> ::/0, iif: n/a, oif: n/a, table: 254
Sep 24 13:51:13 debiantest systemd-networkd[198]: Remembering foreign routing policy rule: priority: 0, ::/0 -> ::/0, iif: n/a, oif: n/a, table: 255
Sep 24 13:51:13 debiantest systemd-networkd[198]: rtnl: received rule message with invalid family 129, ignoring.
Sep 24 13:51:13 debiantest systemd-networkd[198]: rtnl: received rule message with invalid family 128, ignoring.
Sep 24 13:51:13 debiantest systemd-networkd[198]: Remembering foreign routing policy rule: priority: 32767, 0.0.0.0/0 -> 0.0.0.0/0, iif: n/a, oif: n/a, table: 253
Sep 24 13:51:13 debiantest systemd-networkd[198]: Remembering foreign routing policy rule: priority: 32766, 0.0.0.0/0 -> 0.0.0.0/0, iif: n/a, oif: n/a, table: 254
Sep 24 13:51:13 debiantest systemd-networkd[198]: Remembering foreign routing policy rule: priority: 0, 0.0.0.0/0 -> 0.0.0.0/0, iif: n/a, oif: n/a, table: 255
Sep 24 13:51:13 debiantest systemd-networkd[198]: Enumeration completed
Sep 24 13:51:13 debiantest systemd-networkd[198]: Bus bus-api-network: changing state AUTHENTICATING → HELLO
Sep 24 13:51:13 debiantest systemd[1]: Started Network Service.
Sep 24 13:51:13 debiantest systemd-networkd[198]: Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=Hello cookie=1 re
...skipping 1 line
Sep 24 13:51:13 debiantest systemd-networkd[198]: Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=RequestName cookie=2 reply_cookie=0 signature=su error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=AddMatch cookie=3 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1/link/_3124 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=4 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1/link/_31 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=5 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1/link/_3124 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=6 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/network1 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=7 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Got message type=method_return sender=org.freedesktop.DBus destination=:1.10 path=n/a interface=n/a member=n/a cookie=1 reply_cookie=1 signature=s error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Bus bus-api-network: changing state HELLO → RUNNING
Sep 24 13:51:13 debiantest systemd-networkd[198]: Got message type=signal sender=org.freedesktop.DBus.Local destination=n/a path=/org/freedesktop/DBus/Local interface=org.freedesktop.DBus.Local member=Connected cookie=4294967295 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a 
Sep 24 13:51:13 debiantest systemd-networkd[198]: Got message type=signal sender=org.freedesktop.DBus destination=:1.10 path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=NameAcquired cookie=2 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Got message type=signal sender=org.freedesktop.DBus destination=:1.10 path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=NameAcquired cookie=3 reply_cookie=0 signature=s error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Got message type=method_return sender=org.freedesktop.DBus destination=:1.10 path=n/a interface=n/a member=n/a cookie=4 reply_cookie=2 signature=u error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Successfully acquired requested service name.
Sep 24 13:51:13 debiantest systemd-networkd[198]: Got message type=method_return sender=org.freedesktop.DBus destination=:1.10 path=n/a interface=n/a member=n/a cookie=5 reply_cookie=3 signature=n/a error-name=n/a error-message=n/a
Sep 24 13:51:13 debiantest systemd-networkd[198]: Match type='signal',sender='org.freedesktop.login1',path='/org/freedesktop/login1',interface='org.freedesktop.login1.Manager',member='PrepareForSleep' successfully installed.

If I remember correctly, the issue was systemd-networkd in containers. On a normal host or a VM, it’ll work just fine.

Hmm. That is strange, because when using images:debian/buster or archlinux in container it works.

Hi, I think I’m stumbling over this same issue. For me too “Link File” and “Network File” are “n/a” in the networkctl output like @pulsarr has shown. And networkctl seems to get this from "/run/systemd/netif/links/. Which unfortunately doesn’t have NETWORK_FILE=… either. It seems to me that somehow bus_method_reload()->link_reconfigure()->…->link_reconfigure_internal() is never called in systemd-networkd? At least I never see the “Re-configuring with /etc/systemd/network/<iface>.network” message in my systemd logs, which I see on another, working, non-containered Debian Sid.

I’m trying with Debian Bullseye, systemd 247.3-6 in the container and a Raspberry Pi Linux kernel 5.9.2-v8+ here. LXC version is 4.19.

One more difference I noticed in a working, non-containered Debian Sid is that it had “Link File: /usr/lib/systemd/network/99-default.link” on my br0 automatically. And this 99-default.link comes from the udev package, which is not installed by default in an LXD Debian Bullseye container. However installing udev in the container with apt and rebooting the container did not seem to help for me.

I did a bit more testing. What works for me:

  • Debian Bullseye + systemd v246, v248 or v249 build from the source

What does not work for me:

  • Debian Bullseye + systemd v247 build from the source (couldn’t build v241 or v244)
  • Debian Sid with systemd v249 provided by the LXD Debian Sid systemd package

I was a bit surprised that systemd v249 works on Debian Bullseye and not on Sid for me.

Protocol of the commands I used for building and testing:

root$ lxc launch images:debian/sid debian-sid-systemd-test
root$ lxc exec debian-sid-systemd-test -- /bin/bash

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ networkctl reload
($ systemctl restart systemd-networkd)
$ reboot

root$ lxc exec debian-sid-systemd-test -- /bin/bash

root@debian-sid-systemd-test:~# networkctl -l status eth0
● 49: eth0                                                           
                     Link File: n/a
                  Network File: n/a
                          Type: ether
                         State: routable (pending)
                  Online state: unknown
                    HW Address: 00:16:3e:2c:96:cb
                           MTU: 1394 (min: 68, max: 65535)
                         QDisc: noqueue
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp
                       Address: 2a03:2260:200f:101:216:3eff:fe2c:96cb
                                fd01:fd59:0:1800:216:3eff:fe2c:96cb
                                fe80::216:3eff:fe2c:96cb
                       Gateway: fe80::ba:7aff:fedf:101
             Activation Policy: up
           Required For Online: yes

Oct 19 21:42:35 debian-sid-systemd-test systemd-networkd[74]: eth0: Link UP
Oct 19 21:42:35 debian-sid-systemd-test systemd-networkd[74]: eth0: Gained carrier
Oct 19 21:42:35 debian-sid-systemd-test systemd-networkd[74]: eth0: Gained IPv6LL
root@debian-sid-systemd-test:~#

=> NOT OK!

$ exit 
root$ lxc stop debian-sid-systemd-test

=====

root$ lxc launch images:debian/bullseye debian-bullseye-systemd-test
root$ lxc exec debian-bullseye-systemd-test -- /bin/bash

$ echo "deb-src http://deb.debian.org/debian bullseye main" > /etc/apt/sources.list.d/deb-src.list
$ apt-get update
$ apt-get install git python3-jinja2
$ apt-get build-dep systemd

$ git clone https://github.com/systemd/systemd.git
$ cd systemd
$ git checkout v241

$ meson build/ && ninja -C build

$ git checkout v249
$ meson setup --wipe build/
$ meson build/ && ninja -C build
$ exit

=====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v249
root$ lxc start debian-bullseye-systemd-test-v249
root$ lxc exec debian-bullseye-systemd-test-v249 -- /bin/bash

$ cd systemd
$ meson install -C build/
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v249 -- /bin/bash

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ networkctl reload
($ systemctl restart systemd-networkd)
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v249 -- /bin/bash

root@debian-bullseye-systemd-test-v249:~# networkctl -l status eth0
● 29: eth0                                                                     
                     Link File: /usr/lib/systemd/network/99-default.link
                  Network File: /etc/systemd/network/eth0.network
                          Type: ether
                         State: routable (configured)
                  Online state: online                                         
                        Driver: veth
                    HW Address: 00:16:3e:e1:7e:b4 (Xensource, Inc.)
                           MTU: 1394 (min: 68, max: 65535)
                         QDisc: noqueue
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp
                       Address: 10.204.32.155 (DHCP4 via 10.204.32.2)
                                2a03:2260:200f:101:216:3eff:fee1:7eb4
                                fd01:fd59:0:1800:216:3eff:fee1:7eb4
                                fe80::216:3eff:fee1:7eb4
                       Gateway: 10.204.32.2
                                fe80::ba:7aff:fedf:101
                           DNS: 10.204.32.1
                                fd01:fd59:0:1800::1:1
             Activation Policy: up
           Required For Online: yes
               DHCP4 Client ID: IAID:0x4dda998c/DUID
             DHCP6 Client IAID: 0x4dda998c
             DHCP6 Client DUID: DUID-EN/Vendor:0000ab11499382881370f8d70000

Oct 19 20:29:56 debian-bullseye-systemd-test-v249 systemd-networkd[85]: eth0: Link UP
Oct 19 20:29:56 debian-bullseye-systemd-test-v249 systemd-networkd[85]: eth0: Gained carrier
Oct 19 20:29:56 debian-bullseye-systemd-test-v249 systemd-networkd[85]: eth0: Gained IPv6LL
Oct 19 20:29:56 debian-bullseye-systemd-test-v249 systemd-networkd[85]: eth0: DHCPv4 address 10.204.32.155/20 via 10.204.32.2

=> OK!

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v249

=====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v247
root$ lxc start debian-bullseye-systemd-test-v247
root$ lxc exec debian-bullseye-systemd-test-v247 -- /bin/bash

$ cd systemd
$ git checkout v247
$ meson setup --wipe build/
$ meson build/ && ninja -C build
$ meson install -C build/
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v247 -- /bin/bash

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ systemctl start systemd-networkd
$ networkctl reload
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v247 -- /bin/bash

root@debian-bullseye-systemd-test-v247:~# networkctl -l status eth0
● 35: eth0                                                           
                     Link File: n/a                                  
                  Network File: n/a                                  
                          Type: ether                                
                         State: routable (pending)      
                    HW Address: 00:16:3e:c7:e7:6a (Xensource, Inc.)  
                           MTU: 1394 (min: 68, max: 65535)           
                         QDisc: noqueue                              
  IPv6 Address Generation Mode: eui64                                
          Queue Length (Tx/Rx): 1/1                                  
              Auto negotiation: no                                   
                         Speed: 10Gbps                               
                        Duplex: full                                 
                          Port: tp                                   
                       Address: 2a03:2260:200f:101:216:3eff:fec7:e76a
                                fd01:fd59:0:1800:216:3eff:fec7:e76a  
                                fe80::216:3eff:fec7:e76a             
                       Gateway: fe80::ba:7aff:fedf:101               

Oct 19 20:56:22 debian-bullseye-systemd-test-v247 systemd-networkd[79]: eth0: Gained IPv6LL
root@debian-bullseye-systemd-test-v247:~#

=> NOT OK!

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v247

=====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v241
root$ lxc start debian-bullseye-systemd-test-v241
root$ lxc exec debian-bullseye-systemd-test-v241 -- /bin/bash

$ cd systemd
$ git checkout v241
$ meson setup --wipe build/
$ meson build/ && ninja -C build
[...]
ninja: Entering directory `build'
[479/1743] Compiling C object src/network/libnetworkd-core.a.p/netdev_tunnel.c.o
../src/network/netdev/tunnel.c: In function ‘ipip_init’:
../src/network/netdev/tunnel.c:765:27: warning: implicit conversion from ‘enum <anonymous>’ to ‘FooOverUDPEncapType’ [-Wenum-conversion]
  765 |         t->fou_encap_type = FOU_ENCAP_DIRECT;
      |                           ^
[550/1743] Generating linux.c.o with a custom command
../src/boot/efi/linux.c:63:72: warning: ‘regparm’ attribute directive ignored [-Wattributes]
   63 | typedef VOID(*handover_f)(VOID *image, EFI_SYSTEM_TABLE *table, struct SetupHeader *setup) __attribute__((regparm(0)));
      |                                                                        ^~~~~~~~~~~
[552/1743] Generating shim.c.o with a custom command
../src/boot/efi/shim.c:21:9: warning: ‘sysv_abi’ attribute directive ignored [-Wattributes]
   21 |         EFI_STATUS __attribute__((sysv_abi)) (*shim_verify) (VOID *buffer, UINT32 size);
      |         ^~~~~~~~~~
../src/boot/efi/shim.c:25:9: warning: ‘sysv_abi’ attribute directive ignored [-Wattributes]
   25 |         EFI_STATUS __attribute__((sysv_abi)) (*generate_hash) (VOID *data, UINT32 datasize, VOID *context, UINT8 *sha256hash, UINT8 *sha1hash);
      |         ^~~~~~~~~~
../src/boot/efi/shim.c:27:9: warning: ‘sysv_abi’ attribute directive ignored [-Wattributes]
   27 |         EFI_STATUS __attribute__((sysv_abi)) (*read_header) (VOID *data, UINT32 datasize, VOID *context);
      |         ^~~~~~~~~~
[556/1743] Generating stub.so with a custom command
FAILED: src/boot/efi/stub.so 
/usr/bin/ld -o src/boot/efi/stub.so -T /usr/lib/elf_aarch64_efi.lds -shared -Bsymbolic -nostdlib -znocombreloc -L /usr/lib /usr/lib/crt0-efi-aarch64.o --defsym=EFI_SUBSYSTEM=0xa src/boot/efi/disk.c.o src/boot/efi/graphics.c.o src/boot/efi/measure.c.o src/boot/efi/pe.c.o src/boot/efi/util.c.o src/boot/efi/linux.c.o src/boot/efi/splash.c.o src/boot/efi/stub.c.o -lefi -lgnuefi /usr/lib/gcc/aarch64-linux-gnu/10/libgcc.a
/usr/bin/ld: src/boot/efi/graphics.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/pe.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/util.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/linux.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/splash.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/stub.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
[557/1743] Generating systemd_boot.so with a custom command
FAILED: src/boot/efi/systemd_boot.so 
/usr/bin/ld -o src/boot/efi/systemd_boot.so -T /usr/lib/elf_aarch64_efi.lds -shared -Bsymbolic -nostdlib -znocombreloc -L /usr/lib /usr/lib/crt0-efi-aarch64.o --defsym=EFI_SUBSYSTEM=0xa src/boot/efi/disk.c.o src/boot/efi/graphics.c.o src/boot/efi/measure.c.o src/boot/efi/pe.c.o src/boot/efi/util.c.o src/boot/efi/boot.c.o src/boot/efi/console.c.o src/boot/efi/shim.c.o -lefi -lgnuefi /usr/lib/gcc/aarch64-linux-gnu/10/libgcc.a
/usr/bin/ld: src/boot/efi/graphics.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/pe.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/util.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/boot.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/console.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
/usr/bin/ld: src/boot/efi/shim.c.o:/root/systemd/build/../src/boot/efi/util.h:54: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:54: first defined here
[561/1743] Generating test-libsystemd-sym.c with a custom command (wrapped by meson to capture output)
ninja: build stopped: subcommand failed.
root@debian-bullseye-systemd-test-v241:~/systemd# echo $?
1
root@debian-bullseye-systemd-test-v241:~/systemd# 

=> build FAILS

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v241

====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v248
root$ lxc start debian-bullseye-systemd-test-v248
root$ lxc exec debian-bullseye-systemd-test-v248 -- /bin/bash

$ cd systemd
$ git checkout v248
$ meson setup --wipe build/
$ meson build/ && ninja -C build
$ meson install -C build/
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v248 -- /bin/bash

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ systemctl start systemd-networkd
$ networkctl reload
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v248 -- /bin/bash

root@debian-bullseye-systemd-test-v248:~# networkctl -l status eth0
● 45: eth0                                                                     
                     Link File: /usr/lib/systemd/network/99-default.link
                  Network File: /etc/systemd/network/eth0.network
                          Type: ether
                         State: routable (configured)
                        Driver: veth
                    HW Address: 00:16:3e:c9:2d:1d (Xensource, Inc.)
                           MTU: 1394 (min: 68, max: 65535)
                         QDisc: noqueue
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp
                       Address: 10.204.33.69 (DHCP4 via 10.204.32.2)
                                2a03:2260:200f:101:216:3eff:fec9:2d1d
                                fd01:fd59:0:1800:216:3eff:fec9:2d1d
                                fe80::216:3eff:fec9:2d1d
                       Gateway: 10.204.32.2
                                fe80::ba:7aff:fedf:101
                           DNS: 10.204.32.1
                                fd01:fd59:0:1800::1:1
             Activation Policy: up
               DHCP4 Client ID: IAID:0x511822df/DUID
             DHCP6 Client IAID: 0x511822df
             DHCP6 Client DUID: DUID-EN/Vendor:0000ab11499382881370f8d70000

Oct 19 21:28:45 debian-bullseye-systemd-test-v248 systemd-networkd[94]: eth0: Gained IPv6LL
Oct 19 21:28:45 debian-bullseye-systemd-test-v248 systemd-networkd[94]: eth0: DHCPv4 address 10.204.33.69/20 via 10.204.32.2
root@debian-bullseye-systemd-test-v248:~#

=> OK!

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v248

=====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v244
root$ lxc start debian-bullseye-systemd-test-v244
root$ lxc exec debian-bullseye-systemd-test-v244 -- /bin/bash

$ cd systemd
$ git checkout v244
$ meson setup --wipe build/
$ meson build/ && ninja -C build
[...]
ninja: Entering directory `build'
[504/1549] Compiling C object src/network/libnetworkd-core.a.p/netdev_tunnel.c.o
../src/network/netdev/tunnel.c: In function ‘ipip_sit_init’:
../src/network/netdev/tunnel.c:732:27: warning: implicit conversion from ‘enum <anonymous>’ to ‘FooOverUDPEncapType’ [-Wenum-conversion]
  732 |         t->fou_encap_type = FOU_ENCAP_DIRECT;
      |                           ^
../src/network/netdev/tunnel.c: In function ‘gre_erspan_init’:
../src/network/netdev/tunnel.c:774:27: warning: implicit conversion from ‘enum <anonymous>’ to ‘FooOverUDPEncapType’ [-Wenum-conversion]
  774 |         t->fou_encap_type = FOU_ENCAP_DIRECT;
      |                           ^
[593/1549] Generating stub.so with a custom command
FAILED: src/boot/efi/stub.so 
/usr/bin/ld -o src/boot/efi/stub.so -T /usr/lib/elf_aarch64_efi.lds -shared -Bsymbolic -nostdlib -znocombreloc -L /usr/lib /usr/lib/crt0-efi-aarch64.o --defsym=EFI_SUBSYSTEM=0xa src/boot/efi/disk.c.o src/boot/efi/graphics.c.o src/boot/efi/measure.c.o src/boot/efi/pe.c.o src/boot/efi/util.c.o src/boot/efi/linux.c.o src/boot/efi/splash.c.o src/boot/efi/stub.c.o -lefi -lgnuefi /usr/lib/gcc/aarch64-linux-gnu/10/libgcc.a
/usr/bin/ld: src/boot/efi/graphics.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
/usr/bin/ld: src/boot/efi/pe.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
/usr/bin/ld: src/boot/efi/util.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
/usr/bin/ld: src/boot/efi/linux.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
/usr/bin/ld: src/boot/efi/splash.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
/usr/bin/ld: src/boot/efi/stub.c.o:/root/systemd/build/../src/boot/efi/util.h:58: multiple definition of `loader_guid'; src/boot/efi/disk.c.o:/root/systemd/build/../src/boot/efi/util.h:58: first defined here
[598/1549] Generating test-libudev-sym.c with a custom command (wrapped by meson to capture output)
ninja: build stopped: subcommand failed.
root@debian-bullseye-systemd-test-v244:~/systemd# echo $?
1
root@debian-bullseye-systemd-test-v244:~/systemd# 

=> build FAILS

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v244

=====

root$ lxc copy debian-bullseye-systemd-test debian-bullseye-systemd-test-v246
root$ lxc start debian-bullseye-systemd-test-v246
root$ lxc exec debian-bullseye-systemd-test-v246 -- /bin/bash

$ cd systemd
$ git checkout v246
$ meson setup --wipe build/
$ meson build/ && ninja -C build
$ meson install -C build/
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v246 -- /bin/bash

$ systemctl unmask systemd-networkd
$ systemctl unmask systemd-networkd.socket
$ systemctl unmask systemd-networkd-wait-online.service
$ systemctl enable systemd-networkd
$ systemctl enable systemd-networkd.socket
$ systemctl enable systemd-networkd-wait-online.service
$ systemctl daemon-reload
$ mv /etc/network/interfaces /etc/network/interfaces.save

$ cat > /etc/systemd/network/eth0.network << EOF
[Match]
Name=eth0
[Network]
DHCP=true
EOF
$ systemctl start systemd-networkd
$ networkctl reload
$ reboot

root$ lxc exec debian-bullseye-systemd-test-v246 -- /bin/bash

root@debian-bullseye-systemd-test-v246:~# networkctl -l status eth0
● 57: eth0                                                                     
                     Link File: /usr/lib/systemd/network/99-default.link       
                  Network File: /etc/systemd/network/eth0.network              
                          Type: ether                                          
                         State: routable (configured)
                        Driver: veth                                           
                    HW Address: 00:16:3e:a6:e3:35 (Xensource, Inc.)            
                           MTU: 1394 (min: 68, max: 65535)                     
                         QDisc: noqueue                                        
  IPv6 Address Generation Mode: eui64                                          
          Queue Length (Tx/Rx): 1/1                                            
              Auto negotiation: no                                             
                         Speed: 10Gbps                                         
                        Duplex: full                                           
                          Port: tp                                             
                       Address: 10.204.32.252 (DHCP4 via 10.204.32.2)          
                                2a03:2260:200f:101:216:3eff:fea6:e335          
                                fe80::216:3eff:fea6:e335                       
                       Gateway: 10.204.32.2                                    
                                fe80::ba:7aff:fedf:101                         
                           DNS: 10.204.32.1                                    
                                fd01:fd59:0:1800::1:1                          
               DHCP4 Client ID: IAID:0x5c902b8f/DUID                           
             DHCP6 Client IAID: 0x5c902b8f                                     
             DHCP6 Client DUID: DUID-EN/Vendor:0000ab11499382881370f8d70000    

Oct 19 22:31:13 debian-bullseye-systemd-test-v246 systemd-networkd[73]: eth0: Gained IPv6LL
Oct 19 22:31:13 debian-bullseye-systemd-test-v246 systemd-networkd[73]: eth0: IPv6 successfully enabled
Oct 19 22:31:13 debian-bullseye-systemd-test-v246 systemd-networkd[73]: eth0: DHCPv4 address 10.204.32.252/20 via 10.204.32.2
root@debian-bullseye-systemd-test-v246:~#

=> OK!

$ exit 
root$ lxc stop debian-bullseye-systemd-test-v246

=====

Edit:

With further bisect’ing I found:

So it seems to be related to the return code of some udevadm trigger call. More details here: https://github.com/systemd/systemd/pull/18559

Edit2:

Could solve the Debian Sid vs. Debian Bullseye + systemd v249 mystery:

The Debian Sid container works for me after installing the udev package (which is build from the systemd v249 source). Otherwise udevadm is missing.

Edit3:

Backporting/cherry-picking just 0e789e6d4 and ensuring that udev/udevadm is installed is sufficient on Debian Bullseye, tested this. Cherry-picking 0e789e6d4 onto v247 does not work cleanly, but fixing it up is straight forward.

1 Like

Just chiming in that this morning I discovered that I can’t get systemd-networkd to work in an Ubuntu 20.04 container, either. Of course the version of systemd on Ubuntu 20.04 is quite old: 245.4. By comparison, I’m running 249.4 on my Arch linux workstation.

I’ve reported this bug upstream at Debian and they are going to backport the mentioned fix to the systemd v247 Debian package.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=997006


@pgoetz v245 is one of the versions I haven’t tested. Can you check that you have the udev package and udevadm tool installed? Other than that I would expect that at least Ubuntu 20.10 (Grovvy, systemd v246) or Ubuntu 21.10 (Impish, systemd v248) work. Ubuntu 21.04 (Hirsute, systemd v247) is probably affected too though and probably needs the same backport of 0e789e6d4.

2 Likes
# dpkg -l | grep udev
ii  libudev1:amd64                 245.4-4ubuntu3.13                amd64        libudev shared library
ii  udev                           245.4-4ubuntu3.13                amd64        /dev/ and hotplug management daemon

No udevadm though. Is this a prerequisite?