Containers not receiving ipv4 upon host reboot. Errors during boot. Debian/12/cloud

I have configured several containers with static ips using systemd-networkd. Config looks like this:

luigi@HomeServer:~$ incus profile show ext_network
config: {}
description: “”
devices:
eth0:
name: eth0
nictype: bridged
parent: mainbr0
type: nic
name: ext_network
used_by:

  • /1.0/instances/Docker
  • /1.0/instances/Debian12Cloud
  • /1.0/instances/Python
  • /1.0/instances/MyDesktop

root@Python:~# cat /etc/systemd/network/10-cloud-init-eth0.network
[Match]
Name=eth0

[Network]
Address=10.10.0.52/24
Gateway=10.10.0.1
DNS=10.10.0.1

Everytime I restart my incus host none of these containers receive an iP, If I restart the container or I restart systemd-networkd they do receive it. I have checked all revelant logs for systemd-networkd that I know of and I still cannot determine whats going on. I started doing some troubleshooting inside the container logs as follows:

root@Python:~# dmesg
dmesg: read kernel buffer failed: Operation not permitted

journalctl shows a bunch of errors like the following:

Apr 17 00:42:14 Python udevadm[136]: event_source: Failed to write ‘add’ to ‘/sys/bus/event_source/uevent’: Permission denied
Apr 17 00:42:14 Python udevadm[136]: gpio: Failed to write ‘add’ to ‘/sys/bus/gpio/uevent’: Permission denied
Apr 17 00:42:14 Python udevadm[136]: hardware_error_device: Failed to write ‘add’ to ‘/sys/bus/acpi/drivers/hardware_error_device/uevent’: Permission denied
Apr 17 00:50:44 MyDesktop udevadm[137]: LNXSYSTM:00: Failed to write ‘add’ to ‘/sys/devices/LNXSYSTM:00/uevent’: Permission denied
Apr 17 00:50:44 MyDesktop udevadm[137]: LNXPWRBN:00: Failed to write ‘add’ to ‘/sys/devices/LNXSYSTM:00/LNXPWRBN:00/uevent’: Permission denied
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8201F Fast Ethernet: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8201F Fast Ethernet/uevent’: Permissio>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8208 Fast Ethernet: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8208 Fast Ethernet/uevent’: Permission >
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8211 Gigabit Ethernet: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8211 Gigabit Ethernet/uevent’: Permi>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8211B Gigabit Ethernet: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8211B Gigabit Ethernet/uevent’: Per>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8211C Gigabit Ethernet: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8211C Gigabit Ethernet/uevent’: Per>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8211DN Gigabit Ethernet: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8211DN Gigabit Ethernet/uevent’: P>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8211E Gigabit Ethernet: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8211E Gigabit Ethernet/uevent’: Per>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8211F Gigabit Ethernet: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8211F Gigabit Ethernet/uevent’: Per>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8211F-VD Gigabit Ethernet: Failed to write ‘add’ to '/sys/bus/mdio_bus/drivers/RTL8211F-VD Gigabit Ethernet/uevent>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8221B-VB-CG 2.5Gbps PHY: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8221B-VB-CG 2.5Gbps PHY/uevent’: P>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8221B-VM-CG 2.5Gbps PHY: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8221B-VM-CG 2.5Gbps PHY/uevent’: P>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8226 2.5Gbps PHY: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8226 2.5Gbps PHY/uevent’: Permission deni>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8226-CG 2.5Gbps PHY: Failed to write ‘add’ to ‘/sys/bus/mdio_bus/drivers/RTL8226-CG 2.5Gbps PHY/uevent’: Permissio>
Apr 17 00:58:28 MyDesktop udevadm[133]: RTL8226B-CG_RTL8221B-CG 2.5Gbps PHY: Failed to write ‘add’ to '/sys/bus/mdio_bus/drivers/RTL

~# systemctl status systemd-networkd
○ systemd-networkd.service - Network Configuration
Loaded: loaded (/lib/systemd/system/systemd-networkd.service; disabled; preset: enabled)
Drop-In: /run/systemd/system/service.d
└─zzz-lxc-service.conf
Active: inactive (dead)
TriggeredBy: ● systemd-networkd.socket
Docs: man:systemd-networkd.service(8)
man:org.freedesktop.network1(5)

if I do a incus restart all I see all seems to work and all containers get either their respective static ip or dynamic ip. This is only happening when I restart the host.

±--------------±--------±---------------------±-----±----------±----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
±--------------±--------±---------------------±-----±----------±----------+
| Debian12Cloud | STOPPED | | | CONTAINER | 1 |
±--------------±--------±---------------------±-----±----------±----------+
| Docker | RUNNING | 172.17.0.1 (docker0) | | CONTAINER | 0 |
| | | 10.10.0.51 (eth0) | | | |
±--------------±--------±---------------------±-----±----------±----------+
| MyDesktop | RUNNING | 10.10.0.106 (eth0) | | CONTAINER | 0 |
±--------------±--------±---------------------±-----±----------±----------+
| Python | RUNNING | 10.10.0.52 (eth0) | | CONTAINER | 0 |
±--------------±--------±---------------------±-----±----------±----------+

That mainbr0 is a bridge external of Incus?

If so, make sure it has a static MAC address or it’s going to be flip/flopping a lot during boot which could cause that kind of issue.

mainbr0 is externally managed at the os using bridge-utils and the interfaces file config below. Also I keep wondering if all the permission denied messages are part of the issue or not related. Below is my network config for the host.

The loopback network interface

auto lo
iface lo inet loopback

The primary network interface

allow-hotplug enp1s0
iface enp1s0 inet manual

auto mainbr0
iface mainbr0 inet static
address 10.10.0.50/24
gateway 10.10.0.1
netmask 255.255.255.0
dns-nameservers 10.10.0.1
bridge_ports enp1s0
bridge_stp off
bridge_fd 0
bridge_maxwait 0

config: {}
description: “”
name: mainbr0
type: bridge
used_by:

  • /1.0/instances/Debian12Cloud
  • /1.0/instances/Docker
  • /1.0/instances/MyDesktop
  • /1.0/instances/Python
  • /1.0/profiles/ext_network
    managed: false
    status: “”
    locations:

root@HomeServer:~# brctl show mainbr0
bridge name bridge id STP enabled interfaces
mainbr0 8000.72ddbd7207de no enp1s0
vethba6083a7
vethbef6a858
vethd7134385

I did further research, I assigned a static MAC to the bridge using the primary network card. That did not seem to help. I went inside the container and did systemctl enable systemd-networkd that seems to have done the trick I now see IP`s on all containers both static and dynamic. Does the could image not use systemd-networkd by default? Anywhere I can read up on this? I still do see the errors I mentioned and the following when I check eth0 in an instance.

Apr 17 03:00:38 MyDesktop systemd-networkd[154]: Failed to increase receive buffer size for general netlink socket, ignoring: Operation not permitted
Apr 17 03:00:38 MyDesktop systemd-networkd[154]: eth0: Link UP
Apr 17 03:00:38 MyDesktop systemd-networkd[154]: eth0: Gained carrier
Apr 17 03:00:38 MyDesktop systemd-networkd[154]: lo: Link UP
Apr 17 03:00:38 MyDesktop systemd-networkd[154]: lo: Gained carrier
Apr 17 03:00:38 MyDesktop systemd-networkd[154]: eth0: Gained IPv6LL
Apr 17 03:00:38 MyDesktop systemd-networkd[154]: Enumeration completed
Apr 17 03:00:38 MyDesktop systemd-networkd[154]: eth0: Configuring with /etc/systemd/network/10-cloud-init-eth0.network.
Apr 17 03:00:38 MyDesktop systemd[1]: Started systemd-networkd.service - Network Configuration.
Apr 17 03:00:38 MyDesktop systemd-networkd[154]: eth0: DHCPv4 address 10.10.0.106/8, gateway 10.10.0.1 acquired from 10.10.0.1

now that I crosscheck my previous post, could it be something is disabling systemd-networkd upon reboot? Maybe the permissions issue?

ok something is definately maybe wrong with my config. I create a new instance, only with the default profile, did not change networking or any type of configuration.

incus init images:debian/12/cloud MyDesktop

then went inside did journalctl -b and see a ton of permission errors.

Placed in pastebin to not flood the post.

The sysctl and module errors are normal, you’re in a container, containers can’t load modules or modify system-wide sysctls, that’s fine.

Why are you using the cloud variant of the image? Do you actually use cloud-init?
I’ve not looked at the Debian cloud images specifically but it wouldn’t surprise me if cloud-init on Debian was still using ifupdown which would then explain why systemd-networkd is disabled.

I was using cloud-init to setup my user, and installed applications. Not totally required though.i am choosing Debian/12 cloud images as I was thinking theyre the most stable and I am familiar with them. Any recommendations?

The base debian/12 image should be basically the same but without cloud-init which I believe also means that it will be using networkd out of the box for networking rather than whatever cloud-init does on Debian.

Hi Just an update so I installed the debian/12 images as instructed yesterday. I just booted up the pc and to my surprise once again my systemd-networkd is disabled :frowning: any other ideas? If I restart networkd or I reboot the client it works, but not upon initial host boot. I do notice one thing, one of my other containers has docker installed, which does seem to work upon boot. I read on the docs that docker can cause problems while running on the host. On the host I only run incus. Does it matter if the host is running ENI etc/network/interfaces and the containers run systems?

root@Bashing:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
6: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:55:81:4a brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::216:3eff:fe55:814a/64 scope link
       valid_lft forever preferred_lft forever
root@Bashing:~# systemctl status systemd-networkd
○ systemd-networkd.service - Network Configuration
     Loaded: loaded (/lib/systemd/system/systemd-networkd.service; disabled; preset: enabled)
    Drop-In: /run/systemd/system/service.d
             └─zzz-lxc-service.conf
     Active: inactive (dead)
TriggeredBy: ● systemd-networkd.socket
       Docs: man:systemd-networkd.service(8)
             man:org.freedesktop.network1(5)
root@Bashing:~# networkctl
WARNING: systemd-networkd is not running, output will be incomplete.

IDX LINK TYPE     OPERATIONAL SETUP
  1 lo   loopback -           unmanaged
  6 eth0 ether    -           unmanaged

2 links listed.

Errors from journal on systemd

I think I might have found the issue looks like my router is acting up and my mask was set to 255.0.0.0 I am hoping it`s that. Ill keep testing.

After several hours of troubleshooting, I changed my router for a switch redid my networking config and looks like all is well.

Sorry for the time spent, hopefully this might help someone else.