Integrate with systemd-resolved inside a container

I have the following setup

  • c1 running on lxd on bare metal with security.nesting=true
  • nested1 running in c1, with a rather standard lxd server setup.

Trying to follow this guide in nested1 but I cannot get it working, lxd-dns-lxdbr0 doesn’t start


$ cat /etc/systemd/system/lxd-dns-lxdbr0.service 
[Unit]
Description=LXD per-link DNS configuration for lxdbr0
BindsTo=sys-subsystem-net-devices-lxdbr0.device
After=sys-subsystem-net-devices-lxdbr0.device

[Service]
Type=oneshot
ExecStart=/usr/bin/resolvectl dns lxdbr0 10.239.107.6
ExecStart=/usr/bin/resolvectl domain lxdbr0 '~localenv.com'

[Install]
WantedBy=sys-subsystem-net-devices-lxdbr0.device

$ systemctl status sys-subsystem-net-devices-lxdbr0.device
● sys-subsystem-net-devices-lxdbr0.device - /sys/subsystem/net/devices/lxdbr0
     Loaded: loaded
     Active: inactive (dead)

May 25 11:18:43 ncenv systemd[1]: sys-subsystem-net-devices-lxdbr0.device: Job sys-subsystem-net-devices-lxdbr0.device/start timed out.
May 25 11:18:43 ncenv systemd[1]: Timed out waiting for device /sys/subsystem/net/devices/lxdbr0.
May 25 11:18:43 ncenv systemd[1]: sys-subsystem-net-devices-lxdbr0.device: Job sys-subsystem-net-devices-lxdbr0.device/start failed wit>

Is there some setting that I’m missing to make it work? Manually issuing the resolvectl commands is successful, but as pointed out in that guide, not reboot-persistent.

Any update on this topic?

Just realized that the fact of using nested1 is not relevant in this case.

What I want to achieve is to integrate with systemd-resolved inside a container.

The above code listings run in c1, not in nested1.

Hi,

Please can you show the output of lxc list on the “host” (nested container?) that you want to integrate with systemd-resolved?

Also can you show the output of ip a and ip r on that host?

root@ncenv:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:be:9a:26 brd ff:ff:ff:ff:ff:ff
    inet 10.158.189.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
    inet6 fd42:c391:91b0:6e5e::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:febe:9a26/64 scope link 
       valid_lft forever preferred_lft forever
4: veth8c21db1c@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether fe:34:9d:ac:26:0b brd ff:ff:ff:ff:ff:ff link-netnsid 1
6: veth163c2f2a@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether 32:9e:d5:ef:d8:e0 brd ff:ff:ff:ff:ff:ff link-netnsid 2
22: eth0@if23: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:57:fa:39 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.208.133.56/24 metric 100 brd 10.208.133.255 scope global dynamic eth0
       valid_lft 3411sec preferred_lft 3411sec
    inet6 fd42:42fc:e347:ce20:216:3eff:fe57:fa39/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 3596sec preferred_lft 3596sec
    inet6 fe80::216:3eff:fe57:fa39/64 scope link 
       valid_lft forever preferred_lft forever
root@ncenv:~# ip r
default via 10.208.133.1 dev eth0 proto dhcp src 10.208.133.56 metric 100 
10.158.189.0/24 dev lxdbr0 proto kernel scope link src 10.158.189.1 
10.208.133.0/24 dev eth0 proto kernel scope link src 10.208.133.56 metric 100 
10.208.133.1 dev eth0 proto dhcp scope link src 10.208.133.56 metric 100 

Does /sys/subsystem/net/devices/lxdbr0 exist?

No…

What does sudo systemctl | grep subsystem show inside the nested host?

So I think the issue here is udev not updating the systemctl units list with the subsystem devices.
If you run systemctl daemon-reload after the lxdbr0 interface is created, you’ll see sys-subsystem-net-devices-lxdbr0 in the systemctl list.

@stgraber @brauner I’m using images:ubuntu/22.04 and ubuntu 22.04 on the host, is it expected that udev work OK for network devices?

Hello,

I can confirm that running systemctl daemon-reload after the lxdbr0 interface is created, the configuration works fine.

The full instructions sequence:

$ lxc launch ubuntu:22.04 c1 -c security.nesting=true
Creating c1
Starting c1

$ lxc exec c1 bash

root@c1:~# lxd init
[Accept defaults]

root@c1:~# systemctl | grep subsystem
sys-subsystem-net-devices-eth0.device   loaded active plugged   /sys/subsystem/net/devices/eth0

root@c1:~# systemctl daemon-reload

root@c1:~# systemctl | grep subsystem
sys-subsystem-net-devices-eth0.device   loaded active plugged   /sys/subsystem/net/devices/eth0
sys-subsystem-net-devices-lxdbr0.device loaded active plugged   /sys/subsystem/net/devices/lxdbr0

Great, although on reboot of the instance this won’t work AFAIK, as udev appears to be broken currently (reproduced myself too).

In my case, after a reboot:

root@c1:~# systemctl | grep subsystem
root@c1:~# systemctl daemon-reload
root@c1:~# systemctl | grep subsystem
  sys-subsystem-net-devices-eth0.device   loaded active plugged   /sys/subsystem/net/devices/eth0
root@c1:~# lxc network list
+--------+--------+---------+----------------+------+-------------+---------+---------+
|  NAME  |  TYPE  | MANAGED |      IPV4      | IPV6 | DESCRIPTION | USED BY |  STATE  |
+--------+--------+---------+----------------+------+-------------+---------+---------+
| lxdbr0 | bridge | YES     | 10.104.65.1/24 | none |             | 1       | CREATED |
+--------+--------+---------+----------------+------+-------------+---------+---------+
root@c1:~# systemctl daemon-reload
root@c1:~# systemctl | grep subsystem
  sys-subsystem-net-devices-eth0.device   loaded active plugged   /sys/subsystem/net/devices/eth0
  sys-subsystem-net-devices-lxdbr0.device loaded active plugged   /sys/subsystem/net/devices/lxdbr0

It seems that lxc network list is needed also to materialize /sys/subsystem/net/devices/lxdbr0

Oh nice, well that is expected because LXD creates the lxdbr0, and LXD won’t start until needed or if there is an instance starting on boot.