I have a ubuntu focal host, which is running lxd 4.4 (self compiled, not snap). On the host lxd is running fine!
On this host there is a ubuntu:20.04 container with nesting allowed:
# lxc config show testdenis --expanded
architecture: x86_64
config:
image.architecture: amd64
image.description: ubuntu 20.04 LTS amd64 (release) (20201117)
image.label: release
image.os: ubuntu
image.release: focal
image.serial: "20201117"
image.type: squashfs
image.version: "20.04"
security.nesting: "true"
security.privileged: "true"
volatile.base_image: c141ba91f766aab428b64f0e2f64b11e583093c2f4e52b4f6c8baa32021d413d
volatile.eth0.host_name: veth2c340b6f
volatile.eth0.hwaddr: 00:16:3e:14:07:52
volatile.idmap.base: "0"
volatile.idmap.current: '[]'
volatile.idmap.next: '[]'
volatile.last_state.idmap: '[]'
volatile.last_state.power: RUNNING
devices:
eth0:
name: eth0
network: lxdbr0
type: nic
root:
path: /
pool: default
type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""
In this nested container testdenis there is also a lxd 4.4 (self compiled, not snap) running. After wondering, why its container are not getting an IP, I found out that dnsmasq is not running for the managed lxdbr0:
root@testdenis:~# lxc network show lxdbr0
config:
ipv4.address: 10.55.71.1/24
ipv4.nat: "true"
ipv6.address: fd42:245b:7d9f:89e8::1/64
ipv6.nat: "true"
volatile.bridge.hwaddr: 00:16:3e:2e:7d:cd
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/instances/test
- /1.0/profiles/default
managed: true
status: Created
locations:
- none
root@testdenis:~# ps -auxf | grep dns
root 11333 0.0 0.0 8160 584 ? S+ 19:04 0:00 \_ grep --color=auto dns
There is no lxd error output even with --debug, I only get the log entry:
Nov 20 18:17:50 testdenis lxd[8370]: t=2020-11-20T18:17:48+0000 lvl=dbug msg="Setting up network" driver=bridge network=lxdbr0
After looking around the source code I found this line: https://github.com/lxc/lxd/blame/303353dc9617f1a653c66837d2275735597063a0/lxd/network/driver_bridge.go#L1376
So that means, if I set raw.dnsmasq to something not empty it ignores apparmor. And tada, now I have dnsmasq running and after restarting the container gets an IP.
root@testdenis:~# lxc list
+------+---------+------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+---------+------+------+-----------+-----------+
| test | RUNNING | | | CONTAINER | 0 |
+------+---------+------+------+-----------+-----------+
root@testdenis:~# lxc network set lxdbr0 "raw.dnsmasq= "
root@testdenis:~# lxc restart test
root@testdenis:~# lxc list
+------+---------+---------------------+----------------------------------------------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+---------+---------------------+----------------------------------------------+-----------+-----------+
| test | RUNNING | 10.55.71.113 (eth0) | fd42:245b:7d9f:89e8:216:3eff:fec1:c2a (eth0) | CONTAINER | 0 |
+------+---------+---------------------+----------------------------------------------+-----------+-----------+
root@testdenis:~# ps -auxf | grep dns
root 13436 0.0 0.0 8160 592 ? S+ 19:08 0:00 \_ grep --color=auto dns
lxd 13000 1.6 0.0 13936 2912 ? Ss 19:08 0:00 \_ dnsmasq --keep-in-foreground --strict-order --bind-interfaces --except-interface=lo --pid-file= --no-ping --interface=lxdbr0 --dhcp-rapid-commit --quiet-dhcp --quiet-dhcp6 --quiet-ra --listen-address=10.55.71.1 --dhcp-no-override --dhcp-authoritative --dhcp-leasefile=/var/lib/lxd/networks/lxdbr0/dnsmasq.leases --dhcp-hostsfile=/var/lib/lxd/networks/lxdbr0/dnsmasq.hosts --dhcp-range 10.55.71.2,10.55.71.254,1h --listen-address=fd42:245b:7d9f:89e8::1 --enable-ra --dhcp-range ::,constructor:lxdbr0,ra-stateless,ra-names -s lxd -S /lxd/ --conf-file=/var/lib/lxd/networks/lxdbr0/dnsmasq.raw -u lxd
root@testdenis:~# lxc network set lxdbr0 "raw.dnsmasq="
root@testdenis:~# lxc restart test
root@testdenis:~# lxc list
+------+---------+------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+---------+------+------+-----------+-----------+
| test | RUNNING | | | CONTAINER | 0 |
+------+---------+------+------+-----------+-----------+
root@testdenis:~# ps -auxf | grep dns
root 14071 0.0 0.0 8160 588 ? S+ 19:09 0:00 \_ grep --color=auto dns
I’m unsure if this is a bug, but I have zero experience with apparmor… I hope somebody is able to reproduce this issue using these infos.