TLDR; I have a container, in which lxd is running with help of nesting.
In that container I’m gonna build additional containers and convert them later to images and uploading them to my lxd “hub”. But the inner container never runs the final step of cloud-init, independent which ubuntu version i’m running (tested with xenial, bionic, focal).
I tried playing around with raw.lxc=lxc.apparmor.profile=xyz but that makes things worse. I’ve read somewhere, that nesting works “out of the box” with systemd, which is the case here.
FULL/steps to reproduce; Lets go quickly through the setup.
Host:
# snap version
snap 2.44.3+20.04
snapd 2.44.3+20.04
series 16
ubuntu 20.04
kernel 5.4.0-31-generic
with lxd init --auto (except zfs as storage pool for default)
… and has the container for building additional containers:
# lxc config show runner
architecture: x86_64
config:
image.architecture: amd64
image.description: ubuntu 20.04 LTS amd64 (release) (20200513)
image.label: release
image.os: ubuntu
image.release: focal
image.serial: "20200513"
image.type: squashfs
image.version: "20.04"
security.nesting: "true"
security.privileged: "true"
volatile.base_image: 89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56
volatile.eth0.host_name: veth0a273226
volatile.eth0.hwaddr: 00:16:3e:b3:95:c0
volatile.idmap.base: "0"
volatile.idmap.current: '[]'
volatile.idmap.next: '[]'
volatile.last_state.idmap: '[]'
volatile.last_state.power: RUNNING
devices: {}
ephemeral: false
profiles:
- default
stateful: false
description: ""
In that “runner” container:
# snap version
snap 2.44.3+20.04
snapd 2.44.3+20.04
series 16
ubuntu 20.04
kernel 5.4.0-31-generic
with lxd init --auto
… which has the container I want to manipulate:
# lxc config show merged
architecture: x86_64
config:
image.architecture: amd64
image.description: ubuntu 18.04 LTS amd64 (release) (20200518.1)
image.label: release
image.os: ubuntu
image.release: bionic
image.serial: "20200518.1"
image.type: squashfs
image.version: "18.04"
user.user-data: |-
#cloud-config
packages:
- apt-transport-https
- jq
runcmd:
- export HOME=/root
- curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
- apt-get install git-lfs -y
- git lfs install
- curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | bash
- apt-get install gitlab-runner -y
volatile.base_image: 5539cdd71a7b2f56a75d9bb8f45c70b6c1618217ae4df5977215c5636bf8adce
volatile.eth0.host_name: veth15dc5d2d
volatile.eth0.hwaddr: 00:16:3e:58:71:01
volatile.idmap.base: "0"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
volatile.last_state.power: RUNNING
devices: {}
ephemeral: false
profiles:
- default
stateful: false
description: ""
That “merged” container should run cloud-init, but it never reaches cloud-final
# tail -f /var/log/cloud-init-output.log
| .o o * o..... |
| o + * . . o .|
| . + = . o . .|
| o o + ES . . |
|o . = ..o . |
| o o +. o . |
| . ..o+. . |
| . oo.. |
+----[SHA256]-----+
Cloud-init v. 19.4-33-gbb4131a2-0ubuntu1~18.04.1 running 'modules:config' at Wed, 20 May 2020 14:24:24 +0000. Up 9.56 seconds.
where systemd thinks (i think i interpret it correctly), that it should be fine to run:
# systemctl list-dependencies cloud-final
cloud-final.service
● ├─cloud-config.service // green
● ├─snapd.seeded.service // green
● ├─system.slice // green
● ├─network-online.target // green
● │ └─systemd-networkd-wait-online.service // green
● └─sysinit.target // green
● ├─apparmor.service // red
● ├─blk-availability.service // green
● ├─dev-hugepages.mount // white
...
When forcing manually to run that cloud-init final step:
# systemctl start cloud-final
// hangs
^C
# cloud-init modules --mode final
Cloud-init v. 19.4-33-gbb4131a2-0ubuntu1~18.04.1 running 'modules:final' at Wed, 20 May 2020 14:52:08 +0000. Up 1673.43 seconds.
Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease
... // and continues to finish it
I have a lot of errors according to journalctl:
-- Logs begin at Wed 2020-05-20 15:10:45 UTC, end at Wed 2020-05-20 15:10:57 UTC. --
May 20 15:10:45 merged systemd-journald[85]: Journal started
May 20 15:10:45 merged systemd-journald[85]: Runtime journal (/run/log/journal/7366d1d6787348c0a8a2b4e4c42fa4f1) is 8.0M, max 398.0M, 390.0M free.
May 20 15:10:45 merged systemd-modules-load[81]: Failed to lookup alias 'iscsi_tcp': Function not implemented
May 20 15:10:45 merged systemd-modules-load[81]: Failed to lookup alias 'ib_iser': Function not implemented
May 20 15:10:45 merged systemd-sysctl[87]: Couldn't write '4 4 1 7' to 'kernel/printk', ignoring: Permission denied
May 20 15:10:45 merged systemd-sysctl[87]: Couldn't write '1' to 'kernel/kptr_restrict', ignoring: Permission denied
May 20 15:10:45 merged keyboard-setup.sh[82]: Couldn't get a file descriptor referring to the console
May 20 15:10:45 merged systemd-sysctl[87]: Couldn't write '1' to 'fs/protected_hardlinks', ignoring: Permission denied
May 20 15:10:45 merged keyboard-setup.sh[82]: Couldn't get a file descriptor referring to the console
May 20 15:10:45 merged systemd-sysctl[87]: Couldn't write '1' to 'fs/protected_symlinks', ignoring: Permission denied
May 20 15:10:45 merged systemd-sysctl[87]: Couldn't write '1024' to 'fs/inotify/max_user_instances', ignoring: Permission denied
May 20 15:10:45 merged systemd-sysctl[87]: Couldn't write '176' to 'kernel/sysrq', ignoring: Permission denied
May 20 15:10:45 merged systemd-sysctl[87]: Couldn't write '1' to 'kernel/yama/ptrace_scope', ignoring: Permission denied
May 20 15:10:45 merged systemd-sysctl[87]: Couldn't write '65536' to 'vm/mmap_min_addr', ignoring: Permission denied
May 20 15:10:45 merged systemd-journald[85]: Forwarding to syslog missed 1 messages.
May 20 15:10:45 merged systemd-sysctl[87]: Couldn't write 'fq_codel' to 'net/core/default_qdisc', ignoring: No such file or directory
May 20 15:10:45 merged systemd[1]: systemd-journal-flush.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:45 merged systemd[1]: Starting Flush Journal to Persistent Storage...
May 20 15:10:45 merged systemd-journald[85]: Time spent on flushing to /var is 2.634ms for 18 entries.
May 20 15:10:45 merged systemd-journald[85]: System journal (/var/log/journal/7366d1d6787348c0a8a2b4e4c42fa4f1) is 512B, max 4.0G, 3.9G free.
May 20 15:10:45 merged systemd[1]: Started udev Kernel Device Manager.
May 20 15:10:46 merged keyboard-setup.sh[82]: setupcon: We are not on the console, the console is left unconfigured.
May 20 15:10:45 merged systemd[1]: Started Set the console keyboard layout.
May 20 15:10:45 merged systemd[1]: Reached target Local File Systems (Pre).
May 20 15:10:46 merged apparmor[118]: * Starting AppArmor profiles
May 20 15:10:45 merged systemd[1]: Reached target Local File Systems.
May 20 15:10:46 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "/usr/bin/lxc-start". Permission denied; attempted to load a profile while confined?
May 20 15:10:45 merged systemd[1]: ebtables.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:45 merged systemd[1]: Starting ebtables ruleset management...
May 20 15:10:45 merged systemd[1]: apparmor.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:45 merged systemd[1]: Starting AppArmor initialization...
May 20 15:10:45 merged systemd[1]: plymouth-read-write.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:45 merged systemd[1]: Starting Tell Plymouth To Write Out Runtime Data...
May 20 15:10:45 merged systemd[1]: console-setup.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:46 merged systemd-udevd[156]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
May 20 15:10:46 merged systemd[1]: Starting Update UTMP about System Boot/Shutdown...
May 20 15:10:46 merged systemd[1]: Reached target System Time Synchronized.
May 20 15:10:46 merged systemd-udevd[157]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
May 20 15:10:46 merged systemd[1]: Started Update UTMP about System Boot/Shutdown.
May 20 15:10:46 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "/usr/bin/man". Permission denied; attempted to load a profile while confined?
May 20 15:10:46 merged apparmor[118]: Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
May 20 15:10:46 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "/sbin/dhclient". Permission denied; attempted to load a profile while confined?
May 20 15:10:46 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "/usr/sbin/tcpdump". Permission denied; attempted to load a profile while confined?
May 20 15:10:46 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "mount-namespace-capture-helper". Permission denied; attempted to load a profile while confined?
May 20 15:10:46 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "/usr/lib/snapd/snap-confine". Permission denied; attempted to load a profile while confined?
May 20 15:10:46 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "lxc-container-default". Permission denied; attempted to load a profile while confined?
May 20 15:10:46 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "/usr/bin/lxc-start". Permission denied; attempted to load a profile while confined?
May 20 15:10:46 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "/usr/bin/man". Permission denied; attempted to load a profile while confined?
May 20 15:10:46 merged apparmor[118]: Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
May 20 15:10:46 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "/sbin/dhclient". Permission denied; attempted to load a profile while confined?
May 20 15:10:47 merged console-setup.sh[120]: setupcon: We are not on the console, the console is left unconfigured.
May 20 15:10:47 merged systemd[1]: Started Set console font and keymap.
May 20 15:10:47 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "mount-namespace-capture-helper". Permission denied; attempted to load a profile while confined?
May 20 15:10:47 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "/usr/lib/snapd/snap-confine". Permission denied; attempted to load a profile while confined?
May 20 15:10:47 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "/usr/sbin/tcpdump". Permission denied; attempted to load a profile while confined?
May 20 15:10:47 merged apparmor[118]: /sbin/apparmor_parser: Unable to replace "lxc-container-default". Permission denied; attempted to load a profile while confined?
May 20 15:10:47 merged apparmor[118]: ...fail!
May 20 15:10:47 merged systemd[1]: apparmor.service: Main process exited, code=exited, status=123/n/a
May 20 15:10:47 merged systemd[1]: apparmor.service: Failed with result 'exit-code'.
May 20 15:10:47 merged systemd[1]: Failed to start AppArmor initialization.
May 20 15:10:47 merged systemd[1]: cloud-init-local.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:47 merged systemd[1]: Starting Initial cloud-init job (pre-networking)...
May 20 15:10:48 merged cloud-init[323]: Cloud-init v. 19.4-33-gbb4131a2-0ubuntu1~18.04.1 running 'init-local' at Wed, 20 May 2020 15:10:47 +0000. Up 3.27 seconds.
May 20 15:10:48 merged systemd[1]: Started Initial cloud-init job (pre-networking).
May 20 15:10:48 merged systemd[1]: Reached target Network (Pre).
May 20 15:10:48 merged systemd[1]: systemd-networkd.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:48 merged systemd[1]: Starting Network Service...
May 20 15:10:48 merged systemd-networkd[359]: eth0: IPv6 successfully enabled
May 20 15:10:48 merged systemd-networkd[359]: eth0: Gained IPv6LL
May 20 15:10:48 merged systemd-networkd[359]: Enumeration completed
May 20 15:10:48 merged systemd[1]: Started Network Service.
May 20 15:10:48 merged systemd[1]: systemd-networkd-wait-online.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:48 merged systemd[1]: Starting Wait for Network to be Configured...
May 20 15:10:48 merged systemd[1]: systemd-resolved.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:48 merged systemd[1]: Failed to set devices.allow on /system.slice/systemd-resolved.service: Operation not permitted
May 20 15:10:48 merged systemd[1]: Failed to set devices.allow on /system.slice/systemd-resolved.service: Operation not permitted
May 20 15:10:48 merged systemd[1]: Failed to set devices.allow on /system.slice/systemd-resolved.service: Operation not permitted
May 20 15:10:48 merged systemd[1]: Failed to set devices.allow on /system.slice/systemd-resolved.service: Operation not permitted
May 20 15:10:48 merged systemd[1]: Failed to set devices.allow on /system.slice/systemd-resolved.service: Operation not permitted
May 20 15:10:48 merged systemd[1]: Failed to set devices.allow on /system.slice/systemd-resolved.service: Operation not permitted
May 20 15:10:48 merged systemd[1]: Failed to set devices.allow on /system.slice/systemd-resolved.service: Operation not permitted
May 20 15:10:48 merged systemd[1]: Failed to set devices.allow on /system.slice/systemd-resolved.service: Operation not permitted
May 20 15:10:48 merged systemd[1]: Starting Network Name Resolution...
May 20 15:10:48 merged systemd-networkd[359]: eth0: DHCPv4 address 10.165.162.172/24 via 10.165.162.1
May 20 15:10:48 merged systemd-networkd[359]: Not connected to system bus, not setting hostname.
May 20 15:10:48 merged systemd-resolved[361]: Positive Trust Anchors:
May 20 15:10:48 merged systemd-resolved[361]: . IN DS 19036 8 2 49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
May 20 15:10:48 merged systemd-resolved[361]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
May 20 15:10:48 merged systemd-resolved[361]: Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal intranet lan local private test
May 20 15:10:48 merged systemd-resolved[361]: Using system hostname 'merged'.
May 20 15:10:48 merged systemd[1]: Started Network Name Resolution.
May 20 15:10:48 merged systemd[1]: Reached target Host and Network Name Lookups.
May 20 15:10:48 merged systemd[1]: Reached target Network.
May 20 15:10:50 merged systemd-networkd[359]: eth0: Configured
May 20 15:10:50 merged systemd-networkd-wait-online[360]: managing: eth0
May 20 15:10:50 merged systemd-networkd-wait-online[360]: ignoring: lo
May 20 15:10:50 merged systemd[1]: Started Wait for Network to be Configured.
May 20 15:10:50 merged systemd[1]: cloud-init.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:50 merged systemd[1]: Starting Initial cloud-init job (metadata service crawler)...
May 20 15:10:51 merged cloud-init[364]: Cloud-init v. 19.4-33-gbb4131a2-0ubuntu1~18.04.1 running 'init' at Wed, 20 May 2020 15:10:51 +0000. Up 6.71 seconds.
...
May 20 15:10:52 merged systemd[1]: Started Initial cloud-init job (metadata service crawler).
May 20 15:10:52 merged systemd[1]: Reached target System Initialization.
May 20 15:10:52 merged systemd[1]: Listening on ACPID Listen Socket.
May 20 15:10:52 merged systemd[1]: Listening on D-Bus System Message Bus Socket.
May 20 15:10:52 merged systemd[1]: Started ACPI Events Check.
May 20 15:10:52 merged systemd[1]: Reached target Paths.
May 20 15:10:52 merged systemd[1]: lxd.socket: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting LXD - unix socket.
May 20 15:10:52 merged systemd[1]: Started Daily apt download activities.
May 20 15:10:52 merged systemd[1]: Started Daily apt upgrade and clean activities.
May 20 15:10:52 merged systemd[1]: snapd.socket: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting Socket activation for snappy daemon.
May 20 15:10:52 merged systemd[1]: Listening on Open-iSCSI iscsid Socket.
May 20 15:10:52 merged systemd[1]: Started Message of the Day.
May 20 15:10:52 merged systemd[1]: Started Daily Cleanup of Temporary Directories.
May 20 15:10:52 merged systemd[1]: Reached target Timers.
May 20 15:10:52 merged systemd[1]: Listening on Unix socket for apport crash forwarding.
May 20 15:10:52 merged systemd[1]: Listening on UUID daemon activation socket.
May 20 15:10:52 merged systemd[1]: Reached target Cloud-config availability.
May 20 15:10:52 merged systemd[1]: Reached target Network is Online.
May 20 15:10:52 merged systemd[1]: blk-availability.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting Availability of block devices...
May 20 15:10:52 merged systemd[1]: Reached target Remote File Systems (Pre).
May 20 15:10:52 merged systemd[1]: Reached target Remote File Systems.
May 20 15:10:52 merged systemd[1]: Listening on LXD - unix socket.
May 20 15:10:52 merged systemd[1]: Listening on Socket activation for snappy daemon.
May 20 15:10:52 merged systemd[1]: Reached target Sockets.
May 20 15:10:52 merged systemd[1]: Reached target Basic System.
May 20 15:10:52 merged systemd[1]: lxd-containers.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting LXD - container startup/shutdown...
May 20 15:10:52 merged systemd[1]: systemd-user-sessions.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting Permit User Sessions...
May 20 15:10:52 merged systemd[1]: accounts-daemon.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting Accounts Service...
May 20 15:10:52 merged systemd[1]: cron.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Started Regular background program processing daemon.
May 20 15:10:52 merged systemd[1]: dbus.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Started D-Bus System Message Bus.
May 20 15:10:52 merged cron[413]: (CRON) INFO (pidfile fd = 3)
May 20 15:10:52 merged cron[413]: (CRON) INFO (Running @reboot jobs)
May 20 15:10:52 merged dbus-daemon[414]: [system] AppArmor D-Bus mediation is enabled
May 20 15:10:52 merged systemd[1]: Failed to get initial list of names: Permission denied
May 20 15:10:52 merged systemd[1]: Failed to set up API bus: Permission denied
May 20 15:10:52 merged systemd[1]: Error occured during D-Bus APIs initialization: Permission denied
May 20 15:10:52 merged systemd[1]: atd.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Started Deferred execution scheduler.
May 20 15:10:52 merged systemd[1]: systemd-logind.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting Login Service...
May 20 15:10:52 merged systemd[1]: snapd.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting Snappy daemon...
May 20 15:10:52 merged systemd[1]: ssh.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting OpenBSD Secure Shell server...
May 20 15:10:52 merged systemd[1]: rsyslog.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting System Logging Service...
May 20 15:10:52 merged systemd[1]: networkd-dispatcher.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting Dispatcher daemon for systemd-networkd...
May 20 15:10:52 merged systemd[1]: apport.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting LSB: automatic crash report generation...
May 20 15:10:52 merged systemd[1]: Started Availability of block devices.
May 20 15:10:52 merged systemd[1]: Started Permit User Sessions.
May 20 15:10:52 merged systemd[1]: Started Accounts Service.
May 20 15:10:52 merged systemd[1]: plymouth-quit-wait.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting Hold until boot process finishes up...
May 20 15:10:52 merged systemd[1]: plymouth-quit.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting Terminate Plymouth Boot Screen...
May 20 15:10:52 merged systemd[1]: Started Hold until boot process finishes up.
May 20 15:10:52 merged systemd[1]: system-getty.slice: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Created slice system-getty.slice.
May 20 15:10:52 merged systemd[1]: console-getty.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Started Console Getty.
May 20 15:10:52 merged systemd[1]: Reached target Login Prompts.
May 20 15:10:52 merged systemd[1]: Started Terminate Plymouth Boot Screen.
May 20 15:10:52 merged rsyslogd[427]: imuxsock: Acquired UNIX socket '/run/systemd/journal/syslog' (fd 3) from systemd. [v8.32.0]
May 20 15:10:52 merged rsyslogd[427]: imklog: cannot open kernel log (/proc/kmsg): Permission denied.
May 20 15:10:52 merged rsyslogd[427]: activation of module imklog failed [v8.32.0 try http://www.rsyslog.com/e/2145 ]
May 20 15:10:52 merged rsyslogd[427]: rsyslogd's groupid changed to 106
May 20 15:10:52 merged rsyslogd[427]: rsyslogd's userid changed to 102
May 20 15:10:52 merged rsyslogd[427]: [origin software="rsyslogd" swVersion="8.32.0" x-pid="427" x-info="http://www.rsyslog.com"] start
May 20 15:10:52 merged systemd[1]: Started System Logging Service.
May 20 15:10:52 merged sshd[436]: Server listening on 0.0.0.0 port 22.
May 20 15:10:52 merged sshd[436]: Server listening on :: port 22.
May 20 15:10:52 merged systemd[1]: Started OpenBSD Secure Shell server.
May 20 15:10:52 merged systemd[1]: Started LSB: automatic crash report generation.
May 20 15:10:52 merged systemd[1]: Started LXD - container startup/shutdown.
May 20 15:10:52 merged systemd-logind[419]: New seat seat0.
May 20 15:10:52 merged snapd[421]: AppArmor status: apparmor is enabled and all features are available
May 20 15:10:52 merged networkd-dispatcher[428]: No valid path found for iwconfig
May 20 15:10:52 merged networkd-dispatcher[428]: No valid path found for iw
May 20 15:10:52 merged snapd[421]: daemon.go:346: started snapd/2.42.1+18.04 (series 16; classic) ubuntu/18.04 (amd64) linux/5.4.0-31-generic.
May 20 15:10:52 merged snapd[421]: daemon.go:439: adjusting startup timeout by 30s (pessimistic estimate of 30s plus 5s per snap)
May 20 15:10:52 merged snapd[421]: helpers.go:104: error trying to compare the snap system key: system-key missing on disk
May 20 15:10:52 merged systemd[1]: Started Snappy daemon.
May 20 15:10:52 merged systemd[1]: snapd.seeded.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:52 merged systemd[1]: Starting Wait until snapd is fully seeded...
May 20 15:10:52 merged networkd-dispatcher[428]: Traceback (most recent call last):
May 20 15:10:52 merged networkd-dispatcher[428]: File "/usr/bin/networkd-dispatcher", line 483, in <module>
May 20 15:10:52 merged networkd-dispatcher[428]: main()
May 20 15:10:52 merged networkd-dispatcher[428]: File "/usr/bin/networkd-dispatcher", line 468, in main
May 20 15:10:52 merged networkd-dispatcher[428]: dispatcher.register()
May 20 15:10:52 merged networkd-dispatcher[428]: File "/usr/bin/networkd-dispatcher", line 237, in register
May 20 15:10:52 merged networkd-dispatcher[428]: bus = dbus.SystemBus()
May 20 15:10:52 merged networkd-dispatcher[428]: File "/usr/lib/python3/dist-packages/dbus/_dbus.py", line 194, in __new__
May 20 15:10:52 merged networkd-dispatcher[428]: private=private)
May 20 15:10:52 merged networkd-dispatcher[428]: File "/usr/lib/python3/dist-packages/dbus/_dbus.py", line 100, in __new__
May 20 15:10:52 merged networkd-dispatcher[428]: bus = BusConnection.__new__(subclass, bus_type, mainloop=mainloop)
May 20 15:10:52 merged networkd-dispatcher[428]: File "/usr/lib/python3/dist-packages/dbus/bus.py", line 122, in __new__
May 20 15:10:52 merged networkd-dispatcher[428]: bus = cls._new_for_bus(address_or_type, mainloop=mainloop)
May 20 15:10:52 merged networkd-dispatcher[428]: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.AccessDenied: Failed to query AppArmor policy: Permission denied
May 20 15:10:52 merged systemd[1]: networkd-dispatcher.service: Main process exited, code=exited, status=1/FAILURE
May 20 15:10:52 merged systemd[1]: networkd-dispatcher.service: Failed with result 'exit-code'.
May 20 15:10:52 merged systemd[1]: Failed to start Dispatcher daemon for systemd-networkd.
May 20 15:10:53 merged systemd[1]: Started Wait until snapd is fully seeded.
May 20 15:10:53 merged systemd[1]: cloud-config.service: Failed to reset devices.list: Operation not permitted
May 20 15:10:53 merged systemd[1]: Starting Apply the settings specified in cloud-config...
May 20 15:10:54 merged cloud-init[494]: Cloud-init v. 19.4-33-gbb4131a2-0ubuntu1~18.04.1 running 'modules:config' at Wed, 20 May 2020 15:10:54 +0000. Up 9.43 seconds.
May 20 15:10:54 merged systemd[1]: Started Apply the settings specified in cloud-config.
May 20 15:10:57 merged snapd[421]: daemon.go:540: gracefully waiting for running hooks
May 20 15:10:57 merged snapd[421]: daemon.go:542: done waiting for running hooks
May 20 15:10:57 merged snapd[421]: daemon stop requested to wait for socket activation