Is it normal for cloud-init VMs to crash and reboot during provisioning?

candlerb · August 26, 2026, 4:07pm

One-line reproducer (incus 7.0.1)

nsrc@host1:~$ time incus launch images:ubuntu/26.04/cloud --vm testvm --console
Launching testvm
To detach from the console, press: <ctrl>+a q
BdsDxe: loading Boot0002 "UEFI QEMU QEMU HARDDISK " from PciRoot(0x0)/Pci(0x1,0x1)/Pci(0x0,0x0)/Scsi(0x0,0x1)
BdsDxe: starting Boot0002 "UEFI QEMU QEMU HARDDISK " from PciRoot(0x0)/Pci(0x1,0x1)/Pci(0x0,0x0)/Scsi(0x0,0x1)
[    2.244817] systemd[1]: Failed to preset all unit: Unit /etc/systemd/system/ua-auto-attach.service is masked
[    2.248110] systemd[1]: Failed to preset all unit: Unit chronyd.service is an unresolvable alias
[    2.248651] systemd[1]: Failed to preset all unit: Unit /run/systemd/generator/boot-efi.mount is transient or generated
[    2.249282] systemd[1]: Failed to preset all unit: Unit /run/systemd/generator/-.mount is transient or generated
[    2.249890] systemd[1]: Failed to preset all unit: Unit /usr/lib/systemd/system/hwclock.service is masked
[    2.250448] systemd[1]: Failed to preset all unit: Unit /usr/lib/systemd/system/cryptdisks.service is masked
[    2.251025] systemd[1]: Failed to preset all unit: Unit /usr/lib/systemd/system/x11-common.service is masked
[    2.251589] systemd[1]: Failed to preset all unit: Unit /usr/lib/systemd/system/cryptdisks-early.service is masked
[    2.252188] systemd[1]: Failed to preset all unit: Unit /usr/lib/systemd/system/sudo.service is masked
[    2.252727] systemd[1]: Failed to preset all unit: Unit /run/systemd/generator.late/netplan-ovs-cleanup.service is transient or generated
[    2.253431] systemd[1]: Failed to preset all unit /etc/systemd/system/ctrl-alt-del.target: Read-only file system
[    2.254011] systemd[1]: Failed to preset all unit /etc/systemd/system/dracut-pre-udev.service: Read-only file system
[    2.254594] systemd[1]: Failed to preset all unit /etc/systemd/system/multi-user.target.wants/remote-veritysetup.target: Read-only file system
[    2.255311] systemd[1]: Failed to preset all unit /etc/systemd/system/dracut-cmdline.service: Read-only file system
[    2.255910] systemd[1]: Failed to preset all unit /etc/systemd/system/sysinit.target.wants/systemd-sysext.service: Read-only file system
[    2.256627] systemd[1]: Failed to preset all unit /etc/systemd/system/sockets.target.wants/systemd-sysext.socket: Read-only file system
[    2.257297] systemd[1]: Failed to preset all unit /etc/systemd/system/sysinit.target.wants/systemd-pcrlock-machine-id.service: Read-only file system
[    2.258111] systemd[1]: Failed to preset all unit /etc/systemd/system/sysinit.target.wants/systemd-pcrlock-file-system.service: Read-only file system
[    2.258921] systemd[1]: Failed to preset all unit: Unit /etc/systemd/system/factory-reset.target.wants/systemd-tpm2-clear.service does not exist
[    2.259658] systemd[1]: Failed to preset all unit /etc/systemd/system/dracut-pre-trigger.service: Read-only file system
[    2.260268] systemd[1]: Failed to preset all unit /etc/systemd/system/sysinit.target.wants/systemd-pcrlock-firmware-code.service: Read-only file system
[    2.261026] systemd[1]: Failed to preset all unit /etc/systemd/system/dracut-pre-pivot.service: Read-only file system
[    2.261614] systemd[1]: Failed to preset all unit /etc/systemd/system/sysinit.target.wants/systemd-pcrlock-make-policy.service: Read-only file system
[    2.262375] systemd[1]: Failed to preset all unit /etc/systemd/system/sysinit.target.wants/systemd-udev-load-credentials.service: Read-only file system
[    2.263137] systemd[1]: Failed to preset all unit /etc/systemd/system/dracut-pre-mount.service: Read-only file system
[    2.263730] systemd[1]: Failed to preset all unit /etc/systemd/system/dracut-shutdown-onfailure.service: Read-only file system
[    2.264380] systemd[1]: Failed to preset all unit /etc/systemd/system/sysinit.target.wants/systemd-pcrlock-firmware-config.service: Read-only file system
[    2.265144] systemd[1]: Failed to preset all unit /etc/systemd/system/sysinit.target.wants/systemd-pcrlock-secureboot-policy.service: Read-only file system
[    2.265922] systemd[1]: Failed to preset all unit /etc/systemd/system/sockets.target.wants/systemd-journald-audit.socket: Read-only file system
[    2.266637] systemd[1]: Failed to preset all unit: Unit /etc/systemd/system/systemd-journald.service.wants/systemd-journald-audit.socket does not exist
[    2.267407] systemd[1]: Failed to preset all unit /etc/systemd/system/multi-user.target.wants/remote-cryptsetup.target: Read-only file system
[    2.268115] systemd[1]: Failed to preset all unit /etc/systemd/system/dracut-shutdown.service: Read-only file system
[    2.268700] systemd[1]: Failed to preset all unit /etc/systemd/system/dracut-mount.service: Read-only file system
[    2.269274] systemd[1]: Failed to preset all unit /etc/systemd/system/sockets.target.wants/systemd-pcrextend.socket: Read-only file system
[    2.270026] systemd[1]: Failed to preset all unit /etc/systemd/system/sockets.target.wants/systemd-factory-reset.socket: Read-only file system
[    2.270754] systemd[1]: Failed to preset all unit /etc/systemd/system/sysinit.target.wants/systemd-pcrlock-secureboot-authority.service: Read-only file system
[    2.271574] systemd[1]: Failed to preset all unit /etc/systemd/system/sysinit.target.wants/systemd-confext.service: Read-only file system
[    2.272289] systemd[1]: Failed to preset all unit /etc/systemd/system/multi-user.target.wants/remote-integritysetup.target: Read-only file system
[    2.273058] systemd[1]: Failed to preset all unit /etc/systemd/system/dracut-initqueue.service: Read-only file system
[    3.168307] cloud-init[384]: Cloud-init v. 26.1-0ubuntu3~26.04.1 running 'init-local' at Wed, 26 Aug 2026 15:49:56 +0000. Up 3.14 seconds.
[    3.180040] cloud-init[384]: 2026-08-26 15:49:57,007 - DataSourceLXD.py[WARNING]: /dev/lxd/sock does not exist.
[    4.124334] cloud-init[384]: Cloud-init v. 26.1-0ubuntu3~26.04.1 running 'init' at Wed, 26 Aug 2026 15:49:57 +0000. Up 4.10 seconds.
[    4.262078] cloud-init[384]: 2026-08-26 15:49:58,089 - ubuntu.py[ERROR]: Failed to wait for network: Unexpected error while running command.
[    4.323605] cloud-init[384]: Command: ['systemctl', 'start', 'systemd-networkd-wait-online.service']
[    4.327079] cloud-init[384]: Exit code: 1
[    4.327430] cloud-init[384]: Reason: -
[    4.327716] cloud-init[384]: Stdout:
[    4.327982] cloud-init[384]: Stderr: Job for systemd-networkd-wait-online.service canceled.
[    4.336032] cloud-init[384]: ci-info: +++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++
[    4.346093] cloud-init[384]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+
[    4.346794] cloud-init[384]: ci-info: | Device |   Up  |  Address  |    Mask   | Scope |     Hw-Address    |
[    4.357111] cloud-init[384]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+
[    4.360845] cloud-init[384]: ci-info: | enp5s0 | False |     .     |     .     |   .   | 10:66:6a:b7:da:24 |
[    4.366160] cloud-init[384]: ci-info: |   lo   |  True | 127.0.0.1 | 255.0.0.0 |  host |         .         |
[    4.369838] cloud-init[384]: ci-info: |   lo   |  True |  ::1/128  |     .     |  host |         .         |
[    4.373056] cloud-init[384]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+
[    4.377161] cloud-init[384]: ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++
[    4.381600] cloud-init[384]: ci-info: +-------+-------------+---------+-----------+-------+
[    4.383093] cloud-init[384]: ci-info: | Route | Destination | Gateway | Interface | Flags |
[    4.386158] cloud-init[384]: ci-info: +-------+-------------+---------+-----------+-------+
[    4.389068] cloud-init[384]: ci-info: +-------+-------------+---------+-----------+-------+
[    4.445501] cloud-init[384]: Received signal 15 resulting in exit. Cause:
[    4.446053] cloud-init[384]:   Filename: /usr/lib/python3.14/selectors.py
Received signal 15 resulting in exit. Cause:
  Filename: /usr/lib/python3.14/selectors.py
  Function: select
  Line number: 398
    Filename: /usr/lib/python3.14/subprocess.py
    Function: _communicate
    Line number: 2153
      Filename: /usr/lib/python3.14/subprocess.py
      Function: communicate
      Line number: 1221
[    4.450067] cloud-init[384]:   Function: select
[    4.453963] cloud-init[384]:   Line number: 398
[    4.455787] cloud-init[384]:     Filename: /usr/lib/python3.14/subprocess.py
[    4.456750] cloud-init[384]:     Function: _communicate
[    4.457979] cloud-init[384]:     Line number: 2153
[    4.460670] cloud-init[384]:       Filename: /usr/lib/python3.14/subprocess.py
[    4.462346] cloud-init[384]:       Function: communicate
[    4.464047] cloud-init[384]:       Line number: 1221
[    4.465077] cloud-init[384]: Traceback (most recent call last):
[    4.466175] cloud-init[384]:   File "/usr/bin/cloud-init", line 9, in <module>
[    4.469036] cloud-init[384]:     sys.exit(main.main())
[    4.469450] cloud-init[384]:              ~~~~~~~~~^^
[    4.470885] cloud-init[384]:   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 1309, in main
[    4.473343] cloud-init[384]:     return all_stages(parser)
[    4.474036] cloud-init[384]:   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 1331, in all_stages
[    4.476790] cloud-init[384]:     with sync("network"):
[    4.477318] cloud-init[384]:          ~~~~^^^^^^^^^^^
[    4.479053] cloud-init[384]:   File "/usr/lib/python3/dist-packages/cloudinit/socket.py", line 161, in __exit__
[    4.479791] cloud-init[384]:     sock.sendall(
[    4.481034] cloud-init[384]:     ~~~~~~~~~~~~^
[    4.481422] cloud-init[384]:         f"echo '{message}'; exit {self.systemd_exit_code};".encode()
[    4.483038] cloud-init[384]:         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[    4.483673] cloud-init[384]:     )
[    4.485036] cloud-init[384]:     ^
[    4.485323] cloud-init[384]: BrokenPipeError: [Errno 32] Broken pipe
[    4.652023] reboot: Restarting system


real	0m12.605s
user	0m0.080s
sys	0m0.046s

The VM reboots and comes up successfully on the second attempt. This looks a bit disconcerting though.

ubuntu/26.04 does not have this issue. This suggests it’s cloud-init specific.

Aside: I also tried freebsd/15.1/cloud. It also reboots, but I think this is unrelated, as it’s due to auto-applying updates. In /var/log/messages I see:

...
Aug 26 15:57:41 testvm3 pkg[1789]: [80/80] Upgrading FreeBSD-zoneinfo from 15.1 to 15.1p2...
Aug 26 15:57:41 testvm3 pkg[1789]: [80/80] Extracting FreeBSD-zoneinfo-15.1p2: .......... done
Aug 26 15:57:41 testvm3 pkg[1800]: FreeBSD-zoneinfo upgraded: 15.1 -> 15.1p2
Aug 26 15:57:41 testvm3 pkg[1789]: ==> Running trigger: mandoc.ucl
Aug 26 15:57:41 testvm3 pkg[1789]: Generating apropos(1) database for /usr/share/man...
Aug 26 15:57:42 testvm3 pkg[1789]: Generating apropos(1) database for /usr/share/openssl/man...
Aug 26 15:57:43 testvm3 pkg[1789]: Requesting reboot after installing updates.
Aug 26 15:57:43 testvm3 syslogd: exiting on signal 15
Aug 26 15:57:59 testvm3 syslogd: kernel boot file is /boot/kernel/kernel
Aug 26 15:57:59 testvm3 kernel: ---<<BOOT>>---
Aug 26 15:57:59 testvm3 kernel: Copyright (c) 1992-2025 The FreeBSD Project.
Aug 26 15:57:59 testvm3 kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
...

However, I can’t find any trace of cloud-init on the system:

root@testvm3:~ # find / -iname '*cloud*'
root@testvm3:~ #

This seems strange for a /cloud variant.

stgraber · August 26, 2026, 7:49pm

It’s normal for cloud VMs to reboot during provisioning, yes.

Basically we need the incus-agent process to put the actual cloud-init configuration in place. At which point it triggers a reboot to have cloud-init run and apply it correctly.

We don’t do that in containers as we can control the filesystem externally and so avoid any potential race with cloud-init.

github.com/lxc/incus

cmd/incus-agent/main_agent.go

main


      
          			return err
          		}
          	} else if util.PathExists("/etc/rc.conf.d/hostname") && slices.Contains(files, "/etc/rc.conf.d/hostname") {
          		// Set the hostname.
          		_, err := subprocess.RunCommand("service", "hostname", "restart")
          		if err != nil {
          			return err
          		}
          	}
          
          	// Run cloud-init.
          	if util.PathExists("/etc/cloud") && slices.Contains(files, "/var/lib/cloud/seed/nocloud-net/meta-data") {
          		logger.Info("Seeding cloud-init")
          
          		cloudInitPath := "/run/cloud-init"
          		if util.PathExists(cloudInitPath) {
          			logger.Info(fmt.Sprintf("Removing %q", cloudInitPath))
          			err = os.RemoveAll(cloudInitPath)
          			if err != nil {
          				return err
          			}