Cloud-init not being run on second boot

I wonder if anyone can help here. cloud-init is correctly running on first start, but if I clean it all out with “cloud-init clean …” and restart the container then it’s not running a second time.

More specifically:

$ incus create images:ubuntu/22.04/cloud foo
$ sudo ls /var/lib/incus/containers/foo/rootfs/etc/netplan /var/lib/incus/containers/foo/rootfs/var/lib/cloud
... confirms those directories are not present
$ incus start foo
$ sudo ls /var/lib/incus/containers/foo/rootfs/etc/netplan /var/lib/incus/containers/foo/rootfs/var/lib/cloud

data  handlers	instance  instances  scripts  seed  sem

$ incus shell foo
root@foo:~# cloud-init clean --seed --logs --configs all
2024-04-19 14:38:46,174 -[WARNING]: skipping /etc/cloud/clean.d/README as its not executable or the underlying file system is mounted without executable permissions.
root@foo:~# ls /etc/netplan
root@foo:~# ls /var/lib/cloud
root@foo:~# exit

$ incus stop foo
$ incus start foo

I find that /etc/netplan and /var/lib/cloud are not populated, and as a result, the instance does not pick up an (IPv4) address.

What’s different between the second startup and the first?

I think I found it. Before first boot, the container has volatile.apply_template: create. The image templates (in metadata.yaml and *.tpl) unpack various cloud-init files under /var/lib/cloud/seed/nocloud-net/

If I understand this correctly, it means there could be up to three different cloud-init mechanisms at play, depending on what you’re doing with incus:

  • nocloud files stuffed directly into the container
  • an ISO device that you attach to the VM
  • the lxd/incus data source via the incus agent (although I’m now not sure if or when that’s actually used)

I believe that’s correct, I did find somewhere you could reinitialize but it’s like creating a new instance. What are you trying to do?


The cloud-init actions are run only once on the first start of the instance. Rebooting the instance does not re-trigger the actions

Yeah, for the case where cloud-init is fed configuration through on-disk files, which includes the default NoCloud files in containers and VMs (created by the agent in the VM case), then you need to set volatile.apply_template to create to have Incus re-generate those files.

The native LXD datasource is meant to better handle this but it’s not quite working with Incus nor is it available in all distributions yet.

And indeed, when attaching a cloud-init ISO to a VM, those files aren’t templated and so will not need a reset of the volatile key.

1 Like

What are you trying to do?

It’s a multi-stage build which involves creating a bunch of different containers, and then assembling them into a master VM image (this is done as a separate stage, so that if I change one container I don’t have to rebuild all the others).

I was fully cleaning the containers, but when the VM started they weren’t picking up addresses via DHCP. For now I’ve just removed the cleaning step.

I thought that was referring to the behaviour of cloud-init itself - which maintains state between boots.