[LXD] First class cloud-init support

monstermunchkin · August 31, 2022, 2:18pm

Both ubuntu:focal and images:ubuntu/cloud images use cloud-init templates (see here). The difference is that ubuntu:focal only considers user.* keys whereas images:ubuntu/cloud considers both user.* and cloud-init.* keys.

This is indeed confusing as e.g. ubuntu:jammy uses cloud-init.* keys.

@stgraber would it be possible to update the template files for older ubuntu: images to consider cloud-init.* keys?

stgraber · August 31, 2022, 3:40pm

Should be, yeah. Can you file a bug at cloud-images in Launchpad ?

monstermunchkin · September 1, 2022, 9:00am

j5awry · September 28, 2022, 7:12pm

Updates related to Bug #1988401 “LXD image metadata: Update nocloud-net templates i...” : Bugs : cloud-images

On LXD < 5.1 based hosts (which included Ubuntu Focal and Ubuntu Bionic by default), there is a bug related to rendering of specific characters. This was fixed upstream, but not ported to the LTS snap channels yet

github.com/lxc/lxd

user-data gets broken replacing `%s` with `%!s(MISSING)`

opened 02:46AM - 12 Apr 22 UTC

closed 07:47AM - 12 Apr 22 UTC

freyes

Bug Easy

# Required information * Distribution: Ubuntu * Distribution version: 22.0…4 (Jammy) * The output of "lxc info": https://paste.ubuntu.com/p/qhNjbNXqyM/ * Storage backend in use: dir # Issue description Launching a container with a user-data that contains `%s` gets replaced with `%!s(MISSING)`, this is breaking the launch of juju containers since those are launched with a bash script passed in the `bootcmd` stanza, more details about the analysis of the symptoms in the juju side can be found https://bugs.launchpad.net/bugs/1968384 . # Steps to reproduce 1. Create a test profile: ``` cat <<EOF>test-profile.yaml config: cloud-init.user-data: | #cloud-config bootcmd: - |- printf '%s\n' 'network: version: 2 ' > '/tmp/test.txt' description: "" devices: {} name: test EOF lxc profile create test lxc profile edit test < test-profile.yaml ``` 2. Launch a container with the test profile on top of default profile. (the image should have cloud-init in it). ``` lxc launch -p default -p test juju/jammy/s390x test-jammy ``` Expected result: There is a file inside the container in the path `/tmp/test.txt` with the content: ``` network: version: 2 ``` Actual result: The file `/tmp/test.txt` is empty and checking the log /var/log/cloud-init-output.log we find: ``` /run/cloud-init/tmp/tmpdoppfy12.sh: 2: printf: %!: invalid directive 2022-04-12 02:15:08,490 - util.py[WARNING]: Failed to run bootcmd module bootcmd 2022-04-12 02:15:08,491 - util.py[WARNING]: Running module bootcmd (<module 'cloudinit.config.cc_bootcmd' from '/usr/lib/python3/dist- packages/cloudinit/config/cc_bootcmd.py'>) failed ``` The content of the user-data file that cloud-init wrote contains the broken '%s': ``` root@test-jammy:~# cat /var/lib/cloud/instances/f59e3749-9836-42e3- b84e-50267bb710e9/user-data.txt #cloud-config bootcmd: - |- printf '%!s(MISSING)\n' 'network: version: 2 ' > '/tmp/test.txt' ``` # Information to attach - [ ] Any relevant kernel output (`dmesg`) - [ ] Container log (`lxc info NAME --show-log`) - [ ] Container configuration (`lxc config show NAME --expanded`) - [ ] Main daemon log (at /var/log/lxd/lxd.log or /var/snap/lxd/common/lxd/logs/lxd.log) - [ ] Output of the client with --debug - [ ] Output of the daemon with --debug (alternatively output of `lxc monitor` while reproducing the issue)

So we had to revert our move to the LXD datasource on 18.04 and 20.04 guests as there appears most use cases are matching Host and Guest version. Note that 18.04 hosts running LXD 4.0 will still hit this bug with 22.04+ Ubuntu guests.

When the patch is backported to 4.0 channel, we’ll re-add the LXD datasource move once the fix is backported. Specific users that raised the issue are also working on changing their cloud-init scripts to stop using the offending symbols.