How to merge profiles' user.vendor-data?

LXD
22

user.meta-data doesn’t get merged, it’s a different much more limited format that pretty much just tells cloud-init what the instance name and id is. In the new world, meta-data cannot be altered by the user anymore (that’s why it doesn’t have a cloud-init.meta-data equivalent).

23

Added to Linux Containers - LXD - Has been moved to Canonical

25

Any news? How to apply cloud-init.user-data consistently from many profiles or may be to merge they today?

26

There hasn’t been a change to this.

You can have at most two Incus profiles with cloud-init instructions.
One profile would list those instructions in a user.user-data and the other profile in a user.vendor-data section. Then, by specifying both these two profiles when launching an instance, you would get these cloud-init instructions merged together.

The command line could look like the following.

incus launch images:debian/12/cloud --profile default --profile addwebserver --profile adddatabaseserver
1 Like
27

I tested it and for me don’t work (( what am I doing wrong?

I created two profiles:

incus profile create vendor
incus profile create user

and created content of these profiles:

incus profile set vendor cloud-init.vendor-data - << EOF
#cloud-config
runcmd:
  - echo DISPLAY="$DISPLAY" >> /etc/environment
  - echo WAYLAND_DISPLAY="$WAYLAND_DISPLAY" >> /etc/environment
write_files:
  - path: /home/ubuntu/vendor-content.txt
    permissions: 0755
    content: |
      vendor content
EOF

incus profile set user cloud-init.user-data - << EOF
#cloud-config
write_files:
  - path: /home/ubuntu/user-content.txt
    permissions: 0755
    content: |
      user content
runcmd:
  - echo XDG_SESSION_TYPE="wayland" >> /etc/environment
  - echo QT_QPA_PLATFORM="wayland" >> /etc/environment
EOF

Then launched the container:

incus create ubuntu2310cloud websurf --profile=vendor --profile=user
incus start websurf

After finished cloud-init:

incus exec websurf -- su -l ubuntu -c 'cloud-init analyze show'

I got configuration only last profile (user). Why? Env variables created only from user profile:

incus exec websurf -- su -l ubuntu -c 'cat /etc/environment'

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
XDG_SESSION_TYPE=wayland
QT_QPA_PLATFORM=wayland

and only one file created in home directory. The file vendor-content.txt is missed:

incus exec websurf -- su -l ubuntu -c 'ls -l $HOME'

total 4
-rwxr-xr-x 1 root root 13 Apr  3 10:54 user-content.txt
28

Have a look in the container in the directory /var/lib/cloud/seed/nocloud-net/.

The files are in there, verbatim, just as they came from the Incus profile.

debian@mycloud:/var/lib/cloud/seed/nocloud-net$ ls -l
total 4
-rw-r--r-- 1 root root  46 Apr  3 11:33 meta-data
-rw-r--r-- 1 root root 107 Apr  3 11:33 network-config
-rw-r--r-- 1 root root 230 Apr  3 11:33 user-data
-rw-r--r-- 1 root root 213 Apr  3 11:33 vendor-data
debian@mycloud:/var/lib/cloud/seed/nocloud-net$

This means that now cloud-init took over, has the proper user-data and vendor-data files and can do its magic. It’s off the hands of Incus.

I tried your instructions. It looks like cloud-init might perform some merging of its own. Because it runs part of the user-data and part of the vendor-data.

29

I didn’t understand. Is it now impossible to merge cloud-init.vendor-data and cloud-init.user-data? It now don’t work?

30

Incus does what is required so that the cloud-init.vendor-data and cloud-init.user-data are placed in the container at the proper location ( /var/lib/cloud/seed/nocloud-net) for cloud-init to do its magic.
I do not have deep knowledge of cloud-init; you would need to check the logs or perhaps enable more detailed logs to figure out what’s going on.

Your example files are mostly fine. I wouldn’t put my test files in /home/ubuntu/ because I am not sure in what sequence the non-root account is created. There’s a possibility that /home/ubuntu has not been created yet.

31

Ok, thanks! But env variables to /etc/environment also don’t sets from my cloud-init.vendor-data, so I think it’s not a matter of ubuntu user that hasn’t been created yet. I will look at the logs of cloud-init

32

The incus documentation says:

If both vendor-data and user-data are supplied for an instance, cloud-init merges the two configurations. However, if you use the same keys in both configurations, merging might not be possible. In this case, configure how cloud-init should merge the provided data. See Merging user data sections for instructions.

Both your configs are providing “runcmd” and “write_files”, so if you don’t want one to overwrite the other, you need to tell cloud-init how to merge them, as per this example.

2 Likes
33

For the scenario where NoCloud provider is used & you have the same section such as runcmd in both vendor and user data, there are still problems with merging stuff. The format of user-data has to be in #cloud-config-jsonp if you want to achieve any merging by cloud-init at all.

JSONP = JSON Patch

Repro:

architecture: x86_64
config:
  cloud-init.user-data: |
    #cloud-config-archive
    - type: text/cloud-config
      content: |
        #cloud-config
        merge_how:
          - name: list
            settings: [append]
          - name: dict
            settings: [no_replace, recurse_list]
        runcmd:
          - echo 2 > /from-user-data-1
    - type: text/cloud-config
      content: |
        #cloud-config
        merge_how:
          - name: list
            settings: [append]
          - name: dict
            settings: [no_replace, recurse_list]
        runcmd:
          - echo 3 > /from-user-data-2
  cloud-init.vendor-data: |
    #cloud-config
    merge_how:
      - name: list
        settings: [append]
      - name: dict
        settings: [no_replace, recurse_list]
    runcmd:
      - echo 1 > /from-vendor-data

Gets the following:

➜  ~ incus stop cloud-init-test; incus rebuild images:debian/trixie/cloud cloud-init-test; incus start cloud-init-test; incus exec cloud-init-test -- cloud-init status --wait; incus exec cloud-init-test -- bash
......................status: done
root@cloud-init-test:~# ls /
bin  boot  dev  etc  from-user-data-1  from-user-data-2  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

The behaviour is technically correct as the design philosophy of vendor data is that you can override all of it in user data cleanly. So it goes back to the problem of having access to only 1 user data in 1 of many profiles. You can not achieve separation of concerns by using multiple profiles.

So IMO a new custom Incus data source for cloud-init that has the correct merging support is really the best solution compared to bodging together some kind of archive which wraps a curl script and spits out another file etc etc.

The alternative would be to use a proper configuration mangement tool such as Ansible or Terraform.

34

After more digging, I’ve reached the conclusion that the JSON patch support is buggy and fast-forward an hour or so, I’ve found #cloud-config-jsonp is almost completely useless · Issue #5549 · canonical/cloud-init · GitHub . The cloud-init documentation has also been updated to remove the mention of jsonpatch.

For my use case which is simple, I’ll probably end up using the ability to feed YAML directly to incus launch when I get incus >= 6.1.

35

I actually did something today. This reuses Kusalananda’s answer to merge arrays in JSON with jq. I haven’t understood the limitation (nor have I been able to use the alternative posted later on in the thread) and I need to filter out yq’s --- but that’s minor.

Here is a snippet of the code:

echo '#cloud-config'

yq \
  --yaml-output \
  --slurp \
  '(.[0] | keys[]) as $k | reduce .[] as $item (null; .[$k] += $item[$k])' \
  "${vendor_data_file}" \
  "${new_data_file}" \
  | \
  sed \
  '/^---$/ d'

And you can use it as

incus profile set "${profile}" cloud-init.user-data="$(foo.sh cloud-init-vendor-data.yaml cloud-init-user-data.yaml)"

I can’t say if there are issues or limitations but it’s been working for me and my simple usage.