How to merge profiles' user.vendor-data?

LXD
11

Has there been any changes to this?
trying to have 2 profiles with a #cloud-config work together, basically appending one another.
Which i take is what this discussion was about?

12

Nope, no change, your two options still are user.user-data and user.vendor-data, if you need more than two then you’re out of luck.

There’s been an item for proper integration of LXD with cloud-init on the cloud-init team’s roadmap for a couple of years, though that work always seems to get scheduled and then not actually make it… Once they finally get to implement their side of it, we’ll have more options to provide configuration to cloud-init than what we do today.

13

how does the user.vendor-data work?
i could never get both it and user.user-data to work together,
not with yaml or jsonp (which i don’t get if it’s just for runcmd or not).
i am guessing i must be doing something wrong?

14

vendor-data uses the same format as user-data and gets merged with it. So as long as you don’t use the same config keys in both, cloud-init will merge them together

15

Ah that’s the problem then, so if one has “users” the other can’t append to it but it skipped or replace it.

Well hope the day comes when they add the support you mentioned before as it seems quite useful.

Thanks!

16

Following on from previous questions – how is user.network-config merged (in my experiment on 4.0.9) it didn’t seem to be. And also is there much change in the situation with version 5; I’ve noticed the rename to cloud-init.vendor-data, cloud-init.user-data and cloud-init.network-config happened.

17

Still don’t know in detail how all magic works, but when I add the merge_how section (https://cloudinit.readthedocs.io/en/latest/topics/merging.html) it works – without the merge_how, one file wins!

config:
  user.user-data: |
    #cloud-config
    merge_how:
    - name: list
      settings: [append]
    - name: dict
      settings: [no_replace, recurse_list]
...etc

Realising that data is now provided though a dedicated lxd datasource (and not files) helped with the debugging!
https://cloudinit.readthedocs.io/en/latest/topics/datasources/lxd.html

2 Likes
18

@olx Would you mind sharing your example profiles for how this works for you? Does ordering matter?

@stgraber would this be something that could be added to the documentation around cloud-init and profiles? It seems there is a path forward now and it would be a good way to show you could layer profiles. my use case is to have a user profile and then an app profile … I am trying to follow a dry strategy. In the past I just had external template that I would generate a bunch of different profiles but that is a clunky long term strategy.

19

@ru-fu the merge behafvior for user-data + vendor-data is something that could probably be added to our docs.

21

Sure!

So cloud-init.user-data and cloud-init.vendor-data get merged together.

What about user.meta-data?
According to the current Instance options docs, that one is also appended:

Additionally, those user keys have become common with images (support isn’t guaranteed):

KEY TYPE DEFAULT DESCRIPTION
user.meta-data string - Cloud-init meta-data, content is appended to seed value

But that might be outdated?

22

user.meta-data doesn’t get merged, it’s a different much more limited format that pretty much just tells cloud-init what the instance name and id is. In the new world, meta-data cannot be altered by the user anymore (that’s why it doesn’t have a cloud-init.meta-data equivalent).

23

Added to Linux Containers - LXD - Has been moved to Canonical

25

Any news? How to apply cloud-init.user-data consistently from many profiles or may be to merge they today?

26

There hasn’t been a change to this.

You can have at most two Incus profiles with cloud-init instructions.
One profile would list those instructions in a user.user-data and the other profile in a user.vendor-data section. Then, by specifying both these two profiles when launching an instance, you would get these cloud-init instructions merged together.

The command line could look like the following.

incus launch images:debian/12/cloud --profile default --profile addwebserver --profile adddatabaseserver
1 Like
27

I tested it and for me don’t work (( what am I doing wrong?

I created two profiles:

incus profile create vendor
incus profile create user

and created content of these profiles:

incus profile set vendor cloud-init.vendor-data - << EOF
#cloud-config
runcmd:
  - echo DISPLAY="$DISPLAY" >> /etc/environment
  - echo WAYLAND_DISPLAY="$WAYLAND_DISPLAY" >> /etc/environment
write_files:
  - path: /home/ubuntu/vendor-content.txt
    permissions: 0755
    content: |
      vendor content
EOF

incus profile set user cloud-init.user-data - << EOF
#cloud-config
write_files:
  - path: /home/ubuntu/user-content.txt
    permissions: 0755
    content: |
      user content
runcmd:
  - echo XDG_SESSION_TYPE="wayland" >> /etc/environment
  - echo QT_QPA_PLATFORM="wayland" >> /etc/environment
EOF

Then launched the container:

incus create ubuntu2310cloud websurf --profile=vendor --profile=user
incus start websurf

After finished cloud-init:

incus exec websurf -- su -l ubuntu -c 'cloud-init analyze show'

I got configuration only last profile (user). Why? Env variables created only from user profile:

incus exec websurf -- su -l ubuntu -c 'cat /etc/environment'

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
XDG_SESSION_TYPE=wayland
QT_QPA_PLATFORM=wayland

and only one file created in home directory. The file vendor-content.txt is missed:

incus exec websurf -- su -l ubuntu -c 'ls -l $HOME'

total 4
-rwxr-xr-x 1 root root 13 Apr  3 10:54 user-content.txt
28

Have a look in the container in the directory /var/lib/cloud/seed/nocloud-net/.

The files are in there, verbatim, just as they came from the Incus profile.

debian@mycloud:/var/lib/cloud/seed/nocloud-net$ ls -l
total 4
-rw-r--r-- 1 root root  46 Apr  3 11:33 meta-data
-rw-r--r-- 1 root root 107 Apr  3 11:33 network-config
-rw-r--r-- 1 root root 230 Apr  3 11:33 user-data
-rw-r--r-- 1 root root 213 Apr  3 11:33 vendor-data
debian@mycloud:/var/lib/cloud/seed/nocloud-net$

This means that now cloud-init took over, has the proper user-data and vendor-data files and can do its magic. It’s off the hands of Incus.

I tried your instructions. It looks like cloud-init might perform some merging of its own. Because it runs part of the user-data and part of the vendor-data.

29

I didn’t understand. Is it now impossible to merge cloud-init.vendor-data and cloud-init.user-data? It now don’t work?

30

Incus does what is required so that the cloud-init.vendor-data and cloud-init.user-data are placed in the container at the proper location ( /var/lib/cloud/seed/nocloud-net) for cloud-init to do its magic.
I do not have deep knowledge of cloud-init; you would need to check the logs or perhaps enable more detailed logs to figure out what’s going on.

Your example files are mostly fine. I wouldn’t put my test files in /home/ubuntu/ because I am not sure in what sequence the non-root account is created. There’s a possibility that /home/ubuntu has not been created yet.

31

Ok, thanks! But env variables to /etc/environment also don’t sets from my cloud-init.vendor-data, so I think it’s not a matter of ubuntu user that hasn’t been created yet. I will look at the logs of cloud-init

32

The incus documentation says:

If both vendor-data and user-data are supplied for an instance, cloud-init merges the two configurations. However, if you use the same keys in both configurations, merging might not be possible. In this case, configure how cloud-init should merge the provided data. See Merging user data sections for instructions.

Both your configs are providing “runcmd” and “write_files”, so if you don’t want one to overwrite the other, you need to tell cloud-init how to merge them, as per this example.

2 Likes