User-data config is being repeatedly ignored - no errors that make sense

1

lxd v4.0.4 installed via snapd on ubuntu 20.04 …

Been trying with images:

  • ubuntu:20.04
  • ubuntu:18.04
  • images:ubuntu/focal/cloud

Network configs are ok … and the correct userdata gets copied to /var/lib/cloud/seed/nocloud-net.

Separating the userdat into a yaml file makes no difference either.

I have run validate-yaml.py on the profile as well as the userdata.yaml, and it says they are OK

Have been through some of the debugging suggestions listed at:
https://cloudinit.readthedocs.io/en/latest/topics/debugging.html

The only error I see in the cloud-init logs in the container is the following:

2021-01-22 22:10:22,958 - __init__.py[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloud'>
2021-01-22 22:10:22,958 - __init__.py[DEBUG]: Update datasource metadata and network config due to events: New instance first boot
2021-01-22 22:10:22,958 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/user-data (quiet=False)
2021-01-22 22:10:22,958 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/meta-data (quiet=False)
2021-01-22 22:10:22,959 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/vendor-data (quiet=False)
2021-01-22 22:10:22,959 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/network-config (quiet=False)
2021-01-22 22:10:22,959 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/user-data (quiet=False)
2021-01-22 22:10:22,959 - util.py[DEBUG]: Read 18 bytes from /var/lib/cloud/seed/nocloud-net/user-data
2021-01-22 22:10:22,959 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/meta-data (quiet=False)
2021-01-22 22:10:22,959 - util.py[DEBUG]: Read 50 bytes from /var/lib/cloud/seed/nocloud-net/meta-data
2021-01-22 22:10:22,959 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/vendor-data (quiet=False)
2021-01-22 22:10:22,959 - util.py[DEBUG]: Read 18 bytes from /var/lib/cloud/seed/nocloud-net/vendor-data
2021-01-22 22:10:22,959 - util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/network-config (quiet=False)
2021-01-22 22:10:22,959 - util.py[DEBUG]: Read 5 bytes from /var/lib/cloud/seed/nocloud-net/network-config
2021-01-22 22:10:22,959 - DataSourceNoCloud.py[DEBUG]: Using seeded data from /var/lib/cloud/seed/nocloud-net
2021-01-22 22:10:22,959 - util.py[DEBUG]: Attempting to load yaml from string of length 50 with allowed root types (<class 'dict'>,)
2021-01-22 22:10:22,959 - util.py[DEBUG]: Attempting to load yaml from string of length 5 with allowed root types (<class 'dict'>,)
2021-01-22 22:10:22,960 - util.py[WARNING]: Failed loading yaml blob. Yaml load allows (<class 'dict'>,) root types, but got str instead
2021-01-22 22:10:22,960 - handlers.py[DEBUG]: finish: init-local/search-NoCloud: FAIL: no local data found from DataSourceNoCloud
2021-01-22 22:10:22,960 - util.py[WARNING]: Getting data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloud'> failed
2021-01-22 22:10:22,960 - util.py[DEBUG]: Getting data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloud'> failed

It’s not obvious which of the config files it’s failing on, but the sequence of error messages suggest it might be the having issues with network-config.

The following are the corresponding entries for network-config:

# in profile (snipped off the user-data)
  user.network-config: dhcp
  user.networking_mode: dhcp
  user.user-data: |
    #cloud-config

# in /var/lib/cloud/seed/nocloud-net/network-config
dhcp

I am running macvlan, and the network comes up and obtains dhcp ip address from the external dhcp server … but no section of the user-data gets processed.

I guess the most frustrating thing is not even being sure which config file it’s having issues with, and what the issue is …

Would really appreciate any insights / suggestions I can get … have given this thing the better part of 2 days and still no joy.

Note: I am not new to using cloud-config to provision containers … and while troubleshooting I stripped down the user config to really simple package installs, just to be sure nothing else in there was tripping cloud init.

2

Have tried practically every way of debugging this … and finally narrowed the problem down to this fact …

  • If user.network-config: dhcp is set
    • then user.user-data is completely ignored …
  • If user.network-config: dhcp is NOT set
    • then user.user-data is processed, but then I get no dhcp networking

So, it seems the way user.networking-config is processed has completely changed in later versions of lxd, because this has been a standard way I managed all my profiles in some use cases for years now.

I am just relieved that I have understood the cause of this baffling situation

My big question now is:

  • How to simply get dhcp address to macvlan interfaces, already configured in the devices: section ?
  • Alternatively, how to adapt my usage of user.network-config so it does what I am expecting it to do?

Will really appreciate any suggestions

3

:man_facepalming:

  • dhcp is provided by default for macvlan interfaces but device has to be named eth0
  • so I don’t need the user.network-config section for that
  • I was using specially named interface names, like: dmz0, svc1, etc … and cloudinit just ignored them
  • user.network-config disables user.user-data processing … and this was the origin of my troubles and wasted days of my time. I still haven’t understood why, and hope someone can help me understand why.

What a relief to get to the bottom of this … because I so much need to get this working

Task for another day: comprehend the user.network-config v2 usage