How to customize Incus containers with cloud-init

Tutorials
1

This tutorial replaces by old 2018 tutorial on cloud-init.
I use Incus containers in the title to make it search-engine friendly, though I show how all these also work in virtual machines.

In summary,

  1. I talk about the /cloud variant of images, that they have cloud-init baked in them. Also how to list them properly using incus list filters.
  2. Then, there is a bit of talk about Incus profiles.
  3. cloud-init requires Incus profiles, and I show how to put the configuration in separate profiles, then launch instances with --profile default --profile my-cloudinit-profile. I think it’s cleaner that way as a way to introduce cloud-init.
  4. Then, first demonstration using AlpineLinux and creating a simple file in /tmp.
  5. Then, I show how to launch AlpineLinux instances where IPv6 is disabled. This relates to a recent discussion on the forum. I show with both container and virtual machine. It demonstrates that it’s mostly the same between the two.
  6. Then, I switch to Debian. Install nginx and modify the index.html file to mention Incus. Repeat with --vm, works exactly the same.
  7. The Troubleshooting covers two issues. If you add extra spaces at the end of any of the cloud-init lines, then the cloud-init text is messed up wrt formatting. Why and how to avoid.
  8. The other is when connecting to the Web server in an Incus instance, you may get Unable to connect. This is a my browser switches automatically to https issue. It is good to get people to know about this because it will come up at some point when using Incus, even if they do not do cloud-init.
4 Likes
2

Thanks for writing that. I did not know you could stack profiles like that.

3

Depends on what you mean by stacking profiles. Devices will add up, but there is no automatic merging of cloud-init.user-data sections. This means only cloud-init.user-data section from the last profile will be applied to an instance.

But cloud-init.vendor-data uses the same format as cloud-init.user-data. So, as long as you don’t use the same config keys in both, cloud-init will apply them together to the instance. To sum up, only keys from the last cloud-init.vendor-data and cloud-init.user-data sections will be used.

Unless you set up a proper merging of cloud-init.user-data by hand.

1 Like
4

In the tutorial I put the cloud-init stuff in a separate profile that only has cloud-init configuration.
I think that’s the straightforward way to deal with cloud-init for Incus in several use-cases, considering that Incus does not parse that data at all.

Also, the naming of those cloud-init profiles, like cloud-something.

I omitted doing something with description: so that each profile has some nice description. That is, show incus profile set cloud-dev --property description="Run demo cloud-init commands"

As new use-cases appear, that would be useful to many users, I would like to deal with the merging/stacking details in a new tutorial.

1 Like
5

I use cloud-config files to configure all my instances, but I don’t use cloud-init. Instead, I have written a tool that applies cloud-config files to instances using the Incus or LXD API (like “incus exec” and “incus file push” do). That way I don’t need to use images that have cloud-init, and I can apply such cloud-config scripts to running instances at any time, not only when they are launched.

I have implemented 4 cloud-config modules: packages, write_files, users, runcmd. These are enough for my configuration needs.

For example, to create the file “/tmp/I_was_here” in instance C, use this test.cfg file:

#cloud-config
runcmd:
- touch /tmp/I_was_here

or this:

#cloud-config
runcmd:
- [ "touch", "/tmp/I_was_here2" ]

or this:

#cloud-config
write_files:
- path: /tmp/I_was_here3
  content: |
    Indeed.
    I was.

and run:

lxops cloudconfig -f test.cfg C

or

lxops cloudconfig -i C test.cfg

You can find the Go source of this tool at GitHub - melato/lxops
It does several other things related to launching and configuring containers, but you can just use the “cloudconfig” command.
@simos, You’ve tried to compile a predecessor of this tool before, and it did not compile. I’ve simplified the compilation process since then, and I added the cloudconfig command. I just compiled it in a new Debian container with Go:

git clone https://github.com/melato/lxops
cd lxops/impl/incus/main
go install lxops-incus.go

Then rename the resulting lxops-incus to lxops.

1 Like
6

I gave lxops a try and it worked easily.

  1. You are using images from the images: remote, apply changes to them on the fly, then launch the container.
  2. The commands are not equivalent to Incus which can cause some usage friction.
  3. The warning that lxops may mess up your system is harsh. The way I understand this, lxops should only deal with instances that were created with lxops but refuse to work with others.
  4. You should frame lxops as a companion utility that is used for special cases of image creation. lxops makes it easier to create a custom image, compared to a) creating one with distrobuilder (too much effort), b) cloud-init through profiles (here is your competition and requires some analysis).
  5. You do not mention only Incus, and it complicates the messaging. Can lxops detect that internally so that a single executable will work in both cases?
$ incus config get wordpress image.lxops

$ incus config set wordpress image.lxops=true
$ incus config get wordpress image.lxops
true
user@user-desktop:~$
7

Thank you for the review.

A clarification: lxops is not a replacement for distrobuilder. Distrobuilder creates images directly from external artifacts (releases and repositories of the various distributions). lxops does not do this. But incus and lxd can easily create images from other images: Launch a container from an image, install some packages or do any other configuration, take a snapshot, publish the snapshot as an image, export the image to a file, and import the image to another system. lxops automates parts of this procedure with a build file, which specifies the starting image (or instance snapshot), profiles, a list of cloud-config files, etc…

“lxops cloudconfig” works with any existing instance, not instances created by lxops.

I used to have my own home-grown file format for configuring instances. But I switched to the cloud-config format and semantics, which saved me from trying to maintain yet another configuration specification. I understand many people use Ansible instead, perhaps with some bootstrap configuration done with cloud-init. I don’t know Ansible. The cloud-config format is simple (at least the basic modules), easy to learn, straightforward, and can be implemented with almost no requirements on the instance (no python or ssh are required).

Other aspects of lxops can be discussed elsewhere (e.g. here).

8

Why doesn’t the default image server provide desktop VMs with cloud-init?

9

It has been a while since I looked. I think the images: remote at https://images.linuxcontainers.org only has a couple desktop images because they are expensive to build.

Which images are you most interested in? I think the last one I looked at was an Ubuntu image.

You could try building an image with cloud-init by adding desktop to all the places that have the cloud variant.

I am not sure if it would work, but I suspect the reason that it was not added is because most people would not need it for a desktop image.

10

Thanks, I’ll give it a try.