Run commands at first container start (something like runcmd but with non-cloud-init containers)

Incus
1

I would like to embed commands in profiles that should be run once container is first started. runcmd is like this but there’s dependency on cloud-init. Is there a way to create profiles with these sorts of instructions in it? If not, is it planned to add this feature? Or just no, one should either use cloud-init enabled containers or automate this outside of Incus profiles.

2

It’s not planned and not really something we want to deal with in Incus proper.
cloud-init works pretty well for that.

Alternatively if using the Terraform provider, there is support for pushing a file as part of the instance creation which would then let you put a boot script in place to perform a one-time action.

2 Likes
3

This is also how im currently solving it. Terraform to create containers from official images, and I didn’t want to deal with cloud-init.
I had some issues with incus exec and piping output to authorized_keys etc. Which eventually led me to the much simpler and easier to maintain incus file push ... setup.sh variant. I wish we had a very lightweight cloud-init alternative so I can set up openssh, user and ssh key. And then move to ansible for the rest.

1 Like
4

You can build your own images. That is what I do.

But what is wrong with cloud-init? It is ubiquitous across the industry.

5

Nothing’s wrong with it but in some use cases it’s bloat.

6

Nothing against cloud-init, but I would need to swap the images to the cloud-variant, and then put yaml in my Terraform resource definitions. And all I need is for [user + group + ssh key] to be set up. cloud-init is just a big gun for such a simple task.

As for alternatives:

I use packer for a few edge cases involving edge (hehe) hardware. It has the same problem distrobuilder does: I either need to host the images myself, or build them when I need them. It is just not as practical in my use case, where the infra is very heterogeneous and spread across Europe.
TL DR: my current workflow is the following:

  1. run incus playbook on bare metal host
  2. terraform apply to create containers and set up forwards
  3. provisioner "local-exec" { command = "incus file push .... incus exec"
  4. user and ssh public key set up
  5. run whatever ansible playbook I need depending on the role

So on my workstation I only need three tools: incus-client, ansible and terraform.
If anyone has another idea, let me know, im open to constructive criticism.

1 Like
7

Do you always use the same Linux distro for your deployments? I try to standardize mine with Debian Bookworm. I used to use Ubuntu LTS. Are you always configuring the same user, group and ssh keys? I generally do. I like to bootstrap instances with a non-root user.

You might be able to move your cloud-init scripts to your Incus profiles. This keeps them out of your Terraform/ToFu configurations and still fits into the three tools you have on your workstation.

It sounds like you have a pretty light weight solution. :+1:

Using the cloud variant of Debian is not much bloat. In time, you might even use some of the other features offered by cloud-init. That is where it really pays off.

Don’t get me wrong. A simple bash script might be the perfect solution. I don’t always use cloud-init.

8

Yeah im also on Debian, and its usually the same user/group and ssh key for the access. At least that is the case for anything running application containers, so most of the LXC instances.

Actually this is an excellent idea. I will give it a try over the holidays.

Now im starting to think about replacing ansible at least for incus/lxc. For example, Python is not even present in the usual lxc images, so for the initial access I need to install it. By avoiding cloud-init I end up pulling some actual bloat. And for app containers I only need the runtime, and basic user setup. I don’t really profit from existing playbooks at all.

1 Like