I really like using LXC/LXD, but i can’t find a solution for my setups.
The question in short: Is there a best practice to do configuration that mulitple containers share, and do additional specific config for each container?
The long story: I gonna run a couple of LXD containers with different tasks. There is a basic set of customizations, that every container has, and then for each task there are different package installations and configurations. Now I’m looking for a way, to freeze this in infrastructure code.
Initially I thougt about using distrobuilder to build a complete image for every task. That would mean, that I have to duplicate the whole image configuration and the basic steps in each *.yaml configuration file. Then I thought about cloud-init, but I couldn’t find a way to do includes there in a LXD container either. Next idea was to set the basic customizations as cloud-config in the profile in user.user-data, and for each container apply a cloud-config in user.user-data in the container config. but the container config overrides the one from the profile.
I could do the basic steps by building a custom image, use that image for every container and do the task-specific steps in cloud-init. That would mean, that I use diferent approaches for preparing the containers (distrobuilder yaml for the basic image, cloud-init config for the specific configuration), that also kinda smells.
Another solution would be to skip the configuration on that layer entirely and just do it in Ansible. With Ansible comes the need to do something after
lxc launch foo: bar, but I want the images to be complete and self contained, just start it and be done for starters (nope, not gonna use docker :D).
Any thoughts about this?