Newbie question: what's the lxc version of a Dockerfile?

I normally use Docker, but I’m working on a gig that requires lxc. I’m used to configuring docker images with a Dockerfile: start with this image, run this command, copy that file, etc. It does simple dependency management so rebuilds go quickly.

What are my options for doing nice repeatable builds of lxc containers? Should I just start the image and run a chef/ansible/puppet recipe on it? Are there some examples, best-practices I should check out?

Cloud-init is what your looking for

Write your yaml to a file then a command like

lxc profile set PROFILE_NAME user.user-data - < path/to/file

Then lxc launch ubuntu: c1 -p PROFILE_NAME

Alternativly my web app https://github.com/turtle0x1/LxdMosaic has support for

  • creating and storing revisions of these files
  • storing a target image
  • storing ENV variables
  • deploying it all to a host
2 Likes

This is something that is missing from LXD. I think third party tools like the ones you mention are the way to go, for now.

Launching and configuring a container with a single command + a configuration script would be nice. I have done something like it with my own tools, but I’m not happy with it yet. The general idea is to “push” configuration scripts, ssh keys, and other files from the host to the container and run them or install them in the container.

I reject cloud-init because it does not work with alpine and other images, nor would I want to bloat my small alpine images with cloud-init. I also find the clould-init configuration file too monolithic and different from everything else linux. I only use it for very simple things, like preventing the “ubuntu” user from being created on ubuntu images.

If you launch several containers with the same configuration, you can configure one “template” container and launch others by copying (cloning) the template container (or its snapshot). But there is still the problem of scripting the configuration of the template container.

It becomes even more complicated if you want to also configure devices with each container you launch. My goal is to have all my data in separate disk devices, and be able to swap in a new version of the container (the root filesystem) when I need to update any software. It requires a lot of housekeeping and it complicates taking snapshots.

I haven’t used Docker much, but I believe it is designed to do this. They say that LXD is for OS containers while Docker is for application containers, but in a production environment, you use OS containers to run applications, so I would like to see a convergence of the two approaches.

@votsalo

So I reject your rejection of cloud-init :laughing:

Given the following scenario:

  • Script to setup webhost
  • Additional drives & devices

You could do the following

Profile A - Setup Webhost

Create a file containing

#cloud-config

# Apply updates using apt
package_update: true
package_upgrade: true

# Install packages
packages:
 - apache2
 - php-common
 - php7.2
 - libapache2-mod-php7.2

runcmd:
 - timedatectl set-timezone Europe/London
 - rm -rf /etc/apache2/sites-available/default-ssl.conf
 - DO_A_BUNCH_OF_OTHER STUFF

then import this into a profile lxc profile PROFILE_NAME set user.user-data - < FILE_LOCATION

How this is less “monolithic and different from everything else linux.” than a DockerFile which may contain lines like the one below is beyond me

RUN apt-get update -y && apt-get install -y - apache2 php-common php7.2 libapache2-mod-php7.2
RUN timedatectl set-timezone Europe/London &&  rm -rf /etc/apache2/sites-available/default-ssl.conf

The lines can also get harder to read (if people don’t properly line break them) as you try to reduce the number of “layers” in your docker image

Now if your going to argue “id just make my own docker image” then just go and make your own LXD image and optionally don’t bother with cloud-init.

Profile B - Disks & Devices

Create as many profiles as you need with different device & disk declarations (I’m not gonna repeat the documentation here)

You could even include the extra disk definitions in the same profile as the cloud-config instructions (as all containers of this “type” need the same disks right?)

Creating a Container

Well now all our provisioning info is in profiles its as simple as

lxc launch ubuntu: c1 -p PROFILE_WITH_CLOUD_INIT -p PROFILE/S_WITH_DISKS_OR_DEVICES

VS Docker

Compared with docker-compose this is not as simple no, you would just run docker-composer and it would locate the file/s and run it.

But docker still requires a docker-composer.yml (disks & devices) & DockerFile (user.user-data) the only difference between LXD & Docker is that LXD stores this info in a database instead of flat files!

Lack Cloud-init Images

This used to be true, but there was a lot of work to bundle in cloud-init to the images

A lot of the provided images now have cloud-init variants and alpine was / is one of them (3.12 support coming soon?)

Thanks @turtle0x1 this is super helpful. I haven’t seen such a simple explanation of a cloud-init config file. Something like this should be in the docs!

Hi Gary,

I would advise you use Packer. It is a well done open-source project that is made to generate images for a lot of platforms. It has lots of builders to generate images (Docker, lxc/lxd, AMIs, etc) and multiple provisioners to configure those. You can for instance run the LXC builder to create an LXC container and run the shell provisioner to configure your image. Or change the provisioner to Ansible to do the same.

In your case, behind the scenes, it will indeed start an lxc container and run a shell script (in the case of the shell provisioner) or Ansible. This makes it easy to switch from one type of runtime to another.

Dockerfile parallel in lxd/lxc are distrobuilder yaml configs. https://github.com/lxc/distrobuilder

1 Like

There is no such thing as a Dockerfile for lxc/lxd.

Yes the Answer with cloud-init is correct, however the image has to be configured to be compatible with cloud-init. For RLX we use custom images so we need to make the lxd images ready
for for cloud-init you can find the code here:

https://stash.intranet.roche.com/stash/projects/RLX/repos/rlx-deploy/browse/deploy/defaults.lxd.inc?at=cloud-init#6

If you use the ubuntu: images you are fine with cloud-init, if you use any other image you can’t use cloud-init.

What works quite well is to use ansible, simply create a playbook and apply it to any container (yes the comma at the end is needed):

ansible-playbook playbook.yml -c lxd -i CONTAINER_NAME,