Puppet Bolt LXD transport (WIP)

I am working on an LXD transport plugin for the Puppet Bolt config management tool.

Draft PR is here https://github.com/puppetlabs/bolt/pull/2335

Warning: I am not a Ruby programmer. I merely am a fan of Puppet Bolt and LXD, and I think the concept of “system containers” benefits from a configuration management solution.

What a Bolt transport plugin allows is exposing the full range of bolt features in a way that feels like orchestrating VMs over SSH. Features that bolt provides:

  1. remote command execution bolt command run -t $TARGET 'sudo /tmp/foo.sh
  2. upload and download files
  3. install packages
  4. orchestrate all of the above in ‘plans’ composed of granular ‘tasks’

The reason I think bolt is a good fit for LXD containers:

  • Named “targets” in bolt map nicely to instances in lxd.
  • The UID/GID configurations for lxc file push, and lxc exec, have analogues in bolt file upload and bolt command run

This is going to take me a while, because I am bad at Ruby. But if you use Puppet Bolt, follow the PR. I will try to remember to post an update here when it’s ready.

2 Likes

The associated github issue: https://github.com/puppetlabs/bolt/issues/2311

1 Like

Continuing work on this. Some Puppet Labs employees say they’ll give this a review in the new year.

1 Like

If you are adventurous, and have some familiarity with ruby + bundler, and bolt, you can try this.

git clone git@github.com:dontlaugh/bolt.git
cd bolt
git checkout 2311_lxd_transport
bundle install
bundle binstubs bolt --path=~/bin  # if ~/bin is on your PATH

switch to your desired remote. (I will allow targeting multiple remotes from inventory at some point).

Example bolt inventory with some container targets, configured with lxd transport, under a single group

# my remote is called "infra"
targets:
  # containers
  - name: cli
    uri: infra
  - name: edge
    uri: infra
  - name: gm-control
    uri: infra
  - name: gm-control-api
    uri: infra
  - name: svc
    uri: infra


groups:
  - name: containers
    config:
      transport: lxd
    targets:
      - cli
      - edge
      - gm-control
      - gm-control-api
      - svc

example command:

$ bolt command run -t containers 'uptime'
Finished on svc:
  STDOUT:
     19:54:05 up 4 days, 18:35,  0 users,  load average: 0.01, 0.05, 0.00
Finished on gm-control:
  STDOUT:
     19:54:05 up 4 days, 18:36,  0 users,  load average: 0.01, 0.05, 0.00
Finished on gm-control-api:
  STDOUT:
     19:54:06 up 4 days, 18:36,  0 users,  load average: 0.00, 0.00, 0.00
Finished on cli:
  STDOUT:
     19:54:06 up 4 days, 18:35,  0 users,  load average: 0.00, 0.00, 0.00
Finished on edge:
  STDOUT:
     19:54:06 up 4 days, 18:35,  0 users,  load average: 0.00, 0.00, 0.00
Successful on 5 targets: cli,edge,gm-control,gm-control-api,svc
Ran on 5 targets in 2.63 sec

Unfortunately, “uri” right now doesn’t really configure anything. I intend to make it such that it can address the container like this <remote>:<container>, in the typical lxc fashion. This should allow setting arbitrary target names in bolt that do not necessarily have to match the lxd container name proper.

A good overview of puppet bolt project structure is here https://www.youtube.com/watch?v=F5BRRgFE2fo

1 Like

This feature is getting closer. I was hacking on and off for a few weeks, and then Puppet Bolt dropped a major 3.0 release.

One of their developers has been kind enough to squash all my work and open a new PR against the new release. I was falling behind.

There is still some work to do, but I’m feeling good about it

1 Like

Now other Puppet labs employees are looking at the PR. It seems likely to be merged.

After it is merged, it will take a little while to land in a proper release I think. I will keep this thread updated.

Also: not every feature of the lxc command line is implemented. We’re merging an MVP

The PR has been approved :slight_smile:

List of features tested

(commment near the bottom)

Nice!

So, the initial feature that wraps the lxc command line tool was merged, but some essential work remains: supporting remotes.

I mentioned to Lucy and Tom at Puppet Labs that this is a real killer feature for LXD. I came to check on it today, and they’re already working on it.

Yeah, remotes and projects are two concepts I’ve had to push for a bit in the Ansible integration, it’s there now and definitely makes it much easier to manage larger deployments.

The experimental feature docs (without remotes) for the next release:

PR for remotes is open

The remote features got merged. They are still unreleased, but they’ve been merged to the main branch of bolt.

Here is a short terminal screencast demonstrating running commands in containers across two remotes.

https://asciinema.org/a/400047

I am working on a demonstration project with the bleeding-edge bolt here:

It runs two vagrant VMs, each one with a separate LXD remote. I’ll try to build out something somewhat sophisticated, to boost my bolt skills, and show the LXD community the full possibilities. I’ll likely open a new forum post with the title Puppet Bolt Demo or something.

The LXD transport, with remote features, is released in 3.4.0

This is marked as experimental, but I don’t think there will be radical changes in config required.

I’ll make a new post at some point to show some usage examples.