Opinions on the LXD Terraform provider?

My organization is preparing for a migration of several resources to Azure. Some colleagues are eager to use Terraform.

I have a lot of Terraform experience, but in my opinion it’s best to use it for VPC networking, subnets, and static resources, and maybe virtual machines.

That said, LXD instances (so-called “system containers”) do blur the lines between VM workload and container. And I want to continue to encourage LXD adoption.

I tried the LXD Terraform provider exactly once a couple of months ago, without much success. But this was on my local LAN, and I didn’t try too hard to be honest :). Has anyone used the LXD Terraform provider a lot on a cloud? Can you give your impressions of whether/how I should pursue it?

Link to terraform-provider-lxd on GitHub.

1 Like

Searching Terraform Registry for “project” shows 0 results - this isn’t a great start. I have no personal XP but that is something worth considering.

Okay TBF the repo linked does return some results, but it links to https://github.com/terraform-lxd/terraform-provider-lxd/pull/238 which shows it wasn’t merged.

LXD is an ever evolving landscape, i’d never discourage its use, but without projects IMO its worth considering if your provider is ever going to “keep up”.

1 Like

That’s an excellent point. Perhaps not a dealbreaker, though. We don’t use anything besides the default project right now. And we don’t use clustering. So we might be lucky. Also, we’re all Go devs, so perhaps we can contribute this feature.

I’m more concerned with the basic mechanics of managing a container’s lifecycle. Terraform providers are notoriously buggy. Even quasi-official ones like AWS have frustrated me in the past. A big cloud provider’s API is more of a moving target than LXD, though.

Update. This reminds me. I need to add projects support to the Bolt provider.

2 Likes

Then you might be alright, its so environment dependent - its hard to know :smile:

Sounds like your already ready to rebuttal against Terraform.

Seems like every provider has this problem, very annoying.

Ill stop commenting with the hope the LXD team themselves comment on the preferred provider.

Sounds like your already ready to rebuttal against Terraform.

Believe me, I have major criticisms of Terraform as a tool. I am a programmer at heart, but I’ve had full-time ops roles. There’s simply no denying it’s a widely-used tool. I think Terraform gets you into trouble if you use it for the following:

  • Any kind of active lifecycle management beyond create/destroy
  • Anything involving the internals of a Linux system (use Ansible/Chef/Puppet for that!)
  • DNS (kind of scary, imo)

But LXD presents an interesting possibility. Containers + profiles are reasonably declarative. And if you have a private image server solution, you could imagine launching large, beefy servers with lots of containers in Terraform.

The trouble is “Day 2”. How do you evolve the system? The UX of this is provider dependent. If the provider is reliable with nice config, it can be good.

1 Like

Oh please, its a Saturday, dont make think about Active Directory / Service Discovery :laughing:!

The only “problem” I see here is storage, we should be able to compose LXD instances Ubuntu 18.04 or Ubuntu 22.04 its the same.

cloud-config gets very close and probably works (but requires extra dev steps to apply to a profile then deploy & I understand the worry, at work I haven’t mitigated this problem)

Completely agree, its hard to build a system that isn’t competing with SNAP - many have tried and failed :frowning_face:

Snap-LXD-Terraform by smashse

Google translation of the above w/parts that are in Portuguese to English

1 Like

Isn’t Hashicorp the orginator of terraform-provider-lxd.

Looking at the Github repository Releases… its being kept up to date:

May 8, 2022:
Releases · terraform-lxd/terraform-provider-lxd · GitHub

Mentioning Hashicorp… they appear to do a lot of work with their Tools/Applications and LXD:

hashicorp lxd - Google Search

Here is a Youtube video:

We will setup Terraform to use LXD provider for faster container launching and infrastructure creation

ComputingforGeeks has another installation/config guide for Terraform/LXD on Ubuntu:

https://computingforgeeks.com/create-lxc-containers-using-terraform/

1 Like

I think it’s a good idea.

Terraform sure has its share of bugs, but people often blame Terraform when they make random upgrades and stuff breaks. Don’t make random upgrades and it won’t break! The other case is when the cloud provider changes stuff on their side - that’s more difficult to work around, but I’d argue I’d try and provision my infra with a tool that works in any cloud rather than give up and start using hyperscaler’s own tools (fewer problems, but more lock-in).

There’s an LXD provider for Nomad. Not sure if it works, but maybe it doesn’t require a lot of fixing to get it to work - https://lxd-nomad.readthedocs.io. I mention Nomad because if you have DNS and service discovery problems, Nomad and Consul may be able to solve that.

It’s OK to use Ansible after Terraform deploy, but you can’t use it in a way that ingores the fact that certain settings mustn’t be changed from Ansible if you plan to have the resources also managed from Terraform later on.

I wouldn’t personally recommend using terraform-provider-lxd in professional setting without being willing to contribute some code. There are some rough edges with using it. As mentioned it’s missing projects, but I’ve also run into some option guards that need updating around devices. The provider also exits with error if a container in the state was deleted external to terraform.


─❯ lxc delete -f build1
─❯ terraform apply -target lxd_container.build1
╷
│ Error: Instance not found
│ 
│   with lxd_container.build1,
│   on config.tf.json line 55, in resource.lxd_container.build1:
│   55:       },
│ 
╵
1 Like

Project support for the TF provider should be coming soon.

3 Likes