Create a cluster bridge network with terraform

1

Ok, so I have a test cluster: 3 nodes, one drive (btrfs), one NIC each. No ceph or ovn, just Incus.

I’m trying to use TerraformOpenTofu to create projects and resources within it, and I’m struggling to create a bridge network. (My projects so far have features.networks set to false.)

terraform {
  required_providers {
    incus = {
      source  = "lxc/incus"
      version = "~> 1"
    }
  }
}

data "incus_cluster" "cluster" {
}

# Generate a random /24 subnet
module "netaddr4" {
  source  = "./randnet"
  subnet  = "172.16.0.0/12"
  netbits = 12
}

locals {
  # FIXME: Pull dynamically
  node_interfaces = {
    # Don't worry about it
  }
  defaultnet = {
    "ipv4.address" = module.netaddr4.incus_addr # eg 172.27.96.1/24
    "ipv4.nat"     = "true"
  }
}

variable "project" {
  description = "ID of current project"
  type        = string
}

variable "name" {
  description = "Name of network"
  type        = string
}

variable "config" {
  description = "Network configuration"
  type        = map(string)
  default     = {}
}


resource "incus_network" "net_node" {
  for_each = data.incus_cluster.cluster.members
  project  = var.project
  name     = "${var.project}_${var.name}"
  target   = each.key
  config = merge(var.config, {
    parent = local.node_interfaces[each.key]
  })
  type = "bridge"
  # FIXME: Destroy if net fails to create
}

resource "incus_network" "net" {
  project = var.project
  depends_on = [
    incus_network.net_node,
  ]

  name   = "${var.project}_${var.name}"
  type   = "bridge"
  config = merge(local.defaultnet, var.config)
}

output "id" {
  description = "Identifier to use elsewhere"
  value       = var.name
  depends_on  = [incus_network.net]
}

I think this does the right thing, where it’ll create the network on each node, and then create the cluster-wide network.

But, two things:

  • If I specify the network config in the node-specific network (not in the example), I get Config key "ipv4.address" may not be used as member-specific key, which is different from the cluster example
  • If I run the example as-is, I get Invalid option for network "forge2_default" option "parent" on the cluster-wide network
2

Actually, I’m starting to think that OpenTofu Registry is just wrong.

Comparing it to How to configure networks for a cluster - Incus documentation and How to create a network - Incus documentation , the suggested values are just red herring levels of wrong.

3

@maveonair though @breml may have some pointers on this one too as we’re doing something pretty similar in Operations Center I believe.