Unable to create multiple container from an image using Terraform in a 7 node LXD Cluster

LXD
1

Using terraform, In modules main .tf is as below:

variable “name” {}
variable “cluster_name” {}
variable “image” {}
variable “cpu” { default = 1 }
variable “memory” { default = “1GB” }

resource “lxd_container” “conatiner” {
config = {}
ephemeral = false
limits = {
“memory” = “128MB”
“cpu” = 1
}
config = {
“boot.autostart” = true
“security.privileged” = true
}
device {
name = “eth0”
type = “nic”

properties = {
  nictype = "bridged"
  parent  = "virbr0"
}

image = “${lxd_cached_image.centos7.fingerprint}”
wait_for_network = false
}
resource “lxd_cached_image” “centos7” {
source_remote = “images”
source_image = “centos/7”
copy_aliases = true
}

using the above module creating 7 container, few containers start getting created but later fails with the below error messages:

module.a.lxd_container.container[0]: Creating…
module.e.lxd_container.container[0]: Creating…
module.x.lxd_container.container[0]: Creating…
module.d.lxd_container.container[0]: Creating…
module.t.lxd_container.container[0]: Creating…
module.y.lxd_container.container[0]: Creating…
module.h.lxd_container.container[0]: Creating…
module.d.lxd_cached_image.centos7: Still creating… [40s elapsed]
module.y.lxd_cached_image.centos7: Still creating… [40s elapsed]
module.a.lxd_cached_image.centos7: Still creating… [40s elapsed]
module.x.lxd_cached_image.centos7: Still creating… [40s elapsed]
module.y.lxd_cached_image.centos7: Creation complete after 47s [id=cluster/7c5350a25b04b85b3e05aea6b1f83a95ab88]
module.d.lxd_container.container[0]: Creating…
module.d.lxd_container.container[0]: Still creating… [10s elapsed]
module.d.lxd_container.container[0]: Creation complete after 19s [id=testing-d-1]
Error: Failed remote image download: Image sync between nodes: UNIQUE constraint failed: images_nodes.image_id, images_nodes.node_id

on modules/main.tf line 35, in resource “lxd_cached_image” “centos7”:
35: resource “lxd_cached_image” “centos7” {

Error: Failed remote image download: Image sync between nodes: UNIQUE constraint failed: images_nodes.image_id, images_nodes.node_id

on modules/main.tf line 35, in resource “lxd_cached_image” “centos7”:
35: resource “lxd_cached_image” “centos7” {

Error: Failed remote image download: Image sync between nodes: UNIQUE constraint failed: images_nodes.image_id, images_nodes.node_id

on modules/main.tf line 35, in resource “lxd_cached_image” “centos7”:
35: resource “lxd_cached_image” “centos7” {

Error: Failed remote image download: Image sync between nodes: UNIQUE constraint failed: images_nodes.image_id, images_nodes.node_id

on modules/main.tf line 35, in resource “lxd_cached_image” “centos7”:
35: resource “lxd_cached_image” “centos7” {

Error: Failed remote image download: Image sync between nodes: UNIQUE constraint failed: images_nodes.image_id, images_nodes.node_id

on modules/main.tf line 35, in resource “lxd_cached_image” “centos7”:
35: resource “lxd_cached_image” “centos7” {

Error: Failed remote image download: Image sync between nodes: UNIQUE constraint failed: images_nodes.image_id, images_nodes.node_id

on modules/main.tf line 35, in resource “lxd_cached_image” “centos7”:
35: resource “lxd_cached_image” “centos7” {

Creating one container works fine.
Im not sure If im wrong with my lxd_cached_image configurations, i just want to create 7 container from the same image using terraform. Kindly advice. Thanks

2

This is odd given the locking we have in place to prevent an image to be imported multiple times in parallel so I’m not sure exactly what’s causing the DB error there.

It sounds like it’s an issue in the logic to transfer images within the cluster but I would still expect this to be under our normal image download lock.

@freeekanayaka thoughts?

3

Does it eventually resolve? Meaning if you run it again does it still show those errors?

4

This isn’t related to the problem but you had a misspelled:

resource “lxd_container” “conatiner” {

5

No it doesnt resolve it throws the same error, is it that we can’t pull the same image more than once ? Coz the above main.tf is executed 7 times, so the centos7 image will be pulled 7 times, so is it that error ?
Later i removed the resource “lxd_cached_image” code and hard coded the image node in variable “image” {default=centos7} then it worked fine creating 7 containers, but i had to pull and keep the centos7 image.

6

@bmullan that was an accident while copying the code, to keep it simple i replaced the name of the resource, i crossed checked too !