How to autoprovision additional disks using profiles or Ansible in a cluster?

Hi

I’m trying to deploy containers/VMs with additional disks being created in different pools, then attached and mounted at the time of deployment. Is this possible?

The long story…

We split some application servers this way:

disk0 = OS, binaries, configs, logs…
disk1 = translogs
disk2 = databases

I’m expecting to be able for these additional disks to be automatically created on the fly on the host where LXD chooses to deploy the instance to. If set it in a profile, LXD won’t let us save the profile without specifying the “source” volume, which wont exist yet and the destination host hasn’t been determined yet:

Config parsing error: Device validation failed for “translogs_disk1”: Disk entry is missing the required “source” property`

The profile or Ansible LXD_Container playbook for a Linux container:

devices:
disk1:
path: “/mnt-appsrv/translogs_disk1”
pool: “lxd_local_disk1”
size: 10GB
type: disk
disk2:
path: “/mnt-appsrv/databases_disk2”
pool: “lxd_local_disk2”
size: 20GB
type: disk

There will be Windows VMs as well.

Host and storage pool e.g.:

lxdhost1

disk0 = SAS RAID10, primary OS files, app binaries
disk1 = NVMe, data
disk2 = NVMe, data

lxdhost2

disk0 = SAS RAID10, primary OS files, app binaries
disk1 = NVMe, data
disk2 = NVMe, data

lxdhost3

disk0 = SAS RAID10, primary OS files, app binaries
disk1 = SAS RAID10, data
disk2 = SAS RAID10, data

LXD Cluster Storage Pools

lxd_local_disk0 = lxdhost1.disk0, lxdhost2.disk0, lxdhost3.disk0
lxd_local_disk1 = lxdhost1.disk1, lxdhost2.disk1, lxdhost3.disk1
lxd_local_disk2 = lxdhost1.disk2, lxdhost2.disk2, lxdhost3.disk2

Thanks

So what this is looking like to me for the above scenario of additional disks in instances (containers & virtual-machines), is that these additional disks are best to be on storage managed externally to LXD and then their backends referenced in source.

Configuration of this seems to be:

  • figure out whether to use at least 1 LXD managed storage pool for the instances, or go fully external managed storage… probably on ZFS
  • figure out how to reference source dymanically
  • prepare additional storage using external storage management tools
  • potentially have to configure cgroup & apparmor permissions
  • stipulate --target for all instances
  • get all of the above to work in Ansible

References

Instance configuration > device > disks

https://github.com/lxc/lxd/blob/master/doc/instances.md#type-disk
size is only supported for the root disk
pool is not required
source is required and can refer to one of several storage backends

Device cgroups & apparmor in LXC article:

Permission denied for mount filesystem - #6 by JeToJedno

Anything else or other pointers that you can think of?

Thanks