Homelab storage and network questions for a cluster

LXD
, , ,
#1

I am preparing to setup a homelab cluster with three nodes. I can’t use MicroCloud, as each node only has a 1TB NVMe disk, so I am going to use local storage on each. Is using directory my only (or best) choice?

I also want to assign different subnets for VM/containers instead of letting the init on auto. I want to use the 100.100.0.0 one; will it work if when setting up each node in the cluster I assign a different subnet to each? For example:

  • node01: 100.100.10.0/24
  • node02: 100.100.20.0/24
  • node03: 100.100.30.0/24

Or do all nodes in a cluster must be on the same subnet, that is, use 100.100.0.0/16 when setting up network in each of them?

#2

MicroCloud (by way of LXD cluser) uses the fan network by default:

This gives the overlay network as a whole a specific subnet and then uses smaller subnets within that larger subnet for each cluster member. Then the instances on that cluster member get an IP within that smaller subnet.

You can also choose to create additional networks, however in LXD cluster those networks are built on all cluster members. So say if you created a bridge network called foo with a subnet of 10.100.10.0/24 it would create a managed bridge interface with the same IP on every cluster member.

You wouldn’t have to connect every instance to it though.

OVN networks may also be an option for you.

But you’ve not explained why you are trying to do this, so I think I would be able to give a better answer if I understood the why of what you’re trying to do.