LXD storage setting of cluster questions

I have a one big storage server, but other nodes don’t have similar size of disk space

So I want to assign a storage role to that specific node, make containers on any

node of cluster can access to it. But storage setup of cluster requires certain object to

exist on every node of cluster to be defined, not existing on only one place. How can I achieve this?

Second question is, can I make containers to share a directory? like NFS, but without

extra privileges to container

Current LXD clustering requires all nodes to provide the same set of storage pools so that they are available in the cluster.

You can’t have one storage node have one more storage pool than the others. That’s a current requirement of the API which requires everything be consistent. You could create another storage pool in the cluster and give it a very small loop partition on all nodes except the one that’s got that big disk and then place the containers that need the extra space on that node.

Unless you’re using CEPH, containers are always attached to local storage pools, you can’t have a container on NODE1 use the storage pool that’s on NODE2, even for custom volumes.

If you don’t mind reading a lot of documentation, I guess you could technically setup a small CEPH cluster in your case. Effectively using your nodes as both compute and storage, providing one CEPH pool which uses the smaller disks of all your nodes and one second CEPH pool that uses that extra disk. You can then attach both pools to LXD and place containers whatever way you want as at that point they wouldn’t be tied to local storage anymore.

2 Likes

Thank you so much

So for LXD cluster requires every nodes to have configuration of specific storage pool,

I need to setup storage pool for every node, but that pool is using CEPH storage which contains

little size disks for every other nodes in the cluster but using entire disks of storage node

This way I can utilize my independent network attached storage node did I get it right?