Does anyone have any experience setting up the lxd storage backend on a minIO distributed setup?
I am looking to setup something thats easier to maintain than Ceph and i found MinIO which is a really nice distributed object storage solution. The nice thing about minIO is that it works on any type of file system (ext4,xfs,etc.).
My initial thoughts were to create a distributed object storage (because i am looking for very fast I/O performance), format it and then use it as the storage backend for my LXD installation, I could do the format as LVM and then just use all of my data available , but for some reason when i am asked to provide the path to the block device it simply doesn’t want to work. Is there something I am doing wrong ? maybe i should use btrfs ? or zfs (although i try to stay away from zfs)?
Right now i have 3 nodes that each carry 8 Micron 7300 MAX U.2 drives . Each node has a Mellanox ConnectX6-DX card (2x100Gb qfsp+) and they are connected to a Mellanox SN2700 Switch (32 x 100Gb ports).
Again the reason I am staying away from ceph is just for simplicity. I may have to switch over to it from what i have been reading (and also because i just cant find much on using MinIO with LXD, although it sells itself as the perfect Object Storage solution for containers).