Hello all. I need a little help figuring out the best course to take
I have acquired a Dell R820 with 10 X 1GB drives to build out a new OpenStack AIO lab. My goal is to build out the containers manually as I have never been able to get JUJU to provision my unique environment properly. I plan to build out an LXD cluster of containers for CEPH first before I start building the OpenStack containers. I intend on using BTRFS, Bluestore, and erasure coding when setting up the CEPH containers. I plan on using 1 harddrive for the OS and the rest for LXD/OpenStack.
I need help mapping out the storage to maximize it’s use. To that end I have a number of options and I need a little more understanding of how much to put into the LXD default storage pool.
Option #1 - Take the 9 remaining harddrives and pre-create a BTRFS Storage pool and use it as the default storage pool when setting up LXD. This seems to maximize my storage pool but when creating ceph-osd’s I will need to mount block devices into the container in order to set up CEPH. I believe I could use a subvolume for each ceph-osd but I don’t know if the mount into the container will be what CEPH needs when using the CEPH tooling to build out the ceph-osds.
Option #2 - Mount each drive into a ceph-osd directly. This seems like the logical choice but it brings up a storage usage issue. How much space do I need for LXD’s default storage pool? If I am using CEPH for Glance, Cinder, etc., what actually uses the default storage pool? I assume the top level OpenStack containers use it so the config, the database, etc., but I don’t think that would require a very large footprint. If that is true, I could just take the 1 drive I plan to use for the host OS and split it into two partitions and use one for the default storage pool. That still leaves me with 9TB of harddrive space mounted into 9 ceph-osds.
Any insight would be appreciated as well as explanation of what the default storage pool is used for in an Openstack implementation and how much space it really needs to have allocated.