Microceph ram requirements/recommendation?

for a small 3 node hyperconverged cluster how much ram would you recommend for microceph and lxd?
the workloads would be a few dns servers, prod/dev mysql and prod/dev websites and a email server.

Do we have any recommendations for this @stgraber ?

On the LXD side, you basically need 128MB of RAM for LXD itself and you need something that’s got at least two cores so that you don’t end up in scheduling craziness.

The rest is whatever your workloads themselves need. In your case, mysql is likely to be the biggest user, unless your mail server is something other than a basic postfix/exim type deal.

Now Ceph is a different beast. I believe the official recommendation is for 4GB of RAM per OSD (disk), so that’s a LOT of memory. Now on a system that isn’t particularly busy, you can go much lower than that. I’ve run Ceph clusters with I believe as low as 512MB per OSD, though note that performance may suffer.

This can be tweaked through osd_memory_target. Note that Ceph when dealing with a bunch of I/O or expensive operations like re-balancing or recovery traffic can also use a lot of CPUs. Ideally you’d budget almost a full CPU core per OSD.

Anyway, don’t be too intimidated by some of the higher recommended specs especially on the Ceph side. Going with something much smaller, like recycled disks, slow CPU and limited memory will not provide great performance but should still provide something functional and with suitable redundancy. I’d just recommend you do a good stress test to know what you’re dealing with :slight_smile:

1 Like