Hi all, a search didn’t show up much about this, so looking for your thoughts on distributed file systems.
We have geographically distributed Alpine hosts with btrfs housing LXC containers (but I guess LXC or LXD doesn’t make a significant difference here). We’d like to mount a distributed file system inside the container so that changes to files in one container are replicated to other containers. But I can’t find a good option, so looking for ideas.
So far we considered:
- GlusterFS: Not available for Alpine.
- Ceph: Memory requirements too high (there’s 1GB RAM or less on some hosts)
- MooseFS: Stores chunks and has a single master for metadata, which makes it less suitable for a geographically distributed system
- NFS/pNFS: Doesn’t store the file locally, so issues with read speeds in a geographically distributed setup
- lsync: We currently use this and designate one container as the ‘master’ but we want to be able to perform the change in an arbitrary container
Some more information:
- Must be usable in unprivileged containers
- Not looking to put the entire container on a distributed file system; just certain directories inside the container
- Whether everything happens inside the container or whether it happens on the host and is bind-mounted into LXC is not too important, although preference for the former
- In this particular case the workload is heavily skewed towards reads
Any ideas are very welcome, thanks.