Cluster FS for replicated FC Storage

1

I’m curious if there is native support for traditional cluster filesystems like GFS2 or OCFS2.

I fully get why Ceph is essential in hyper-converged setups, but it’s impractical for synchronously replicated FC storage. Shared LVM only works with placement logic and heavy storage overcommit at the blockstorage. Big VGs are also risky because of metadata updates and well known risks with multiple PVEs in VGs.

On top of that, each snapshot reserves space equal to the VM’s full disk size. If I have ten 1 TB VMs on a datastore, keeping 3 snapshots each means reserving 30 TB.

Is there any real Cluster FS in the pipeline comparable to VMFS?

This would be essential and a good argument for enterprise customers with legacy FC storage layouts to migrate to incusOS.

I know that there would be also split-brain issues with GFS or OCFS but maybe someone knows a good and supported solution.

2

Not something that’s currently planned and indede GFS2/OCFS2 can be pretty tricky beasts to get right.

We do have some rather large environments running on FC with lvmcluster. You’re right that accounting, especially for snapshots is the biggest problem, but because arrays typically do internal thin provisioning, deduplication and compression, all you really need to do is export LUNs that are significantly larger than you’d normally do, then monitor your actual LUN disk usage through the SAN’s API rather than through Incus.

Incus correctly plumbs discard/trim operations, so only actually written bytes will be taking up space on the SAN drives.