Incus Cluster -> Kubernetes Clusters Advice

Incus
1

We have three physical nodes. We would like to create an incus cluster with these physical servers that runs a kubernetes cluster on top of it.

Technology Considerations:

  • We will be running eBPF based Cilium CNI networking on the cluster, and
  • ZFS on the underlying physical hosts

Are there any how to guides or specific advise on potential problems that we may encounter?

1 Like
2

Iā€™m looking to do the same, right now all my Kubernetes clusters are hosted on Harvester, want to play a bit with Incus but the Storage is stopping me.

3

The area of most concern for us is the networking and working out how OVN/OVS will handle kubernetes cluster traffic managed by Cilium. In particular the BGP peering starts to look rather tricky.

4

So would the idea be to develop kubernetes incus shims that allow kubelets and kube-proxy to hook into CNI and CSI incus provisioning services?