I have four servers interconnected with direct 10-Gb fiber links, forming a peer-to-peer backend network. The hosts exchange routes using BIRD, and each server owns a portion of the 10.10.10.x network.
I recently installed Incus and created a cluster across these four machines. On top of the routed backend, I configured OVN networking. The cluster itself works fine and the containers/VMs can communicate internally.
However, containers and VMs cannot reach the internet.
My goal is for all Incus traffic (cluster communication and container networking) to stay on the 10-Gb backend network while still allowing outbound internet access.
Current design:
-
4 hosts connected in a peer-to-peer 10-Gb fiber mesh
-
Routing handled by BIRD
-
Each host owns part of the 10.10.10.x network
-
Incus cluster running across the hosts
-
OVN configured over the routed backend network
Questions:
-
Is this a reasonable architecture for an Incus cluster?
-
What is the recommended way to provide internet access for containers/VMs in this setup?
-
Should I be using OVN in this case, or would a simpler routed bridge approach be better?
I’d appreciate any advice from people running similar multi-host Incus setups. Maybe I’m over-engineering this which is likely.