I am struggling to finalize my 3-node bare metal Incus cluster in the following topology (this is how it has been provided to me from my hosting service provider):
Incus clustering works like a charm between the 3 nodes communicating to each other through the interfaces using the private IP range. Also OVN cluster works fine through the same private IPs.
I followed the documentation here and also looked at this post.
I ended up with a bridge that I assigned 10.0.101.1/24 IP and an OVN network with 10.0.102.1/24. VMs deployed in the OVN network are able to communicate with each other and to the Internet, seemingly without any issue, but only through a single egress interface.
The odd behaviour I have is that the VMs are not getting registered in the DNS so I can only use their IP addresses to reach them. The DNS service seems to be on the bridge interface. Am I missing something? If I use a standard bridge interface (non-OVN), the machines are getting properly registered and I can leverage DNS, which is convenient.
What I would also like to do is to be able to leverage each cluster node direct Internet access instead of having all VMs going through a single link, improving bandwidth availability for all guests on each node. Is this even achievable?
Hi @Ismahel. I see it’s a old post but did you find a solution to this issue? It sounds firewall related.
I have a similar setup but using a bridge for the public / ovn network as I only have one physical interface per host. What did you do to allow internet access “through a single egress interface”?
@jochumdev I don’t believe my issue is DNS related as I can’t ping external IP’s. When I created my incus / ovn uplink I could not use a shared ipv4 gateway / subnet as my three hosts are on their own public IP’s / VPS’s. I created a new RFC 1918 network like so:
Unless ovn etc does something magic, there is no route for 10.245.39.1 to each hosts external IP. I presume that is why the containers using the uplink can’t access the internet. They can ping each other across on different hosts.
For the record, I was able to make this work by using the regular “incus admin init” / incusbr0 interface as the parent for the ovn network. I can now ping containers on other hosts plus access the wider internet from the containers.
