Incus OVN with ipv4.nat=false: external route is added without next-hop on bridge network

Incus
,
1

Hi everyone,

We are currently evaluating Incus OVN as a replacement for our existing LXD + bridge setup.

Our goal is to make the physical network and the OVN network reachable directly from each other, without NAT. Because of that, we configured the OVN network with:

ipv4.nat: "false"

At this stage this is only a test environment, and the underlying OVN parent network is using bridge mode.

Background

We noticed that when creating an OVN network with a specific IPv4 subnet, the OVN network’s ipv4.address needs to be included in the parent bridge network’s ipv4.routes.

For example, our parent bridge network has:

ipv4.ovn.ranges: 10.10.1.200-10.10.1.255
ipv4.routes: 10.80.1.0/24

The OVN network is configured roughly like this:

ipv4.address: 10.80.1.0/24
ipv4.nat: “false”
volatile.network.ipv4.address: 10.10.1.200

After creating the OVN network, the host gets a route like this:

10.80.1.0/24 dev bridge

However, with this route, the host cannot directly reach the instances inside the OVN network.

If we manually delete that route and add it back with an explicit next-hop, then the host can reach the instances:

ip route del 10.80.1.0/24 dev bridge
ip route add 10.80.1.0/24 via 10.10.1.200 dev bridge

Question

I would like to ask whether this is the expected design when using Incus OVN with ipv4.nat=false, or whether we are misunderstanding how the parent bridge network, ipv4.routes, and the OVN router address are supposed to work together.

In particular:

  1. Should the route to the OVN subnet be added as a direct dev bridge route?

  2. Or should it normally point to the OVN external/router address as the next-hop?

  3. If manual route replacement is required, is there a recommended Incus/OVN way to configure this instead of changing the host route manually?

I may be missing something basic in the intended network model, so any clarification would be very appreciated.

Thank you very much.

2

Hmm why do you use a 0 address not 1 for example?

ipv4.address: 10.80.1.1/24

Maybe cause of ipv4.routes ? I have no clue about OVN.

As I’m reading the docs I get more and more confused, OVN network - Incus documentation doesn’t say anything about ipv4.routes.

Maybe just ignore this “answer” :slight_smile:

Kind regards,
René

3

Please ignore the specific IP addresses in my example, as they are only meant to illustrate the topology.

From what I understand, when an OVN network is created on top of a bridge network, and ipv4.nat=false is used, the OVN network’s IPv4 subnet needs to be listed in the underlying bridge network’s ipv4.routes.

So it seems this may be an expected requirement or limitation of this setup, rather than something caused by the particular IP addresses I used in the example.

4

Hey :slight_smile:

The address is quite important as 0 is normaly the network address maybe thats why something fails in between.

Did you try to set a valid ipv4.address for OVN Network?

5

I may have written that incorrectly earlier.

From what I remember, incus network list shows the network address as something like:
10.80.1.1/24

So to be more precise, the OVN network itself has an address such as 10.80.1.1/24, and the Incus VMs inside that network receive addresses from the same subnet, for example 10.80.1.15.

The main issue I am trying to describe is still the routing behavior: the host cannot directly reach the VM addresses in the OVN network unless the route to that subnet points to the correct next-hop address.

1 Like