Understanding OVN UPLINK configuration

Incus
1

The configuration of a PHYSICAL network for use as OVN’s uplink has an odd restriction: the ipv6.gateway address must be within the ipv6.routes subnet.

I use fd5b:7e71:a0dc:00<vlan_id>::1/64 as the gateway address, configured at my router.
I also set aside fd5b:7e71:a0dc:1000::1/56 for use with Incus; I’m using that CIDR in my BGP configuration. For example I created an Incus managed bridge with the subnet fd5b:7e71:a0dc:1001::/64 which is available on the rest of my VLANs through my router.

I’d like allocate fd5b:7e71:a0dc:1020::/60 to my UPLINK, and allocate /64s for the OVN networks. But I’m blocked from doing so because the gateway address is not in this range.

I’m really not sure why that limitation exists, especially since Incus supports BGP. Maybe I’m missing something.

2

Hmm, I’m surprised that there is any relation between ipv6.gateway and ipv6.routes on an uplink. I’d expect the only restriction be that ipv6.gateway must be within the subnet of ipv6.address. ipv6.routes are usually a completely distinct set from the network used for ipv6.address and ipv6.gateway.

3

Physical networks don’t have an ipv6.address configuration option. Should I be using a Bridge as my uplink?

4

Oh, right, sorry.

5

Just checked my production environments and they all do have the gateway’s subnet within their ipv6.routes.

6

Ah, no, just found one that isn’t.

stgraber@castiana:~ (incus:n-cloud/default)$ incus network show OVN-CLOUD
config:
  bgp.peers.fw-wan01.address: 2602:fc62:ef:100::100
  bgp.peers.fw-wan01.asn: "64600"
  dns.nameservers: 45.45.148.195,2602:fc62:ef:8::1
  ipv4.gateway: 172.20.1.1/24
  ipv4.ovn.ranges: 172.20.1.10-172.20.1.254
  ipv4.routes: 45.45.148.200/29,45.45.148.208/29
  ipv4.routes.anycast: "true"
  ipv6.gateway: 2602:fc62:ef:301::1/64
  ipv6.routes: 2602:fc62:ee::/48
  ipv6.routes.anycast: "true"
  ovn.ingress_mode: routed
  volatile.last_state.created: "false"
description: ""
name: OVN-CLOUD
type: physical
used_by:
  - /1.0/networks/default?project=montrehack
  - /1.0/networks/default?project=ringzer0ctf
  - /1.0/networks/default?project=ringzer0ctf-spawn2pwn
  - /1.0/networks/dlebrun-net?project=ringzer0ctf
managed: true
status: Created
locations:
  - vm02
  - vm03
  - infra01
  - infra02
  - infra03
  - infra04
  - infra05
  - vm01
project: default
7

Looks like you aren’t using ipv6.ovn.ranges - removing that from my UPLINK config gives me a different error when I try to save my OVN network config:

Config parsing error: Failed setting up BGP prefixes: can't find a specified path(s) with the given UUID(s)

I get this error even if my UPLINK’s BGP configuration is deleted.
It looks like this though:

  bgp.peers.opnsense.address: fd5b:7e71:a0dc::1
  bgp.peers.opnsense.asn: "65123"

Not sure if there’s an issue with using the same ASN number as my bridge network.

8

That sounds like an issue with the built-in BGP server having gotten a bit out of sync somehow. You may want to restart Incus to see if that clears it.

1 Like
9

Alright! Fixed that issue, BGP is now “working” – my router is now routing the OVN subnet to my Incus host.
Unfortunately the IncusOS host isn’t responding, but at least that’s a different error.