Layman's walk through of open virtual network setup (on a single host)

Student · November 20, 2024, 6:01am

Disclaimer:

I am writing all this based on my 2 days of understanding of OVN setup
So please do your own homework
These are my notes based on how I understood
In no way, Iam claiming that I got all of it right (hardly!)

Note:

I am skipping the install/config part of incus as there is enough documentation out there.
I am not using on a cluster setup yet (though that is where the real power of ovn networking is)
I am starting with installing the ovn packages that are needed to have openvswitch on a single host

My setup:

I have an ARM embedded board (RADXA’s Rockpi4c) running Dietpi (debian based)
I Installed incus virtualization software platform from zabbly.com repository

Credits first:

incus documentation: OVN network - Incus documentation
Zabbly: https://www.youtube.com/@TheZabbly/videos
Simos: https://blog.simos.info/
Scott: https://www.youtube.com/playlist?list=PLWq_GaENnwX9KnyTsAP3-oZFfIoSaN-L7
incus: https://discuss.linuxcontainers.org/
DietPi: DietPi Community Forum - Welcome to the DietPi OS Community Forum
Armbian: https://forum.armbian.com/
Search: https://perplexity.ai

Oveall goal:

To get a basic understanding of ovn setup on incus, document the steps and share it

Layout:

Details of my host os:

host# uname -a

Linux rock4c 6.6.56-current-rockchip64 #1 SMP PREEMPT Thu Oct 10 10:50:06 UTC 2024 aarch64 GNU/Linux

host# cat /etc/os-release

PRETTY_NAME=“Debian GNU/Linux 12 (bookworm)”
NAME=“Debian GNU/Linux”
VERSION_ID=“12”
VERSION=“12 (bookworm)”
VERSION_CODENAME=bookworm
ID=debian
HOME_URL=“https://www.debian.org/”
SUPPORT_URL=“Debian -- User Support”
BUG_REPORT_URL=“https://bugs.debian.org/”

OVN packages installation:

host# apt install ovn-host ovn-central -y

Step 01: I ran the initial setup. I am not sure exactly what it does. Couldn’t find much details on what happens under the hood. This command is from incus documentation

host#  ovs-vsctl set open_vswitch . \
   external_ids:ovn-remote=unix:/run/ovn/ovnsb_db.sock \
   external_ids:ovn-encap-type=geneve \
   external_ids:ovn-encap-ip=127.0.0.1

Step 02: I am assigning a specific ip address of my choice (10.1.1.253) to incusbr0 bridge, replacing the original ip address assigned to it (when I did ‘incus admin init’)

host# incus network set incusbr0 ipv4.address=10.1.1.253/24

Step 03: I am seperating two different address spaces on 10.1.1.X (incusbr0 bridge) network. One address range (10.1.1.31 to 10.1.1.51) can be used for static ip address allocation for the vms and containers that connect to this network. And the other ip address range (10.1.1.131 to 10.1.1.151) is to dole them out via dhcp service onto the same network. The idea is to keep static and dhcp addresses seperately so that there will be no conflicts ( At least that is my understanding)

host# incus network set incusbr0 ipv4.ovn.ranges=10.1.1.31-10.1.1.51 ipv4.dhcp.ranges=10.1.1.131-10.1.1.151

Step 04: To test the above command, Iam launching a container

host# incus launch images:ubuntu/22.04 a1

Step 04a: I got inside the container to check its ip address

host:# incus shell a1

root@a1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
17: eth0@if18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:16:3e:e1:a1:d7 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.1.1.148/24 metric 100 brd 10.1.1.255 scope global dynamic eth0
valid_lft 3462sec preferred_lft 3462sec
inet6 fe80::216:3eff:fee1:a1d7/64 scope link
valid_lft forever preferred_lft forever
root@a1:~#exit

Step 04b: I see that an ip address (10.1.1.148) from the dhcp range that I set before is now being served. I now know that the dhcp address range part is working as expected

Step 04c: Iam now verifying that (from within the container) the traffic can reach incus bridge (10.1.1.253) and then to a host on the internet (zabbly.com)

root@a1:~# ping 10.1.1.253
PING 10.1.1.253 (10.1.1.253) 56(84) bytes of data.
64 bytes from 10.1.1.253: icmp_seq=1 ttl=64 time=0.229 ms
64 bytes from 10.1.1.253: icmp_seq=2 ttl=64 time=0.307 ms
64 bytes from 10.1.1.253: icmp_seq=3 ttl=64 time=0.310 ms
^C
— 10.1.1.253 ping statistics —
4 packets transmitted, 4 received, 0% packet loss, time 3066ms
rtt min/avg/max/mdev = 0.229/0.281/0.310/0.032 ms
root@a1:~#
root@a1:~# ping zabbly.com
PING zabbly.com (45.45.148.7) 56(84) bytes of data.
64 bytes from rproxy.dcmtl.stgraber.org (45.45.148.7): icmp_seq=1 ttl=50 time=88.7 ms
64 bytes from rproxy.dcmtl.stgraber.org (45.45.148.7): icmp_seq=2 ttl=50 time=79.0 ms
64 bytes from rproxy.dcmtl.stgraber.org (45.45.148.7): icmp_seq=3 ttl=50 time=81.9 ms
^C
— zabbly.com ping statistics —
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 78.500/82.206/88.680/3.650 ms
root@a1:~# exit
logout
host#

All seems good with the traffic on incusbr0 bridge. Now moving onto ovn setup

Step 05a: I am creating a new network named ‘ovnnet-coke’ of type ‘ovn’

host# incus network create ovnnet-coke --type=ovn

Step 05b: I am assigning an ip address 172.16.10.153 to it (assuming that a linux bridge will be created underneath on that name. But I had a surprise. Will come to that in a bit)

host# incus network set ovnnet-coke ipv4.address=172.16.10.253/24

Step 05c: I am enabling the dhcp service on the newly created network (by setting ipv4.nat=true)

host# incus network set ovnnet-coke ipv4.nat=true

Step 05d: I am setting incusbr0 bridge to be the parent of the newly created ovnnet-coke network

host# incus network set ovnnet-coke network=incusbr0

Step 05e (optional): I believe all the above 3 commands can be combined into one command (will try it next time)

host# incus network set ovnnet-coke ipv4.address=172.16.10.253/24 ipv4.nat=true network=incusbr0

Note: Just to recap where I am right now:

incusbr0 bridge is serving 10.1.1.X network
ovnnet-coke is serving 172.16.10.X network
incusbr0 bridge is the parent of ovnnet-coke

Step 05f: I am explicitly setting the incusbr0 bridge to do the routing for ovnnet-coke (172.16.10.X network)

host# incus network set incusbr0 ipv4.routes=172.16.10.0/24

Step 06: I am launching a container and connecting it to ovnnet-coke network (fingers crossed!)

host# incus launch images:ubuntu/22.04 c1 --network ovnnet-coke

Step 07: I am listing all the networks to make sure the static ip addresses that I assigned are indeed reflecting right. I believe br-int, incusovn1, incusovn1a, incusovn1b, ovs-system listed below are the ones that do the real OVN magic

host# incus network list
+-------------+----------+---------+------------------+------+-------------+---------+---------+
|    NAME     |   TYPE   | MANAGED |       IPV4       | IPV6 | DESCRIPTION | USED BY |  STATE  |
+-------------+----------+---------+------------------+------+-------------+---------+---------+
| br-int      | bridge   | NO      |                  |      |             | 0       |         |
+-------------+----------+---------+------------------+------+-------------+---------+---------+
| eth0        | physical | NO      |                  |      |             | 0       |         |
+-------------+----------+---------+------------------+------+-------------+---------+---------+
| incusbr0    | bridge   | YES     | 10.1.1.253/24    | none |             | 2       | CREATED |
+-------------+----------+---------+------------------+------+-------------+---------+---------+
| incusovn1   | bridge   | NO      |                  |      |             | 0       |         |
+-------------+----------+---------+------------------+------+-------------+---------+---------+
| incusovn1a  | unknown  | NO      |                  |      |             | 0       |         |
+-------------+----------+---------+------------------+------+-------------+---------+---------+
| incusovn1b  | unknown  | NO      |                  |      |             | 0       |         |
+-------------+----------+---------+------------------+------+-------------+---------+---------+
| lo          | loopback | NO      |                  |      |             | 0       |         |
+-------------+----------+---------+------------------+------+-------------+---------+---------+
| ovnnet-coke | ovn      | YES     | 172.16.10.253/24 |      |             | 1       | CREATED |
+-------------+----------+---------+------------------+------+-------------+---------+---------+
| ovs-system  | unknown  | NO      |                  |      |             | 0       |         |
+-------------+----------+---------+------------------+------+-------------+---------+---------+

Step 08: I am listing the instances. And Voila! The container did receive an ip address on 172.16.10.X network

host# incus ls
+------+---------+--------------------+------+-----------+-----------+
| NAME |  STATE  |        IPV4        | IPV6 |   TYPE    | SNAPSHOTS |
+------+---------+--------------------+------+-----------+-----------+
| c1   | RUNNING | 172.16.10.2 (eth0) |      | CONTAINER | 0         |
+------+---------+--------------------+------+-----------+-----------+

Step 09: I am checking on the bridges created underneath so far

host# ip -br -c a

lo UNKNOWN 127.0.0.1/8
eth0 UP 192.168.1.14/24
ovs-system DOWN
br-int DOWN
incusbr0 UP 10.1.1.253/24
incusovn1b@incusovn1a UP
incusovn1a@incusovn1b UP
incusovn1 DOWN
veth74dfa876@if15 UP

Step 10: My surprise! I don’t see any bridge named ovnnet-coke on the host that I created! So what it tells me is that the entire ovn networking paradigm lives within the realm of databases(s) and other constructs that incus maintains and the host OS is completely ignorant of its existence. I believe all the ovn networking magic happens outide the purvue of host os. (Hope someone can shed some light on this - if my understanding is right)

host# ip -br -4 a

lo               UNKNOWN        127.0.0.1/8
eth0             UP             192.168.1.14/24
incusbr0         UP             10.1.1.253/24

host# brctl show

bridge name     bridge id               STP enabled     interfaces
incusbr0        8000.00163eb5a19b       no              incusovn1a
                                                        veth48dae2d2

Step 11: The routing table on the host shows me what I was hoping to see.

My host is on my home network 192.168.1.14
The default router for my home network is 192.168.1.1
All traffic on 10.1.1.X/24 network goes via incusbr0 bridge to the physical nic (eth0) on my host
All traffic on 172.16.10.X/24 ovn network gets routed to incusbr0 bridge
So I believe there happens a double NAT’ing. One on ovnnet-coke level and one on incusbr0 bridge level

host# ip route

default via 192.168.1.1 dev eth0 onlink
10.1.1.0/24 dev incusbr0 proto kernel scope link src 10.1.1.253
172.16.10.0/24 dev incusbr0 proto static scope link
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.14

Step 12: Finally, I got inside the newly launched container that is connected to ovnnet-coke (step 06)

host# incus shell c1

Step 13: I am listing the interfaces inside the container

root@c1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
15: eth0@if16: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:16:3e:68:0d:5c brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.16.10.2/24 metric 100 brd 172.16.10.255 scope global dynamic eth0
valid_lft 2882sec preferred_lft 2882sec
inet6 fe80::216:3eff:fe68:d5c/64 scope link
valid_lft forever preferred_lft forever

Step 14a: I am able to ping the ip address that I assigned (step 05b) while creating ovnnet-coke

root@c1:~# ping 172.16.10.253
PING 172.16.10.253 (172.16.10.253) 56(84) bytes of data.
64 bytes from 172.16.10.253: icmp_seq=1 ttl=254 time=3.28 ms
64 bytes from 172.16.10.253: icmp_seq=2 ttl=254 time=2.78 ms
64 bytes from 172.16.10.253: icmp_seq=3 ttl=254 time=2.77 ms
^C
— 172.16.10.253 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 2.769/2.942/3.284/0.241 ms

Step 14b: I am able to ping the ip address of the incusbr0 bridge as well (step 02)

root@c1:~# ping 10.1.1.253
PING 10.1.1.253 (10.1.1.253) 56(84) bytes of data.
64 bytes from 10.1.1.253: icmp_seq=1 ttl=63 time=4.58 ms
64 bytes from 10.1.1.253: icmp_seq=2 ttl=63 time=0.457 ms
64 bytes from 10.1.1.253: icmp_seq=3 ttl=63 time=0.449 ms
^C
— 10.1.1.253 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2023ms
rtt min/avg/max/mdev = 0.449/1.828/4.580/1.945 ms

Step 14c: Finally I am able to ping an external host on the internet (zabbly.com)

root@c1:~# ping zabbly.com
PING zabbly.com (45.45.148.7) 56(84) bytes of data.
64 bytes from rproxy.dcmtl.stgraber.org (45.45.148.7): icmp_seq=1 ttl=49 time=96.1 ms
64 bytes from rproxy.dcmtl.stgraber.org (45.45.148.7): icmp_seq=2 ttl=49 time=79.6 ms
64 bytes from rproxy.dcmtl.stgraber.org (45.45.148.7): icmp_seq=3 ttl=49 time=79.6 ms
^C
— zabbly.com ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 79.559/85.093/96.137/7.809 ms

Step 14d: I installed traceroute package to verify the traffic path from the container running on ovnnet-coke to all the way to zabbly.com. Worked like charm!

root@c1:~# apt install traceroute
root@c1:~# traceroute zabbly.com
traceroute to zabbly.com (45.45.148.7), 30 hops max, 60 byte packets
1 _gateway (172.16.10.253) 20.458 ms 21.679 ms 23.083 ms
2 _gateway.incus (10.1.1.253) 34.633 ms 34.662 ms 34.728 ms
3 192.168.1.1 (192.168.1.1) 37.595 ms 37.673 ms 37.653 ms
…
…
…
root@c1:~# exit

Step 15: I am listing all the network interfaces on my host for reference

host# ifconfig -a
br-int: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether de:fa:39:18:8f:04  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.14  netmask 255.255.255.0  broadcast 192.168.1.255
        ether 56:36:df:fc:4f:d0  txqueuelen 1000  (Ethernet)
        RX packets 155107  bytes 110518899 (105.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 13362  bytes 1313843 (1.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 76

incusbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.1.1.253  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 00:16:3e:b5:a1:9b  txqueuelen 1000  (Ethernet)
        RX packets 813  bytes 48646 (47.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 833  bytes 3440697 (3.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

incusovn1: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether 72:d5:24:80:2c:42  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 11  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

incusovn1a: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 46:24:03:42:29:56  txqueuelen 1000  (Ethernet)
        RX packets 794  bytes 59006 (57.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 818  bytes 3439843 (3.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

incusovn1b: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 32:8c:6d:dc:33:9d  txqueuelen 1000  (Ethernet)
        RX packets 818  bytes 3439843 (3.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 794  bytes 59006 (57.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 10  bytes 952 (952.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 10  bytes 952 (952.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ovs-system: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether be:98:67:a0:13:4b  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth74dfa876: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 1a:f2:a9:e0:3c:e7  txqueuelen 1000  (Ethernet)
        RX packets 847  bytes 67018 (65.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 852  bytes 3446733 (3.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Step 16: I am listing all the services related to ovn networking on my host for reference

host# systemctl status -l ovn-
ovn-central.service          ovn-host.service             ovn-northd.service           ovn-ovsdb-server-sb.service
ovn-controller.service       ovn-nb-ovsdb.service         ovn-ovsdb-server-nb.service  ovn-sb-ovsdb.service

host# systemctl status -l ovn-host.service
● ovn-host.service - Open Virtual Network host components
     Loaded: loaded (/lib/systemd/system/ovn-host.service; enabled; preset: enabled)
     Active: active (exited) since Thu 2024-11-14 22:47:36 PST; 1 day 9h ago
   Main PID: 651 (code=exited, status=0/SUCCESS)
        CPU: 5ms

Nov 14 22:47:36 rock4c systemd[1]: Starting ovn-host.service - Open Virtual Network host components...
Nov 14 22:47:36 rock4c systemd[1]: Finished ovn-host.service - Open Virtual Network host components.

host# systemctl status -l ovn-central.service
● ovn-central.service - Open Virtual Network central components
     Loaded: loaded (/lib/systemd/system/ovn-central.service; enabled; preset: enabled)
     Active: active (exited) since Thu 2024-11-14 22:47:36 PST; 1 day 9h ago
   Main PID: 647 (code=exited, status=0/SUCCESS)
        CPU: 6ms

Nov 14 22:47:36 rock4c systemd[1]: Starting ovn-central.service - Open Virtual Network central components...
Nov 14 22:47:36 rock4c systemd[1]: Finished ovn-central.service - Open Virtual Network central components.

host# systemctl status -l ovn-controller.service
● ovn-controller.service - Open Virtual Network host control daemon
     Loaded: loaded (/lib/systemd/system/ovn-controller.service; static)
     Active: active (running) since Thu 2024-11-14 22:47:37 PST; 1 day 9h ago
   Main PID: 775 (ovn-controller)
      Tasks: 4 (limit: 4458)
     Memory: 7.4M
        CPU: 1min 40.160s
     CGroup: /system.slice/ovn-controller.service
             └─775 ovn-controller unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --no-chdir --log-file=/var/log/ovn/ovn-controller.log --pidfile=/var/run/ovn/ovn-controller.pid --detach

Nov 14 22:47:36 rock4c systemd[1]: Starting ovn-controller.service - Open Virtual Network host control daemon...
Nov 14 22:47:37 rock4c ovn-ctl[650]: Starting ovn-controller.
Nov 14 22:47:37 rock4c systemd[1]: Started ovn-controller.service - Open Virtual Network host control daemon.

host# systemctl status -l ovn-ovsdb-server-sb.service
● ovn-ovsdb-server-sb.service - Open vSwitch database server for OVN Southbound database
     Loaded: loaded (/lib/systemd/system/ovn-ovsdb-server-sb.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-11-14 22:47:36 PST; 1 day 9h ago
   Main PID: 656 (ovsdb-server)
      Tasks: 1 (limit: 4458)
     Memory: 4.0M
        CPU: 33.028s
     CGroup: /system.slice/ovn-ovsdb-server-sb.service
             └─656 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-sb.log --remote=punix:/var/run/ovn/ovnsb_db.sock --pidfile=/var/run/ovn/ovnsb_db.pid --unixctl=/var/run/ovn/ovnsb_db.ctl --remote=db:OVN_Southbound,SB_Global,connections --private-key=db:OVN_Southbound,SSL,private_key --certificate=db:OVN_Southbound,SSL,certificate --ca-cert=db:OVN_Southbound,SSL,ca_cert --ssl-protocols=db:OVN_Southbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Southbound,SSL,ssl_ciphers /var/lib/ovn/ovnsb_db.db

Nov 14 22:47:36 rock4c systemd[1]: Started ovn-ovsdb-server-sb.service - Open vSwitch database server for OVN Southbound database.
Nov 14 22:47:36 rock4c ovsdb-server[656]: ovs|00001|vlog|INFO|opened log file /var/log/ovn/ovsdb-server-sb.log
Nov 14 22:47:36 rock4c ovsdb-server[656]: ovs|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 3.1.0
Nov 14 22:47:46 rock4c ovsdb-server[656]: ovs|00003|memory|INFO|11136 kB peak resident set size after 10.1 seconds
Nov 14 22:47:46 rock4c ovsdb-server[656]: ovs|00004|memory|INFO|atoms:4796 cells:4469 monitors:4 n-weak-refs:43 sessions:3

host# systemctl status -l ovn-northd.service
● ovn-northd.service - Open Virtual Network central control daemon
     Loaded: loaded (/lib/systemd/system/ovn-northd.service; static)
     Active: active (running) since Thu 2024-11-14 22:47:37 PST; 1 day 9h ago
   Main PID: 776 (ovn-northd)
      Tasks: 3 (limit: 4458)
     Memory: 6.7M
        CPU: 915ms
     CGroup: /system.slice/ovn-northd.service
             └─776 ovn-northd -vconsole:emer -vsyslog:err -vfile:info --ovnnb-db=unix:/var/run/ovn/ovnnb_db.sock --ovnsb-db=unix:/var/run/ovn/ovnsb_db.sock --no-chdir --log-file=/var/log/ovn/ovn-northd.log --pidfile=/var/run/ovn/ovn-northd.pid --detach

Nov 14 22:47:36 rock4c systemd[1]: Starting ovn-northd.service - Open Virtual Network central control daemon...
Nov 14 22:47:37 rock4c ovn-ctl[660]: Starting ovn-northd.
Nov 14 22:47:37 rock4c systemd[1]: Started ovn-northd.service - Open Virtual Network central control daemon.

host# systemctl status -l ovn-controller.service
● ovn-controller.service - Open Virtual Network host control daemon
     Loaded: loaded (/lib/systemd/system/ovn-controller.service; static)
     Active: active (running) since Thu 2024-11-14 22:47:37 PST; 1 day 9h ago
   Main PID: 775 (ovn-controller)
      Tasks: 4 (limit: 4458)
     Memory: 7.4M
        CPU: 1min 40.160s
     CGroup: /system.slice/ovn-controller.service
             └─775 ovn-controller unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --no-chdir --log-file=/var/log/ovn/ovn-controller.log --pidfile=/var/run/ovn/ovn-controller.pid --detach

Nov 14 22:47:36 rock4c systemd[1]: Starting ovn-controller.service - Open Virtual Network host control daemon...
Nov 14 22:47:37 rock4c ovn-ctl[650]: Starting ovn-controller.
Nov 14 22:47:37 rock4c systemd[1]: Started ovn-controller.service - Open Virtual Network host control daemon.

host# systemctl status -l ovn-ovsdb-server-sb.service
● ovn-ovsdb-server-sb.service - Open vSwitch database server for OVN Southbound database
     Loaded: loaded (/lib/systemd/system/ovn-ovsdb-server-sb.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-11-14 22:47:36 PST; 1 day 9h ago
   Main PID: 656 (ovsdb-server)
      Tasks: 1 (limit: 4458)
     Memory: 4.0M
        CPU: 33.028s
     CGroup: /system.slice/ovn-ovsdb-server-sb.service
             └─656 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-sb.log --remote=punix:/var/run/ovn/ovnsb_db.sock --pidfile=/var/run/ovn/ovnsb_db.pid --unixctl=/var/run/ovn/ovnsb_db.ctl --remote=db:OVN_Southbound,SB_Global,connections --private-key=db:OVN_Southbound,SSL,private_key --certificate=db:OVN_Southbound,SSL,certificate --ca-cert=db:OVN_Southbound,SSL,ca_cert --ssl-protocols=db:OVN_Southbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Southbound,SSL,ssl_ciphers /var/lib/ovn/ovnsb_db.db

Nov 14 22:47:36 rock4c systemd[1]: Started ovn-ovsdb-server-sb.service - Open vSwitch database server for OVN Southbound database.
Nov 14 22:47:36 rock4c ovsdb-server[656]: ovs|00001|vlog|INFO|opened log file /var/log/ovn/ovsdb-server-sb.log
Nov 14 22:47:36 rock4c ovsdb-server[656]: ovs|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 3.1.0
Nov 14 22:47:46 rock4c ovsdb-server[656]: ovs|00003|memory|INFO|11136 kB peak resident set size after 10.1 seconds
Nov 14 22:47:46 rock4c ovsdb-server[656]: ovs|00004|memory|INFO|atoms:4796 cells:4469 monitors:4 n-weak-refs:43 sessions:3

host# systemctl status -l ovn-ovsdb-server-nb.service
● ovn-ovsdb-server-nb.service - Open vSwitch database server for OVN Northbound database
     Loaded: loaded (/lib/systemd/system/ovn-ovsdb-server-nb.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-11-14 22:47:36 PST; 1 day 9h ago
   Main PID: 652 (ovsdb-server)
      Tasks: 1 (limit: 4458)
     Memory: 2.2M
        CPU: 31.584s
     CGroup: /system.slice/ovn-ovsdb-server-nb.service
             └─652 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-nb.log --remote=punix:/var/run/ovn/ovnnb_db.sock --pidfile=/var/run/ovn/ovnnb_db.pid --unixctl=/var/run/ovn/ovnnb_db.ctl --remote=db:OVN_Northbound,NB_Global,connections --private-key=db:OVN_Northbound,SSL,private_key --certificate=db:OVN_Northbound,SSL,certificate --ca-cert=db:OVN_Northbound,SSL,ca_cert --ssl-protocols=db:OVN_Northbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Northbound,SSL,ssl_ciphers /var/lib/ovn/ovnnb_db.db

Nov 14 22:47:36 rock4c systemd[1]: Started ovn-ovsdb-server-nb.service - Open vSwitch database server for OVN Northbound database.
Nov 14 22:47:36 rock4c ovsdb-server[652]: ovs|00001|vlog|INFO|opened log file /var/log/ovn/ovsdb-server-nb.log
Nov 14 22:47:36 rock4c ovsdb-server[652]: ovs|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 3.1.0
Nov 14 22:47:46 rock4c ovsdb-server[652]: ovs|00003|memory|INFO|9728 kB peak resident set size after 10.1 seconds
Nov 14 22:47:46 rock4c ovsdb-server[652]: ovs|00004|memory|INFO|atoms:596 cells:746 monitors:3 n-weak-refs:0 sessions:2

host# systemctl status -l ovn-nb-ovsdb.service
● ovn-ovsdb-server-nb.service - Open vSwitch database server for OVN Northbound database
     Loaded: loaded (/lib/systemd/system/ovn-ovsdb-server-nb.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-11-14 22:47:36 PST; 1 day 9h ago
   Main PID: 652 (ovsdb-server)
      Tasks: 1 (limit: 4458)
     Memory: 2.2M
        CPU: 31.602s
     CGroup: /system.slice/ovn-ovsdb-server-nb.service
             └─652 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-nb.log --remote=punix:/var/run/ovn/ovnnb_db.sock --pidfile=/var/run/ovn/ovnnb_db.pid --unixctl=/var/run/ovn/ovnnb_db.ctl --remote=db:OVN_Northbound,NB_Global,connections --private-key=db:OVN_Northbound,SSL,private_key --certificate=db:OVN_Northbound,SSL,certificate --ca-cert=db:OVN_Northbound,SSL,ca_cert --ssl-protocols=db:OVN_Northbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Northbound,SSL,ssl_ciphers /var/lib/ovn/ovnnb_db.db

Nov 14 22:47:36 rock4c systemd[1]: Started ovn-ovsdb-server-nb.service - Open vSwitch database server for OVN Northbound database.
Nov 14 22:47:36 rock4c ovsdb-server[652]: ovs|00001|vlog|INFO|opened log file /var/log/ovn/ovsdb-server-nb.log
Nov 14 22:47:36 rock4c ovsdb-server[652]: ovs|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 3.1.0
Nov 14 22:47:46 rock4c ovsdb-server[652]: ovs|00003|memory|INFO|9728 kB peak resident set size after 10.1 seconds
Nov 14 22:47:46 rock4c ovsdb-server[652]: ovs|00004|memory|INFO|atoms:596 cells:746 monitors:3 n-weak-refs:0 sessions:2

host# systemctl status -l ovn-sb-ovsdb.service
● ovn-ovsdb-server-sb.service - Open vSwitch database server for OVN Southbound database
     Loaded: loaded (/lib/systemd/system/ovn-ovsdb-server-sb.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-11-14 22:47:36 PST; 1 day 9h ago
   Main PID: 656 (ovsdb-server)
      Tasks: 1 (limit: 4458)
     Memory: 4.0M
        CPU: 33.061s
     CGroup: /system.slice/ovn-ovsdb-server-sb.service
             └─656 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-sb.log --remote=punix:/var/run/ovn/ovnsb_db.sock --pidfile=/var/run/ovn/ovnsb_db.pid --unixctl=/var/run/ovn/ovnsb_db.ctl --remote=db:OVN_Southbound,SB_Global,connections --private-key=db:OVN_Southbound,SSL,private_key --certificate=db:OVN_Southbound,SSL,certificate --ca-cert=db:OVN_Southbound,SSL,ca_cert --ssl-protocols=db:OVN_Southbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Southbound,SSL,ssl_ciphers /var/lib/ovn/ovnsb_db.db

Nov 14 22:47:36 rock4c systemd[1]: Started ovn-ovsdb-server-sb.service - Open vSwitch database server for OVN Southbound database.
Nov 14 22:47:36 rock4c ovsdb-server[656]: ovs|00001|vlog|INFO|opened log file /var/log/ovn/ovsdb-server-sb.log
Nov 14 22:47:36 rock4c ovsdb-server[656]: ovs|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 3.1.0
Nov 14 22:47:46 rock4c ovsdb-server[656]: ovs|00003|memory|INFO|11136 kB peak resident set size after 10.1 seconds
Nov 14 22:47:46 rock4c ovsdb-server[656]: ovs|00004|memory|INFO|atoms:4796 cells:4469 monitors:4 n-weak-refs:43 sessions:3

Step 17: I am glad to see you coming down this far Good luck and best wishes!

moseph · April 16, 2025, 3:57pm

This was a fantastic read and helpful walk-through!

I’ve done as you did, and am able to reach from the inner network out as much as I want.

Have you come across a method for allowing the reverse? As in I want to run some servers in that network, and allow some web traffic to pass into them for processing (keycloak, for example).

Student · April 16, 2025, 8:56pm

Thank you for the compliment Reverse would be an equally interesting exercise. For now, I am playing with multi-tenancy on incus using Projects, allocating its own dedicated storage and network isolation. Will be happy to post when I am done.