Unfortunately, nothing helps to complete microovn integration with the Incus cluster,
Below is details on my test environment:
- 3x Ubuntu VMs under Hyper-V
- basic
microovn cluster setup over these 3 nodes:microovn: a2c59c105b
ovn: 24.03.2-0ubuntu0.24.04.1
openvswitch: 3.3.0-1ubuntu3
$ sudo microovn status
MicroOVN deployment summary:
- vm-01 (172.25.81.250)
Services: central, chassis, switch
- vm-02 (172.25.81.251)
Services: central, chassis, switch
- vm-03 (172.25.81.252)
Services: central, chassis, switch
OVN Database summary:
OVN Northbound: OK (7.3.0)
OVN Southbound: OK (20.33.0)
$ sudo microovn.ovn-sbctl show
Chassis vm-01
hostname: vm-01
Encap geneve
ip: "172.25.81.250"
options: {csum="true"}
Chassis vm-02
hostname: vm-02
Encap geneve
ip: "172.25.81.251"
options: {csum="true"}
Chassis vm-03
hostname: vm-03
Encap geneve
ip: "172.25.81.252"
options: {csum="true"}
- basic
Incus cluster setup over these 3 nodes:$ sudo incus cluster list
+-------+----------------------------+-----------------+--------------+----------------+-------------+--------+-------------------+
| NAME | URL | ROLES | ARCHITECTURE | FAILURE DOMAIN | DESCRIPTION | STATUS | MESSAGE |
+-------+----------------------------+-----------------+--------------+----------------+-------------+--------+-------------------+
| vm-01 | https://172.25.81.250:8443 | database-leader | x86_64 | default | | ONLINE | Fully operational |
| | | database | | | | | |
+-------+----------------------------+-----------------+--------------+----------------+-------------+--------+-------------------+
| vm-02 | https://172.25.81.251:8443 | database | x86_64 | default | | ONLINE | Fully operational |
+-------+----------------------------+-----------------+--------------+----------------+-------------+--------+-------------------+
| vm-03 | https://172.25.81.252:8443 | database | x86_64 | default | | ONLINE | Fully operational |
+-------+----------------------------+-----------------+--------------+----------------+-------------+--------+-------------------+
$ sudo incus network ls
+----------------+----------+---------+---------------+---------------------------+-------------+---------+---------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY | STATE |
+----------------+----------+---------+---------------+---------------------------+-------------+---------+---------+
| br-int | bridge | NO | | | | 0 | |
+----------------+----------+---------+---------------+---------------------------+-------------+---------+---------+
| eth0 | physical | NO | | | | 0 | |
+----------------+----------+---------+---------------+---------------------------+-------------+---------+---------+
| genev_sys_6081 | unknown | NO | | | | 0 | |
+----------------+----------+---------+---------------+---------------------------+-------------+---------+---------+
| ibr0 | bridge | YES | 10.172.5.1/24 | fd42:62d9:e672:bf19::1/64 | | 0 | CREATED |
+----------------+----------+---------+---------------+---------------------------+-------------+---------+---------+
| lo | loopback | NO | | | | 0 | |
+----------------+----------+---------+---------------+---------------------------+-------------+---------+---------+
| ovs-system | unknown | NO | | | | 0 | |
+----------------+----------+---------+---------------+---------------------------+-------------+---------+---------+
Preparation steps for microovn integration
- I configured
network.ovs.connection variable on every node:
$ sudo incus config show --target vm-01
config:
cluster.https_address: 172.25.81.250:8443
core.https_address: 172.25.81.250:8443
network.ovs.connection: unix:/var/snap/microovn/common/run/switch/db.sock
$ sudo incus config show --target vm-02
config:
core.https_address: 172.25.81.251:8443
network.ovs.connection: unix:/var/snap/microovn/common/run/switch/db.sock
$ sudo incus config show --target vm-03
config:
core.https_address: 172.25.81.252:8443
network.ovs.connection: unix:/var/snap/microovn/common/run/switch/db.sock
- I was tried to configure
network.ovn.ca_cert, network.ovn.client_cert and network.ovn.northbound_connection by editing cluster node configs, using different combinations but every attempt fails. For example:
config:
cluster.https_address: 172.25.81.250:8443
core.https_address: 172.25.81.250:8443
network.ovs.connection: unix:/var/snap/microovn/common/run/switch/db.sock
network.ovn.northbound_connection: /var/snap/microovn/common/run/ovn/ovnnb_db.sock
fails with Config parsing error: failed to notify peer 172.25.81.251:8443: failed to connect to /var/snap/microovn/common/run/ovn/ovnnb_db.sock: failed to open connection: unknown network protocol;
another attempt:
config:
cluster.https_address: 172.25.81.250:8443
core.https_address: 172.25.81.250:8443
network.ovs.connection: unix:/var/snap/microovn/common/run/switch/db.sock
network.ovn.ca_cert: /var/snap/microovn/common/data/pki/cacert.pem
network.ovn.client_cert: /var/snap/microovn/common/data/pki/client-cert.pem
fails with Config parsing error: failed to notify peer 172.25.81.251:8443: failed to connect to unix:/run/ovn/ovnnb_db.sock: failed to open connection: dial unix /run/ovn/ovnnb_db.sock: connect: no such file or directory, which is weird as network.ovs.connection is configured already.
and even one-shot config change fails:
config:
cluster.https_address: 172.25.81.250:8443
core.https_address: 172.25.81.250:8443
network.ovs.connection: unix:/var/snap/microovn/common/run/switch/db.sock
network.ovn.ca_cert: /var/snap/microovn/common/data/pki/cacert.pem
network.ovn.client_cert: /var/snap/microovn/common/data/pki/client-cert.pem
network.ovn.northbound_connection: /var/snap/microovn/common/run/ovn/ovnnb_db.sock
Error: Config parsing error: failed to notify peer 172.25.81.251:8443: failed to connect to /var/snap/microovn/common/run/ovn/ovnnb_db.sock: failed to open connection: unknown network protocol
