OVN setup examples

[tomp (Thomas Parrott)]

I’m not really clear on what you’re trying to achieve, for instance, why are you using a mixture of geneve and vxlans at the same time?

However, what I can say is that:

1. You must certainly not use the OVN southbound connection details in lxc config set network.ovn.northbound_connection - this will not work, and if it is for you, then something else is wrong.
2. I’m confused why your ovn-sbctl show command is showing different things on different hosts. This seems wrong to me, like they are not using the same northbound database as their source. Also I’m not clear what hosts are involved, do you have separate hosts for the OVN NB and SB database cluster vs the chassis hosts that run the actual LXD instances?

My recommendation is to start simple and then work up the complexity from there.
So I would start with a simple 4 node setup; 3x LXD nodes in a cluster and a separate OVN DB node.
Or use one of the nodes as a single NB/SB DB node.

This is what I described here:

Virtual/Cloud based network

Sounds like using OVN (https://www.ovn.org) from Openvswitch which LXD has recently added support for would be appropriate here. It allows multiple virtual networks to be setup (across multiple machines) and manages the tunneling of traffic between machines (using geneve tunnels). It then uses OVS on each host to connect to the instances via veth pairs (like a normal native bridge does). A simple single-node setup can be used to experiment with pretty quickly, we have a basic guide as part of …

Once you’re happy with that, then you can look at adding clustering to the OVN DB itself.
@stgraber has done this with the LXD infrastructure so he may be able to show you the connection strings he used for LXD’s northbound setting and OVS’s external_ids:ovn-remote setting?

[tomp (Thomas Parrott)]

I’m going to have a go at setting up a LXD and OVN cluster on 3 nodes later so will post my findings here once done.

[JeToJedno (David Hodgson)]

I’ve given up, for now, trying to get a 3 node OVN cluster working. I’ve gone the simple route: creating a controller node and setting the other 2 to sync nb & sb from that.

Took about 2 hours, mainly because I had to repeat it 3 times to get it right, and I got a log flood & high CPU because of a typo in the controller config.

If anyone wants to repeat it, the primary sources are:
https://www.ovn.org/support/dist-docs/ovn-ctl.8.html
https://blog.oddbit.com/post/2019-12-19-ovn-and-dhcp/

[tomp (Thomas Parrott)]

I’ve gotten chance now to put down some notes on setting up an OVN cluster:

OVN high availability cluster tutorial

This tutorial describes how to setup a 3 node OVN and LXD high availability cluster. I’m doing this with 3 LXD VMs connected to a private bridge with subnet 10.98.30.0/24, so lets create them first: lxc init images:ubuntu/focal v1 --vm lxc init images:ubuntu/focal v2 --vm lxc init images:ubuntu/focal v3 --vm We want to ensure they have statically assigned IPs: lxc config device override v1 eth0 ipv4.address=10.98.30.2 lxc config device override v2 eth0 ipv4.address=10.98.30.3 lxc config devi…