Rotate Out Old Cluster Node for Newer Node with OVN and LXD

LXD
1

Hello LXD Friends.

I tried to find this information specific to OVN but I couldn’t find it anywhere, So I thought I would ask here to see if someone could point me in the right direction.

I currently have a 3 node cluster that sits atop of OVN. I bought some newer hardware and want to rotate out the cluster and not add to it. I have other ideas for the older hardware.

My first approach was to just add the new 4th node into the cluster and remove the older stuff. However, I got hung up on the config of OVN part. Would it be as simple as just including the new node in the config and restart the service… I don’t want to blow up my cluster?

Any guidance would be great.

Thakn you,
Adrean

2

Joining the OVN cluster on the new servers is indeed as simple as it was adding the second or third server initially. However you will need to remove the old servers from the OVN cluster once the new ones are in place.

I’ve done this once and it’s sadly not exactly well documented. I had a quick look at my server bash history but don’t have those commands left in there.

Basically you have to use ovsdb commands to trigger cluster/leave on the correct systems.
http://www.openvswitch.org/support/dist-docs/ovsdb-server.1.txt

You’ll have to do that on both the northd and southd databases. You should be able to use cluster/status to check that things look correct before and after the change.

Hope that helps!

1 Like
3

Thank you @stgraber thats very helpful . I couldn’t find anything anywhere!!!

4

Adding this for others that might need it.

I was able to remove the server from the cluster by using

udo ovs-appctl -t /var/run/ovn/ovnnb_db.ctl cluster/leave OVN_Northbound

the before

sudo ovs-appctl -t /var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound
39ad
Name: OVN_Northbound
Cluster ID:
Server ID: 
Address: 
Status: cluster member
Role: follower
Term: 11788
Leader: 6c25
Vote: 6c25

Last Election started 9316812 ms ago, reason: timeout
Election timer: 1000
Log: [2147, 2320]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: ->6c25 ->6dae ->c334 <-6dae <-c334 <-6c25
Disconnections: 1
Servers:
    6c25 (6c25 at tcp:192.168.20.50:6643) last msg 30 ms ago
    6dae (6dae at tcp:192.168.20.52:6643) last msg 9318498 ms ago
    39ad (39ad at tcp:192.168.20.51:6643) (self)
    c334 (c334 at tcp:192.168.20.53:6643) last msg 9316798 ms ago

and the after

Name: OVN_Northbound
Cluster ID: 7202 (7202b2bf-a6e9-4bfa-ac7f-54e3faaaa23e)
Server ID: c334 (c33435a2-94cc-4ffe-8f2a-99c0d870f2d5)
Address: tcp:192.168.20.53:6643
Status: cluster member
Role: follower
Term: 11788
Leader: 6c25
Vote: unknown

Last Election started 72805406 ms ago, reason: timeout
Election timer: 1000
Log: [2147, 2321]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: ->6c25 ->6dae <-6dae <-6c25
Disconnections: 3
Servers:
    6c25 (6c25 at tcp:192.168.20.50:6643) last msg 109 ms ago
    6dae (6dae at tcp:192.168.20.52:6643) last msg 9400780 ms ago
    c334 (c334 at tcp:192.168.20.53:6643) (self)

Hope this helps others!

1 Like