OVN cluster init issue

LXD
1

I’m again having issues getting OVN clustered. :frowning:
I’ve tried a complete uninstall, reboot and reinstall … the same result.

I have 3 hosts, all three don’t look right, but in different ways. The first host has different behaviour to the other two.

I’ve checked again my setup commands and compared them to those in OVN high availability cluster tutorial, and can’t see the difference. Nor can I see why they’re behaving differently.

Any pointers to the wrong config (or should I just go back to the single ovn-central setup which I had working successfully)?

All three are complaining about bind: Cannot assign requested address. On the first host, it seems to then bind to ports 6643 & 6644 (in addition to 6641 & 6642). On the other two … not.

on the “first” host:

# ovs-vsctl list open_vswitch
[sudo] password for albans: 
_uuid               : 4ff0e1cf-2c23-40cb-a803-53baf2ddec4a
bridges             : [be31ed83-2ca2-409c-ae8b-93102eccd124]
cur_cfg             : 1
datapath_types      : [netdev, system]
datapaths           : {}
db_version          : "8.2.0"
dpdk_initialized    : false
dpdk_version        : none
external_ids        : {hostname=albans.domuz, ovn-encap-ip="10.1.0.215", ovn-encap-type=geneve, ovn-openflow-probe-interval="15000", ovn-remote="unix:/var/run/ovn/ovnsb_db.sock", ovn-remote-probe-interval="5000", rundir="/var/run/openvswitch", system-id="f015fb56-439c-4d56-bfc6-d9087f6753f5"}
iface_types         : [erspan, geneve, gre, internal, ip6erspan, ip6gre, lisp, patch, stt, system, tap, vxlan]
manager_options     : []
next_cfg            : 1
other_config        : {}
ovs_version         : "2.13.1"
ssl                 : []
statistics          : {}
system_type         : ubuntu
system_version      : "20.04"

# ovs-vsctl show
4ff0e1cf-2c23-40cb-a803-53baf2ddec4a
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
    ovs_version: "2.13.1"

$ cat /etc/default/ovn-central
# This is a POSIX shell fragment                -*- sh -*-

OVN_CTL_OPTS= \
  --db-nb-create-insecure-remote=yes \
  --db-sb-create-insecure-remote=yes \
  --db-nb-addr=10.1.0.215 \
  --db-sb-addr=10.1.0.215 \
  --db-nb-cluster-local-addr=10.1.0.215 \
  --db-sb-cluster-local-addr=10.1.0.215 \
  --ovn-northd-nb-db=tcp:10.1.0.215:6641,tcp:10.1.0.213:6641,tcp:10.1.0.214:6641 \
  --ovn-northd-sb-db=tcp:10.1.0.215:6642,tcp:10.1.0.213:6642,tcp:10.1.0.214:6642

# ss -tuxlpn \
>         | grep -e '^\s*tcp\s.*\b:664[0-5]\b' -e '^\s*u_str\s.*\bovn\b' \
>         | sed -r -e 's/\s+$//'
[sudo] password for albans: 
u_str LISTEN  0       64                        /var/run/ovn/ovnnb_db.ctl 32322                                           * 0                                    users:(("ovsdb-server",pid=1816,fd=19))
u_str LISTEN  0       64                 /var/run/ovn/ovn-northd.1804.ctl 32880                                           * 0                                    users:(("ovn-northd",pid=1804,fd=9))
u_str LISTEN  0       64             /var/run/ovn/ovn-controller.1745.ctl 32810                                           * 0                                    users:(("ovn-controller",pid=1745,fd=9))
u_str LISTEN  0       64                       /var/run/ovn/ovnnb_db.sock 32315                                           * 0                                    users:(("ovsdb-server",pid=1816,fd=14))
u_str LISTEN  0       64                       /var/run/ovn/ovnsb_db.sock 32317                                           * 0                                    users:(("ovsdb-server",pid=1824,fd=14))
u_str LISTEN  0       64                        /var/run/ovn/ovnsb_db.ctl 32326                                           * 0                                    users:(("ovsdb-server",pid=1824,fd=21))
tcp   LISTEN  0       10                                       10.1.0.215:6641                                      0.0.0.0:*                                    users:(("ovsdb-server",pid=1816,fd=20))
tcp   LISTEN  0       10                                       10.1.0.215:6642                                      0.0.0.0:*                                    users:(("ovsdb-server",pid=1824,fd=25))
tcp   LISTEN  0       10                                       10.1.0.215:6643                                      0.0.0.0:*                                    users:(("ovsdb-server",pid=1816,fd=23))
tcp   LISTEN  0       10                                       10.1.0.215:6644                                      0.0.0.0:*                                    users:(("ovsdb-server",pid=1824,fd=26))

==> /var/log/ovn/ovsdb-server-nb.log <==
2021-05-16T15:40:42.277Z|00009|socket_util|ERR|6643:10.1.0.215: bind: Cannot assign requested address
2021-05-16T15:40:42.277Z|00010|raft|WARN|ptcp:6643:10.1.0.215: listen failed (Cannot assign requested address)
2021-05-16T15:40:42.292Z|00011|socket_util|ERR|6641:10.1.0.215: bind: Cannot assign requested address
2021-05-16T15:40:42.292Z|00012|ovsdb_jsonrpc_server|ERR|ptcp:6641:10.1.0.215: listen failed: Cannot assign requested address
2021-05-16T15:40:42.293Z|00013|socket_util|ERR|6641:10.1.0.215: bind: Cannot assign requested address
2021-05-16T15:40:42.294Z|00014|ovsdb_jsonrpc_server|ERR|ptcp:6641:10.1.0.215: listen failed: Cannot assign requested address
2021-05-16T15:40:42.301Z|00015|ovsdb_jsonrpc_server|ERR|ptcp:6641:10.1.0.215: listen failed: Cannot assign requested address
2021-05-16T15:40:43.278Z|00016|raft|WARN|ptcp:6643:10.1.0.215: listen failed (Cannot assign requested address)
2021-05-16T15:40:52.277Z|00017|memory|INFO|6708 kB peak resident set size after 10.1 seconds
2021-05-16T15:40:52.277Z|00018|memory|INFO|cells:31 monitors:2 sessions:1

==> /var/log/ovn/ovsdb-server-sb.log <==
2021-05-16T15:40:42.261Z|00009|socket_util|ERR|6644:10.1.0.215: bind: Cannot assign requested address
2021-05-16T15:40:42.261Z|00010|raft|WARN|ptcp:6644:10.1.0.215: listen failed (Cannot assign requested address)
2021-05-16T15:40:42.278Z|00011|socket_util|ERR|6642:10.1.0.215: bind: Cannot assign requested address
2021-05-16T15:40:42.278Z|00012|ovsdb_jsonrpc_server|ERR|ptcp:6642:10.1.0.215: listen failed: Cannot assign requested address
2021-05-16T15:40:42.294Z|00013|socket_util|ERR|6642:10.1.0.215: bind: Cannot assign requested address
2021-05-16T15:40:42.295Z|00014|ovsdb_jsonrpc_server|ERR|ptcp:6642:10.1.0.215: listen failed: Cannot assign requested address
2021-05-16T15:40:42.296Z|00015|ovsdb_jsonrpc_server|ERR|ptcp:6642:10.1.0.215: listen failed: Cannot assign requested address
2021-05-16T15:40:43.262Z|00016|raft|WARN|ptcp:6644:10.1.0.215: listen failed (Cannot assign requested address)
2021-05-16T15:40:52.263Z|00017|memory|INFO|6368 kB peak resident set size after 10.1 seconds
2021-05-16T15:40:52.263Z|00018|memory|INFO|cells:43 monitors:2 sessions:1

on the other two:

# ovs-vsctl list open_vswitch
[sudo] password for grantham: 
_uuid               : 3cb6267d-184f-480e-ad20-d7dcc752d294
bridges             : [fbe424d6-4eb1-471d-9f38-9011901ed544]
cur_cfg             : 1
datapath_types      : [netdev, system]
datapaths           : {}
db_version          : "8.2.0"
dpdk_initialized    : false
dpdk_version        : none
external_ids        : {hostname=grantham.domuz, ovn-encap-ip="10.1.0.213", ovn-encap-type=geneve, ovn-openflow-probe-interval="15000", ovn-remote="unix:/var/run/ovn/ovnsb_db.sock", ovn-remote-probe-interval="5000", rundir="/var/run/openvswitch", system-id="36d534c9-dc90-45a7-94c6-af3b1dc41697"}
iface_types         : [erspan, geneve, gre, internal, ip6erspan, ip6gre, lisp, patch, stt, system, tap, vxlan]
manager_options     : []
next_cfg            : 1
other_config        : {}
ovs_version         : "2.13.1"
ssl                 : []
statistics          : {}
system_type         : ubuntu
system_version      : "20.04"

# ovs-vsctl show
3cb6267d-184f-480e-ad20-d7dcc752d294
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
    ovs_version: "2.13.1"

$ cat /etc/default/ovn-central
# This is a POSIX shell fragment                -*- sh -*-

OVN_CTL_OPTS= \
  --db-nb-create-insecure-remote=yes \
  --db-sb-create-insecure-remote=yes \
  --db-nb-addr=10.1.0.213 \
  --db-sb-addr=10.1.0.213 \
  --db-nb-cluster-local-addr=10.1.0.213 \
  --db-sb-cluster-local-addr=10.1.0.213 \
  --db-nb-cluster-remote-addr=10.1.0.215 \
  --db-sb-cluster-remote-addr=10.1.0.215 \
  --ovn-northd-nb-db=tcp:10.1.0.215:6641,tcp:10.1.0.213:6641,tcp:10.1.0.214:6641 \
  --ovn-northd-sb-db=tcp:10.1.0.215:6642,tcp:10.1.0.213:6642,tcp:10.1.0.214:6642

# ss -tuxlpn \
>         | grep -e '^\s*tcp\s.*\b:664[0-5]\b' -e '^\s*u_str\s.*\bovn\b' \
>         | sed -r -e 's/\s+$//'
[sudo] password for grantham: 
u_str LISTEN  0       64             /var/run/ovn/ovn-controller.2482.ctl 35393                                           * 0                                    users:(("ovn-controller",pid=2482,fd=9))
u_str LISTEN  0       64                       /var/run/ovn/ovnsb_db.sock 41988                                           * 0                                    users:(("ovsdb-server",pid=2340,fd=14))
u_str LISTEN  0       64                 /var/run/ovn/ovn-northd.2679.ctl 1843                                            * 0                                    users:(("ovn-northd",pid=2679,fd=9))
u_str LISTEN  0       64                       /var/run/ovn/ovnnb_db.sock 31570                                           * 0                                    users:(("ovsdb-server",pid=2332,fd=14))
u_str LISTEN  0       64                        /var/run/ovn/ovnsb_db.ctl 41990                                           * 0                                    users:(("ovsdb-server",pid=2340,fd=15))
u_str LISTEN  0       64                        /var/run/ovn/ovnnb_db.ctl 31572                                           * 0                                    users:(("ovsdb-server",pid=2332,fd=15))
tcp   LISTEN  0       10                                       10.1.0.213:6641                                      0.0.0.0:*                                    users:(("ovsdb-server",pid=2332,fd=16))
tcp   LISTEN  0       10                                       10.1.0.213:6642                                      0.0.0.0:*                                    users:(("ovsdb-server",pid=2340,fd=17))

==> /var/log/ovn/ovsdb-server-nb.log <==
2021-05-16T15:40:43.847Z|00005|socket_util|ERR|6641:10.1.0.213: bind: Cannot assign requested address
2021-05-16T15:40:43.847Z|00006|ovsdb_jsonrpc_server|ERR|ptcp:6641:10.1.0.213: listen failed: Cannot assign requested address
2021-05-16T15:40:43.848Z|00007|socket_util|ERR|6641:10.1.0.213: bind: Cannot assign requested address
2021-05-16T15:40:43.848Z|00008|ovsdb_jsonrpc_server|ERR|ptcp:6641:10.1.0.213: listen failed: Cannot assign requested address
2021-05-16T15:40:46.349Z|00009|socket_util|ERR|6641:10.1.0.213: bind: Cannot assign requested address
2021-05-16T15:40:46.349Z|00010|ovsdb_jsonrpc_server|ERR|ptcp:6641:10.1.0.213: listen failed: Cannot assign requested address
2021-05-16T15:40:48.850Z|00011|socket_util|ERR|6641:10.1.0.213: bind: Cannot assign requested address
2021-05-16T15:40:48.850Z|00012|ovsdb_jsonrpc_server|ERR|ptcp:6641:10.1.0.213: listen failed: Cannot assign requested address
2021-05-16T15:40:53.856Z|00013|memory|INFO|6600 kB peak resident set size after 10.0 seconds
2021-05-16T15:40:53.857Z|00014|memory|INFO|cells:31 monitors:0

==> /var/log/ovn/ovsdb-server-sb.log <==
2021-05-16T15:40:43.845Z|00005|socket_util|ERR|6642:10.1.0.213: bind: Cannot assign requested address
2021-05-16T15:40:43.845Z|00006|ovsdb_jsonrpc_server|ERR|ptcp:6642:10.1.0.213: listen failed: Cannot assign requested address
2021-05-16T15:40:43.845Z|00007|socket_util|ERR|6642:10.1.0.213: bind: Cannot assign requested address
2021-05-16T15:40:43.845Z|00008|ovsdb_jsonrpc_server|ERR|ptcp:6642:10.1.0.213: listen failed: Cannot assign requested address
2021-05-16T15:40:44.799Z|00009|socket_util|ERR|6642:10.1.0.213: bind: Cannot assign requested address
2021-05-16T15:40:44.799Z|00010|ovsdb_jsonrpc_server|ERR|ptcp:6642:10.1.0.213: listen failed: Cannot assign requested address
2021-05-16T15:40:44.799Z|00011|socket_util|ERR|6642:10.1.0.213: bind: Cannot assign requested address
2021-05-16T15:40:44.799Z|00012|ovsdb_jsonrpc_server|ERR|ptcp:6642:10.1.0.213: listen failed: Cannot assign requested address
2021-05-16T15:40:53.855Z|00013|memory|INFO|6600 kB peak resident set size after 10.0 seconds
2021-05-16T15:40:53.855Z|00014|memory|INFO|cells:248 monitors:2 sessions:1
2021-05-16T15:51:07.114Z|00015|jsonrpc|WARN|unix#1: send error: Broken pipe
2021-05-16T15:51:07.114Z|00016|reconnect|WARN|unix#1: connection dropped (Broken pipe)

This is after all the hosts have been restarted (the “first” first).

2

I saw something similar when setting up my initial cluster for the tutorial.

It turned out to be from a left over manual setting for these:

ovn-nbctl set-connection ptcp:6641:0.0.0.0
ovn-sbctl set-connection ptcp:6642:0.0.0.0

I had to run the equivalent del-connection command and then it seemed to start up without any errors, and only opened the connections specified in the config file.

3

both the connections, on all machines, were blank - so I though that if I set the connection then cleared it, than may help.

It didn’t work, even in the slightest!

# ovn-nbctl get-connection
<<nothing>>
# ovn-sbctl get-connection
<<nothing>>

# ovn-nbctl set-connection unix:/var/run/ovn/ovnnb_db.sock ptcp:6641:0.0.0.0
# ovn-sbctl set-connection unix:/var/run/ovn/ovnsb_db.sock ptcp:6642:0.0.0.0

# ovn-nbctl get-connection
ptcp:6641:0.0.0.0
unix:/var/run/ovn/ovnnb_db.sock
# ovn-sbctl get-connection
read-write role="" ptcp:6642:0.0.0.0
read-write role="" unix:/var/run/ovn/ovnsb_db.sock

# ss -tuxlpn | grep -e '^\s*tcp\s.*\b:664[0-5]\b' -e '^\s*udp\s.*\b:6081\b' -e '^\s*u_str\s.*\bovn\b' | sed -r -e 's/\s+$//'
u_str LISTEN 0      64                       /var/run/ovn/ovnnb_db.sock 6561164                                           * 0                                    users:(("ovsdb-server",pid=10056,fd=18))
u_str LISTEN 0      64                        /var/run/ovn/ovnnb_db.ctl 6560592                                           * 0                                    users:(("ovsdb-server",pid=10056,fd=20))
u_str LISTEN 0      64                 /var/run/ovn/ovn-northd.9872.ctl 6535459                                           * 0                                    users:(("ovn-northd",pid=9872,fd=9))
u_str LISTEN 0      64                        /var/run/ovn/ovnsb_db.ctl 6561195                                           * 0                                    users:(("ovsdb-server",pid=10088,fd=16))
u_str LISTEN 0      64                       /var/run/ovn/ovnsb_db.sock 6561190                                           * 0                                    users:(("ovsdb-server",pid=10088,fd=15))
tcp   LISTEN 0      10                                       10.1.0.215:6641                                        0.0.0.0:*                                    users:(("ovsdb-server",pid=10056,fd=19))
tcp   LISTEN 0      10                                       10.1.0.215:6642                                        0.0.0.0:*                                    users:(("ovsdb-server",pid=10088,fd=14))
tcp   LISTEN 0      10                                       10.1.0.215:6643                                        0.0.0.0:*                                    users:(("ovsdb-server",pid=10056,fd=25))
tcp   LISTEN 0      10                                       10.1.0.215:6644                                        0.0.0.0:*                                    users:(("ovsdb-server",pid=10088,fd=21))

 # systemctl restart ovn-central.service

# ss -tuxlpn | grep -e '^\s*tcp\s.*\b:664[0-5]\b' -e '^\s*udp\s.*\b:6081\b' -e '^\s*u_str\s.*\bovn\b' | sed -r -e 
<<the same, with different pids>>

# systemctl list-units --type=service --all ovn-*
  UNIT                        LOAD   ACTIVE   SUB     DESCRIPTION                                             
  ovn-central.service         loaded active   exited  Open Virtual Network central components                 
  ovn-controller.service      loaded inactive dead    Open Virtual Network host control daemon                
  ovn-host.service            loaded inactive dead    Open Virtual Network host components                    
  ovn-northd.service          loaded active   running Open Virtual Network central control daemon             
  ovn-ovsdb-server-nb.service loaded active   running Open vSwitch database server for OVN Northbound database
  ovn-ovsdb-server-sb.service loaded active   running Open vSwitch database server for OVN Southbound database

what’s weird about this is that it’s still listening on 10.1.0.215 instead of 0.0.0.0!

4

and here’s another odd thing:

the man page for ovn-ctl doesn’t list --db-nb-addr & --db-sb-addr as options (just --db-nb-cluster-local-addr & --db-sb-cluster-local-addr) although the example given at the end is

#  ovn-ctl --db-nb-addr=x.x.x.x --db-nb-create-insecure-remote=yes --db-sb-addr=x.x.x.x --db-sb-create-insecure-remote=yes --db-nb-cluster-local-addr=x.x.x.x  --db-sb-cluster-local-addr=x.x.x.x --ovn-northd-nb-db=tcp:x.x.x.x:6641,tcp:y.y.y.y:6641,tcp:z.z.z.z:6641  --ovn-northd-sb-db=tcp:x.x.x.x:6642,tcp:y.y.y.y:6642,tcp:z.z.z.z:6642 start_northd
5

You shouldn’t need to use ovn-nbctl set-connection at all, just del-connection to remove them from ovn-nbctl and ovn-sbctl and then restart the cluster. The command line arguments setup the specific IP listeners you need instead.

6

It doesn’t surprise me, the OVN documentation tends to be somewhat haphazard.

7

@tomp : could I ask for some advice.

I seem to have created a script, based on your tutorial’s instructions, which creates double connections to the northd databases / hosts. One pair of connections (in the ovn-northd.log) is to / from ports 6641 & 6642. The next connections are ovsdb-server-nb.log 6643 & ovsdb-server-sb.log 6644.

This gives the listening as:

# ss -tuxlpn     | grep     -e '^\s*tcp\s.*\b:664[0-5]\b'         -e '^\s*udp\s.*\b:6081\b'         -e '^\s*u_str\s.*\bovn\b'     | sed -r -e 's/\s+$//' ;  echo
u_str LISTEN 0       64                        /var/run/ovn/ovnnb_db.ctl 27541                                            * 0                                    users:(("ovsdb-server",pid=2299,fd=15))
u_str LISTEN 0       64                 /var/run/ovn/ovn-northd.2580.ctl 26415                                            * 0                                    users:(("ovn-northd",pid=2580,fd=9))
u_str LISTEN 0       64                       /var/run/ovn/ovnnb_db.sock 27539                                            * 0                                    users:(("ovsdb-server",pid=2299,fd=14))
u_str LISTEN 0       64                       /var/run/ovn/ovnsb_db.sock 40053                                            * 0                                    users:(("ovsdb-server",pid=2304,fd=14))
u_str LISTEN 0       64                        /var/run/ovn/ovnsb_db.ctl 40055                                            * 0                                    users:(("ovsdb-server",pid=2304,fd=15))
tcp   LISTEN 0       10                                       10.1.0.213:6641                                       0.0.0.0:*                                    users:(("ovsdb-server",pid=2299,fd=24))
tcp   LISTEN 0       10                                       10.1.0.213:6642                                       0.0.0.0:*                                    users:(("ovsdb-server",pid=2304,fd=24))
tcp   LISTEN 0       10                                       10.1.0.213:6643                                       0.0.0.0:*                                    users:(("ovsdb-server",pid=2299,fd=25))
tcp   LISTEN 0       10                                       10.1.0.213:6644                                       0.0.0.0:*                                    users:(("ovsdb-server",pid=2304,fd=25))

it’s consistent across all 3 hosts in the cluster. Is this “normal”?

How can I get a list / table of the hosts participating in the OVN cluster?

8

I have found the same thing in my test cluster (which makes sense since we followed the same steps):

root@v1:~# cat /etc/default/ovn-central 
OVN_CTL_OPTS= \
  --db-nb-addr=10.98.30.2 \
  --db-sb-addr=10.98.30.2 \
  --db-nb-cluster-local-addr=10.98.30.2 \
  --db-sb-cluster-local-addr=10.98.30.2 \
  --db-nb-create-insecure-remote=yes \
  --db-sb-create-insecure-remote=yes \
  --ovn-northd-nb-db=tcp:10.98.30.2:6641,tcp:10.98.30.3:6641,tcp:10.98.30.4:6641 \
  --ovn-northd-sb-db=tcp:10.98.30.2:6642,tcp:10.98.30.3:6642,tcp:10.98.30.4:6642

root@v1:~# ss -tlpn
State    Recv-Q   Send-Q                          Local Address:Port       Peer Address:Port   Process                                      
LISTEN   0        10                                 10.98.30.2:6641            0.0.0.0:*       users:(("ovsdb-server",pid=812,fd=15))      
LISTEN   0        10                                 10.98.30.2:6642            0.0.0.0:*       users:(("ovsdb-server",pid=817,fd=15))      
LISTEN   0        10                                 10.98.30.2:6643            0.0.0.0:*       users:(("ovsdb-server",pid=812,fd=18))      
LISTEN   0        10                                 10.98.30.2:6644            0.0.0.0:*       users:(("ovsdb-server",pid=817,fd=18))

I had a quick search for “ovn port 6643” and found this: https://github.com/openshift/ovn-kubernetes/blob/master/FAQ.md

Which suggests that when using the RAFT cluster mode (as we are) OVN will open 1 more port for each database (northbound and southbound):

  port 6641 (for OVN Northbound OVSDB Server)
  port 6642 (for OVN Southbound OVSDB Server)
  port 6643 (when using RAFT and is required for NB RAFT control plane)
  port 6644 (when using RAFT and is required for SB RAFT control plane)
9

@tomp it may amuse you to learn - I finally traced the defect in what I was doing originally: POSIX shell script quoting (or the lack of it).

in /etc/default/ovn-central I had no quotes, like this:

OVN_CTL_OPTS= \
  --db-nb-create-insecure-remote=yes \
  --db-sb-create-insecure-remote=yes \
  --db-nb-addr=10.1.0.215 \
  --db-sb-addr=10.1.0.215 \
  --db-nb-cluster-local-addr=10.1.0.215 \
  --db-sb-cluster-local-addr=10.1.0.215 \
  --ovn-northd-nb-db=tcp:10.1.0.213:6641,tcp:10.1.0.214:6641,tcp:10.1.0.215:6641 \
  --ovn-northd-sb-db=tcp:10.1.0.213:6642,tcp:10.1.0.214:6642,tcp:10.1.0.215:6642

now, I have:

OVN_CTL_OPTS=" \
  --db-nb-create-insecure-remote=yes \
  --db-sb-create-insecure-remote=yes \
  --db-nb-addr=10.1.0.215 \
  --db-sb-addr=10.1.0.215 \
  --db-nb-cluster-local-addr=10.1.0.215 \
  --db-sb-cluster-local-addr=10.1.0.215 \
  --ovn-northd-nb-db=tcp:10.1.0.213:6641,tcp:10.1.0.214:6641,tcp:10.1.0.215:6641 \
  --ovn-northd-sb-db=tcp:10.1.0.213:6642,tcp:10.1.0.214:6642,tcp:10.1.0.215:6642 \
"

and it works properly.
$£%%^!**&!

There must have been an error message (I got one when I tried sourcing the file in /bin/sh) but this didn’t appear in the logs.

… but your file is the same as mine was, and yours works. Is your /bin/sh really dash (i.e. are you on Ubuntu) or something else?

$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Jul 18  2019 /bin/sh -> dash

what does /bin/sh -c '. /etc/default/ovn-central' show? Mine produced an error (--db-nb-create-insecure-remote=yes: not found) with the unquoted file.

10

Yes I’m using Ubuntu 20.04 with an unmodified shell. I did find that when testing with your original /etc/default/ovn-central file (that dynamically produced the values) that that didn’t work, so resorted to just defining them statically.

One thing I think is relevant is that that file isn’t run through a shell as far as I understand it, instead its used as part of the systemd unit via EnvironmentFile:

e.g.

cat /lib/systemd/system/ovn-northd.service
[Unit]
Description=Open Virtual Network central control daemon
After=network.target ovn-nb-ovsdb.service ovn-sb-ovsdb.service
PartOf=ovn-central.service
DefaultDependencies=no

[Service]
Type=forking
EnvironmentFile=-/etc/default/ovn-central
PIDFile=%t/ovn/ovn-northd.pid
ExecStart=/usr/share/ovn/scripts/ovn-ctl start_northd --ovn-manage-ovsdb=no --no-monitor $OVN_CTL_OPTS
ExecStop=/usr/share/ovn/scripts/ovn-ctl stop_northd --no-monitor
Restart=on-failure
LimitNOFILE=65535
TimeoutStopSec=15

See systemd.exec

11

Thanks. Bad assumption on my part.