OVN networking attempt

LXD
,
1

Hi All!

I’m attempting to get OVN working again, but this time on an ubuntu 20:04 cloud host in hetzner (last time I failed as I was using proxmox/debian buster).

However, still having a few issues, but I’m not sure if its down to the lack of resources of the server, it only has 2GB of RAM not sure if that makes a difference.

So far I’ve tried the following:

    sudo apt install ovn-host ovn-central
    ovs-vsctl set open_vswitch .    external_ids:ovn-remote=unix:/var/run/ovn/ovnsb_db.sock    external_ids:ovn-encap-type=geneve    external_ids:ovn-encap-ip=10.55.0.42
    lxc network set lxdbr0 ipv4.dhcp.ranges=10.0.0.10-10.0.0.200 ipv4.ovn.ranges=10.42.10.5-10.42.10.200
    lxc network create ovn0 --type=ovn network=lxdbr0
    lxc launch ubuntu:20.04 mattersmost -s zfs1 -n ovn0

it seems to suceed, no errors, but then the container won’t start because it looks as though the ovn0 bridge has not been created?

I see the following:

root@p42:~# ip -br a
lo               UNKNOWN        127.0.0.1/8 ::1/128 
eth0             UP             159.69.42.91/32 2a01:4f8:c2c:7bbc::1/64 fe80::9400:ff:fe75:d6a9/64 
ovs-system       DOWN           
lxdovn1          DOWN           
br-int           DOWN           
ztks5yas3r       UNKNOWN        10.55.0.42/24 fd17:d709:436c:8908:1d99:9359:fe6b:8460/88 fe80::1c51:77ff:fe07:c769/64 
docker0          DOWN           172.17.0.1/16 
br-30b0e255e043  UP             172.19.0.1/16 fe80::42:bfff:fea6:b3c8/64 
docker_gwbridge  DOWN           172.18.0.1/16 
vethd54da4a@if10 UP             fe80::449e:a8ff:feb4:5226/64 
lxdbr0           UP             10.42.10.1/24 fd42:5bf8:7fd5:119a::1/64 fe80::216:3eff:fe86:d191/64 
lxdovn1b@lxdovn1a UP             
lxdovn1a@lxdovn1b UP             
lxdbr1           UP             10.42.11.1/24 fd42:23a3:d094:170d::1/64 fe80::216:3eff:fe6e:43af/64 
lxdovn3b@lxdovn3a UP             
lxdovn3a@lxdovn3b UP             
lxdovn3          DOWN

root@p42:~# lxc network list
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
|      NAME       |   TYPE   | MANAGED |      IPV4       |           IPV6            | DESCRIPTION | USED BY |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| br-30b0e255e043 | bridge   | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| br-int          | bridge   | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| docker0         | bridge   | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| docker_gwbridge | bridge   | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| eth0            | physical | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| lxdbr0          | bridge   | YES     | 10.42.10.1/24   | fd42:5bf8:7fd5:119a::1/64 |             | 1       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| lxdbr1          | bridge   | YES     | 10.42.11.1/24   | fd42:23a3:d094:170d::1/64 |             | 1       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| lxdovn1         | bridge   | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| lxdovn3         | bridge   | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| ovn0            | ovn      | YES     | 10.113.240.1/24 | fd42:24d6:6ce6:b0f2::1/64 |             | 1       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| ovn1            | ovn      | YES     | 10.50.215.1/24  | fd42:617a:48ee:2454::1/64 |             | 1       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+

root@p42:~# ovs-vsctl show
eb0d809a-cdf8-440a-8200-929412dee20a
    Bridge lxdovn1
        Port lxdovn1
            Interface lxdovn1
                type: internal
        Port lxdovn1b
            Interface lxdovn1b
        Port patch-lxd-net2-ls-ext-lsp-provider-to-br-int
            Interface patch-lxd-net2-ls-ext-lsp-provider-to-br-int
                type: patch
                options: {peer=patch-br-int-to-lxd-net2-ls-ext-lsp-provider}
    Bridge br-int
        fail_mode: secure
        Port patch-br-int-to-lxd-net2-ls-ext-lsp-provider
            Interface patch-br-int-to-lxd-net2-ls-ext-lsp-provider
                type: patch
                options: {peer=patch-lxd-net2-ls-ext-lsp-provider-to-br-int}
        Port br-int
            Interface br-int
                type: internal
        Port patch-br-int-to-lxd-net4-ls-ext-lsp-provider
            Interface patch-br-int-to-lxd-net4-ls-ext-lsp-provider
                type: patch
                options: {peer=patch-lxd-net4-ls-ext-lsp-provider-to-br-int}
    Bridge lxdovn3
        Port lxdovn3b
            Interface lxdovn3b
        Port lxdovn3
            Interface lxdovn3
                type: internal
        Port patch-lxd-net4-ls-ext-lsp-provider-to-br-int
            Interface patch-lxd-net4-ls-ext-lsp-provider-to-br-int
                type: patch
                options: {peer=patch-br-int-to-lxd-net4-ls-ext-lsp-provider}
    ovs_version: "2.13.0"

root@p42:~# ovn-nbctl show
switch c8f82ec8-454b-49e3-9197-9b8218e131fe (net0)
    port port1
        addresses: ["dynamic"]
switch 22676f1d-fe14-4dec-a730-41cd1c9cdeb2 (lxd-net4-ls-ext)
    port lxd-net4-ls-ext-lsp-router
        type: router
        router-port: lxd-net4-lr-lrp-ext
    port lxd-net4-ls-ext-lsp-provider
        type: localnet
        addresses: ["unknown"]
switch d4e4c4a2-139f-4625-b473-2a3e35bed875 (lxd-net2-ls-int)
    port lxd-net2-ls-int-lsp-router
        type: router
        router-port: lxd-net2-lr-lrp-int
switch 43657317-d1aa-4af0-b378-2b8cce8541ae (lxd-net2-ls-ext)
    port lxd-net2-ls-ext-lsp-router
        type: router
        router-port: lxd-net2-lr-lrp-ext
    port lxd-net2-ls-ext-lsp-provider
        type: localnet
        addresses: ["unknown"]
switch 9223579f-fd16-42dc-bfb1-f79caf9bcdfe (lxd-net4-ls-int)
    port lxd-net4-ls-int-lsp-router
        type: router
        router-port: lxd-net4-lr-lrp-int
router 3e096a94-9d8a-4667-8b4c-8775e9c0a1a4 (lxd-net2-lr)
    port lxd-net2-lr-lrp-ext
        mac: "00:16:3e:0d:ac:29"
        networks: ["10.42.10.5/24", "fd42:5bf8:7fd5:119a:216:3eff:fe0d:ac29/64"]
    port lxd-net2-lr-lrp-int
        mac: "00:16:3e:0d:ac:29"
        networks: ["10.113.240.1/24", "fd42:24d6:6ce6:b0f2::1/64"]
    nat 06946f37-b934-47c4-8550-705c39cc2fe6
        external ip: "fd42:5bf8:7fd5:119a:216:3eff:fe0d:ac29"
        logical ip: "fd42:24d6:6ce6:b0f2::/64"
        type: "snat"
    nat 8500c690-7239-4896-92d6-e78c41745818
        external ip: "10.42.10.5"
        logical ip: "10.113.240.0/24"
        type: "snat"
router b202cf2f-8c40-4fcb-b5cf-a34008e998f9 (lxd-net4-lr)
    port lxd-net4-lr-lrp-ext
        mac: "00:16:3e:f4:76:04"
        networks: ["10.42.11.201/24", "fd42:23a3:d094:170d:216:3eff:fef4:7604/64"]
    port lxd-net4-lr-lrp-int
        mac: "00:16:3e:f4:76:04"
        networks: ["10.50.215.1/24", "fd42:617a:48ee:2454::1/64"]
    nat 58fde284-2601-45e5-b6c7-315058ed44b9
        external ip: "10.42.11.201"
        logical ip: "10.50.215.0/24"
        type: "snat"
    nat ae739baf-b210-4874-ac55-b99142e5ba07
        external ip: "fd42:23a3:d094:170d:216:3eff:fef4:7604"
        logical ip: "fd42:617a:48ee:2454::/64"
        type: "snat

root@p42:~# ovs-vsctl list open_vswitch
_uuid               : eb0d809a-cdf8-440a-8200-929412dee20a
bridges             : [4d480b5a-ad40-401b-a13f-e031d1aac443, 6ca35c00-536f-4924-8ef6-98b4893427cc, f0d1f553-634f-4cfb-ad08-95c184b9324c]
cur_cfg             : 9
datapath_types      : [netdev, system]
datapaths           : {}
db_version          : "8.2.0"
dpdk_initialized    : false
dpdk_version        : none
external_ids        : {hostname=p42, ovn-bridge-mappings="lxdbr0:lxdovn1,lxdbr1:lxdovn3", ovn-encap-ip="10.55.0.42", ovn-encap-type=geneve, ovn-remote="tcp:10.55.0.42:6642", rundir="/var/run/openvswitch", system-id="d3002a31-8cd5-44ee-8e08-271cba3ca0c0"}
iface_types         : [erspan, geneve, gre, internal, ip6erspan, ip6gre, lisp, patch, stt, system, tap, vxlan]
manager_options     : []
next_cfg            : 9
other_config        : {}
ovs_version         : "2.13.0"
ssl                 : []
statistics          : {}
system_type         : ubuntu
system_version      : "20.04"

I have also tried the following:


sudo ovs-vsctl set open_vswitch .   external_ids:ovn-remote=unix:/var/run/ovn/ovnsb_db.sock   external_ids:ovn-encap-type=geneve   external_ids:ovn-encap-ip=127.0.0.1

lxc network set lxdbr1 ipv4.dhcp.ranges=10.42.11.5-10.42.11.200 ipv4.ovn.ranges=10.42.11.201-10.42.11.250

and the logs

root@p42:/var/log/openvswitch# ls
ovsdb-server.log  ovs-vswitchd.log
root@p42:/var/log/openvswitch# cat ovsdb-server.log 
2020-12-21T14:51:45.740Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovsdb-server.log
2020-12-21T14:51:45.765Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.13.0
2020-12-21T14:51:55.774Z|00003|memory|INFO|4732 kB peak resident set size after 10.0 seconds
2020-12-21T14:51:55.774Z|00004|memory|INFO|cells:39 monitors:3 sessions:2
2020-12-21T14:58:35.207Z|00005|jsonrpc|WARN|unix#10: receive error: Connection reset by peer
2020-12-21T14:58:35.207Z|00006|reconnect|WARN|unix#10: connection dropped (Connection reset by peer)
2020-12-21T15:05:40.042Z|00007|jsonrpc|WARN|unix#23: send error: Broken pipe
2020-12-21T15:05:40.042Z|00008|reconnect|WARN|unix#23: connection dropped (Broken pipe)
2020-12-21T15:11:37.412Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovsdb-server.log
2020-12-21T15:11:37.438Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.13.0
2020-12-21T15:11:47.447Z|00003|memory|INFO|4640 kB peak resident set size after 10.0 seconds
2020-12-21T15:11:47.447Z|00004|memory|INFO|cells:382 monitors:3 sessions:2
2020-12-21T15:19:09.761Z|00005|jsonrpc|WARN|unix#21: send error: Broken pipe
2020-12-21T15:19:09.761Z|00006|reconnect|WARN|unix#21: connection dropped (Broken pipe)
root@p42:/var/log/openvswitch# cat ovs-vswitchd.log 
2020-12-21T14:51:45.944Z|00002|vlog|INFO|opened log file /var/log/openvswitch/ovs-vswitchd.log
2020-12-21T14:51:45.955Z|00003|ovs_numa|INFO|Discovered 2 CPU cores on NUMA node 0
2020-12-21T14:51:45.955Z|00004|ovs_numa|INFO|Discovered 1 NUMA nodes and 2 CPU cores
2020-12-21T14:51:45.955Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connecting...
2020-12-21T14:51:45.955Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected
2020-12-21T14:51:45.966Z|00007|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.13.0
2020-12-21T14:54:50.338Z|00008|memory|INFO|13144 kB peak resident set size after 184.4 seconds
2020-12-21T14:54:50.344Z|00009|ofproto_dpif|INFO|system@ovs-system: Datapath supports recirculation
2020-12-21T14:54:50.344Z|00010|ofproto_dpif|INFO|system@ovs-system: VLAN header stack length probed as 2
2020-12-21T14:54:50.344Z|00011|ofproto_dpif|INFO|system@ovs-system: MPLS label stack length probed as 1
2020-12-21T14:54:50.344Z|00012|ofproto_dpif|INFO|system@ovs-system: Datapath supports truncate action
2020-12-21T14:54:50.344Z|00013|ofproto_dpif|INFO|system@ovs-system: Datapath supports unique flow ids
2020-12-21T14:54:50.344Z|00014|ofproto_dpif|INFO|system@ovs-system: Datapath supports clone action
2020-12-21T14:54:50.345Z|00015|ofproto_dpif|INFO|system@ovs-system: Max sample nesting level probed as 10
2020-12-21T14:54:50.345Z|00016|ofproto_dpif|INFO|system@ovs-system: Datapath supports eventmask in conntrack action
2020-12-21T14:54:50.345Z|00017|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_clear action
2020-12-21T14:54:50.345Z|00018|ofproto_dpif|INFO|system@ovs-system: Max dp_hash algorithm probed to be 0
2020-12-21T14:54:50.345Z|00019|ofproto_dpif|INFO|system@ovs-system: Datapath supports check_pkt_len action
2020-12-21T14:54:50.345Z|00020|ofproto_dpif|INFO|system@ovs-system: Datapath supports timeout policy in conntrack action
2020-12-21T14:54:50.345Z|00021|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_state
2020-12-21T14:54:50.345Z|00022|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_zone
2020-12-21T14:54:50.345Z|00023|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_mark
2020-12-21T14:54:50.345Z|00024|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_label
2020-12-21T14:54:50.345Z|00025|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_state_nat
2020-12-21T14:54:50.345Z|00026|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_orig_tuple
2020-12-21T14:54:50.345Z|00027|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_orig_tuple6
2020-12-21T14:54:50.345Z|00028|ofproto_dpif|INFO|system@ovs-system: Datapath does not support IPv6 ND Extensions
2020-12-21T14:54:50.394Z|00029|bridge|INFO|bridge br-int: added interface br-int on port 65534
2020-12-21T14:54:50.395Z|00030|bridge|INFO|bridge br-int: using datapath ID 0000025ca36c2449
2020-12-21T14:54:50.395Z|00031|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
2020-12-21T14:55:00.339Z|00032|memory|INFO|peak resident set size grew 189% in last 10.0 seconds, from 13144 kB to 37956 kB
2020-12-21T14:55:00.339Z|00033|memory|INFO|handlers:1 ofconns:2 ports:1 revalidators:1 rules:9
2020-12-21T14:55:01.346Z|00034|connmgr|INFO|br-int<->unix#0: 6 flow_mods 10 s ago (5 adds, 1 deletes)
2020-12-21T14:58:35.858Z|00035|bridge|INFO|bridge lxdovn1: added interface lxdovn1 on port 65534
2020-12-21T14:58:35.858Z|00036|bridge|INFO|bridge lxdovn1: using datapath ID 00005a0b484d1b40
2020-12-21T14:58:35.858Z|00037|connmgr|INFO|lxdovn1: added service controller "punix:/var/run/openvswitch/lxdovn1.mgmt"
2020-12-21T14:58:35.881Z|00038|bridge|INFO|bridge lxdovn1: added interface lxdovn1b on port 1
2020-12-21T14:58:35.920Z|00039|bridge|INFO|bridge lxdovn1: added interface patch-lxd-net2-ls-ext-lsp-provider-to-br-int on port 2
2020-12-21T14:58:35.921Z|00040|bridge|INFO|bridge br-int: added interface patch-br-int-to-lxd-net2-ls-ext-lsp-provider on port 1
2020-12-21T14:58:45.407Z|00041|connmgr|INFO|br-int<->unix#0: 627 flow_mods in the 1 s starting 10 s ago (625 adds, 2 modifications)
2020-12-21T15:11:28.431Z|00042|bridge|INFO|bridge lxdovn1: deleted interface lxdovn1 on port 65534
2020-12-21T15:11:28.431Z|00043|bridge|INFO|bridge lxdovn1: deleted interface lxdovn1b on port 1
2020-12-21T15:11:28.431Z|00044|bridge|INFO|bridge lxdovn1: deleted interface patch-lxd-net2-ls-ext-lsp-provider-to-br-int on port 2
2020-12-21T15:11:28.432Z|00045|bridge|INFO|bridge br-int: deleted interface br-int on port 65534
2020-12-21T15:11:28.432Z|00046|bridge|INFO|bridge br-int: deleted interface patch-br-int-to-lxd-net2-ls-ext-lsp-provider on port 1
2020-12-21T15:11:37.618Z|00002|vlog|INFO|opened log file /var/log/openvswitch/ovs-vswitchd.log
2020-12-21T15:11:37.637Z|00003|ovs_numa|INFO|Discovered 2 CPU cores on NUMA node 0
2020-12-21T15:11:37.637Z|00004|ovs_numa|INFO|Discovered 1 NUMA nodes and 2 CPU cores
2020-12-21T15:11:37.638Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connecting...
2020-12-21T15:11:37.638Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected
2020-12-21T15:11:37.650Z|00007|ofproto_dpif|INFO|system@ovs-system: Datapath supports recirculation
2020-12-21T15:11:37.650Z|00008|ofproto_dpif|INFO|system@ovs-system: VLAN header stack length probed as 2
2020-12-21T15:11:37.650Z|00009|ofproto_dpif|INFO|system@ovs-system: MPLS label stack length probed as 1
2020-12-21T15:11:37.650Z|00010|ofproto_dpif|INFO|system@ovs-system: Datapath supports truncate action
2020-12-21T15:11:37.650Z|00011|ofproto_dpif|INFO|system@ovs-system: Datapath supports unique flow ids
2020-12-21T15:11:37.650Z|00012|ofproto_dpif|INFO|system@ovs-system: Datapath supports clone action
2020-12-21T15:11:37.650Z|00013|ofproto_dpif|INFO|system@ovs-system: Max sample nesting level probed as 10
2020-12-21T15:11:37.650Z|00014|ofproto_dpif|INFO|system@ovs-system: Datapath supports eventmask in conntrack action
2020-12-21T15:11:37.650Z|00015|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_clear action
2020-12-21T15:11:37.650Z|00016|ofproto_dpif|INFO|system@ovs-system: Max dp_hash algorithm probed to be 0
2020-12-21T15:11:37.650Z|00017|ofproto_dpif|INFO|system@ovs-system: Datapath supports check_pkt_len action
2020-12-21T15:11:37.650Z|00018|ofproto_dpif|INFO|system@ovs-system: Datapath supports timeout policy in conntrack action
2020-12-21T15:11:37.650Z|00019|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_state
2020-12-21T15:11:37.650Z|00020|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_zone
2020-12-21T15:11:37.650Z|00021|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_mark
2020-12-21T15:11:37.650Z|00022|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_label
2020-12-21T15:11:37.650Z|00023|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_state_nat
2020-12-21T15:11:37.650Z|00024|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_orig_tuple
2020-12-21T15:11:37.650Z|00025|ofproto_dpif|INFO|system@ovs-system: Datapath supports ct_orig_tuple6
2020-12-21T15:11:37.650Z|00026|ofproto_dpif|INFO|system@ovs-system: Datapath does not support IPv6 ND Extensions
2020-12-21T15:11:37.711Z|00027|dpif|WARN|system@ovs-system: failed to query port patch-lxd-net2-ls-ext-lsp-provider-to-br-int: Invalid argument
2020-12-21T15:11:37.712Z|00028|dpif|WARN|system@ovs-system: failed to query port patch-br-int-to-lxd-net2-ls-ext-lsp-provider: Invalid argument
2020-12-21T15:11:37.715Z|00029|bridge|INFO|bridge lxdovn1: added interface lxdovn1 on port 65534
2020-12-21T15:11:37.722Z|00030|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:37.722Z|00031|bridge|INFO|bridge lxdovn1: added interface patch-lxd-net2-ls-ext-lsp-provider-to-br-int on port 2
2020-12-21T15:11:37.726Z|00032|bridge|INFO|bridge br-int: added interface br-int on port 65534
2020-12-21T15:11:37.726Z|00033|bridge|INFO|bridge br-int: added interface patch-br-int-to-lxd-net2-ls-ext-lsp-provider on port 1
2020-12-21T15:11:37.726Z|00034|bridge|INFO|bridge lxdovn1: using datapath ID 00005a0b484d1b40
2020-12-21T15:11:37.726Z|00035|connmgr|INFO|lxdovn1: added service controller "punix:/var/run/openvswitch/lxdovn1.mgmt"
2020-12-21T15:11:37.728Z|00036|bridge|INFO|bridge br-int: using datapath ID 0000025ca36c2449
2020-12-21T15:11:37.728Z|00037|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
2020-12-21T15:11:37.746Z|00038|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:37.748Z|00039|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.13.0
2020-12-21T15:11:40.987Z|00040|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:41.001Z|00041|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:41.012Z|00042|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:41.145Z|00043|bridge|INFO|bridge lxdovn1: deleted interface patch-lxd-net2-ls-ext-lsp-provider-to-br-int on port 2
2020-12-21T15:11:41.145Z|00044|bridge|INFO|bridge br-int: deleted interface patch-br-int-to-lxd-net2-ls-ext-lsp-provider on port 1
2020-12-21T15:11:41.167Z|00045|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:41.167Z|00046|dpif|WARN|Dropped 2 log messages in last 3 seconds (most recently, 3 seconds ago) due to excessive rate
2020-12-21T15:11:41.167Z|00047|dpif|WARN|system@ovs-system: failed to query port patch-lxd-net2-ls-ext-lsp-provider-to-br-int: Invalid argument
2020-12-21T15:11:41.167Z|00048|dpif|WARN|system@ovs-system: failed to query port patch-br-int-to-lxd-net2-ls-ext-lsp-provider: Invalid argument
2020-12-21T15:11:42.123Z|00049|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:42.123Z|00050|bridge|INFO|bridge lxdovn1: added interface patch-lxd-net2-ls-ext-lsp-provider-to-br-int on port 1
2020-12-21T15:11:42.123Z|00051|bridge|INFO|bridge br-int: added interface patch-br-int-to-lxd-net2-ls-ext-lsp-provider on port 2
2020-12-21T15:11:42.261Z|00052|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:42.337Z|00053|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:42.352Z|00054|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:42.411Z|00055|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:42.421Z|00056|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:42.477Z|00057|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:42.613Z|00058|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:43.271Z|00059|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:43.501Z|00060|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:43.527Z|00061|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:11:47.749Z|00062|memory|INFO|38480 kB peak resident set size after 10.1 seconds
2020-12-21T15:11:47.749Z|00063|memory|INFO|handlers:1 ofconns:2 ports:4 revalidators:1 rules:332
2020-12-21T15:11:51.142Z|00064|connmgr|INFO|br-int<->unix#0: 636 flow_mods in the 1 s starting 10 s ago (633 adds, 1 deletes, 2 modifications)
2020-12-21T15:11:58.683Z|00065|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:12:36.920Z|00066|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:12:36.936Z|00067|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:12:36.942Z|00068|bridge|WARN|could not open network device lxdovn1b (No such device)
2020-12-21T15:12:37.673Z|00069|bridge|INFO|bridge lxdovn1: added interface lxdovn1b on port 3
2020-12-21T15:12:50.391Z|00070|connmgr|INFO|br-int<->unix#0: 6 flow_mods 12 s ago (6 adds)
2020-12-21T15:19:09.839Z|00071|bridge|INFO|bridge lxdovn3: added interface lxdovn3 on port 65534
2020-12-21T15:19:09.839Z|00072|bridge|INFO|bridge lxdovn3: using datapath ID 000052f5d1f0fb4c
2020-12-21T15:19:09.840Z|00073|connmgr|INFO|lxdovn3: added service controller "punix:/var/run/openvswitch/lxdovn3.mgmt"
2020-12-21T15:19:09.903Z|00074|bridge|INFO|bridge lxdovn3: added interface lxdovn3b on port 1
2020-12-21T15:19:09.952Z|00075|bridge|INFO|bridge lxdovn3: added interface patch-lxd-net4-ls-ext-lsp-provider-to-br-int on port 2
2020-12-21T15:19:09.952Z|00076|bridge|INFO|bridge br-int: added interface patch-br-int-to-lxd-net4-ls-ext-lsp-provider on port 3
2020-12-21T15:19:19.822Z|00077|connmgr|INFO|br-int<->unix#0: 622 flow_mods in the 1 s starting 10 s ago (620 adds, 2 modifications)
root@p42:/var/log/openvswitch#
2

Quick update, I realised the geneve kernel module wasn’t loaded so I’ve loaded that and rebooted.
Will test again…

1 Like
3

The LXD docs contain an example of how to setup a minimal local-only OVN network, it would be worth confirming that works before adding LAN geneve tunnels.

1 Like
4

OK I just tried that on another fresh install, this time on my desktop with more resources and the same thing happens. I don’t get any bridge created so the container fails to start

root@jon-linux-mint-pc:/home/jon# lxc storage create zfs1 zfs source=/dev/sda2
Storage pool zfs1 created
root@jon-linux-mint-pc:/home/jon# lxc network set lxdbr0 ipv4.dhcp.ranges=10.12.202.5-10.12.202.100 ipv4.ovn.ranges=10.12.202.220-10.12.202.252
root@jon-linux-mint-pc:/home/jon# lxc network create ovntest --type=ovn network=lxdbr0
Network ovntest created
root@jon-linux-mint-pc:/home/jon# lxc list
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+
root@jon-linux-mint-pc:/home/jon# lxc network list
+---------+----------+---------+-----------------+---------------------------+-------------+---------+
|  NAME   |   TYPE   | MANAGED |      IPV4       |           IPV6            | DESCRIPTION | USED BY |
+---------+----------+---------+-----------------+---------------------------+-------------+---------+
| br-int  | bridge   | NO      |                 |                           |             | 0       |
+---------+----------+---------+-----------------+---------------------------+-------------+---------+
| enp3s0  | physical | NO      |                 |                           |             | 0       |
+---------+----------+---------+-----------------+---------------------------+-------------+---------+
| lxdbr0  | bridge   | YES     | 10.12.202.1/24  | fd42:f90e:5774:aaf1::1/64 |             | 1       |
+---------+----------+---------+-----------------+---------------------------+-------------+---------+
| lxdovn1 | bridge   | NO      |                 |                           |             | 0       |
+---------+----------+---------+-----------------+---------------------------+-------------+---------+
| ovntest | ovn      | YES     | 10.140.213.1/24 | fd42:497:403d:ac57::1/64  |             | 0       |
+---------+----------+---------+-----------------+---------------------------+-------------+---------+
| wlp6s0  | physical | NO      |                 |                           |             | 0       |
+---------+----------+---------+-----------------+---------------------------+-------------+---------+
root@jon-linux-mint-pc:/home/jon# ip -br a
lo               UNKNOWN        127.0.0.1/8 ::1/128 
enp3s0           DOWN           
wlp6s0           UP             10.12.10.55/24 fd9f:3eca:664a:1:1126:44c9:1898:ac09/64 fd9f:3eca:664a:1:a8a2:a2ff:34e4:5c53/64 fe80::6d58:d50c:3764:2a80/64 
ovs-system       DOWN           
br-int           DOWN           
lxdbr0           UP             10.12.202.1/24 fd42:f90e:5774:aaf1::1/64 fe80::216:3eff:fe01:af6e/64 
lxdovn1b@lxdovn1a UP             
lxdovn1a@lxdovn1b UP             
lxdovn1          DOWN           
root@jon-linux-mint-pc:/home/jon# lxc launch ubuntu:20.04 mattersmost -s zfs1 -n ovntest
Creating mattersmost
Starting mattersmost                        
Error: Failed preparing container for start: Failed to start device "ovntest": Parent device 'ovntest' doesn't exist
Try `lxc info --show-log local:mattersmost` for more info
5

What’s in your container’s expanded config?
OVN networks don’t show up as traditional bridges, so if you’re using nictype=bridged this won’t work.

Instead you should attach the nic using the network= key.

1 Like
6

OK thanks, looks like I’ve got further now, actually got it to start, the old nic was macvlan? :

     lxc config device add mattersmost eth1 nic network=ovntest
     lxc config device show mattersmost

     lxc config device remove mattersmost ovntest    # removed the old macvlan default nic
     lxc start mattersmost

root@jon-linux-mint-pc:/home/jon# lxc list
+-------------+---------+------+----------------------------------------------+-----------+-----------+
|    NAME     |  STATE  | IPV4 |                     IPV6                     |   TYPE    | SNAPSHOTS |
+-------------+---------+------+----------------------------------------------+-----------+-----------+
| mattersmost | RUNNING |      | fd42:497:403d:ac57:216:3eff:fef5:f662 (eth1) | CONTAINER | 0         |
+-------------+---------+------+----------------------------------------------+-----------+-----------+

Looks like possibly I have to enable the dhcp service for ipv4 as current its not getting any address.

Cheers,
Jon.

7

it would help if I also enabled dhcp client on eth1 !!
Seems to have an address now.
All good. just need to tinker and test things.
Cheers!
Jon. :desktop_computer: :blush:

8

One thing I noticed, showing the details of ovntest. I can see the MTU is 1500 and the nic of the lxd container is also 1500. Should this not be set slightly lower by default to compensate for any extra overhead used by Geneve encapsulation?
This is assuming running on standard 1500 byte size networks and not having the luxury of jumbo frames.

I’m pretty sure the kube overlays automatically set lower MTU’s for things like weave etc to compensate.

Cheers!
Jon.

9

LXD will calculate a suitable MTU based on the MTU of the device carrying the geneve traffic.

If you’re dealing with a single node, you don’t have such an interface so I would expect it to use 1500.

1 Like
10

We calculate the MTU to use based on the MTU of the interface that owns the ovn-encap-ip setting (and we take into account the IP version used for ovn-encap-ip as well).

1 Like