I am not able to establish tcp connectivity from host towards the container, that resides on an ovn cluster

Incus
1

I am not able to establish tcp connectivity from host towards the container, that resides on an ovn cluster

I am using the method recommended here:

What is the best approach access from host to ovn containers, with a route toward the ovn gateway address

I have a cluster setup with 3+1 nodes:

incus network list
±----------±---------±--------±---------------±--------------------------±------------±--------±--------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY | STATE |
±----------±---------±--------±---------------±--------------------------±------------±--------±--------+
| UPLINK | physical | YES | | | | 1 | CREATED |
±----------±---------±--------±---------------±--------------------------±------------±--------±--------+
| br0 | bridge | NO | | | | 1 | |
±----------±---------±--------±---------------±--------------------------±------------±--------±--------+
| br-int | bridge | NO | | | | 0 | |
±----------±---------±--------±---------------±--------------------------±------------±--------±--------+
| ens3 | physical | NO | | | | 0 | |
±----------±---------±--------±---------------±--------------------------±------------±--------±--------+
| ens4 | physical | NO | | | | 0 | |
±----------±---------±--------±---------------±--------------------------±------------±--------±--------+
| ens5 | physical | NO | | | | 0 | |
±----------±---------±--------±---------------±--------------------------±------------±--------±--------+
| incusovn1 | bridge | NO | | | | 0 | |
±----------±---------±--------±---------------±--------------------------±------------±--------±--------+
| my-ovn | ovn | YES | 10.206.16.1/24 | fd42:dc1d:edde:fdb7::1/64 | | 2 | CREATED |
±----------±---------±--------±---------------±--------------------------±------------±--------±--------+

root@ubuntu:~# incus list
±-----±--------±-------------------±----------------------------------------------±----------±----------±---------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | LOCATION |
±-----±--------±-------------------±----------------------------------------------±----------±----------±---------+
| c1 | RUNNING | 10.206.16.3 (eth0) | fd42:dc1d:edde:fdb7:216:3eff:fe2a:12ae (eth0) | CONTAINER | 0 | ubuntu01 |
±-----±--------±-------------------±----------------------------------------------±----------±----------±---------+
| c11 | RUNNING | 10.206.16.2 (eth0) | fd42:dc1d:edde:fdb7:216:3eff:fe2d:d7e7 (eth0) | CONTAINER | 0 | ubuntu03 |
±-----±--------±-------------------±----------------------------------------------±----------±----------±---------+

root@ubuntu:~# incus network show my-ovn
config:
bridge.mtu: “1442”
ipv4.address: 10.206.16.1/24
ipv4.nat: “true”
ipv6.address: fd42:dc1d:edde:fdb7::1/64
ipv6.nat: “true”
network: UPLINK
volatile.network.ipv4.address: 10.255.239.140
description: “”
name: my-ovn
type: ovn
used_by:

  • /1.0/instances/c1
  • /1.0/instances/c11
    managed: true
    status: Created
    locations:
  • ubuntu02
  • ubuntu03
  • ubuntu04
  • ubuntu01

10.206.16.2 via 10.255.239.140 dev br0 << routes toward ovn gateway
10.206.16.3 via 10.255.239.140 dev br0

I can ping the containers:

ping 10.206.16.2
PING 10.206.16.2 (10.206.16.2) 56(84) bytes of data.
64 bytes from 10.206.16.2: icmp_seq=1 ttl=63 time=5.65 ms
64 bytes from 10.206.16.2: icmp_seq=2 ttl=63 time=2.25 ms

but no tcp:

Connection reset by 10.206.16.2 port 22

tcpdump output on containers, show the R flag sent:

14:13:23.143516 eth0 In IP 10.255.239.181.47456 > 10.206.16.2.22: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 3857057851 ecr 1692397751], length 41: SSH: SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.6
14:13:23.143567 eth0 Out IP 10.206.16.2.22 > 10.255.239.181.47456: Flags [.], ack 42, win 511, options [nop,nop,TS val 1692397757 ecr 3857057851], length 0
14:13:23.147105 eth0 In IP 10.255.239.181.47456 > 10.206.16.2.22: Flags [R], seq 4000848859, win 0, length 0
14:13:23.353320 eth0 In IP 10.255.239.181.47456 > 10.206.16.2.22: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 3857058063 ecr 1692397751], length 41: SSH: SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.6
14:13:23.353348 eth0 Out IP 10.206.16.2.22 > 10.255.239.181.47456: Flags [R], seq 2610967809, win 0, length 0 <<<<

Seems like there is a firewall, but no firewall present on the container also no ACL was set.

Traffic between containers works normally

2

Could be the NAT on that network causing issues?
That is, you connect to an IP which shouldn’t be visible externally due to NAT on your OVN network.

3

I eventually managed establish TCP connectivity, by using network forward, as you described here:

https://www.youtube.com/watch?v=B-Uzo9WldMs - very helpful.

Is there any other way to establish TCP connectivity from outside towards the container, except this forwarding mechanism?

4

As mentioned earlier, the issue you’re running into is most likely because of your use of NAT on the network. By directly routing to what should be a hidden network, you end up confusing the heck out of the virtual router as it’s seeing external packets headed for internal IPs but doesn’t find a matching NAT table entry for the traffic.

That’s not really an OVN specific problem so much as a network design problem.
You can indeed use something like a network forward or a load-balancer to expose internal IPs to the outside through their own or a shared external address.

The alternative is to turn off NAT on your network, which will then allow for those communications to go through just fine, but you’ll then need your router further down the line (outside of OVN) to understand how to route to the OVN networks and to handle NAT for you.