Lxc forward and nat address can not be the same

LXD
#1

Hi,
I want to set same ip for lxc forward and ipv4.nat.address?
But when is set in vm I can not see any port outside but ping is ok.

#2

Please can you show the config you are using?

#3 
config:
  bridge.mtu: "1442"
  ipv4.address: 192.168.5.1/24
  ipv4.nat: "true"
  ipv4.nat.address: XX.XX.61.138
  ipv6.address: fd42:d745:d286:a33c::1/64
  ipv6.nat: "true"
  network: UPLINK
  volatile.network.ipv4.address: 10.0.0.110
description: ""
name: *****
type: ovn
used_by:
- /1.0/instances/**
managed: true



lxc network forward create ** XX.XX.61.138 target_address=192.168.5.2
#4

when remove one of them 1) forward or 2) ipv4.nat.address wget google.com respond.

#5

Interesting, I’ll try and recreate to see what is going on.

1 Like
#7

OK so I’ve gotten round to trying this now, and it seems to work on my system (using Ubuntu Jammy):

I’m using a physical network as the uplink but reusing the lxdbr0 as the uplink interface.
I’ve assigned a single IP from the lxdbr0 subnet for use ipv4.routes on the uplink network:

lxc network show uplink
config:
  ipv4.gateway: 10.21.203.1/24
  ipv4.ovn.ranges: 10.21.203.21-10.21.203.30
  ipv4.routes: 10.21.203.12/32
  ovn.ingress_mode: routed
  parent: lxdbr0
  volatile.last_state.created: "false"
description: ""
name: uplink
type: physical

I created an OVN network ontop of that:

lxc network create ovn1 --type=ovn network=uplink

Then set the NAT address:

lxc network set ovn1 ipv4.nat.address=10.21.203.12
lxc network show ovn1
config:
  bridge.mtu: "1500"
  ipv4.address: 10.25.187.1/24
  ipv4.nat: "true"
  ipv4.nat.address: 10.21.203.12
  ipv6.address: fd42:1a9d:35b8:690e::1/64
  ipv6.nat: "true"
  network: uplink
  volatile.network.ipv4.address: 10.21.203.21
description: ""
name: ovn1
type: ovn

Then attached a new instance to the OVN network:

lxc launch images:ubuntu/jammy c1 -n ovn1
lxc ls
+--------+---------+--------------------+-----------------------------------------------+-----------+-----------+
|  NAME  |  STATE  |        IPV4        |                     IPV6                      |   TYPE    | SNAPSHOTS |
+--------+---------+--------------------+-----------------------------------------------+-----------+-----------+
| c1     | RUNNING | 10.25.187.2 (eth0) | fd42:1a9d:35b8:690e:216:3eff:fe37:7ab5 (eth0) | CONTAINER | 0         |
+--------+---------+--------------------+-----------------------------------------------+-----------+-----------+
lxc exec c1 -- ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=7.57 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=8.22 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=119 time=7.16 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=119 time=6.94 ms

Then to check NAT is working, ran tcpdump -i lxdbr0 -nn host 10.21.203.12:

sudo tcpdump -i lxdbr0 -nn host 10.21.203.12
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on lxdbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:50:37.220141 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 34945, seq 1, length 64
16:50:37.225976 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 34945, seq 1, length 64
16:50:38.221402 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 34945, seq 2, length 64
16:50:38.227926 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 34945, seq 2, length 64
16:50:39.223341 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 34945, seq 3, length 64
16:50:39.229147 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 34945, seq 3, length 64
16:50:40.224471 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 34945, seq 4, length 64
16:50:40.230682 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 34945, seq 4, length 64

So NAT is working as expected.

Now lets add the forward and check NAT still working:

lxc network forward create ovn1 10.21.203.12 target_address=10.25.187.2
lxc exec c1 -- ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=7.57 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=8.22 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=119 time=7.16 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=119 time=6.94 ms

sudo tcpdump -i lxdbr0 -nn host 10.21.203.12
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
16:51:43.653895 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 5757, seq 1, length 64
16:51:43.659671 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 5757, seq 1, length 64
16:51:44.655699 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 5757, seq 2, length 64
16:51:44.662518 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 5757, seq 2, length 64
16:51:45.656124 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 5757, seq 3, length 64
16:51:45.663108 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 5757, seq 3, length 64
16:51:46.657567 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 5757, seq 4, length 64
16:51:46.664352 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 5757, seq 4, length 64

Lets check forward is working:

From uplink network:

ping 10.21.203.12
PING 10.21.203.12 (10.21.203.12) 56(84) bytes of data.
64 bytes from 10.21.203.12: icmp_seq=1 ttl=63 time=3.74 ms
64 bytes from 10.21.203.12: icmp_seq=2 ttl=63 time=0.405 ms
64 bytes from 10.21.203.12: icmp_seq=3 ttl=63 time=0.228 ms

Check TCP works:

lxc exec c1 -- apt install nginx
curl http://10.21.203.12
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...

Lets check that another instance connected to the OVN network has its NAT working OK:

lxc launch images:ubuntu/jammy c2 -n ovn1
lxc exec c2 -- ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=7.04 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=7.12 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=119 time=5.81 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=119 time=5.98 ms

sudo tcpdump -i lxdbr0 -nn host 10.21.203.12
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
16:54:37.250531 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 2498, seq 1, length 64
16:54:37.256446 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 2498, seq 1, length 64
16:54:38.251913 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 2498, seq 2, length 64
16:54:38.257987 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 2498, seq 2, length 64
16:54:39.252461 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 2498, seq 3, length 64
16:54:39.258083 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 2498, seq 3, length 64
16:54:40.253818 IP 10.21.203.12 > 8.8.8.8: ICMP echo request, id 2498, seq 4, length 64
16:54:40.259637 IP 8.8.8.8 > 10.21.203.12: ICMP echo reply, id 2498, seq 4, length 64

So seems to work OK using OVN version:

ovn-nbctl --version
ovn-nbctl 22.03.0
Open vSwitch Library 2.17.0
DB Schema 6.1.0
1 Like
#8

I am using ubuntu 20.04 is this reason?

and may you test lxc exec c1 – wget google.com ?

#9

Is this a related bug to Broken TCP egress when same address is used for network NAT and forward (OVN) · Issue #9765 · lxc/lxd · GitHub

My lxd version is 5.5

#10

I switched to ubuntu 22.04 and everything is ok.

1 Like