Have you checked outbound NAT rules on your host?
Yes, there nothing for the mentioned ip alias or the host.
Please can you show me the iptables (or nftables) NAT rules, also please can you show a tcpdump running on the host’s external interface showing the packets leaving with the incorrect source IP.
Thanks
Tom
offcourse,
sudo iptables -L -v -n -t nat
Chain PREROUTING (policy ACCEPT 967 packets, 104K bytes)
pkts bytes target prot opt in out source destination
985 104K PREROUTING_direct all -- * * 0.0.0.0/0 0.0.0.0/0
985 104K PREROUTING_ZONES_SOURCE all -- * * 0.0.0.0/0 0.0.0.0/0
985 104K PREROUTING_ZONES all -- * * 0.0.0.0/0 0.0.0.0/0
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 1996 packets, 443K bytes)
pkts bytes target prot opt in out source destination
2034 450K OUTPUT_direct all -- * * 0.0.0.0/0 0.0.0.0/0
Chain POSTROUTING (policy ACCEPT 1996 packets, 443K bytes)
pkts bytes target prot opt in out source destination
2034 450K POSTROUTING_direct all -- * * 0.0.0.0/0 0.0.0.0/0
2034 450K POSTROUTING_ZONES_SOURCE all -- * * 0.0.0.0/0 0.0.0.0/0
2034 450K POSTROUTING_ZONES all -- * * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT_direct (1 references)
pkts bytes target prot opt in out source destination
Chain POSTROUTING_ZONES (1 references)
pkts bytes target prot opt in out source destination
0 0 POST_public all -- * vlan0 0.0.0.0/0 0.0.0.0/0 [goto]
1937 433K POST_public all -- * br0 0.0.0.0/0 0.0.0.0/0 [goto]
1 40 POST_public all -- * eth0 0.0.0.0/0 0.0.0.0/0 [goto]
93 16374 POST_public all -- * + 0.0.0.0/0 0.0.0.0/0 [goto]
Chain POSTROUTING_ZONES_SOURCE (1 references)
pkts bytes target prot opt in out source destination
Chain POSTROUTING_direct (1 references)
pkts bytes target prot opt in out source destination
Chain POST_external (0 references)
pkts bytes target prot opt in out source destination
1 108 POST_external_log all -- * * 0.0.0.0/0 0.0.0.0/0
1 108 POST_external_deny all -- * * 0.0.0.0/0 0.0.0.0/0
1 108 POST_external_allow all -- * * 0.0.0.0/0 0.0.0.0/0
Chain POST_external_allow (1 references)
pkts bytes target prot opt in out source destination
1 108 MASQUERADE all -- * !lo 0.0.0.0/0 0.0.0.0/0
Chain POST_external_deny (1 references)
pkts bytes target prot opt in out source destination
Chain POST_external_log (1 references)
pkts bytes target prot opt in out source destination
Chain POST_public (4 references)
pkts bytes target prot opt in out source destination
2033 449K POST_public_log all -- * * 0.0.0.0/0 0.0.0.0/0
2033 449K POST_public_deny all -- * * 0.0.0.0/0 0.0.0.0/0
2033 449K POST_public_allow all -- * * 0.0.0.0/0 0.0.0.0/0
Chain POST_public_allow (1 references)
pkts bytes target prot opt in out source destination
Chain POST_public_deny (1 references)
pkts bytes target prot opt in out source destination
Chain POST_public_log (1 references)
pkts bytes target prot opt in out source destination
Chain PREROUTING_ZONES (1 references)
pkts bytes target prot opt in out source destination
0 0 PRE_public all -- vlan0 * 0.0.0.0/0 0.0.0.0/0 [goto]
967 104K PRE_public all -- br0 * 0.0.0.0/0 0.0.0.0/0 [goto]
18 576 PRE_public all -- eth0 * 0.0.0.0/0 0.0.0.0/0 [goto]
0 0 PRE_public all -- + * 0.0.0.0/0 0.0.0.0/0 [goto]
Chain PREROUTING_ZONES_SOURCE (1 references)
pkts bytes target prot opt in out source destination
Chain PREROUTING_direct (1 references)
pkts bytes target prot opt in out source destination
Chain PRE_external (0 references)
pkts bytes target prot opt in out source destination
0 0 PRE_external_log all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 PRE_external_deny all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 PRE_external_allow all -- * * 0.0.0.0/0 0.0.0.0/0
Chain PRE_external_allow (1 references)
pkts bytes target prot opt in out source destination
Chain PRE_external_deny (1 references)
pkts bytes target prot opt in out source destination
Chain PRE_external_log (1 references)
pkts bytes target prot opt in out source destination
Chain PRE_public (4 references)
pkts bytes target prot opt in out source destination
985 104K PRE_public_log all -- * * 0.0.0.0/0 0.0.0.0/0
985 104K PRE_public_deny all -- * * 0.0.0.0/0 0.0.0.0/0
985 104K PRE_public_allow all -- * * 0.0.0.0/0 0.0.0.0/0
Chain PRE_public_allow (1 references)
pkts bytes target prot opt in out source destination
Chain PRE_public_deny (1 references)
pkts bytes target prot opt in out source destination
Chain PRE_public_log (1 references)
pkts bytes target prot opt in out source destination
am not using nftables. just firewalld, and iptables.
about:
please can you show a tcpdump running on the host’s external interface showing the packets leaving with the incorrect source IP.
apology, have no idea how to do that.
This line is the issue ^, it says to NAT all outbound traffic that doesn’t come from the local loopback interface to the host’s IP.
Brilliant 
now a container running on ipvlan, runs nginx dose it require proxy in order to allow inbound?
without proxy its not working.
No you shouldn’t need a proxy device as the container should have a public IP. If ping is working to/from the public IP to/from external destinations then you have connectivity, and its likely a firewall rule blocking inbound traffic somewhere.
ping already works, as the ip is an alias on the host side.
am digging more …
Can you show output of:
ip a on host
ip r on host
ip a in container
ip r in container
Thanks
ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 5000
link/ether MAC_ADD brd ff:ff:ff:ff:ff:ff
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether MAC_ADD brd ff:ff:ff:ff:ff:ff
inet HOST_IP/26 brd HOST_BRD scope global br0
valid_lft forever preferred_lft forever
inet ALIAS_1/29 brd ALIAS_1_BRD scope global br0
valid_lft forever preferred_lft forever
inet ALIAS_2/29 brd ALIAS_2_BRD scope global secondary br0
4: testing: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether e2:02:33:e2:23:19 brd ff:ff:ff:ff:ff:ff
inet 11.11.11.1/24 scope global testing
valid_lft forever preferred_lft forever
inet6 fe80::e002:33ff:fee2:2319/64 scope link
valid_lft forever preferred_lft forever
ip r
default via HOST_BRD dev br0
11.11.11.0/24 dev testing proto kernel scope link src 11.11.11.1
HOST_BRD/26 dev br0 proto kernel scope link src HOST_IP
----------------------------------------------------------------------------------------------------
ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
16: eth0@if3: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether MAC_AD brd ff:ff:ff:ff:ff:ff
inet IP_ALIAS/32 brd 255.255.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 IPV6/64 scope link
valid_lft forever preferred_lft forever
ip r
default dev eth0
EDITE:
after reboot i can see this on host. in addition to the other results.
ALIAS_1 dev lo scope link
Is IP_ALIAS the same as ALIAS_1 or ALIAS_2? I.e is the IP alias you have inside the container also bound the host’s br0 interface?
IP_ALIAS is same as IP_ALIAS_1 .
by bound? not sure there. do i need to make the host ip as default gateway?
You shouldn’t have the public IP that you’re adding to the container existing on the host at all.
Otherwise there will be confusion as to where the packets should go when destined for that IP, should it go to the host or the container?
It should go to the container, i imagined i needed to add it on the host at first then make it available on the container. am going to remove it from host and try.
please check this edit:
That is normal, it is a static route of the public IP alias to the local loopback interface, it is added by LXD when using ipvlan, its only purpose is to activate the proxy ARP on the host so that your host responds to ARP requests for your public IP alias that you’re passing through into the container.
However it is important that the IP itself is not bound (setup) on the host.
Very informative.
I just removed from the host. / removed host bond for ALIAS_1 . still can not ping it at all now. or visit port 80.
Restart the container, and then gather output of following please:
On host:
ip a
ip r
ip neigh show proxy
And then inside the container;
ip a
ip r
Thanks
Tom
Hi @tomp
ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 5000
link/ether MAC_ADD brd ff:ff:ff:ff:ff:ff
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether MAC_ADD brd ff:ff:ff:ff:ff:ff
inet HOST_IP/26 brd HOST_BRD scope global br0
valid_lft forever preferred_lft forever
inet ALIAS_2/29 brd ALIAS_2_BRD scope global secondary br0
valid_lft forever preferred_lft forever
4: testing: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether e2:02:33:e2:23:19 brd ff:ff:ff:ff:ff:ff
inet 11.11.11.1/24 scope global testing
valid_lft forever preferred_lft forever
inet6 fe80::e002:33ff:fee2:2319/64 scope link
valid_lft forever preferred_lft forever
ip r
default via HOST_BRD dev br0
11.11.11.0/24 dev testing proto kernel scope link src 11.11.11.1
IP_ALIAS_SUBNET_ENDING_WITH0/29 dev br0 proto kernel scope link src IP_ALIAS_SUBNET_STARTING_WITH_0
ALIAS_1 dev lo scope link
HOST_BRD/26 dev br0 proto kernel scope link src HOST_IP
ip neigh show proxy
ALIAS_1 dev br0 proxy
NOTE:
IP_ALIAS_SUBNET_ENDING_WITH0/29
all added on ip aliases /32
_____________________________________________________________________________
ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
16: eth0@if3: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether MAC_AD brd ff:ff:ff:ff:ff:ff
inet IP_ALIAS/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 IPV6/64 scope link
valid_lft forever preferred_lft forever
ip r
default via IP_ALIAS_SUBNET_STARTING_WITH_0 dev eth0 proto static onlink
EDITE:
i can ping it from outside.
i can ping google.com from within the container => dns working.
nginx not accessible from outside.
ufw status
Status: inactive
Can you show output of netstat -tlpn in container running nginx please.
netstat -tlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 ALIAS_1:80 0.0.0.0:* LISTEN 247/nginx: master p
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 176/systemd-resolve
tcp6 0 0 :::80 :::* LISTEN 247/nginx: master p