Hi, am trying to allow nat go through an ip alias , OR added on ip that my service provider allocated for me.
Afterwards created containers can not ping in/out. in other words all internet activity is blocked.
more informations, can be found on this thread.
the reason of opening this new thread, is that it has been 2 days and i got no help.
Please can you try with LXD 3.22 that has support for nftables in case that is the reason your iptables rules are not being matched.
Hi, am not using nftables, am using firewalld.
I’ve checked that ipv4.nat.address is working OK locally on LXD 3.21, here is my test plan:
Check current IP addresses of my ‘external’ interface (in this case the wifi adapter on my laptop):
ip -4 a show dev wlp2s0
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.1.128/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp2s0
valid_lft 85678sec preferred_lft 85678sec
Confirm default lxd network settings:
lxc network show lxdbr0
config:
ipv4.address: 10.96.212.1/24
ipv4.nat: "true"
ipv6.address: fd42:465d:fb7e:2ebd::1/64
ipv6.nat: "true"
Add IP alias to external interface:
ip a add 192.168.1.99/32 dev wlp2s0
Confirm added:
ip -4 a show dev wlp2s0
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
inet 192.168.1.128/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp2s0
valid_lft 85678sec preferred_lft 85678sec
inet 192.168.1.99/32 scope global wlp2s0
valid_lft forever preferred_lft forever
Check pingable from another host in same network segment:
home-lxc01:~# ping 192.168.1.99
PING 192.168.1.99 (192.168.1.99): 56 data bytes
64 bytes from 192.168.1.99: seq=0 ttl=64 time=3.003 ms
64 bytes from 192.168.1.99: seq=1 ttl=64 time=26.586 ms
64 bytes from 192.168.1.99: seq=2 ttl=64 time=50.038 ms
64 bytes from 192.168.1.99: seq=3 ttl=64 time=75.565 ms
^C
--- 192.168.1.99 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 3.003/38.798/75.565 ms
Change outbound NAT address for lxdbr0:
lxc network set lxdbr0 ipv4.nat.address 192.168.1.99
Check iptables rules added:
iptables -L -v -n -t nat
Chain PREROUTING (policy ACCEPT 16 packets, 3297 bytes)
pkts bytes target prot opt in out source destination
Chain INPUT (policy ACCEPT 14 packets, 3177 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 120 packets, 11119 bytes)
pkts bytes target prot opt in out source destination
Chain POSTROUTING (policy ACCEPT 109 packets, 9462 bytes)
pkts bytes target prot opt in out source destination
13 1777 SNAT all -- * * 10.96.212.0/24 !10.96.212.0/24 /* generated for LXD network lxdbr0 */ to:192.168.1.99
Setup tcpdump on external interface to check outbound packets from 192.168.1.99:
tcpdump -i wlp2s0 icmp and host 192.168.1.99 -nn
Launch new container:
lxc launch ubuntu:18.04 c1
Start an outbound ping inside c1 to 8.8.8.8 and observe tcpdump output:
lxc exec c1 -- ping 8.8.8.8 -c 5
09:37:42.213155 IP 192.168.1.99 > 8.8.8.8: ICMP echo request, id 325, seq 1, length 64
09:37:42.223199 IP 8.8.8.8 > 192.168.1.99: ICMP echo reply, id 325, seq 1, length 64
09:37:43.214394 IP 192.168.1.99 > 8.8.8.8: ICMP echo request, id 325, seq 2, length 64
09:37:43.223718 IP 8.8.8.8 > 192.168.1.99: ICMP echo reply, id 325, seq 2, length 64
09:37:44.215959 IP 192.168.1.99 > 8.8.8.8: ICMP echo request, id 325, seq 3, length 64
09:37:44.228288 IP 8.8.8.8 > 192.168.1.99: ICMP echo reply, id 325, seq 3, length 64
09:37:45.217489 IP 192.168.1.99 > 8.8.8.8: ICMP echo request, id 325, seq 4, length 64
09:37:45.229827 IP 8.8.8.8 > 192.168.1.99: ICMP echo reply, id 325, seq 4, length 64
09:37:46.219040 IP 192.168.1.99 > 8.8.8.8: ICMP echo request, id 325, seq 5, length 64
09:37:46.229046 IP 8.8.8.8 > 192.168.1.99: ICMP echo reply, id 325, seq 5, length 64
So seems to be working OK in a fresh empty environment at least.
am going to do another test, just same way you did it on fresh environment.
quick question though. the type of nat enabled for this containers, will it allow inbound and outbound.
if a container on 10.96.212.8 run ssh server, will sshing to 192.168.1.99 works?
am using firewalld, what zone is the interfacewlp2s0 should be in? iny other or specific rules should we do on host side? routes maybe?
No it is only for outbound traffic (that is why there is only a single address specified at the network level and allows multiple containers to share the same IP).
For inbound you would need to add the LXD proxy device to a container, and there is an optional NAT mode for that device type that will result in setting up an inbound DNAT rule on the host that will forward to the specific IP. Note you will need to use static DHCP assignments to use the proxy in NAT mode by specifying an ipv4.address on the container’s NIC device in LXD.
got it.
Is this what is needed for inbound, iptables -t nat -A POSTROUTING -s 10.96.212.8 -j DNAT --to 192.168.1.99
Unfortunate, i just reformatted my server using opensuse 15.1, installed lxd, and done all nating + adding ip, exactly the way you did. but i can not ping outside the container.
Suggest using an LXD proxy device:
lxc stop c1
lxc config device override c1 eth0 ipv4.address=10.96.212.100
lxc config device add c1 myproxy proxy listen=tcp:192.168.1.128:80 connect=tcp:0.0.0.0:80 nat=true
lxc start c1
The connect=tcp:0.0.0.0:80 part means to use the static IP assignment of the container for the DNAT.
Check the iptables DNAT rule is added:
sudo iptables -L -v -n -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DNAT tcp -- * * 0.0.0.0/0 192.168.1.128 tcp dpt:80 /* generated for LXD container c1 (myproxy) */ to:10.96.212.100:80
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DNAT tcp -- * * 0.0.0.0/0 192.168.1.128 tcp dpt:80 /* generated for LXD container c1 (myproxy) */ to:10.96.212.100:80
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
10 1334 SNAT all -- * * 10.96.212.0/24 !10.96.212.0/24 /* generated for LXD network lxdbr0 */ to:192.168.1.99
Did you get the IP alias working yet (without LXD and NAT)?
Yes and No.
if i do ip a
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 5000 link/ether MAC_ADDRESS brd ff:ff:ff:ff:ff:ff inet HOST_IP/26 brd HOST_IP_BORADCAST scope global eth0 valid_lft forever preferred_lft forever inet ADD_ON_IP/29 brd ADD_ON_BORADCAST scope global eth0 valid_lft forever preferred_lft forever
I can ping it on host terminal. but can not ping it from home.
the method i used to add it. is yast2 lan , simply the file of /etc/sysconfig/network/ifcfg-eth0
was modified with an entry for IPADDR_0='ADD_ON_IP'
I confirm this tutorial works on my home machine, but not the server.
will mark it as solved, as soon as i get the server working.
#9 using post number 9, here easier than creating new thread.
lets assume the gate way for 10.96.212.100 is 10.96.212.1 .
i notice visiting 192.168.1.128:80 is coming fist through the gateway 10.96.212.1 then to the meant ip 10.96.212.100 . can i prevent that completely . because its causing issue with email relay.
instead have 192.168.1.128:80 mapped without passing through gateway of 10.96.212.1 directly to 10.96.212.100:80.
Thanks in advance.
I think you are saying that the source address of the proxied packets appears to be coming from the host IP of the LXD bridge? Is that correct.
If so then this likely means you’re using the proxy device, but without the the nat=true option.
exactly, what i mean.
now taking off nat=false cause the ports to be down, and can not reach them from the public ip.
how would i fix this please. its causing me a lot of problems.
nat=true needs to be enabled, so that is correct.
Please can you provide more info and importantly, examples, of the issue.
Things to provide:
tcpdumps of inbound traffic to host and traffic arriving at container showing the source addresses.
Examples of the container config lxc config show <container> --expanded showing the proxy config.
Thanks
Tom
lxc config show my_container --expanded
architecture: x86_64
config:
image.architecture: amd64
image.description: ubuntu 18.04 LTS amd64 (release) (20200218)
image.label: release
image.os: ubuntu
image.release: bionic
image.serial: "20200218"
image.type: squashfs
image.version: "18.04"
linux.kernel_modules: nf_conntrack
volatile.base_image: 8c4e87e53c024e0449003350f0b0626b124b68060b73c0a7ad9547670e00d4b3
volatile.eth0.host_name: vethba52d42c
volatile.eth0.hwaddr: 00:16:3e:02:6a:1f
volatile.idmap.base: "0"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":400000000,"Nsid":0,"Maprange":500000001},{"Isuid":false,"Isgid":true,"Hostid":400000000,"Nsid":0,"Maprange":500000001}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":400000000,"Nsid":0,"Maprange":500000001},{"Isuid":false,"Isgid":true,"Hostid":400000000,"Nsid":0,"Maprange":500000001}]'
volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":400000000,"Nsid":0,"Maprange":500000001},{"Isuid":false,"Isgid":true,"Hostid":400000000,"Nsid":0,"Maprange":500000001}]'
volatile.last_state.power: RUNNING
devices:
eth0:
ipv4.address: local_ip
name: eth0
nictype: bridged
parent: mail
type: nic
myproxy25:
connect: tcp:0.0.0.0:25
listen: tcp:public_ip:25
nat: "true"
type: proxy
myproxy80:
connect: tcp:0.0.0.0:80
listen: tcp:public_ip:80
nat: "true"
type: proxy
myproxy443:
connect: tcp:0.0.0.0:443
listen: tcp:public_ip:443
nat: "true"
type: proxy
myproxyIMAP2:
connect: tcp:0.0.0.0:995
listen: tcp:public_ip:995
nat: "true"
type: proxy
myproxyIMAPtlsSSL:
connect: tcp:0.0.0.0:993
listen: tcp:public_ip:993
nat: "true"
type: proxy
myproxyTCP:
connect: tcp:0.0.0.0:4190
listen: tcp:public_ip:4190
nat: "true"
type: proxy
myproxysmtpTLS:
connect: tcp:0.0.0.0:587
listen: tcp:public_ip:587
nat: "true"
type: proxy
root:
path: /
pool: default
type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""
tcp traces is out of knowledge for time being.
the importance here, in my case runing email server. the server is listening on port 25 for incoming emails. also using filtering for spam. thus everything coming from bridge ip is rejected. and allowing everything coming from the bridge ip would basically make my server and open relay.
so, i want to have the port to be mapped to that local ip without going through bridge ip first.
Yes this config looks fine. All inbound connections to the host’s public IP on port 25 will be forwarded to the container’s internal IP but with the source address still set to the original external source address.
Its likely the MASQUERADE rule you have from the other thread is causing the forwarded packets to be translated to appear to be from the host’s mail interface.