Iptables rule to access site from within? (proxy NAT mode)

You were likely running proxy in non-nat mode, nat mode will preserve source IP.

See Making sure that IP's "connected" to the containers gameserver proxy shows users real IP? for examples.

Just tried this kind of setup:

lxc config device add alp p7 proxy listen=tcp:a.b.c.d:7777 connect=tcp:0.0.0.0:7777 
lxc config device set alp p7 nat=true listen=tcp:a.b.c.d:7777 connect=tcp:0.0.0.0:7777

In this case telnet a.b.c.d 7777 works ok from outside, it works from within the container itself, but it does not work from inside the host or from inside other containers.

I run internal DNS on local IP 172.16.172.233:53. I did this kind of setup for port 53, tcp and udp. Now container can talk to its dns via external IP address, I can connect to external IP and resolve from outside, but a) host itself can’t query DNS via external address and other containers can’t do that as well.

Can you check your LXD logs and see if there is a warning about br_netfilter not being enabled?

Jul  6 15:38:09 ns lxd.daemon[31233]: t=2020-07-06T15:38:09+0300 lvl=warn msg="Proxy bridge netfilter not enabled: br_netfilte
r not loaded: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory. Instances using the bridge will no
t be able to connect to the proxy's listen IP"
ls -al /etc/alternatives|grep iptables                                              [/etc/alternatives 15:41 Mon 6] 
lrwxrwxrwx  1 root root   25 Jun 13 03:31 iptables -> /usr/sbin/iptables-legacy
lrwxrwxrwx  1 root root   33 Jun 13 03:31 iptables-restore -> /usr/sbin/iptables-legacy-restore
lrwxrwxrwx  1 root root   30 Jun 13 03:31 iptables-save -> /usr/sbin/iptables-legacy-save

OK so you will need that enabled, see Resolve Host Top Level Domain to Bridge IP inside LXD network? for a similar issue.

Yeah, with

modprobe br_filter

it works. BTW, does that proxy mechanism work with ipv6 addresses?

Yes it should do.

1 Like

Ok, thanks. I think it should be documented somewhere clearly, because whole process is not obvious and scattered around this forum… including this tiny detail about kernel module not loaded by default. And that you have to use two commands to create a proxy and then to set it into nat mode.

Yes the br_netfilter optional module could be documented here https://linuxcontainers.org/lxd/docs/master/instances#type-proxy

What do you mean about “you have to use two commands to create a proxy and then to set it into nat mode.”?

Nevermind. For some reasons I used add and then set as two commands. I looked up my zsh history and it seems that first hints I found on the net used ‘lxc config device set’, lxc complained about missing device, I became confused and used two commands – created proxy without nat and then set it into nat mode.

Anyway, clear guide how to set up port forwarding and preserve remote IPs would be great.

Great thanks.

I will look at adding a mention for br_netfilter to the proxy doc.

If you were able to write a post on the forum for setting up a proxy in nat mode then I could promote that into the Tutorials category for the benefit of other users.

I’ll try to write something, but my English is lousy :slight_smile:

1 Like
lxc config device add alp proxy80v6 proxy listen=tcp:2604:280:1:5f8::2:80 connect=tcp:0.0.0.0:80
Error: Invalid devices: Device validation failed "proxy80v6": Invalid value for device option "listen": address 2604:280:1:5f8::2:80: too many colons in address

Ugh.

You need to use square brackets, e.g. listen=tcp:[2604:280:1:5f8::2]:80

lxc config device add alp proxy80v6 proxy nat=true "listen=tcp:[2604:280:1:5f8::2]:80" "connect=tcp:[::]:80"   
Error: Failed to start device "proxy80v6": Proxy connect IP cannot be used with any of the instance NICs static IPs

I was able to add v4 proxy to this container without problems.

You need to use static IPs for both IPv4 and IPv6 (when using proxy in nat mode), please show the output of lxc config show <container> --expanded for the container you added the IPv4 proxy device on.

architecture: x86_64
config:
  image.architecture: amd64
  image.description: Alpine 3.9 amd64 (20190321_13:00)
  image.os: Alpine
  image.release: "3.9"
  image.serial: "20190321_13:00"
  volatile.base_image: 0cbd911b5a203c7e475241b8b22cc5332d10fd30ae27916bae1558bcb118c9ce
  volatile.eth0.host_name: veth196022e7
  volatile.eth0.hwaddr: 00:16:3e:6b:04:12
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    ipv4.address: 172.16.172.116
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
  root:
    path: /
    pool: tank
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""

Yep, so your eth0 NIC has a static IP defined already. So when you added a proxy in NAT mode with a connect option of tcp:0.0.0.0:7777 will use the static IP to setup the NAT rule.

In order to define a static IPv6 address, you also need to enable stateful DHCPv6 on the LXD managed network (otherwise the container may use SLAAC to generate a random IP). You will get an error to that effect when you try and define a static IPv6 address on a bridged NIC.