How to get real client ip when using lxd to forward port 80

,
#1

I have setup haproxy in a lxd container and then on the host forward port 80 and 443 to container. I am using following commands for it:

/snap/bin/lxc config device add "{{container_name}}" http proxy listen=tcp:0.0.0.0:80 connect=tcp:127.0.0.1:80
/snap/bin/lxc config device add "{{container_name}}" http proxy listen=tcp:0.0.0.0:443 connect=tcp:127.0.0.1:443

When doing this haproxy logs show traffic coming from remote ip address 127.0.0.1 which is not right. I need the actual client ip, to be passed to haproxy even if its in X-Forward-For header.
Is it possible?
If its not possible what will be your suggestion?
The simple solution I can think of at present is to install haproxy from container to physical host.

1 Like
#2

Docker containers face the same issue in swarm mode.
Unable to retrieve user’s IP address in docker swarm mode
The discussion how haproxy solves this issue:
Preserve source IP address despite reverse proxies

#3

It’s not possible with the way the proxy device works right now.

There is a protocol that exists for sending the client address in such cases and is supported by nginx, haproxy and a few others. We could in theory add support for that as an option that can be turned on in the proxy, but it’d need the receiving end to be aware of it.

We do have another issue open right now to support doing straight iptables in cases where forwarding with it is possible.

#4

That should be the proxy protocol (also in the second link that @roka wrote earlier),
http://www.haproxy.org/download/1.8/doc/proxy-protocol.txt (implementations in Section 3 or https://www.haproxy.com/blog/haproxy/proxy-protocol/).

There are two versions, 1 (textual) and 2 (binary).
Here is a Go implementation of both versions, https://github.com/pires/go-proxyproto (Apache License 2.0).

#5

Hi. Sorry to dig this old topic. But is there a ticket for this problem to follow on git? Can it be solved with device or iptables now in the latest version of lxd/lxc?

#6

Sorry. It looks lxd already supports proxy protocol. Found it here in @simos blog.

#7

I believe you are referring to the proxy_protocol=true setting on the proxy device:

See https://linuxcontainers.org/lxd/docs/master/instances#type-proxy

Alternatively, using nat=true on the proxy device, combined with a static IP allocation on the NIC would also pass the client IP through.

#8

I have an interface on the target instance with a static IP, but that IP is set inside the target instance and not using lxd. When I try to set nat=true on the lxd device, I get this:

$ lxc config device set up-rat http listen=tcp:<host_external_ip>:80 nat=true connect=tcp:<target_instance_ip>:80
Error: Failed to start device “http”: Proxy connect IP cannot be used with any of the instance NICs static IPs

Does this require that lxd be managing the interface? Does nat=true require that iptables be running on the host?

#9

If you tell LXD about the static IP using lxc config device set <instance> eth0 ipv4.address=n.n.n.n then it will know which IP to setup for the DNAT entry (using either iptables or nftables).