Forward port 80 and 443 from WAN to container

LXD
1

Dear all,

I want to run a webserver inside an LXC container. The container is on my Ubuntu 18.04 LTS router. The WAN interface is enp1s0. With the standard setup after LXC init I have access from the lan but not WAN. I have read that the new LXD/LXC has a opportunity to set-up a proxy automatically so that traffic could be forwarded without changing IPTABLES. However, I could not find an explanation of the syntax.
How do I get LXD to do the nexessary modifications to forward port 80 and 443 from enp1s0 to the container?

Thank you

2

Hi!

Here are the commands to run on the host. You only need the name of the container to perform the setup. In my example I’ll use mycontainer.

lxc config device add mycontainer myport80 proxy listen=tcp:0.0.0.0:80 connect=tcp:127.0.0.1:80

lxc config device add mycontainer myport443 proxy listen=tcp:0.0.0.0:443 connect=tcp:127.0.0.1:443

What we do here is:

  1. We add a proxy device on container mycontainer, giving it an arbitrary name (myport80).
  2. We set it up to listen on all (0.0.0.0) network interfaces on the host, port 80. You can change the IP address to something more specific, if you want to.
  3. We set it up to make connections to the container mycontainer on the 127.0.0.1 (localhost) interface at port 80. PREVIOUS GUIDES SUGGESTED TO USE localhost. DOES NOT WORK ANYMORE; PLEASE USE 127.0.0.1 INSTEAD!

To verify that LXD is listening on port 80 (http), run

$ sudo lsof -i -n | grep http
lxd       2157            root    5u  IPv6 668213      0t0  TCP *:http (LISTEN)
lxd       2157            root    8u  IPv6 668213      0t0  TCP *:http (LISTEN)

To remove a proxy device, run

lxc config device remove mycontainer myport80
6 Likes
3

Perfect, thank you very much. That worked! Just one more questions: Is there a performance penalty compared to the “traditional” use of iptables with Forward etc. or is the performance the same?

1 Like
4

Sorry, stupid question. I just noticed that I seem to need to have both? Here is what I did:
enp1s0 is my WAN interface
-A PREROUTING -p tcp -m tcp -i enp1s0 --dport 80 -j DNAT --to-destination 10.175.162.124:80
-A INPUT -i enp1s0 -p tcp --dport 80 -j ACCEPT
-A FORWARD -p tcp -d 10.175.162.124 --dport 80 -j ACCEPT

Plus the setup that you suggested with the LXC proxy. Is this too complicated? Should I use a different LXC bridge?

5

Someone else just asked the same question,

There is a way to test, which I describe. I do not have the setup yet to test myself, so feel free to test and report back.

6

You do not need to add iptables rules.

The two commands that start with lxc config device add mycontainer myport80 proxy…, are commands that set up a LXD Proxy Device. One command is for port 80, the other for port 443.

There is no need to use different network settings like a different bridge.

7

Hello Simos, in ‘listen’ can the destination domain of the packet be used instead of IP (0.0.0.0) and thus use this feature in the same way as we would use in a reverse proxy in nginx?

8

In listen you put a valid IP address of the host, or 0.0.0.0 to specify all interfaces.
Ideally, it is better to put the explicit public IP address of the host and not 0.0.0.0.

When you mention destination domain, is that a domain name? If so then no.

1 Like
9

That’s right, “domain name” such as “www.test.com”.
But anyway thank you very much @simos

10

If the hostname resolves to the appropriate host IP address, then you can use it instead.

11 
lxc config device add mycontainer myport80 proxy listen=tcp:0.0.0.0:80 connect=tcp:localhost:80
lxc config device add mycontainer myport443 proxy listen=tcp:0.0.0.0:443 connect=tcp:localhost:443

i have domain name which resolve IP, but can i use both IPv4 & IPv6

12

Can an addition port for an api be added the same way?
Currently I route all 80 and 443 to a proxy container and then from proxy container to the desired website container… using real_ip_header X-Real-IP; set_real_ip_from proxy.lxd;
I have several several containers each with websites on them. At the moment I use 80 and 443 and it works fine, but on a new website I’m working on (running a nodejs app) I need to pass additional information from port 8117, to the website front end viewable to public.
currently I do this by iptable forward port 8117 to the desired container. Problem I’m having is when I install an ssl cert for the website port 8117 information is not longer viewable.
`server {
listen 80 proxy_protocol;
listen [::]:80 proxy_protocol;

    server_name apache1.example.com;

    location / {
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_pass http://apache1.lxd;
    }

    real_ip_header proxy_protocol;
    set_real_ip_from 127.0.0.1;

}`

image
image.jpg2400×2525 455 KB

1 Like
13

Hi!

If I understand correctly, the nodejs container has to be both accessible by the front-end container BUT also from the Internet? If that is the case, can you describe me a simple example so that I can use and duplicate the issue?

If the nodejs container is ONLY accessible either from the internet, or the front-end container, then it is much simpler.

14

Hi simos,

the container with the website files are in folder /var/www/html (default)

The nodejs app connects to another app running and pulls information from port 8117 and displays it on the website front end at /var/www/html (default) …these are static pages except for the information displayed via port 8117.

So when someone goes to the website, they view the static frontend pages and the live streaming data coming from 8117. Everything works fine when the site is run on “http” , but when its run on “https” that the live stream info is not displayed.

I hope I’m explaining it for some clarity to you. Just to recap…I have a container named “proxy” and I route all 80 and 443 to it. The proxy container has several virtual blocks that point to the other lxd containers. The other containers are the ones hosting the website frontends and running the various apps pushing info on 8117. I just need to figure out how to allow the reverse proxy to display information from 8117

| proxy (lxd container) | —> | lxd container 1 ( website 1) has live stream info 8117 |
—> | lxd container 2 ( website 2) has live stream info 8118 |
—> | lxd container 3 ( website 3) has live stream info 8119 |

so at the moment all website work fine on “http” just not on “https”

currently I use this and all the info displays correctly on “http”
sudo iptables -t nat -A PREROUTING -p tcp -i ens18 --dport 8117 -j DNAT --to-destination 10.130.57.223:8117
sudo iptables -t nat -A PREROUTING -p tcp -i ens18 --dport 8118 -j DNAT --to-destination 10.130.57.25:8118
sudo iptables -t nat -A PREROUTING -p tcp -i ens18 --dport 8119 -j DNAT --to-destination 10.130.57.79:8119

15

Is this URL http://10.130.57.223:8117 used on your front-end? Most likely not, because it is obviously not addressable from the internet and you mentioned it works for http. You use a hostname for this. If you want to use https in addition to http, then you need to specify another port for https.

Alternatively, you could use a service like https://www.softwaretestinghelp.com/ngrok-alternatives/

16

Hi @simos,

Thanks for your reply on this thread. I have a similar (not sure) question.

I got these setup:

2 multipass ubuntu containers:

Multipass #1 —> forward nginx port 80 traffic to Multipass #2 <-- I was able to do this using nginx reverse proxy.

But I have to install nginx on the Multipass #2 container but not sure how to do these:

User visits mysite1.com (hosts file using the IP of Multipass #1) —> forward the request to Multipass #2 LXC container Wordpress-1.

User visits mysite2.com (hosts file using the IP of Multipass #1) —> forward the request to Multipass #2 LXC container Wordpress-2.

Is it possible to detect the domain then forward the traffic to a specific LXC container like the example above?

Thanks in advance.

17

Hi @john_mark,

As far as I know, multipass creates virtual machines. There is an option local.driver=lxd when you create a VM (and I haven’t used it). Does that option create a LXD system container or does it create the virtual machine through the LXD hypervisor? I believe it should be the latter.

If virtual machines are not a hard requirement for your task, I suggest to use system containers instead. Port forwarding is easier with system containers.

Having said that, LXD provides port forwarding between the host and a container (you have the option for both host->container and container->host).

To answer you question, I think you are asking about the reverse proxy functionality. That is, depending on the website domain, you want the same LXD host to have many many websites.
I suggest to use nginx as a reverse proxy as you get Let’s Encrypt for free. See my tutorial at https://www.linode.com/docs/guides/beginners-guide-to-lxd-reverse-proxy/

If your intention is to put WordPress in those websites, see https://www.linode.com/docs/guides/how-to-set-up-multiple-wordpress-sites-with-lxd-containers/ which is a continuation of the first tutorial. The nice feature here is that the mysql database is used by all WordPress installations.

2 Likes
18

Dear @simos, thanks for your detailed explanations on the LXC port-forwarding, which taught me so much and led me to dive deep into this topic. I might have tested all the possibilities of manipulating the forwarding on my 3-node cloud based on Ubuntu 20.04 server, and the LXD/LXC version is 4.0.8.

My nodes use the IP address in the range 10.0.0.0/8, while my LXC instances use the IP address in the range 240.0.0.0/8. By the way, I run a Tomcat 9 instance in the container.

Test 1 - Configure on the very host, the first node with an IP address 10.10.4.86, on which the LXC instance, named mycontainer is running. The IP address of the instance is 240.4.186.19.

Method 1: lxc config device add mycontainer port8082-lxc8080 proxy listen=tcp:0.0.0.0:8082 connect=tcp:127.0.0.1:8080 (Successful as expected)
Note: This is better as my container gets its IP address dynamically.

Method 2: lxc config device add mycontainer port8082-lxc8080 proxy listen=tcp:0.0.0.0:8082 connect=tcp:240.4.186.19:8080 (Successful as expected)

Method 3: lxc config device add mycontainer port8082-lxc8080 proxy listen=tcp:10.10.4.186:8082 connect=tcp:240.4.186.19:8080 (failed)
Error: Failed to start device "port8082-lxc8080": Error occurred when starting proxy device: Error: Failed to listen on 10.10.4.186:8082: listen tcp 10.10.4.186:8082: bind: cannot assign requested address

The following error messages are identical.

Method 4: lxc config device add mycontainer port8082-lxc8080 proxy listen=tcp:10.10.4.186:8082 connect=tcp:127.0.0.1:8080 (failed)

Test 2 - Configure on the second node, whose IP address is 10.10.4.185, rather than the node on which mycontainer is running.
Aim: To test whether we can access the service the other nodes on which the container is NOT running (failed).

Method 1: lxc config device add mycontainer port8082-lxc8080 proxy listen=tcp:0.0.0.0:8082 connect=tcp:240.4.186.19:8080 (Problematic)
Note: The configuration is successful, bu the service is inaccessible.

Method 2: lxc config device add mycontainer port8082-lxc8080 proxy listen=tcp:10.10.4.185:8082 connect=tcp:240.4.186.19:8080 (failed)

Test 3 - Configure on the third node, whose IP address is 10.10.5.113, but specify the host IP address as the very host on which the container is running.
Aim: To test whether we can configure on any node as we did in Test 1 (failed).

Method 1: lxc config device add mycontainer port8082-lxc8080 proxy listen=tcp:10.10.4.186:8082 connect=tcp:240.4.186.19:8080 (failed)

Method 2: lxc config device add mycontainer port8082-lxc8080 proxy listen=tcp:10.10.4.186:8082 connect=tcp:127.0.0.1:8080 (failed)

Test 4 - Manually edit the configuration, but there’s no success when testing the service (failed).
lxc config edit mycontainer

My conclusions:
(1) Port-forwarding can only be configured on the very host that the container is running.
(2) To easily control the request dispatching to the containers, a reverse proxy seems to be the only option as indicated by @simos.

19

Update 1: Test 3 - Configure on the third node is successful. That’s is to say, port forwarding can be added from any node of the cloud. Therefore, my first conclusion above is wrong.

┌─[user01@server4][~]
└─▪lxc config device add mycontainer port80-lxc80 proxy listen=tcp:10.10.5.113:80 connect=tcp:127.0.0.1:80
Error: Failed to start device "port80-lxc80": Error occurred when starting proxy device: Error: Failed to listen on 10.10.5.113:80: listen tcp 10.10.5.113:80: bind: address already in use
┌─[user01@server4][~]
└─▪lxc config device add mycontainer port443-lxc443 proxy listen=tcp:10.10.5.113:443 connect=tcp:127.0.0.1:443
Device port443-lxc443 added to mycontainer
...
┌─[user01@server2][~]
└─|lxc config device list proxysimlab 
port443-lxc443
┌─[user01@server2][~]
└─|lxc config device list tljha      
port80
port8022
port9443
┌─[user01@server2][~]
└─|lxc stop tljha              
┌─[user01@server2][~]
└─|lxc config device remove tljha port80
Device port80 removed from tljha
...
┌─[user01@server4][~]
└─▪lxc config device add mycontainer port80-lxc80 proxy listen=tcp:10.10.5.113:80 connect=tcp:127.0.0.1:80
Device port80-lxc80 added to mycontainer