So, I’m trying to run NGINX inside containers but the main server (not a container) is also running nginx. I’m aware that I need to alter the port 80/443 inside containers, and use proxy pass. But thus far I seem to be failing. I can’t use lxc config device add with listen/connect to port 80 seeing the port is already in use.
So it would be great if someone can help me with this. Also running UFW by the way 
Regards,
Skyrider
Hi!
Indeed, it is an issue when you want to have more than one website being served on the same ports 80/443. What is needed, is a way for whichever service responds to an Internet connection, is able to deal connections that were intended for the other websites.
If you can put the host’s nginx server inside a container, then you can follow the following guide, https://www.linode.com/docs/applications/containers/beginners-guide-to-lxd-reverse-proxy/
This is the most elegant way.
If you really want to keep that nginx running on the host, you need to adapt the above guide to work as a reverse proxy, with an additional web server (host’s web server). The above link should help you as well.
How can I use that guide for a reverse proxy, if I can’t use:
lxc config device add proxy myport80 proxy listen=tcp:0.0.0.0:80 connect=tcp:127.0.0.1:80 proxy_protocol=true
As port 80 is already in use? Can’t really find anything in the tutorial that is within my issue.
Id strongly recommenced you take the time to read and apply the tutorial, it will all seem so simple once your understand whats going on.
2 things can’t listen on the same port at the same time, the proxy device on the host isnt going to be able to listen for requests to port 80 because NGINX is already listening on port 80.
We use a reverse proxy here to distribute the inbound requests on port 80 between one or more hosts / containers typically by domain name (in the context of web)
Simos is saying you can do one of the following
The ideal option
The website running on the host NGINX will need to be put into a container / moved somewhere else because you are going to change NGINX on the host into a reverse proxy as per this part of the tutorial
You will not need to add a proxy device if you follow this solution because NGINX will act as the proxy device
It will look something like this
Even more ideal would be to follow the tutorial and move the reverse proxy into a container & use an additional proxy device which would look like this
Less Ideal
Keep NGINX on the host & move the website to a different port, use your nginx reverse proxy to redirect to port 81
your containers can still listen on port 80 they each have their own interface
Thanks, Guess I’ll make some adjustments to have 1 proxy NGINX container then. I apologize for my edit, but I noticed I was a bit fast in replying. Noticed that it was the proxy only, forwarding to the nginx container, as such the root files are located there as well.
Guess this means some downtime, as I was hoping I could be running nginx on my main host until I swapped them over one by one. But I prefer the ideal method.
Which reminds me to ask another question regarding storage. I’m still new to lxd/lxc, so hope I can ask this in the same thread/post. If I generate more containers for the proxy and nginx, would that mean it would use an existing storage image file for all the containers or generate a new one each time?
Thus far I kept making new storage volumes for each container, alongside with new profiles. Not sure if that’s the ideal method. That’s the only thing I’m confused of right now how it generates storages for each container created.
Small example:
lxc launch ubuntu:18.04 apache1
Great, new ubuntu container. Yet, doesn’t explain where it gains the storage from, nor which type of storage either.
Sorry for the bit of off-topic question, but had to ask as it would be making my container creation for all my nginx containers easier.
Disclaimer: this is just my opinion on “ideal”
storage is just a mystery to me, its another thread, and I think your question has been asked before so do some digging
A proxy isn’t used to serve files - it used to redirect requests to servers that will
Your containers for websites will contain the root directives
I dont use NGINX, so ive guessed done / quick google
// You would have one of these for each domain name you want to proxy
server {
// Listen on ipv4 port 80 - enabling proxy_protocol
listen 80 proxy_protocol;
// Listen ipv6 port 80 - enabling proxy_protocol
listen [::]:80 proxy_protocol;
// This server will handle connections for the domain name xxx
server_name nginx1.example.com;
// Im assuming this is a special charectar for proxy all requests
// you may have to tweak this
location / {
// When you use a reverse proxy you loose the original IP
// address so this provides the clients IP address as a
// new HTTP header
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
// This is where the request is forwarded to your container/s
// running webservers for this domain (because you could run
// more than one, I guess you could also provide an ip
// here aswell but im not sure E.G
// proxy_pass http://MY_CONTAINERS_IP
proxy_pass http://nginx1.lxd;
}
// This is some kind of module enabling in NGINX
real_ip_header proxy_protocol;
// Anyone's guess
set_real_ip_from 127.0.0.1;
}
Regarding the question on the storage, please create a new topic. The practice is to make the questions easier to find to new users when they search the forum.
No worries, I appreciate the replies I’ve been given!
Thanks everyone!
Of which reminds me. I assume that PHP FPM runs on the main container without the need for it to be running inside proxy?
PHP-FPM runs on all the containers serving websites not the proxy.
a reverse proxy just forwards trafic its not related to a specific implementation (PHP) I.E it can also “proxy” (forward) requests other than HTTP like web sockets or UDP / TCP connections
Imagine the reverse proxy as the person working at the train station who tells you what platform your train is on if you dont know - they dont know anything about your particular train they just know how to look it up and sends you off to the right platform
Please take the time todo some reading we are explaining what 1000+ articles on the web do 
I figured as much, but was just making sure.
And I do read before making topics, hence I generated this topic as apparently I was trying to do it in a way that usually isn’t the best nor ideal way. So couldn’t find much helpful to help me out in my situation. Now I know it’s just better to create a single proxy container.
Again, I appreciate the replies. Thank you
Merged to prevent multi posts
I do have one more question, and I’ve googled it up but I couldn’t find any results that would help me.
According to the tutorial, you can set up:
set_real_ip_from proxy.lxd;
and:
proxy_pass http://nginx1.lxd;
But everytime I attempt to do so, I keep getting errors like:
nginx: [emerg] host not found in upstream "containername.lxd" in /etc/nginx/sites-enabled/default:10
And:
nginx: [emerg] host not found in set_real_ip_from "proxy.lxd" in /etc/nginx/conf.d/real-ip.conf:2
And yes, the container names exists. Once I replace the container name with their own IP address, I no longer receive any errors and it seems to be accepting it. How come?
can you ping your hosts from the container ? If not you have a DNS issue, a quick work around will be to put the dns names in /etc/hosts but thats the same as setting the IP’s in the config file
I can ping the containers from the main host, and ping other containers within a container. Nginx (proxy) and nginx container works just fine as well. I’m just unable to use the container hostname, only works by using their IP addresses.
While your solution would work, I’m just curious why the hostname is not recognized.
When you ping a container it appears to use ipv6 resolution, when you plug the IP address in I assume your putting in the ipv4 address,
In your web server are you listening for ipv6 requests I.E listen [::]:80;
Ya, I’m not using ipv6 though it is listening to it. And yup, used ipv4 for the config file/pings.
Are you sure ? can you paste the output of ping your_container_name.lxd from within the proxy container
Oh, that replies with ipv6.
root@sourceop:~# ping sourceop.lxd
PING sourceop.lxd(sourceop.lxd (fd42:1020:56ec:da27:216:3eff:fe67:ae3a)) 56 data bytes
64 bytes from sourceop.lxd (fd42:1020:56ec:da27:216:3eff:fe67:ae3a): icmp_seq=1 ttl=64 time=0.101 ms
64 bytes from sourceop.lxd (fd42:1020:56ec:da27:216:3eff:fe67:ae3a): icmp_seq=2 ttl=64 time=0.072 ms
I merely answered your question if my web server was listening to ipv6, and I mentioned yes. But in general, I’m not using ipv6 so I probably should disable it.
I guess that’s the reason why it’s not working as it should, as it replies back with ipv6 as you mentioned rather than ipv4.
can you post your web server config file ?
You should keep ipv6
I’m using the default web file:
server {
listen 80 default_server;
listen [::]:80 default_server;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name _;
location / {
try_files $uri $uri/ =404;
}
}
Just to make sure it works, which it does. I haven’t changed/altered it yet to my needs. I noticed some web configs have a proxy variable in listen, though not used in the tutorial.
I do have to ask.
Everything is working nicely with a single proxy container, and other nginx containers. But I have noticed is loop errors when I wanted to restore nginx web config files. I know it’s because both proxy and the nginx container forwards http to https. And I know the root directory is added within the nginx container as well. So what exactly is put in the proxy container? Only the port 80 and redirect to 443 alongside with their encryption location files?
I would assume then I would put the web configurations inside the 443 server block on the nginx container, as long it won’t have any redirection (from http to https?)