Nginx, resolver directive: how-to?

LXD
1

I have two containers. web.lxd and reverseproxy.lxd. As you can imagine I use Nginx in reverseproxy.lxd. I use this configuration:

location /test {
        include     conf.d/proxy_set_header.inc;
        proxy_pass http://web.lxd/test;
}

If I restart web.lxd its ip is changed so Nginx can no longer connect to web.lxd because it uses previous ip. Then I tried to use resolver directive

resolver 127.0.0.1 valid=30s;
set $web_backend "http://web.lxd/test";
location /test {
        include     conf.d/proxy_set_header.inc;
        proxy_pass $web_backend;
}

It doesn’t work. Am I correct that 127.0.0.1 where dns resolver lives. And obviously it is not there (I am using alpine image). In /etc/resolv.conf I have the following:

search lxd
nameserver 10.3.13.1

So 10.3.13.1 is my name server. Should I put it there? But I believe 10.3.13.1 is dynamic and can change on restart two. Actually I tried to put it and got infinite loop in Nginx when resolving web.lxd. Should I install name server in reverseproxy.lxd machine? I thought I can use name server available on host somehow and using its host name without using any IP address. Can someone enlighten me on this topic?

2

What is the IP of your host’s lxdbr0 address (Please show output of ip a command on host).

3

I have this for lxdbr0

3: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:5e:04:c1 brd ff:ff:ff:ff:ff:ff
    inet 10.3.13.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
    inet 169.254.229.0/16 brd 169.254.255.255 scope global noprefixroute lxdbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::9729:166e:eeec:d741/64 scope link
       valid_lft forever preferred_lft forever
4

So the resolver should be 10.3.13.1, you can test this by doing dig @10.3.13.1 web.lxd

5

I think I tried it and as I mentioned above I got an infinite loop when accessing web-site. But it could be Nginx problem. I will check it again anyway.
But I was more interested (sorry if I say nonsense) is it possible to specify a host-name for resolver (directive supports domain name or IP). So something like this resolver my.parent.host.lxd;. Am I correct saying that all IPs could be different after a restart of host machine? If yes what is the best/recommended way to approach that nameserver ip can change so I don’t hard code it in various scripts and configs?

6

No I dont think, as the point of a resolver is to resolver domain names to IPs, if you specify a domain name for your resolver, how will the resolver know which IP to resolve it to (chicken and egg comes to mind).

The lxdbr0 interface’s IP is static so it won’t change unless you change it.

7

One more question then. I didn’t try and don’t have a machine to try but I believe I can set a new ip address to a created network object, is that right? In this case will lxd automatically update all created instances’ /etc/resolv.conf files with a new IP address?

8

Not so directly. You can change the lxdbr0’s IP using lxc network set lxdbr0 ipv4.address=... but this will not change anything inside the containers.

However it will change the DNS and gateway IP handed out via DHCP. Which assuming your containers are using DHCP to configure themselves will mean the next time they initiate a DHCP request they will update their resolver and default gateway IP.

9

Even if it is chicken and egg I want to try to use domain name for resolver. I hope Nginx will use resolve.conf first and then only resolve backend ip. But I cannot find if I can assign a host name to lxd host. Is it possible? So I can talk to a parent host from a container using hostname not an ip.

10

I see what you’re asking, you want to create a DNS name for the LXD DNS resolver.

You can do this using the raw.dnsmasq setting on the network config, e.g.

# Get address of lxdbr0 DNS server.
lxc network get lxdbr0 ipv4.address
10.102.242.1/24
# Add this as a DNS name in dnsmasq itself.
lxc network set lxdbr0 raw.dnsmasq="address=/gateway.lxd/10.102.242.1"

Now in a container:

lxc exec c1 -- ping -c1 gateway.lxd
PING gateway.lxd (10.102.242.1) 56(84) bytes of data.
64 bytes from 10.102.242.1 (10.102.242.1): icmp_seq=1 ttl=64 time=0.035 ms

--- gateway.lxd ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.035/0.035/0.035/0.000 ms

p.s. nginx does appear to resolve the address of gateway.lxd at startup when used in the resolver directive. However I do not know whether it will re-resolve that address periodically so if it does ever change you may need to restart nginx.

11

Amazing. That is what I needed. Actually it works as expected.

server {
  resolver gateway.lxd valid=30s;
  location /test {
        set $test_backend "http://test.lxd";
        proxy_pass  $test_backend;
    }
}

Yes. If I change gateway ip Nginx won’t pickup it but if I restart a host machine and all ips are changed the setup above works perfect. Also If I only restart test.lxd and it gets a new ip Nginx will pick up a new ip at least within 30s.
Thanks a lot for your help.

1 Like