Nginx "proxy_pass container.xd;" causes a lot of connection refused - how to resolve to ipv4 instead of ipv6?

1

I have noticed that with example:

    location / {
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_pass http://container.lxd;
    }

There’s a lot of connection refused errors in the nginx error log. It works for one, while refusing it for others and also spinning out random 502’s on the website. But when I change the proxy_pass to the container IP address, all errors disappears.

How do I make it so, so that the container.lxd hostname resolves to the ipv4 instead of ipv6?

2

container.lxd will resolve to an IPv4 address if you are looking up an A record. However I am not sure how you would instruct nginx’s reverse proxy code to only lookup A records (rather than AAAA) records.

Have you considered just making nginx listen on both IPv4 and IPv6 addresses, or using a static IPv4 address and specifying that statically in the nginx config?

3

nginx is actually listening on both ipv4 and ipv6.

server {
        listen 80 proxy_protocol;
        listen [::]:80 proxy_protocol;

        server_name website;

        location / {
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_pass http://ipaddress;
}

        real_ip_header proxy_protocol;
        set_real_ip_from 127.0.0.1;

        listen [::]:443 proxy_protocol ssl ipv6only=on; # managed by Certbot
        listen 443 proxy_protocol ssl; # managed by Certbot
}

Same listen 80/443 (and [::]) is included in the web container running nginx.

Even if I ping a container from within a container, it shows ipv6 results back. So while I could adjust the nginx settings, it appears to a lxd/lxc thing.

4

Where is the nginx reverse proxy running? Is it on the LXD host or inside a LXD container?

What is the output of dig a container.lxd and dig aaaa container.lxd on the node/containe running nginx reverse proxy?

5

Reverse Proxy container and Web containers are running inside a container.

Using lavalink container as example.

root@nginx-proxy:/# dig a lavalink.lxd

; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> a lavalink.lxd
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48956
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;lavalink.lxd. IN A

;; ANSWER SECTION:
lavalink.lxd. 0 IN A 10.75.254.49

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Mon May 18 16:32:23 CEST 2020
;; MSG SIZE rcvd: 57

root@nginx-proxy:/# dig aaaa lavalink.lxd

; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> aaaa lavalink.lxd
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49179
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;lavalink.lxd. IN AAAA

;; ANSWER SECTION:
lavalink.lxd. 0 IN AAAA fd42:903d:d968:c9b3:216:3eff:fef0:3eb0

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Mon May 18 16:33:20 CEST 2020
;; MSG SIZE rcvd: 69

root@nginx-proxy:/# ping lavalink.lxd
PING lavalink.lxd(lavalink.lxd (fd42:903d:d968:c9b3:216:3eff:fef0:3eb0)) 56 data bytes
64 bytes from lavalink.lxd (fd42:903d:d968:c9b3:216:3eff:fef0:3eb0): icmp_seq=1 ttl=64 time=0.544 ms
64 bytes from lavalink.lxd (fd42:903d:d968:c9b3:216:3eff:fef0:3eb0): icmp_seq=2 ttl=64 time=0.086 ms
64 bytes from lavalink.lxd (fd42:903d:d968:c9b3:216:3eff:fef0:3eb0): icmp_seq=3 ttl=64 time=0.078 ms
64 bytes from lavalink.lxd (fd42:903d:d968:c9b3:216:3eff:fef0:3eb0): icmp_seq=4 ttl=64 time=0.097 ms
64 bytes from lavalink.lxd (fd42:903d:d968:c9b3:216:3eff:fef0:3eb0): icmp_seq=5 ttl=64 time=0.089 ms

Unable to run these outside the container (on the main host), as no name / service is known. I would assume then that lxd doesn’t add the hosts of the containers on the main system.

6

I think the nginx resolver directive is what you want as it can take an option to disable ipv6 record look ups.

7

What if I need the host for any other application? It’ll use the ipv6 instead of ipv4.

I’m curious as to why the hostname grabs the ipv6 over ipv4.

8

@turtle0x1

Do you still want me to reply in regards of your reply?

No one seems to know why pinging lxd hosts defaults to ipv6?

9

This is standard behaviour in Linux, it is not specific to LXD. Generally speaking (and this is not true for all applications), if the hostname you specify has an AAAA DNS record (which LXD does generate for the IPv6 addresses) then it will use IPv6 over IPv4.

So when pinging or using an application like nginx, you need to instruct it to use IPv4 specifically if you need to, using the -4 flag with ping and in nginx as I described above.