Can't launch new containers - temporay failure in name resolution

Incus
1

Hello. I have some LXD experience, but starting a new server using Incus. In creating a new container, I am getting a name resolution failure:

$ incus launch images:ubuntu/24.04 
Launching the instance
Error: Failed instance creation: Get "https://images.linuxcontainers.org/images/ubuntu/noble/amd64/default/20241207_07:42/incus.tar.xz": lookup images.linuxcontainers.org: Temporary failure in name resolution

If I ping, no dice:

$ ping images.linuxcontainers.org
ping: images.linuxcontainers.org: Temporary failure in name resolution

But if I use nslookup I get

Server:		172.16.10.1
Address:	172.16.10.1#53

Non-authoritative answer:
Name:	images.linuxcontainers.org
Address: 45.45.148.8
Name:	images.linuxcontainers.org
Address: 2602:fc62:a:1::8

In a browser, I can browse to https://images.linuxcontainers.org/images, so I don’t think it’s my overall network. I think it has to be something to do with my incus server and possibly, resolved. I recently followed How to integrate with systemd-resolved in order to have a local incus-managed nameservice for my incus containers. Is it possible something locally on the incus server has hosed public name service?

(Answer: yes)

$ wget google.com
--2025-01-01 19:48:28--  http://google.com/
Resolving google.com (google.com)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘google.com’

I understand basic networking concepts, even a little DNS, but I do not know anything about incus networking or systemd-resolved, so any help to diagnose this is appreciated.

-brmiller

2

Hi!

Your Web browser has a feature, DNS over HTTPS, which bypasses your system’s DNS configuration. Most likely you got that enabled. Check in the settings of your browser.
Obviously, do not disable DNS over HTTPS yet, at least until you manage to resolve (pun intended) the issue with your system’s DNS configuration.

The file /etc/resolv.conf has the DNS configuration of your system. In newer systems, this file looks like

# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad
search lan

The line nameserver 127.0.0.53 says that your system is using some local (DNS) nameserver, therefore, it tells you to continue your search on what service is running on the local IP address 127.0.0.53 (systemd-resolved).

Although the file says to you not to edit it, you may indeed change it temporarily and replace 127.0.0.53 with either 1.1.1.1 (Cloudflare public DNS) or 8.8.8.8 (Google Public DNS) and get the job done. Obviously, when you reboot, your system will revert back to the original version of the file (and stop working until the issue is resolved).

I am not sure how to troubleshoot but probably someone else will be able to help.
In the meantime, provide the output of the following commands.

Below it says that the interface enp5s0 has a DNS server.

$ resolvectl dns
Global:
Link 1 (enp5s0): 192.168.1.1
Link 2 (virbr0):
Link 3 (lxcbr0):
Link 4 (incusbr0):
Link 5 (incusbr1):
.
.
.

Then, get more info from that interface.

$ resolvectl status enp5s0
Link 1 (enp5s0)
    Current Scopes: DNS
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.1.1
       DNS Servers: 192.168.1.1
        DNS Domain: lan
$

Do the above for your case and someone may be able to direct you to the solution.

3

you need to edit /etc/systemd/resolved.conf or /etc/systemd/resolved.conf.d/dns_servers.conf.
DNS=1.1.1.1
FallbackDNS=1.0.0.1
or whatever dns provider you like.

restart systemd-resolved or start it if not started:
systemctl restart systemd-resolved

run this cmd to check config is applied. if you see dns server list 1.1.1.1, then good to go.
resolvectl status

to test dns name resolve, run:
resolvectl query images.linuxcontainers.org

4

Thanks @simos @catfish. I did neglect one thing last night. My /etc/resolv.conf had

nameserver <ip of lan dns server>

I run pfsense which runs its DNS resolver and has several static IP assignments and registers local DHCP hosts in its DNS, which is given to every host on the network. In some cases (possibly even my incus server), I use a MAC-based DHCP lease so it acts like a static address, but I only have to configure it in one place.

I was hoping that systemd-resolved would “honor” this resolv.conf. Before following @catfish’s suggestions, I had

$ resolvectl dns
Global:
Link 2 (eno1):
Link 3 (eno2):
Link 4 (incusbr0): 172.16.60.1
Link 12 (veth65d01752):
$ resolvectl status eno1
Link 2 (eno1)
Current Scopes: LLMNR/IPv4 LLMNR/IPv6
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

After adding my pfsense DNS server IP address to /etc/systemd/resolved.conf as the DNS directive, I have

$ resolvectl dns
Global: 172.16.10.1
Link 2 (eno1):
Link 3 (eno2):
Link 4 (incusbr0): 172.16.60.1
Link 12 (veth65d01752):
$ resolvectl status
Global
       Protocols: +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub
      DNS Servers 172.16.10.1

Link 2 (eno1)
Current Scopes: LLMNR/IPv4 LLMNR/IPv6
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 3 (eno2)
Current Scopes: none
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 4 (incusbr0)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
   DNS Servers: 172.16.60.1
    DNS Domain: ~incus

Link 12 (veth65d01752)
Current Scopes: none
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
$  resolvectl query images.linuxcontainers.org
images.linuxcontainers.org: 45.45.148.8        -- link: eno1
                            2602:fc62:a:1::8   -- link: eno1

Strangely now, my /etc/resolv.conf was rewritten as

nameserver 127.0.0.53
options edns0 trust-ad
search .

The good news is I can launch containers now! Thanks, all.

-brmiller