Reboot and no bridge, nor veths available

Hi,

I’ve setup four containers using Incus.
For some unknown reasond, the server (VPS) rebooted.
After the reboot, there was no bridge available for networking and no veths.
All containers are stopped.

Is this expected? What shoud I do?

What error are you getting when you’re starting to start a container?
Did you look for errors in the Incus log?

The server (VPS) rebooted at the end of 2024-11-23.
Before the reboot, there was an error connecting to the container of unbound, expected at the time.
The bridge used before reboot was incusbr1.

time="2024-11-22T19:22:43Z" level=error msg="Failed updating instance types" err="Get \"https://images.linuxcontainers.org/meta/instance-types/.yaml\": lookup images.linuxcontainers.org on 10.239.1.181:53: read udp 10.239.1.1:39329->10.239.1.181:53: read: connection refused"
time="2024-11-23T19:22:43Z" level=warning msg="Failed to update instance types: Get \"https://images.linuxcontainers.org/meta/instance-types/.yaml\": lookup images.linuxcontainers.org on 10.239.1.181:53: read udp 10.239.1.1:38318->10.239.1.181:53: read: connection refused"
time="2024-11-23T19:22:43Z" level=error msg="Failed updating instance types" err="Get \"https://images.linuxcontainers.org/meta/instance-types/.yaml\": lookup images.linuxcontainers.org on 10.239.1.181:53: read udp 10.239.1.1:38318->10.239.1.181:53: read: connection refused"
time="2024-11-24T00:09:56Z" level=warning msg="AppArmor support has been disabled because of lack of kernel support"
time="2024-11-24T00:09:56Z" level=warning msg=" - AppArmor support has been disabled, Disabled because of lack of kernel support"
time="2024-11-24T00:09:56Z" level=warning msg=" - Couldn't find the CGroup blkio.weight, disk priority will be ignored"
time="2024-11-24T00:09:56Z" level=warning msg=" - Couldn't find the CGroup hugetlb controller, hugepage limits will be ignored"
time="2024-11-24T00:09:56Z" level=warning msg="Instance type not operational" driver=qemu err="KVM support is missing (no /dev/kvm)" type=virtual-machine
time="2024-11-24T00:09:58Z" level=error msg="Failed initializing network" err="Failed starting: The DNS and DHCP service exited prematurely: exit status 2 (\"dnsmasq: failed to create listening socket for 10.239.1.1: Address already in use\")" network=incusbr1 project=default
time="2024-11-24T00:10:58Z" level=error msg="Failed initializing network" err="Failed starting: The DNS and DHCP service exited prematurely: exit status 2 (\"dnsmasq: failed to create listening socket for 10.239.1.1: Address already in use\")" network=incusbr1 project=default
time="2024-11-24T00:11:58Z" level=error msg="Failed initializing network" err="Failed starting: The DNS and DHCP service exited prematurely: exit status 2 (\"dnsmasq: failed to create listening socket for 10.239.1.1: Address already in use\")" network=incusbr1 project=default
time="2024-11-24T00:12:58Z" level=error msg="Failed initializing network" err="Failed starting: The DNS and DHCP service exited prematurely: exit status 2 (\"dnsmasq: failed to create listening socket for 10.239.1.1: Address already in use\")" network=incusbr1 project=default

These messages continue:

time="2024-11-25T23:29:46Z" level=error msg="Failed initializing network" err="Failed starting: The DNS and DHCP service exited prematurely: exit status 2 (\"dnsmasq: failed to create listening socket for 10.239.1.1: Address already in use\")" network=incusbr1 project=default
time="2024-11-25T23:30:47Z" level=error msg="Failed initializing network" err="Failed starting: The DNS and DHCP service exited prematurely: exit status 2 (\"dnsmasq: failed to create listening socket for 10.239.1.1: Address already in use\")" network=incusbr1 project=default
time="2024-11-25T23:31:47Z" level=error msg="Failed initializing network" err="Failed starting: The DNS and DHCP service exited prematurely: exit status 2 (\"dnsmasq: failed to create listening socket for 10.239.1.1: Address already in use\")" network=incusbr1 project=default

ifconfig or ip addr list only shows eth0 and lo0.

A restart of the incusd service shows in the log:

time="2024-11-25T23:35:33Z" level=warning msg="AppArmor support has been disabled because of lack of kernel support"
time="2024-11-25T23:35:33Z" level=warning msg=" - AppArmor support has been disabled, Disabled because of lack of kernel support"
time="2024-11-25T23:35:33Z" level=warning msg=" - Couldn't find the CGroup blkio.weight, disk priority will be ignored"
time="2024-11-25T23:35:33Z" level=warning msg=" - Couldn't find the CGroup hugetlb controller, hugepage limits will be ignored"
time="2024-11-25T23:35:33Z" level=warning msg="Instance type not operational" driver=qemu err="KVM support is missing (no /dev/kvm)" type=virtual-machine
time="2024-11-25T23:35:35Z" level=error msg="Failed initializing network" err="Failed starting: The DNS and DHCP service exited prematurely: exit status 2 (\"dnsmasq: failed to create listening socket for 10.239.1.1: Address already in use\")" network=incusbr1 project=default

Sounds like you may have installed a DNS or DHCP server on your server which is conflicting with the one that’s built into Incus.

Can you look at netstat -lnp | grep 67 and netstat -lnp | grep 53 for potential candidates?

Hi,

Nothing is shown for port 67.

# netstat -lnp | grep 67

# netstat -lnp | grep 53
tcp6       0      0 :::53                   :::*                    LISTEN      10296/incusd        
udp        0      0 0.0.0.0:5353            0.0.0.0:*                           1021/avahi-daemon:  
udp6       0      0 :::53                   :::*                                10296/incusd        
udp6       0      0 :::5353                 :::*                                1021/avahi-daemon:  

This VPS has not ipv6 address.

Hold on, why is Incus itself listening on port 53?

Can you show incus config show?

I’ve setup unbound on a container and issued this command:

incus config set core.dns_address :53

Maybe I shouldn’t… If not, what’s the use of it?

# incus config show
config:
  core.dns_address: :53

Ah yeah, that’d be the problem, you should either run that on a specific IP address (other than your bridge’s address) or run it on a non-standard port. I commonly use 8553 for that.

You can then easily configure unbound or other DNS servers to query on that non-standard port.

1 Like

Everything is fine now, just had to

incus config set core.dns_address :8553

Thank you very much @stgraber.
All the best to you!