Problems with bridge adapter, no connection to the internet from containers (docker)

Hi,

I have problem with my fresh installation of lxd on physical server.

Hardware details:

  • Asus H110M-A/M.2 Micro ATX
  • Intel Core i7-6700 4 x 3.4 GHz
  • DDR4 HyperX Predator 16GB 3200Mhz CL16

Os nad LXD details:

  • Ubuntu 22.04.1 LTS
  • LXC version: 5.0.1
  • LXD version (snap list): lxd 5.0.1-9dcf35b 23541 5.0/stable/… canonical✓ -

This is how I did the whole setup, first of all I changed default location of lxc to different partition (I’m not sure if that may have any impact on my further problems, but maybe that was worth to mention):

snap stop lxd
mkdir /mnt/drives/virt/lxd

Than I added new mount point to /etc/fstab

/mnt/drives/virt/lxd /var/snap/lxd/common/lxd none bind 0 0

Next step was lxd init, this is how I did answer to the questions from the creator:

root@nas:/root# lxd init
Would you like to use LXD clustering? (yes/no) [default=no]: 
Do you want to configure a new storage pool? (yes/no) [default=yes]: 
Name of the new storage pool [default=default]: 
Name of the storage backend to use (dir, lvm, zfs, ceph, btrfs) [default=zfs]: dir
Would you like to connect to a MAAS server? (yes/no) [default=no]: 
Would you like to create a new local network bridge? (yes/no) [default=yes]: 
What should the new bridge be called? [default=lxdbr0]: 
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: 
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: none
Would you like the LXD server to be available over the network? (yes/no) [default=no]: yes
Address to bind LXD to (not including port) [default=all]: 
Port to bind LXD to [default=8443]: 
Trust password for new clients: 
Again: 
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]: 
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: 

Bridge adapter lxdbr0 was created, please find output of ip a from the host: admin@nas:~$ ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state U - Pastebin.com
This is the output of the ip route list from the host:

admin@nas:~$ ip route list
default via 192.168.1.1 dev enp0s31f6 proto dhcp src 192.168.1.204 metric 100 
10.168.17.0/24 dev lxdbr0 proto kernel scope link src 10.168.17.1 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 
172.18.0.0/16 dev br-7df03143e143 proto kernel scope link src 172.18.0.1 
172.19.0.0/16 dev br-518524b036cd proto kernel scope link src 172.19.0.1 
192.168.1.0/24 dev enp0s31f6 proto kernel scope link src 192.168.1.204 metric 100 
192.168.1.1 dev enp0s31f6 proto dhcp scope link src 192.168.1.204 metric 100 

Remote connection from other lxc server works just fine, I can create both containers and virtual machines, locally and from the remote server. The problem is, that my containers and vms do not have access to the internet and to each other - I cannot install anything from the repos on them, even apt update does not work.

This is the output of ip a from the cntainer:

root@full-elephant:~# ip route
default via 10.168.17.1 dev eth0 proto dhcp src 10.168.17.193 metric 100 
10.168.17.0/24 dev eth0 proto kernel scope link src 10.168.17.193 metric 100 
10.168.17.1 dev eth0 proto dhcp scope link src 10.168.17.193 metric 100 

I can ping from the host to container:

admin@nas:~$ lxc list
+----------+---------+----------------------+------+-----------+-----------+
|   NAME   |  STATE  |         IPV4         | IPV6 |   TYPE    | SNAPSHOTS |
+----------+---------+----------------------+------+-----------+-----------+
| ubuntu01 | RUNNING | 10.168.17.225 (eth0) |      | CONTAINER | 0         |
+----------+---------+----------------------+------+-----------+-----------+
admin@nas:~$ ping 10.168.17.225
PING 10.168.17.225 (10.168.17.225) 56(84) bytes of data.
64 bytes from 10.168.17.225: icmp_seq=1 ttl=64 time=0.088 ms
64 bytes from 10.168.17.225: icmp_seq=2 ttl=64 time=0.061 ms
64 bytes from 10.168.17.225: icmp_seq=3 ttl=64 time=0.059 ms
^C
--- 10.168.17.225 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2053ms
rtt min/avg/max/mdev = 0.059/0.069/0.088/0.013 ms

And from the host to bridge adapter:

admin@nas:~$ ping 10.168.17.1
PING 10.168.17.1 (10.168.17.1) 56(84) bytes of data.
64 bytes from 10.168.17.1: icmp_seq=1 ttl=64 time=0.083 ms
64 bytes from 10.168.17.1: icmp_seq=2 ttl=64 time=0.049 ms
64 bytes from 10.168.17.1: icmp_seq=3 ttl=64 time=0.050 ms
^C
--- 10.168.17.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2044ms
rtt min/avg/max/mdev = 0.049/0.060/0.083/0.015 ms

I can ping bridge adapter from the container:

root@ubuntu01:~# ping 10.168.17.1
PING 10.168.17.1 (10.168.17.1) 56(84) bytes of data.
64 bytes from 10.168.17.1: icmp_seq=1 ttl=64 time=0.099 ms
64 bytes from 10.168.17.1: icmp_seq=2 ttl=64 time=0.073 ms
64 bytes from 10.168.17.1: icmp_seq=3 ttl=64 time=0.072 ms
^C
--- 10.168.17.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2047ms
rtt min/avg/max/mdev = 0.072/0.081/0.099/0.012 ms

However, I cannot update apt database on the container:

root@ubuntu01:~# apt update
Ign:1 http://security.ubuntu.com/ubuntu jammy-security InRelease                                                  
Ign:2 http://archive.ubuntu.com/ubuntu jammy InRelease                                                            
Ign:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Ign:1 http://security.ubuntu.com/ubuntu jammy-security InRelease
Ign:2 http://archive.ubuntu.com/ubuntu jammy InRelease
Ign:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Ign:1 http://security.ubuntu.com/ubuntu jammy-security InRelease
Ign:2 http://archive.ubuntu.com/ubuntu jammy InRelease
Ign:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Err:1 http://security.ubuntu.com/ubuntu jammy-security InRelease
  Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (91.189.91.39), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.39), connection timed out
Err:2 http://archive.ubuntu.com/ubuntu jammy InRelease
  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.39), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.36), connection timed out
Err:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/jammy/InRelease  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.39), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.36), connection timed out
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/jammy-updates/InRelease  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable)
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/jammy-security/InRelease  Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (91.189.91.39), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.39), connection timed out
W: Some index files failed to download. They have been ignored, or old ones used instead.

I can’t ping google’s dns as well:

root@ubuntu01:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
^C
--- 8.8.8.8 ping statistics ---
26 packets transmitted, 0 received, 100% packet loss, time 25598ms

Pinging from one container to another is not possible as well:

admin@nas:~$ lxc list
+----------+---------+----------------------+------+-----------+-----------+
|   NAME   |  STATE  |         IPV4         | IPV6 |   TYPE    | SNAPSHOTS |
+----------+---------+----------------------+------+-----------+-----------+
| ubuntu01 | RUNNING | 10.168.17.225 (eth0) |      | CONTAINER | 0         |
+----------+---------+----------------------+------+-----------+-----------+
| ubuntu02 | RUNNING | 10.168.17.243 (eth0) |      | CONTAINER | 0         |
+----------+---------+----------------------+------+-----------+-----------+

root@ubuntu01:~# ping 10.168.17.243
PING 10.168.17.243 (10.168.17.243) 56(84) bytes of data.
^C
--- 10.168.17.243 ping statistics ---
22 packets transmitted, 0 received, 100% packet loss, time 21504ms

Any idea what could go wrong? I tested exacly the same setup on raspberry pi with Ubuntu Server 22 and everything works just fine, no idea why is it not working on this machine.

It sounds as if docker was getting in the way, please check if that helps:

Thanks for reply. Any idea, how to diagnose and test it? I have desktop with manjaro and docker, and lxd works perfectly fine there.

I’d first check if the iptables suggestion helps.

Quick summary of outputs:

Ouptut of ip a on the host: https://pastebin.com/4ayezDm5
ip a of the container: https://pastebin.com/d0LwLRxu
ip route list on the host: https://pastebin.com/N8BPYT86
ip route list of the container: https://pastebin.com/FrpRDNFH

I have docker, but there is only lxd listening on port 8443 (I use this port for remote conenction to lxd and it works fine): https://pastebin.com/ZGvnL5Rt

Please show output of sudo iptables-save.

Also see How to configure your firewall - LXD documentation as its almost certainly this.

Sorry, I don’t understand why are you asking for iptables-save. I thought that this command is used to save alteration of iptables, but, I haven’t done any manual alteration so far.

Here is the output of iptables -L -n -v, I hope that could give us meaningful information.

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 4 packets, 1256 bytes)
 pkts bytes target     prot opt in     out     source               destination         
17995   16M DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
17995   16M DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           
 7993   15M ACCEPT     all  --  *      br-7df03143e143  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
 1895  256K DOCKER     all  --  *      br-7df03143e143  0.0.0.0/0            0.0.0.0/0           
 7099  658K ACCEPT     all  --  br-7df03143e143 !br-7df03143e143  0.0.0.0/0            0.0.0.0/0           
 1892  256K ACCEPT     all  --  br-7df03143e143 br-7df03143e143  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      br-518524b036cd  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-518524b036cd  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-518524b036cd !br-518524b036cd  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  br-518524b036cd br-518524b036cd  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (3 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  !br-518524b036cd br-518524b036cd  0.0.0.0/0            172.19.0.2           tcp dpt:3000
    0     0 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.2           tcp dpt:3000
    0     0 ACCEPT     tcp  --  !br-518524b036cd br-518524b036cd  0.0.0.0/0            172.19.0.2           tcp dpt:22
    0     0 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.4           tcp dpt:8083
    0     0 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.5           tcp dpt:8096
    1    60 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.6           tcp dpt:55555
    0     0 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.6           tcp dpt:8888
    0     0 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.7           tcp dpt:80
    0     0 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.8           tcp dpt:80
    0     0 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.9           tcp dpt:443
    0     0 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.9           tcp dpt:80
    1    60 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.11          tcp dpt:443
    0     0 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.11          tcp dpt:81
    1    60 ACCEPT     tcp  --  !br-7df03143e143 br-7df03143e143  0.0.0.0/0            172.18.0.11          tcp dpt:80

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
 8103  712K DOCKER-ISOLATION-STAGE-2  all  --  br-7df03143e143 !br-7df03143e143  0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  br-518524b036cd !br-518524b036cd  0.0.0.0/0            0.0.0.0/0           
16991   16M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (3 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      br-7df03143e143  0.0.0.0/0            0.0.0.0/0           
 1004 54240 DROP       all  --  *      br-518524b036cd  0.0.0.0/0            0.0.0.0/0           
 7099  658K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
17995   16M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0         

No, man iptables-save says iptables-save — dump iptables rules. Its a way of displaying all of the active rules and covers more than iptables -L -v -n.

Anyway, what you have provided is sufficient to show that, as expected, you are being affected by the way that docker sets the FORWARD policy to drop.

See How to configure your firewall - LXD documentation for more info.

1 Like