Containers do not have outgoing internet access

Hi,

I was happily using LXD containers for a couple of years now, recently I have noticed problem accessing internet from within my containers, restarting lxd and restarting the host machine previously resolved the problem. But this time nothing is helping, I have even tried uninstalling lxd and re-installing it.

My system:
Host: Ubuntu 20.04
LXD version 4.13

I am using all default configuration values during the installation, nothing is special.

$ lxc ls

+-----------+---------+----------------------+-----------------------------------------------+-----------+-----------+
|   NAME    |  STATE  |         IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+-----------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| ubuntuone | RUNNING | 10.235.48.173 (eth0) | fd42:a188:52d4:442d:216:3eff:fe8f:9757 (eth0) | CONTAINER | 0         |
+-----------+---------+----------------------+-----------------------------------------------+-----------+-----------+

$ lxc exec ubuntuone -- ping -c1 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

Network on the host machine:

root@debashis-ThinkPad:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s31f6: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether e8:6a:64:89:01:31 brd ff:ff:ff:ff:ff:ff
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    link/ether 20:79:18:5f:4d:bc brd ff:ff:ff:ff:ff:ff
    inet 192.168.87.20/24 brd 192.168.87.255 scope global dynamic noprefixroute wlp2s0
       valid_lft 46677sec preferred_lft 46677sec
    inet6 fe80::57d:6815:66b4:c320/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
4: enx3c18a0ba710c: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 3c:18:a0:ba:71:0c brd ff:ff:ff:ff:ff:ff
6: br-2c0eaa4592c2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:66:2e:07:40 brd ff:ff:ff:ff:ff:ff
    inet 172.26.0.1/16 brd 172.26.255.255 scope global br-2c0eaa4592c2
       valid_lft forever preferred_lft forever
7: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:50:85:c8:87 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
11: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:c9:45:47 brd ff:ff:ff:ff:ff:ff
    inet 10.235.48.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
    inet6 fd42:a188:52d4:442d::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fec9:4547/64 scope link 
       valid_lft forever preferred_lft forever
13: vethf473a9d8@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether 32:ab:5f:19:85:b1 brd ff:ff:ff:ff:ff:ff link-netnsid 0
root@debashis-ThinkPad:~# ip r
default via 192.168.87.1 dev wlp2s0 proto dhcp metric 600 
10.235.48.0/24 dev lxdbr0 proto kernel scope link src 10.235.48.1 
169.254.0.0/16 dev wlp2s0 scope link metric 1000 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 
172.26.0.0/16 dev br-2c0eaa4592c2 proto kernel scope link src 172.26.0.1 linkdown 
192.168.87.0/24 dev wlp2s0 proto kernel scope link src 192.168.87.20 metric 600

iptables on the host machine:

root@debashis-ThinkPad:~# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy DROP)
target     prot opt source               destination         
DOCKER-USER  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  10.1.0.0/16          anywhere             /* generated for MicroK8s pods */
ACCEPT     all  --  anywhere             10.1.0.0/16          /* generated for MicroK8s pods */

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain DOCKER (2 references)
target     prot opt source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination         
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere            
DROP       all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-USER (1 references)
target     prot opt source               destination         
RETURN     all  --  anywhere             anywhere

I applied the solution of this other thread Lxd uses iptables-legacy even if iptables-nft is default and that solved my problem :slight_smile:

which is basically to flush all the iptable rules. This is all I did -

# for ipt in iptables iptables-legacy ip6tables ip6tables-legacy; do $ipt --flush; $ipt --flush -t nat; $ipt --delete-chain; $ipt --delete-chain -t nat; $ipt -P FORWARD ACCEPT; $ipt -P INPUT ACCEPT; $ipt -P OUTPUT ACCEPT; done
# systemctl reload snap.lxd.daemon 

And that was it

1 Like