LXD containers can't connect to network. Using ubuntu image ping fails and apt update also fails (docker)

LXD
1

Hi guys,

So I reinstalled my laptop and using arch linux with btrfs.
Noticed that my containers don’t have network access.


$ lxd --version
5.9

$ lxc launch ubuntu:20.04
$ lxc exec learning-man -- sudo --user ubuntu --login
$ ping 8.8.8.8

ping fails and also apt update command fails to get any connection to network.

following commands for how I tried to solve this problem.

$ lxc list
+-----------------+---------+---------------------+------+-----------+-----------+
|      NAME       |  STATE  |        IPV4         | IPV6 |   TYPE    | SNAPSHOTS |
+-----------------+---------+---------------------+------+-----------+-----------+
| enormous-earwig | STOPPED |                     |      | CONTAINER | 0         |
+-----------------+---------+---------------------+------+-----------+-----------+
| fine-mastodon   | RUNNING | 10.27.28.154 (eth0) |      | CONTAINER | 0         |
+-----------------+---------+---------------------+------+-----------+-----------+
| gui1804         | RUNNING | 10.27.28.85 (eth0)  |      | CONTAINER | 0         |
+-----------------+---------+---------------------+------+-----------+-----------+
| learning-man    | RUNNING | 10.27.28.25 (eth0)  |      | CONTAINER | 0         |
+-----------------+---------+---------------------+------+-----------+-----------+
| ubuntu          | RUNNING | 10.27.28.60 (eth0)  |      | CONTAINER | 0         |



$ lxc network list
+---------+----------+---------+---------------+------+-------------+---------+---------+
|  NAME   |   TYPE   | MANAGED |     IPV4      | IPV6 | DESCRIPTION | USED BY |  STATE  |
+---------+----------+---------+---------------+------+-------------+---------+---------+
| docker0 | bridge   | NO      |               |      |             | 0       |         |
+---------+----------+---------+---------------+------+-------------+---------+---------+
| lxdbr0  | bridge   | YES     | 10.27.28.1/24 | none |             | 6       | CREATED |
+---------+----------+---------+---------------+------+-------------+---------+---------+
| wlan0   | physical | NO      |               |      |             | 0       |         |
+---------+----------+---------+---------------+------+-------------+---------+---------+
| wwan0   | physical | NO      |               |      |             | 0       |         |

I think these are more important, but didn’t figure out which one affecting the network connectivity. As it is using bridged network.


$ lxc profile edit default

config: {}
description: Default LXD profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: default
used_by:
- /1.0/instances/enormous-earwig
- /1.0/instances/fine-mastodon
- /1.0/instances/gui1804
- /1.0/instances/learning-man
- /1.0/instances/ubuntu


$ lxc network edit lxdbr0
config:
  ipv4.address: 10.27.28.1/24
  ipv6.address: none
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/profiles/default
- /1.0/instances/gui1804
- /1.0/instances/learning-man
- /1.0/instances/ubuntu
- /1.0/instances/enormous-earwig
- /1.0/instances/fine-mastodon
managed: true
status: Created
locations:
- none


$ systemctl status lxd
Jan 04 13:31:28 linuxfun dnsmasq-dhcp[1816]: read /var/lib/lxd/networks/lxdbr0/dnsmasq.hosts/gui1804.eth0
Jan 04 13:31:28 linuxfun dnsmasq-dhcp[1816]: read /var/lib/lxd/networks/lxdbr0/dnsmasq.hosts/learning-man.eth0
Jan 04 13:31:28 linuxfun dnsmasq-dhcp[1816]: read /var/lib/lxd/networks/lxdbr0/dnsmasq.hosts/ubuntu.eth0
Jan 04 13:31:29 linuxfun lxd[1637]: time="2023-01-04T13:31:29+02:00" level=warning msg="Failed to update instance types: Get \"https://images.linuxcontainers.org/meta/instance-types/.yaml\": lookup images.linuxcontainers.org: Temporary failure in name resolution"
Jan 04 13:31:39 linuxfun dnsmasq[1816]: reading /etc/resolv.conf
Jan 04 13:31:39 linuxfun dnsmasq[1816]: using nameserver 192.168.100.1#53
Jan 04 13:31:39 linuxfun dnsmasq[1816]: using only locally-known addresses for lxd
Jan 04 13:31:41 linuxfun systemd[1]: Started LXD Container Hypervisor.

Problem is most likely with iptables rules I guess, but using these following based on my google-fu it still didn’t help.

# tried
sudo iptables -I DOCKER-USER -i lxdbr0  -o lo -j ACCEPT
sudo iptables -I DOCKER-USER -o lxdbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
sudo ufw allow in on lxdbr0
sudo ufw route allow in on lxdbr0
sudo ufw route allow out on lxdbr0


sudo iptables -L
[sudo] password for penguinland:
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy DROP)
target     prot opt source               destination
DOCKER-USER  all  --  anywhere             anywhere
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain DOCKER (1 references)
target     prot opt source               destination

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target     prot opt source               destination
DROP       all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere

Chain DOCKER-USER (1 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere

I don’t really know what else or how I should try to solve this. Banged my head to the table quite many times. Any help much appreciated.

2
3

iptables -I DOCKER-USER -i <network_bridge> -o <external_interface> -j ACCEPT
iptables -I DOCKER-USER -o <network_bridge> -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

I’ve tried to set those
network bridge I suppose it was like

sudo iptables -I DOCKER-USER -i lxdbr0  -o lo -j ACCEPT
sudo iptables -I DOCKER-USER -o lxdbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

But I might be totally wrong. As it didn’t help.

4

Using
ip addr
command

I can see interfaces, but no clue which is network_bridge nor external_interface.

5

See LXD and Docker Firewall Redux - How to deal with FORWARD policy set to drop - #4 by tomp

6 
 iptables -I DOCKER-USER  -j ACCEPT

Didn’t help. I guess I could try to reboot machine after command or something?

7

I would first start with entirely flushing all firewall rules to confirm that the firewall is indeed the problem.

8

sudo iptables -S
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j ACCEPT
-A DOCKER-USER -j RETURN

So it looks like this, but yeh. I’ll try flushing firewall rules.

9

Flushed my firewall rules

now its just

-P input accept
-P Forward accept
-P Output accept

Still network not working from container. Can’t access internet.

10

Reload LXD so it can add its firewall rules and then show output of sudo iptables-save, further more please show ip a and ip r from host and container.

Please also show tcpdump -i lxdbr0 -nn on host when trying to ping the external network, and then re-run with tcpdump -i <ext_if> -nn to see if the packets are making it out.

11

$ systemctl restart lxd

$ sudo iptables-save

Generated by iptables-save v1.8.8 (nf_tables) on Thu Jan 5 10:26:29 2023

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT

Completed on Thu Jan 5 10:26:29 2023

Generated by iptables-save v1.8.8 (nf_tables) on Thu Jan 5 10:26:29 2023

*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
COMMIT

Completed on Thu Jan 5 10:26:29 2023

HOST MACHINE

$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:5c:29:73:c3 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
6: wwan0: <POINTOPOINT,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/none
7: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 58:ce:2a:fc:33:34 brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.24/24 brd 192.168.100.255 scope global dynamic noprefixroute wlan0
       valid_lft 86259sec preferred_lft 86259sec
    inet6 fe80::e5ce:a2df:d3b3:c42b/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
14: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:61:9a:b9 brd ff:ff:ff:ff:ff:ff
    inet 10.27.28.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
16: veth0138a2f0@if15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether 0a:e0:8e:80:02:00 brd ff:ff:ff:ff:ff:ff link-netnsid 0
18: vethef55b590@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether 9a:d4:a5:24:0c:15 brd ff:ff:ff:ff:ff:ff link-netnsid 1
20: vethd65b5bfe@if19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether ba:16:ae:90:33:b6 brd ff:ff:ff:ff:ff:ff link-netnsid 2
22: veth416b2c6b@if21: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether c6:2f:8c:91:2f:0f brd ff:ff:ff:ff:ff:ff link-netnsid 3


$ ip r
default via 192.168.100.1 dev wlan0 proto dhcp src 192.168.100.24 metric 600
10.27.28.0/24 dev lxdbr0 proto kernel scope link src 10.27.28.1
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.100.0/24 dev wlan0 proto kernel scope link src 192.168.100.24 metric 600


CONTAINER


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
19: eth0@if20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:84:ed:27 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.27.28.25/24 brd 10.27.28.255 scope global dynamic eth0
       valid_lft 3442sec preferred_lft 3442sec
    inet6 fe80::216:3eff:fe84:ed27/64 scope link
       valid_lft forever preferred_lft forever


default via 10.27.28.1 dev eth0 proto dhcp src 10.27.28.25 metric 100
10.27.28.0/24 dev eth0 proto kernel scope link src 10.27.28.25
10.27.28.1 dev eth0 proto dhcp scope link src 10.27.28.25 metric 100

Doing tcpdump while trying to run apt update

sudo tcpdump -i lxdbr0 -nn
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on lxdbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
10:29:19.046206 IP 10.27.28.154.56618 > 185.125.188.59.443: Flags [S], seq 2776693986, win 64240, options [mss 1460,sackOK,TS val 3070580524 ecr 0,nop,wscale 7], length 0
10:29:20.112941 IP 10.27.28.154.40512 > 185.125.188.54.443: Flags [S], seq 3042624339, win 64240, options [mss 1460,sackOK,TS val 961428877 ecr 0,nop,wscale 7], length 0
10:29:20.326288 IP 10.27.28.25.43738 > 185.125.188.59.443: Flags [S], seq 3163562916, win 64240, options [mss 1460,sackOK,TS val 1076167079 ecr 0,nop,wscale 7], length 0
10:29:20.539558 IP 10.27.28.60.49874 > 185.125.188.58.443: Flags [S], seq 520329376, win 64240, options [mss 1460,sackOK,TS val 696014621 ecr 0,nop,wscale 7], length 0
10:29:21.179462 ARP, Request who-has 10.27.28.154 tell 10.27.28.1, length 28
10:29:21.179542 ARP, Reply 10.27.28.154 is-at 00:16:3e:63:00:b6, length 28
10:29:21.197505 IP 10.27.28.25.35982 > 10.27.28.1.53: 33360+ [1au] A? learning-man.lxd. (45)
10:29:21.197577 IP 10.27.28.25.50648 > 10.27.28.1.53: 46413+ [1au] AAAA? learning-man.lxd. (45)
10:29:21.197676 IP 10.27.28.1.53 > 10.27.28.25.35982: 33360* 1/0/1 A 10.27.28.25 (61)
10:29:21.197709 IP 10.27.28.1.53 > 10.27.28.25.50648: 46413 0/0/1 (45)
10:29:21.819551 IP 10.27.28.25.59068 > 185.125.188.58.443: Flags [S], seq 787803648, win 64240, options [mss 1460,sackOK,TS val 3134939501 ecr 0,nop,wscale 7], length 0
10:29:21.819580 IP 10.27.28.60.52388 > 185.125.188.55.443: Flags [S], seq 1955269719, win 64240, options [mss 1460,sackOK,TS val 2999382422 ecr 0,nop,wscale 7], length 0
10:29:22.246183 ARP, Request who-has 10.27.28.25 tell 10.27.28.1, length 28
10:29:22.246312 ARP, Reply 10.27.28.25 is-at 00:16:3e:84:ed:27, length 28
10:29:22.459375 ARP, Request who-has 10.27.28.60 tell 10.27.28.1, length 28
10:29:22.459510 ARP, Reply 10.27.28.60 is-at 00:16:3e:7d:3a:6b, length 28
10:29:22.459471 ARP, Request who-has 10.27.28.1 tell 10.27.28.60, length 28
10:29:22.459649 ARP, Reply 10.27.28.1 is-at 00:16:3e:61:9a:b9, length 28
10:29:22.467058 IP 10.27.28.25.37370 > 10.27.28.1.53: 33230+ [1au] A? learning-man.lxd. (45)
10:29:22.467122 IP 10.27.28.25.52276 > 10.27.28.1.53: 59317+ [1au] AAAA? learning-man.lxd. (45)
10:29:22.467159 IP 10.27.28.1.53 > 10.27.28.25.37370: 33230* 1/0/1 A 10.27.28.25 (61)
10:29:22.467189 IP 10.27.28.1.53 > 10.27.28.25.52276: 59317 0/0/1 (45)
10:29:22.521852 IP 10.27.28.25.59468 > 10.27.28.1.53: 34593+ [1au] SRV? _http._tcp.security.ubuntu.com. (59)
10:29:22.521961 IP 10.27.28.25.48329 > 10.27.28.1.53: 4552+ [1au] SRV? _http._tcp.archive.ubuntu.com. (58)
10:29:22.540732 IP 10.27.28.1.53 > 10.27.28.25.59468: 34593 NXDomain 0/1/1 (120)
10:29:22.540769 IP 10.27.28.1.53 > 10.27.28.25.48329: 4552 0/1/1 (119)
10:29:22.540908 IP 10.27.28.25.59468 > 10.27.28.1.53: 34593+ SRV? _http._tcp.security.ubuntu.com. (48)
10:29:22.540997 IP 10.27.28.1.53 > 10.27.28.25.59468: 34593 NXDomain 0/0/0 (48)
10:29:22.541620 IP 10.27.28.25.50874 > 10.27.28.1.53: 45165+ [1au] A? security.ubuntu.com. (48)
10:29:22.541690 IP 10.27.28.25.56282 > 10.27.28.1.53: 2883+ [1au] A? archive.ubuntu.com. (47)
10:29:22.541747 IP 10.27.28.25.43246 > 10.27.28.1.53: 24172+ [1au] AAAA? security.ubuntu.com. (48)
10:29:22.541797 IP 10.27.28.25.43484 > 10.27.28.1.53: 48358+ [1au] AAAA? archive.ubuntu.com. (47)
10:29:22.561811 IP 10.27.28.1.53 > 10.27.28.25.50874: 45165 4/0/1 A 91.189.91.39, A 185.125.190.39, A 185.125.190.36, A 91.189.91.38 (112)
10:29:22.564986 IP 10.27.28.1.53 > 10.27.28.25.56282: 2883 4/0/1 A 91.189.91.39, A 91.189.91.38, A 185.125.190.36, A 185.125.190.39 (111)
10:29:22.569744 IP 10.27.28.1.53 > 10.27.28.25.43484: 48358 4/0/1 AAAA 2620:2d:4000:1::19, AAAA 2620:2d:4000:1::16, AAAA 2001:67c:1562::15, AAAA 2001:67c:1562::18 (159)
10:29:22.570856 IP 10.27.28.25.45842 > 91.189.91.39.80: Flags [S], seq 3515504760, win 64240, options [mss 1460,sackOK,TS val 3894386980 ecr 0,nop,wscale 7], length 0
10:29:22.592712 IP 10.27.28.1.53 > 10.27.28.25.43246: 24172 4/0/1 AAAA 2620:2d:4000:1::19, AAAA 2620:2d:4000:1::16, AAAA 2001:67c:1562::18, AAAA 2001:67c:1562::15 (160)
10:29:22.594063 IP 10.27.28.25.45846 > 91.189.91.39.80: Flags [S], seq 3392921426, win 64240, options [mss 1460,sackOK,TS val 3894387003 ecr 0,nop,wscale 7], length 0
10:29:22.821764 IP 10.27.28.25.43680 > 91.189.91.38.80: Flags [S], seq 2611796340, win 64240, options [mss 1460,sackOK,TS val 3718178721 ecr 0,nop,wscale 7], length 0
10:29:22.844929 IP 10.27.28.25.36230 > 185.125.190.39.80: Flags [S], seq 3596833942, win 64240, options [mss 1460,sackOK,TS val 3056187222 ecr 0,nop,wscale 7], length 0
10:29:23.072729 IP 10.27.28.25.36342 > 185.125.190.36.80: Flags [S], seq 2770239124, win 64240, options [mss 1460,sackOK,TS val 191908704 ecr 0,nop,wscale 7], length 0
10:29:23.095862 IP 10.27.28.25.36352 > 185.125.190.36.80: Flags [S], seq 2457288259, win 64240, options [mss 1460,sackOK,TS val 191908727 ecr 0,nop,wscale 7], length 0
10:29:23.099542 IP 10.27.28.154.56618 > 185.125.188.59.443: Flags [S], seq 2776693986, win 64240, options [mss 1460,sackOK,TS val 3070584578 ecr 0,nop,wscale 7], length 0
10:29:23.215782 IP6 fe80::216:3eff:fe84:ed27 > ff02::2: ICMP6, router solicitation, length 16
10:29:23.323580 IP 10.27.28.25.36232 > 185.125.190.39.80: Flags [S], seq 1564624620, win 64240, options [mss 1460,sackOK,TS val 3056187701 ecr 0,nop,wscale 7], length 0
10:29:23.346424 IP 10.27.28.25.43694 > 91.189.91.38.80: Flags [S], seq 4183826752, win 64240, options [mss 1460,sackOK,TS val 3718179246 ecr 0,nop,wscale 7], length 0
10:29:23.579559 IP 10.27.28.25.45842 > 91.189.91.39.80: Flags [S], seq 3515504760, win 64240, options [mss 1460,sackOK,TS val 3894387989 ecr 0,nop,wscale 7], length 0
10:29:23.606241 IP 10.27.28.25.45846 > 91.189.91.39.80: Flags [S], seq 3392921426, win 64240, options [mss 1460,sackOK,TS val 3894388015 ecr 0,nop,wscale 7], length 0
10:29:23.846212 IP 10.27.28.25.36230 > 185.125.190.39.80: Flags [S], seq 3596833942, win 64240, options [mss 1460,sackOK,TS val 3056188223 ecr 0,nop,wscale 7], length 0
10:29:23.846228 IP 10.27.28.25.43680 > 91.189.91.38.80: Flags [S], seq 2611796340, win 64240, options [mss 1460,sackOK,TS val 3718179745 ecr 0,nop,wscale 7], length 0
10:29:24.086276 IP 10.27.28.25.36342 > 185.125.190.36.80: Flags [S], seq 2770239124, win 64240, options [mss 1460,sackOK,TS val 191909717 ecr 0,nop,wscale 7], length 0
10:29:24.112949 IP 10.27.28.25.36352 > 185.125.190.36.80: Flags [S], seq 2457288259, win 64240, options [mss 1460,sackOK,TS val 191909744 ecr 0,nop,wscale 7], length 0
10:29:24.326249 IP 10.27.28.25.36232 > 185.125.190.39.80: Flags [S], seq 1564624620, win 64240, options [mss 1460,sackOK,TS val 3056188703 ecr 0,nop,wscale 7], length 0
10:29:24.352868 IP 10.27.28.25.43694 > 91.189.91.38.80: Flags [S], seq 4183826752, win 64240, options [mss 1460,sackOK,TS val 3718180252 ecr 0,nop,wscale 7], length 0
10:29:24.379513 IP 10.27.28.25.43738 > 185.125.188.59.443: Flags [S], seq 3163562916, win 64240, options [mss 1460,sackOK,TS val 1076171133 ecr 0,nop,wscale 7], length 0
10:29:24.592938 IP 10.27.28.60.49874 > 185.125.188.58.443: Flags [S], seq 520329376, win 64240, options [mss 1460,sackOK,TS val 696018674 ecr 0,nop,wscale 7], length 0
10:29:25.659557 IP 10.27.28.25.45842 > 91.189.91.39.80: Flags [S], seq 3515504760, win 64240, options [mss 1460,sackOK,TS val 3894390069 ecr 0,nop,wscale 7], length 0
10:29:25.659565 IP 10.27.28.25.45846 > 91.189.91.39.80: Flags [S], seq 3392921426, win 64240, options [mss 1460,sackOK,TS val 3894390069 ecr 0,nop,wscale 7], length 0
10:29:25.872895 IP 10.27.28.25.43680 > 91.189.91.38.80: Flags [S], seq 2611796340, win 64240, options [mss 1460,sackOK,TS val 3718181772 ecr 0,nop,wscale 7], length 0
10:29:25.872896 IP 10.27.28.25.36230 > 185.125.190.39.80: Flags [S], seq 3596833942, win 64240, options [mss 1460,sackOK,TS val 3056190250 ecr 0,nop,wscale 7], length 0
10:29:26.299599 IP 10.27.28.25.36342 > 185.125.190.36.80: Flags [S], seq 2770239124, win 64240, options [mss 1460,sackOK,TS val 191911931 ecr 0,nop,wscale 7], length 0
10:29:26.299610 IP 10.27.28.25.36352 > 185.125.190.36.80: Flags [S], seq 2457288259, win 64240, options [mss 1460,sackOK,TS val 191911931 ecr 0,nop,wscale 7], length 0
10:29:26.512883 IP 10.27.28.25.36232 > 185.125.190.39.80: Flags [S], seq 1564624620, win 64240, options [mss 1460,sackOK,TS val 3056190890 ecr 0,nop,wscale 7], length 0
10:29:26.512905 IP 10.27.28.25.43694 > 91.189.91.38.80: Flags [S], seq 4183826752, win 64240, options [mss 1460,sackOK,TS val 3718182412 ecr 0,nop,wscale 7], length 0
10:29:27.311498 IP6 fe80::216:3eff:fe63:b6 > ff02::2: ICMP6, router solicitation, length 16
10:29:28.219507 ARP, Request who-has 10.27.28.1 tell 10.27.28.154, length 28
10:29:28.219584 ARP, Reply 10.27.28.1 is-at 00:16:3e:61:9a:b9, length 28
10:29:29.712875 IP 10.27.28.25.45842 > 91.189.91.39.80: Flags [S], seq 3515504760, win 64240, options [mss 1460,sackOK,TS val 3894394122 ecr 0,nop,wscale 7], length 0
10:29:29.712891 IP 10.27.28.25.45846 > 91.189.91.39.80: Flags [S], seq 3392921426, win 64240, options [mss 1460,sackOK,TS val 3894394122 ecr 0,nop,wscale 7], length 0
10:29:29.926217 IP 10.27.28.25.36230 > 185.125.190.39.80: Flags [S], seq 3596833942, win 64240, options [mss 1460,sackOK,TS val 3056194303 ecr 0,nop,wscale 7], length 0
10:29:29.926217 IP 10.27.28.25.43680 > 91.189.91.38.80: Flags [S], seq 2611796340, win 64240, options [mss 1460,sackOK,TS val 3718185825 ecr 0,nop,wscale 7], length 0
10:29:30.352877 IP 10.27.28.154.54038 > 185.125.188.54.443: Flags [S], seq 700742991, win 64240, options [mss 1460,sackOK,TS val 961439117 ecr 0,nop,wscale 7], length 0
10:29:30.352878 IP 10.27.28.25.36342 > 185.125.190.36.80: Flags [S], seq 2770239124, win 64240, options [mss 1460,sackOK,TS val 191915984 ecr 0,nop,wscale 7], length 0
10:29:30.352902 IP 10.27.28.25.36352 > 185.125.190.36.80: Flags [S], seq 2457288259, win 64240, options [mss 1460,sackOK,TS val 191915984 ecr 0,nop,wscale 7], length 0
10:29:30.566219 IP 10.27.28.25.36232 > 185.125.190.39.80: Flags [S], seq 1564624620, win 64240, options [mss 1460,sackOK,TS val 3056194943 ecr 0,nop,wscale 7], length 0
10:29:30.566278 IP 10.27.28.25.43694 > 91.189.91.38.80: Flags [S], seq 4183826752, win 64240, options [mss 1460,sackOK,TS val 3718186465 ecr 0,nop,wscale 7], length 0
10:29:31.206222 IP 10.27.28.154.56618 > 185.125.188.59.443: Flags [S], seq 2776693986, win 64240, options [mss 1460,sackOK,TS val 3070592684 ecr 0,nop,wscale 7], length 0
10:29:32.059499 IP 10.27.28.60.58214 > 185.125.188.55.443: Flags [S], seq 3108191411, win 64240, options [mss 1460,sackOK,TS val 2999392662 ecr 0,nop,wscale 7], length 0
10:29:32.062852 IP 10.27.28.25.33612 > 185.125.188.58.443: Flags [S], seq 152400071, win 64240, options [mss 1460,sackOK,TS val 3134949744 ecr 0,nop,wscale 7], length 0
10:29:32.486226 IP 10.27.28.25.43738 > 185.125.188.59.443: Flags [S], seq 3163562916, win 64240, options [mss 1460,sackOK,TS val 1076179239 ecr 0,nop,wscale 7], length 0
10:29:32.699531 IP 10.27.28.60.49874 > 185.125.188.58.443: Flags [S], seq 520329376, win 64240, options [mss 1460,sackOK,TS val 696026781 ecr 0,nop,wscale 7], length 0
^C
81 packets captured
81 packets received by filter
0 packets dropped by kernel

not sure whats ext_if

 ➜ sudo tcpdump -i  -nn
 by tab I can see this
docker0            lo                 lxdbr0             veth0138a2f0@if15  veth416b2c6b@if21  vethd65b5bfe@if19  vethef55b590@if17  wlan0
12

What does sudo nft list ruleset show? Use sudo apt install nftables if command not found.

ext_if would be your external interface, looks like wlan0 to me.

1 Like
13

Can you use ping 8.8.8.8 for tests to simplify the traces.

14

from host

➜ sudo nft list ruleset

table ip nat {
}
table ip filter {
}
table inet lxd {
        chain fwd.lxdbr0 {
                type filter hook forward priority filter; policy accept;
                ip version 4 oifname "lxdbr0" accept
                ip version 4 iifname "lxdbr0" accept
        }

        chain in.lxdbr0 {
                type filter hook input priority filter; policy accept;
                iifname "lxdbr0" tcp dport 53 accept
                iifname "lxdbr0" udp dport 53 accept
                iifname "lxdbr0" icmp type { destination-unreachable, time-exceeded, parameter-problem } accept
                iifname "lxdbr0" udp dport 67 accept
        }

        chain out.lxdbr0 {
                type filter hook output priority filter; policy accept;
                oifname "lxdbr0" tcp sport 53 accept
                oifname "lxdbr0" udp sport 53 accept
                oifname "lxdbr0" icmp type { destination-unreachable, time-exceeded, parameter-problem } accept
                oifname "lxdbr0" udp sport 67 accept
        }
}

from container

ubuntu@learning-man:~$ sudo nft list ruleset
sudo: nft: command not found

ubuntu@learning-man:~$ sudo apt install nftables
Reading package lists… Done
Building dependency tree
Reading state information… Done
Package nftables is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package ‘nftables’ has no installation candidate

HOST:
➜ sudo tcpdump -i lxdbr0 -nn

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on lxdbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
10:48:15.110917 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 419, seq 1, length 64
10:48:16.112917 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 419, seq 2, length 64
10:48:16.752748 ARP, Request who-has 10.27.28.1 tell 10.27.28.154, length 28
10:48:16.752837 ARP, Reply 10.27.28.1 is-at 00:16:3e:61:9a:b9, length 28
10:48:17.126078 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 419, seq 3, length 64
10:48:18.139459 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 419, seq 4, length 64
10:48:18.459468 IP 10.27.28.154.51712 > 185.125.188.54.443: Flags [S], seq 2614721279, win 64240, options [mss 1460,sackOK,TS val 962567224 ecr 0,nop,wscale 7], length 0
10:48:19.152795 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 419, seq 5, length 64
10:48:20.166107 IP 10.27.28.25.52096 > 185.125.188.54.443: Flags [S], seq 2574664873, win 64240, options [mss 1460,sackOK,TS val 1946150079 ecr 0,nop,wscale 7], length 0
10:48:20.166125 IP 10.27.28.60.46378 > 185.125.188.58.443: Flags [S], seq 318633769, win 64240, options [mss 1460,sackOK,TS val 697154247 ecr 0,nop,wscale 7], length 0
10:48:20.166140 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 419, seq 6, length 64
10:48:21.179453 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 419, seq 7, length 64
10:48:21.872806 IP 10.27.28.154.47818 > 185.125.188.58.443: Flags [S], seq 2972808968, win 64240, options [mss 1460,sackOK,TS val 3461189911 ecr 0,nop,wscale 7], length 0

➜ sudo tcpdump -i wlan0 -nn

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on wlan0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
10:48:41.542344 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 421, seq 1, length 64
10:48:42.566152 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 421, seq 2, length 64
10:48:42.779517 IP 10.27.28.154.57500 > 185.125.188.54.443: Flags [S], seq 1432121300, win 64240, options [mss 1460,sackOK,TS val 962591544 ecr 0,nop,wscale 7], length 0
10:48:43.579506 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 421, seq 3, length 64
10:48:44.486011 IP 10.27.28.60.41636 > 185.125.188.54.443: Flags [S], seq 3466397800, win 64240, options [mss 1460,sackOK,TS val 795662928 ecr 0,nop,wscale 7], length 0
10:48:44.486040 IP 10.27.28.25.55444 > 185.125.188.59.443: Flags [S], seq 3571133551, win 64240, options [mss 1460,sackOK,TS val 1077331239 ecr 0,nop,wscale 7], length 0
10:48:44.596190 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 421, seq 4, length 64
10:48:45.606177 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 421, seq 5, length 64
10:48:46.619536 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 421, seq 6, length 64
15

You don’t appear to be doing SNAT when packets leave by wlan0, so that explains the problem.
There aren’t any SNAT firewall rules in place either.

LXD SNATs by default unless its been turned off.

Please show lxc network show lxdbr0

16

$ lxc network show lxdbr0
config:
ipv4.address: 10.27.28.1/24
ipv6.address: none
description: “”
name: lxdbr0
type: bridge
used_by:

  • /1.0/instances/enormous-earwig
  • /1.0/instances/fine-mastodon
  • /1.0/instances/gui1804
  • /1.0/instances/learning-man
  • /1.0/instances/ubuntu
  • /1.0/profiles/default
    managed: true
    status: Created
    locations:
  • none

I haven’t touched any SNAT things I guess.

17

It should have ipv{n}.nat: true, try doing

lxc network set lxdbr0 ipv4.nat=true ipv6.nat=true
18

Trying by
lxd network edit lxdbr0, and restarting machine.


config:
  ipv4.address: 10.27.28.1/24
  ipv4.nat: "true"
  ipv6.address: none
  ipv6.nat: "true"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/profiles/default
- /1.0/instances/gui1804
- /1.0/instances/learning-man
- /1.0/instances/ubuntu
- /1.0/instances/enormous-earwig
- /1.0/instances/fine-mastodon
managed: true
status: Created
locations:
- none

Restarted my machine, but still fails to ping 8.8.8.8

Are there other steps that I might’ve missed?

19

tcpdumps after the changes looks like this.

➜ sudo tcpdump -i wlan0 -nn
tcpdump: verbose output suppressed, use -v[v]… for full protocol decode
listening on wlan0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
11:21:57.328102 IP 192.168.100.24.38134 > 3.68.175.98.443: Flags [.], ack 2024414709, win 1450, options [nop,nop,TS val 1545945409 ecr 3314062377], length 0
11:21:57.367011 IP 3.68.175.98.443 > 192.168.100.24.38134: Flags [.], ack 1, win 25, options [nop,nop,TS val 3314108455 ecr 1545927982], length 0
11:22:00.085810 IP 192.168.100.24.39710 > 18.168.253.132.443: Flags [P.], seq 3511272936:3511272990, ack 3148927267, win 501, options [nop,nop,TS val 1340561519 ecr 3026399904], length 54
11:22:00.134826 IP 18.168.253.132.443 > 192.168.100.24.39710: Flags [.], ack 54, win 8, options [nop,nop,TS val 3026409903 ecr 1340561519], length 0
11:22:00.134827 IP 18.168.253.132.443 > 192.168.100.24.39710: Flags [P.], seq 1:57, ack 54, win 8, options [nop,nop,TS val 3026409904 ecr 1340561519], length 56
11:22:00.134948 IP 192.168.100.24.39710 > 18.168.253.132.443: Flags [.], ack 57, win 501, options [nop,nop,TS val 1340561568 ecr 3026409904], length 0
11:22:00.483649 IP 3.67.131.16.443 > 192.168.100.24.46868: Flags [.], ack 2932282849, win 8, options [nop,nop,TS val 3076495876 ecr 2183787057], length 0
11:22:00.483702 IP 192.168.100.24.46868 > 3.67.131.16.443: Flags [.], ack 1, win 501, options [nop,nop,TS val 2183848494 ecr 3076465505], length 0
11:22:02.063586 IP 192.168.100.24.39710 > 18.168.253.132.443: Flags [P.], seq 54:110, ack 57, win 501, options [nop,nop,TS val 1340563497 ecr 3026409904], length 56
11:22:02.113306 IP 18.168.253.132.443 > 192.168.100.24.39710: Flags [.], ack 110, win 8, options [nop,nop,TS val 3026411881 ecr 1340563497], length 0

➜ sudo tcpdump -i lxdbr0 -nn
tcpdump: verbose output suppressed, use -v[v]… for full protocol decode
listening on lxdbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
11:22:35.914751 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 392, seq 45, length 64
11:22:36.928076 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 392, seq 46, length 64
11:22:37.941332 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 392, seq 47, length 64
11:22:38.954679 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 392, seq 48, length 64
11:22:39.511963 IP6 fe80::216:3eff:fe60:bf98 > ff02::2: ICMP6, router solicitation, length 16
11:22:39.967987 IP 10.27.28.25 > 8.8.8.8: ICMP echo request, id 392, seq 49, length 64
11:22:39.994597 ARP, Request who-has 10.27.28.1 tell 10.27.28.154, length 28
11:22:39.994691 ARP, Reply 10.27.28.1 is-at 00:16:3e:61:9a:b9, length 28
^C
8 packets captured
8 packets received by filter
0 packets dropped by kernel

20

I removed the nat earlier when trying to debug why it fails to connect to network.