Networking not working in container with docker

For some reason, outbound connectivity isn’t working in my containers:

root@build05:~# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
^C
--- 1.1.1.1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2040ms

The host can ping the container, the container can ping the host, but the container can’t reach the outside Internet.

Network config on host:

root@chi03:~# ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP group default qlen 1000
    link/ether 00:16:3c:c0:ef:fa brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet xxx.xxx.xxx.xxx/24 brd xxx.xxx.xxx.xxx.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 xxxxxxxxx::2/64 scope global
       valid_lft forever preferred_lft forever
    inet6 xxxxxxxxxx:effa/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc fq state UP group default qlen 1000
    link/ether 00:18:3e:7a:98:1e brd ff:ff:ff:ff:ff:ff
    altname enp0s4
    altname ens4
    inet 10.xxx.xxx.xxx/8 brd 10.255.255.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 xxxxxxxxxxxxxxxx:981e/64 scope link
       valid_lft forever preferred_lft forever
4: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 8920 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none
    inet 10.123.1.41/32 scope global wg0
       valid_lft forever preferred_lft forever
5: br-6b057de625dc: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:c7:9a:1a:1d brd ff:ff:ff:ff:ff:ff
    inet 172.21.0.1/16 brd 172.21.255.255 scope global br-6b057de625dc
       valid_lft forever preferred_lft forever
6: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:69:3c:07:8e brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
7: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:42:8d:aa brd ff:ff:ff:ff:ff:ff
    inet 10.121.186.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
    inet6 fd42:de7e:c573:b57b::1/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe42:8daa/64 scope link
       valid_lft forever preferred_lft forever
9: vethd366efd3@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether 8a:8e:b8:4b:5e:57 brd ff:ff:ff:ff:ff:ff link-netnsid 0
11: vethfc34dffa@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
    link/ether 9e:c7:8a:85:ad:ea brd ff:ff:ff:ff:ff:ff link-netnsid 1

lxc config:

root@chi03:~# lxc network list
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
|      NAME       |   TYPE   | MANAGED |      IPV4       |           IPV6            | DESCRIPTION | USED BY |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| br-6b057de625dc | bridge   | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| docker0         | bridge   | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| eth0            | physical | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| eth1            | physical | NO      |                 |                           |             | 0       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
| lxdbr0          | bridge   | YES     | 10.121.186.1/24 | fd42:de7e:c573:b57b::1/64 |             | 3       |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+
root@chi03:~# lxc network show lxdbr0
config:
  ipv4.address: 10.121.186.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:de7e:c573:b57b::1/64
  ipv6.nat: "true"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/instances/build05
- /1.0/instances/tv
- /1.0/profiles/default
managed: true
status: Created
locations:
- none

Network in container:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
8: eth0@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:d0:d7:0a brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.121.186.3/24 brd 10.121.186.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fd42:de7e:c573:b57b:216:3eff:fed0:d70a/64 scope global mngtmpaddr dynamic
       valid_lft 3181sec preferred_lft 3181sec
    inet6 fe80::216:3eff:fed0:d70a/64 scope link
       valid_lft forever preferred_lft forever

Any ideas?

We’ve seen this kind of stuff on systems that also have Docker installed.
Docker has a tendency to put very wide filtering rules in place which may interfere with other applications doing bridge networking on the system.

firewalld on some distros can have a similar efefct.

I guess installing Docker is what broke it. Any suggestions on how to use Docker and LXC at the same time, with networking working in both?

I’m running this on a KVM virtual server so macvlan is not an option.

I believe @tomp posted a link a few weeks ago on this forum about how to configure Docker networking to be less problematic.

In general the main issue is Docker modifying the default policy, so maybe try:

  • iptables -P INPUT ACCEPT
  • iptables -P OUTPUT ACCEPT
  • iptables -P FORWARD ACCEPT

This should reset the policies to the OS default and may be enough to get things to behave again.

Thanks! While you were replying, I found this GitHub issue:

Flushing the FORWARD chain and changing the default policy to ACCEPT was sufficient to get it working again:

iptables -F FORWARD
iptables -P FORWARD ACCEPT

I guess I might have to add a systemd drop-in to run these commands after Docker starts so that it persists after rebooting the system / restarting Docker.

The docker manual has some info and suggestions to fix this: