Can't get IPv4 addresses with docker installed

LXD
1

I recently replaced my server install with 20.04 (was 18.04), and now none of my lxc images are able to get IPV4 addresses. Even running dhclient manually just hangs. I’ve looked around the forums and this seems to be a common problem, but nobody seems to have a solution. It also fails with lxd 4.0, and I can’t install lxd 3.0 since it doesn’t work on ubuntu 20.04.

This is a fresh install with no firewall configured, and only docker, libvirt, qemu, lxd and zfs installed.

karl@thor:~$ sudo snap install lxd
lxd 4.1 from Canonical✓ installed
karl@thor:~$ lxd init
Would you like to use LXD clustering? (yes/no) [default=no]: 
Do you want to configure a new storage pool? (yes/no) [default=yes]: 
Name of the new storage pool [default=default]: 
Name of the storage backend to use (lvm, zfs, ceph, btrfs, dir) [default=zfs]: 
Would you like to create a new zfs dataset under rpool/lxd? (yes/no) [default=yes]: 
Would you like to connect to a MAAS server? (yes/no) [default=no]: 
Would you like to create a new local network bridge? (yes/no) [default=yes]: no
Would you like to configure LXD to use an existing bridge or host interface? (yes/no) [default=no]: yes
Name of the existing bridge or host interface: br0
Would you like LXD to be available over the network? (yes/no) [default=no]: 
Would you like stale cached images to be updated automatically? (yes/no) [default=yes] 
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: 
karl@thor:~$ lxc launch ubuntu
Creating the instance
The local image 'ubuntu' couldn't be found, trying 'ubuntu:' instead.
Instance name is: concrete-bat              
Starting concrete-bat
karl@thor:~$ lxc list
+--------------+---------+------+--------------------------------------------+-----------+-----------+
|     NAME     |  STATE  | IPV4 |                    IPV6                    |   TYPE    | SNAPSHOTS |
+--------------+---------+------+--------------------------------------------+-----------+-----------+
| concrete-bat | RUNNING |      | fdb1:9339:6886::e64 (eth0)                 | CONTAINER | 0         |
|              |         |      | fdb1:9339:6886:0:216:3eff:fed3:c95e (eth0) |           |           |
+--------------+---------+------+--------------------------------------------+-----------+-----------+
karl@thor:~$ lxc exec concrete-bat dhclient
^Ckarl@thor:~$ cat /etc/netplan/01-netcfg.yaml
network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: false
      dhcp6: false
  bridges:
    br0:
      interfaces: [eno1]
      dhcp4: true
      dhcp6: true
      parameters:
        stp: false
        forward-delay: 0
2

Its likely you have a firewall on your host blocking DHCP requests.

Can you show output of iptables-save.

Also please show output on LXD host of ip a and ip r. Thanks

3

No, there’s no firewall.

However, on a hunch I uninstalled docker and now suddenly everything works :confused:

4

Reinstalled docker and it’s broken again. So does LXD not play nice with docker?

5

Docker is well known to break LXD networking.

It is because it sets up a firewall by default (iptables-save would have shown this) and sets the default FORWARD policy to DROP.

See also After installing docker lxd containers no longer getting IP over bridge

6

I can get around the docker issue with this:

iptables -I DOCKER-USER -i br0 -o br0 -j ACCEPT

However, I’m not sure how I can make this permanent… The only options seem to be to save the ENTIRE iptables config, or nothing at all. And I don’t want to store a bunch of iptables rules that are auto-generated since that could break things in future when they change how they get generated…

7

If you can run a script at startup (by some mechanism such as systemd units or rc.local) that adds the rule then that would be a workaround.

8

OK, figured out how to do it. Basically, since I have LXD using my own bridge br0, docker stomps all over it because it wants to control iptables. To stop it from interfering with br0, we create a DOCKER-USER rule (which docker will run before its own rules), and set it up as a one-shot service so that it gets run on every boot.

The key is to use iptables-restore with the -n flag, because that tells it not to flush, then we manually flush DOCKER-USER and add our rules:

cat <<EOF >/etc/iptables-br0.conf
*filter
:DOCKER-USER - [0:0]
-F DOCKER-USER
-A DOCKER-USER -i br0 -o br0 -j ACCEPT
COMMIT
EOF

cat <<EOF >/etc/systemd/system/iptables-br0.service
[Unit]
Description=Prevent docker from interfering with br0
Before=network-pre.target

[Service]
Type=oneshot
ExecStart=/sbin/iptables-restore -n /etc/iptables-br0.conf

[Install]
WantedBy=multi-user.target
EOF

systemctl enable --now iptables-br0

Saving it here in case anyone else gets stuck on this.

1 Like
9

@tomp - I could be wrong but when I first encountered this with Docker I think I then installed Docker before I install LXD and then everyone was happy.

So if I think I “may” use Docker on a new server I do it in that order.

10

It does depends on start order, although as far as I know systemd doesn’t always guarantee the same start order of independent units. Also, it could cause issues if one or other gets restarted while the system is running.

See also Lxd and Docker Firewall Redux - How to deal with FORWARD policy set to drop - #3 by tomp