Setup bridge networking with firewalld

dashohoxha · August 26, 2022, 4:11pm

Fix networking with firewalld

1. Introduction

I usually install firewalld on my hosts, and it interferes with the networking of the LXD containers. If we create a test container, we will notice that the network in the container is not working:

lxc launch images:ubuntu/22.04 u22
lxc ls
lxc exec u22 -- ip addr

The container did not get an IP, as it normally should.

However, if you stop firewalld and restart the container, everything works fine.

systemctl status firewalld
systemctl stop firewalld

lxc restart u22

lxc ls
lxc exec u22 -- ip addr
lxc exec u22 -- ping 8.8.8.8

systemctl start firewalld
systemctl status firewalld

By the way, IP forwarding should already be enabled in the kernel of the host:
sysctl net.ipv4.ip_forward
cat /proc/sys/net/ipv4/ip_forward
If it is not, enable it like this:
echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
sysctl -p

So the problem is that the firewall is not configured properly. Let’s fix it.

2. Allow DHCP requests

firewall-cmd --zone=internal --list-all
firewall-cmd --permanent --zone=internal --add-interface=lxdbr0
firewall-cmd --permanent --zone=internal \
    --remove-service={dhcpv6-client,mdns,samba-client,ssh}
firewall-cmd --permanent --zone=internal --add-service=dhcp
firewall-cmd --reload
firewall-cmd --zone=internal --list-all

By adding the interface lxdbr0 to the zone internal and allowing DHCP requests on this zone, the containers are able to get an automatic IP from lxdbr0.

lxc restart u22
lxc ls
lxc exec u22 -- ip addr

3. Fix forwarding in firewalld

Ping is still not working:

lxc exec u22 -- ping 8.8.8.8

Let’s move the external interface (eth0 in my case) to the zone external, and enable forwarding between the zones internal and external:

firewall-cmd --permanent --zone=external --add-interface=eth0
firewall-cmd --reload
firewall-cmd --zone=external --list-all

firewall-cmd --permanent --new-policy=forward-internal
firewall-cmd --permanent --policy=forward-internal --add-ingress-zone=internal
firewall-cmd --permanent --policy=forward-internal --add-egress-zone=external
firewall-cmd --permanent --policy=forward-internal --set-target=ACCEPT
firewall-cmd --reload
firewall-cmd --list-all-policies

WARNING
By default, the external inteface (eth0 in this case) is associated implicitly to the zone public. If you use an SSH port different from the default one (for example 2201), you have to add this port to the external zone too, along with the eth0 interface, otherwise you may lock yourself out of the server. For example like this:
firewall-cmd --permanent --zone=external --add-port=2201/tcp
firewall-cmd --permanent --zone=public --remove-port=2201/tcp
firewall-cmd --reload

4. Fix the FORWARD chain

If the ping is still not working, usually the problem is the default policy of iptables. If you try iptables-save | head and see something like this: :FORWARD DROP [4:2508], it means that the default policy for the FORWARD chain is DROP.

You can make the default policy ACCEPT, like this: iptables -P FORWARD ACCEPT. However, the next time that the server will be rebooted, or firewalld restarted, you may loose this configuration.

A better way is to add a direct (explicit) rule with firewall-cmd, like this:

firewall-cmd --permanent --direct --add-rule \
    ipv4 filter FORWARD 0 -j ACCEPT
firewall-cmd --reload

firewall-cmd --direct --get-all-rules

This will enable (ACCEPT) forwarding for all the interfaces, the current ones and the ones that will be created in the future. If this is not what you want, you can use more specific rules, like these:

firewall-cmd --permanent --direct --remove-rule \
    ipv4 filter FORWARD 0 -j ACCEPT

firewall-cmd --permanent --direct --add-rule \
    ipv4 filter -i lxdbr0 FORWARD 0 -j ACCEPT
firewall-cmd --permanent --direct --add-rule \
    ipv4 filter -o lxdbr0 FORWARD 0 -j ACCEPT

firewall-cmd --reload
firewall-cmd --direct --get-all-rules

NOTE
I would expect the firewalld policy that we defined on the previous step to take care of enabling the necessary forwarding rules automatically, but apparently it doesn’t do that.

5. Cleanup

Let’s test again and then remove the test container:

lxc exec u22 -- ping 8.8.8.8

lxc stop u22
lxc rm u22
lxc ls

dashohoxha · December 9, 2022, 9:59am

This setup is unnecessarily complicated. The correct way to fix networking with firewalld is to add all the virtual interfaces (virtual networks) to the zone trusted, as explained in this doc page. I had tried this before but it didn’t work for me, maybe because I had installed Docker as well, and I failed to realize that Docker sets the FORWARD policy to DROP.

So, here are the steps that need to be done.

1. Add the bridge interface to the `trusted` zone

Any interface that is not explicitly added to a zone, is handled by the default zone, which is the zone public. This zone is meant for the interfaces that are facing the public internet, so it is restricted. For example DHCP requests are blocked, and the containers cannot get an IP.

To fix this, we can add the bridge interface to the trusted zone, where everything is allowed:

firewall-cmd --permanent --zone=trusted \
             --add-interface=lxdbr0
firewall-cmd --reload

firewall-cmd --zone=trusted --list-all

2. Fix the FORWARD chain

If the ping is still not working, usually the problem is that forwarding is blocked. If you try iptables-save | head and see something like this: :FORWARD DROP [4:2508], it means that the policy for the FORWARD chain is DROP. Maybe it is set by Docker, if you have installed it.