Incus container firewalld is throwing "INVALID_IPV: 'ipv6' is not a valid backend or is unavailable at file"

1

I am running into an issue with incus and firewalld on a few different VPS vms regardless of the incus release version, regardless of the host kernel version. I have tested with incus LTS, stable, and daily and the problem gets reproduced. However, when I remove incus and install lxd at the VPS vm there is no problem with lxd and firewalld.

Running stable incus on ubuntu 24 host

# incus version
Client version: 6.8
Server version: 6.8

I have also tested running incus LTS on a debian 12 host

# incus version
Client version: 6.0.3
Server version: 6.0.3

I have also tested running incus daily on a debian 12 host

# incus version
Client version: 6.8
Server version: 6.8

Linux kernel version 6.8.0-51-generic. Next I installed and tested newest kernel 6.12.6-zabbly+
Incus container image debian/12 Debian GNU/Linux 12 (bookworm)
inside the deb/12 incus container running Firewalld Version 1.3.3

After installing an application that writes ipv="ipv6" into /etc/firewalld/direct.xml
in an incus debian12 container image the firewalld fails.

I initially setup incusbr0 with defaults and the containers are receiving IPV6 fd42:… addresses. There appears to be an issue with incus, IPV6 and firewalld where firewalld fails and won’t function correctly, status shows:

#  firewall-cmd --state
not running
# systemctl status firewalld
Active: active (running)
Dec 21 00:23:26 deb12 firewalld[162]: ERROR: Failed to load user configuration. Falling back to full stock configuration.
Dec 21 00:23:26 deb12 firewalld[162]: ERROR: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable

Inside this container ip a shows

eth0@if33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
   link/ether 00:16:3e:95:08:88 brd ff:ff:ff:ff:ff:ff link-netnsid 0
   inet 10.144.113.50/24 metric 1024 brd 10.144.113.255 scope global dynamic eth0
      valid_lft 3166sec preferred_lft 3166sec
   inet6 fd42:b87f:11d7:b87b:13ea:971e:bff4:2776/128 scope global dynamic noprefixroute
      valid_lft 2394sec preferred_lft 2394sec
   inet6 fe80::216:3eff:fe95:888/64 scope link
      valid_lft forever preferred_lft forever

IPV6 networking works, the incus container can ping6 out

~# ping6 google.com
PING google.com(ord30s22-in-x0e.1e100.net (2607:f8b0:4009:80b::200e)) 56 data bytes
64 bytes from ord30s22-in-x0e.1e100.net (2607:f8b0:4009:80b::200e): icmp_seq=1 ttl=59 time=1.03 ms
64 bytes from ord30s22-in-x0e.1e100.net (2607:f8b0:4009:80b::200e): icmp_seq=2 ttl=59 time=1.16 ms
64 bytes from ord37s35-in-x0e.1e100.net (2607:f8b0:4009:80b::200e): icmp_seq=3 ttl=59 time=1.11 ms

on startup of the incus container the iptables is empty

# iptables -nvxL
Chain INPUT (policy ACCEPT 702 packets, 47001 bytes)
    pkts      bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 702 packets, 46803 bytes)
    pkts      bytes target     prot opt in     out     source               destination

What I find strange is when I reload firewalld then the iptables get generated and begin functioning with rules even though the error continues to appear. When I restart firewalld or restart the container then the iptables are empty again.

# firewall-cmd --reload
Error: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable

# iptables -nvxL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination
       2      104 ACCEPT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED,DNAT
       0        0 ACCEPT     0    --  lo     *       0.0.0.0/0            0.0.0.0/0
       0        0 DROP       0    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
       0        0 INPUT_direct  0    --  *      *       0.0.0.0/0            0.0.0.0/0
       0        0 INPUT_ZONES  0    --  *      *       0.0.0.0/0            0.0.0.0/0
       0        0 REJECT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
    pkts      bytes target     prot opt in     out     source               destination
       0        0 ACCEPT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED,DNAT
       0        0 ACCEPT     0    --  lo     *       0.0.0.0/0            0.0.0.0/0
       0        0 DROP       0    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
       0        0 FORWARD_direct  0    --  *      *       0.0.0.0/0            0.0.0.0/0
       0        0 FORWARD_ZONES  0    --  *      *       0.0.0.0/0            0.0.0.0/0
       0        0 REJECT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited
(...)

The system is running an application that appears to be automatically generating /etc/firewalld/direct.xml
and most of the lines are ipv4 but there are 3 different lines with
ipv=“ipv6”

If I comment out these entire 3 lines with ipv=“ipv6” and then restart firewalld then the firewalld service runs active without error and the iptables rules remain populated after reboots.

<chain table="filter" ipv="ipv6" chain="white_list"/>
 <chain table="filter" ipv="ipv6" chain="fail2ban"/>
 <rule priority="4" table="filter" ipv="ipv6" chain="INPUT_direct">-j fail2ban</rule>

For more testing, I also created an incus network without ipv6.
incus network create netnoipv6 ipv6.address=none
and then created a profile that uses this netnoipv6 and then created a new container using this netnoipv6 and profile with no ipv6. The result is same, firewalld fails with same error messages.

I have already made posts at the firewalld github issues page and wanted to post here too…
Does anyone notice anything different or strange that must be occuring on VPS vms that this

2

This error can be reproduced on any host running incus with a debian12 container. Here are the steps to reproduce:

Install incus GitHub - zabbly/incus: Incus package repository

# wget -O /etc/apt/keyrings/zabbly.asc https://pkgs.zabbly.com/key.asc
# sh -c 'cat <<EOF > /etc/apt/sources.list.d/zabbly-incus-stable.sources
Enabled: yes
Types: deb
URIs: https://pkgs.zabbly.com/incus/stable
Suites: $(. /etc/os-release && echo ${VERSION_CODENAME})
Components: main
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/zabbly.asc

EOF'
# apt-get update
# apt-get install incus
# incus admin init
# incus launch images:debian/12 nameofdebian12container
# incus shell nameofdebian12container

Install this application in debian 12 incus container

# apt install -y wget sudo
# wget https://repo.vitalpbx.com/vitalpbx/v4.5/pbx_installer.sh
# chmod +x pbx_installer.sh
# ./pbx_installer.sh

Check firewalld status will show failed

# firewall-cmd --state
# systemctl status firewalld

I crossposted this issue to firewalld github at "INVALID_IPV: 'ipv6' is not a valid backend or is unavailable at file" · Issue #1439 · firewalld/firewalld · GitHub and I am continuing to troubleshoot this issue so I created a new empty debian 12 incus container and only apt installed firewalld. I edited /etc/firewalld/firewalld.conf with FirewallBackend=iptables to replicate what vitalpbx has set after their installation completes and I created /etc/firewalld/direct.xml with below contents and this does not reproduce the errors.

<?xml version="1.0" encoding="utf-8"?>
<direct>
  <chain table="filter" ipv="ipv6" chain="vpbx_white_list"/>
  <chain table="filter" ipv="ipv6" chain="vpbx_fail2ban"/>
  <rule priority="0" table="filter" ipv="ipv6" chain="INPUT_direct">-j vpbx_fail2ban</rule>
</direct>

This leads me to believe something else is configured that is effecting firewalld and incus after the vitalpbx application installer completes which unfortunately I have not been able to find anything more related to ipv6 and firewalld.

What is most strange is why using lxd/lxc no error is generated but in incus the firewalld throws error.