Hi,
I am running several websites in different containers / apache2 behind another container with nginx.
On the mainserver, I have several scripts that create Ipsets with blocking rules for certain IPs (country based and known offenders)
Currently, I am forwarding traffic through these rules in prerouting.
My Ipset and fail2ban donāt seem to have an effect
It seems to me that the lxd bridge is not required (?) Or is it required for local networking?
What am I missing here? Should I reorder the rules for my blocking to be before the lxdbridge rules? can I delete the lxdbridge rules completely?
Additional info why I use NAT to forward traffic instead of the lxd bridge:
My reason for not using the lxd bridge for forwarding ports 443 and 80 was that I did not manage to make it work with nginx showing the source IP address in my logs.
Thank you very much for your input and helping me securing this
A sidenote: You might wonder why I also configured a proxy device for the ports I already forward by NAT. The reason is that I have an internal python script checking these websites by their domain name and the request did not function without the bridged ports (I assume this is because they also work on the loopback (?).
+-------------+----------+---------+----------------+---------------------------+-------------+---------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY |
+-------------+----------+---------+----------------+---------------------------+-------------+---------+
| eth0 | physical | NO | | | | 0 |
+-------------+----------+---------+----------------+---------------------------+-------------+---------+
| lxd_bridge0 | bridge | YES | INTERNALIP/24 | IN:TER:NAL::4566::1/64 | | 16 |
+-------------+----------+---------+----------------+---------------------------+-------------+---------+
Which logs does fail2ban monitor to add rules to the hostās firewall?
And are you seeing fail2ban successfully add IPs to the set?
Iām also confused why you have proxy device setup, but are using (presumably manually added) NAT rules, when the proxy device itself can operate in nat=true mode so it would add its own NAT rules.
Iām not sure what the internal python script you refer to does, and why this wasnāt working.
With the info so far Iām struggling to come up with a suggestion on how to proceed. However my general recommendation would be to remove your manually added NAT rules, and use the proxy device in nat=true mode. Then the external IP will be passed into your containers, and then hopefully fail2ban will see the correct IP.
Also worth noting, if you are using NAT mode (either manually or via nat=true on the proxy device), then if fail2ban on the host is adding rules to the INPUT firewall chain, then these are not going to match because DNAT will pass these rules through the FORWARD chain in iptables (this is why when using the proxy device in NAT mode we enable bridge iptables filtering and hairpin mode see https://github.com/lxc/lxd/blob/master/lxd/device/proxy.go#L407-L454).
Thanks for your answer. Let me be a bit clearer about it.
Thanks for pointing out that fail2ban is not using the forwarding chain, i oversaw that point.
Letās break the problem down nevertheless and ignore fail2ban for now, because also the other rules donāt block requests.
I use the manually filled ipset COLLECTED_manually in both chains FORWARD and INPUT configured as shown below.
The problem: If I add an IP to this set, it does not get blocked either.
I assume that this is somehow due to the rules generated for the bridge?
In addition to your suggestion to use the bridge in nat mode (which I failed to do so far, I will look into this again to make it work with nginx),
my question is: Should my DROP rule for the set COLLECTED_manually be before the bridge rules to work?
I understood so far, that as soon as a rule in iptables gets matched and has the target āACCEPTā the following rules are not treated anymore?
I guess I overall am not entirely understanding what the rules for the lxc-bridge do.
The ordering in relation to the LXD rules added shouldnāt matter as they are only targeting DNS and DHCP packets, so they wonāt be matching for HTTP/HTTPS traffic. Nor do they affect any interfaces apart from lxdbr0.
Perhaps your system is not configured to pass bridge traffic through iptables at all, please show the output of:
Same result if I try to open a website with a blocked IP in the set COLLECTED_manually, it still goes through.
(as of your earlier post passing through PREROUTING and then FORWARD, which has two rules ACCEPT for the lxd bridge.
Just to see what happens, I temporarily moved the subsequent blocking rule (DROP ALL if in set COLLECTED_mailserver) before the LXD bridge generated rules. That effectively blocked the traffic.
Maybe to precise, the set works for other ports that are not forwarded to the lxd bridge and not in the PREROUTING rules, so it must somewhat be related to the bridge and prerouting.
So thats in encouraging if the br_netfilter module is now loaded, have you tried moving your IP set ruless above the LXD rules in the FORWARD chain (as those should be taking effect now).
If so then you could disable LXDās firewall rules using lxc network set lxdbr0 ipv4.firewall=false and then manually add the INPUT rules for DNS and DHCP.
Ok, just making sure that I understand everything correctly.
1.) move my own iptable rules BEFORE the lxd bridge rules for input and forward
2.) disable LXD firewall
3.) add Iptable rules for DNS and DHCP. (I would appreciate further infos on this, sorry the ignorance) would your suggestion mean:
iptables -A INPUT -p udp -i eth0 --dport 67 -j ACCEPT
iptables -A INPUT -p udp --sport 53 -j ACCEPT
I will do some tests tonight and post the result. Thank you for your help.
Disabling LXD firewall will remove those rules entirely from the FORWARD chain.
Before doing that, if you save a copy of the rules LXD adds (using something like iptables-save ) to and then reproduce those manually, then you can have more control over the firewalling.
Otherwise your default policy of DROP on the INPUT chain will break DHCP and DNS for your containers.
Hi Thomas,
Thanks for all the hints.
Hereās what I did and what I observe:
I added these rules:
/usr/sbin/iptables -A INPUT -j ACCEPT -p TCP --dport domain -m comment --comment "man generated for LXD bridge"
/usr/sbin/iptables -A INPUT -j ACCEPT -p UDP --dport domain -m comment --comment "man generated for LXD bridge"
/usr/sbin/iptables -A INPUT -j ACCEPT -p UDP --dport bootps -m comment --comment "man generated for LXD bridge"
/usr/sbin/iptables -A OUTPUT -j ACCEPT -p TCP --sport domain -m comment --comment "man generated for LXD bridge"
/usr/sbin/iptables -A OUTPUT -j ACCEPT -p UDP --sport domain -m comment --comment "man generated for LXD bridge"
/usr/sbin/iptables -A OUTPUT -j ACCEPT -p UDP --sport bootps -m comment --comment "man generated for LXD bridge"
/usr/sbin/iptables -A FORWARD -j ACCEPT -m comment --comment "man generated for LXD bridge"
/usr/sbin/iptables -t nat -A POSTROUTING -s INTERNALIP/24 ! -d INTERNALIP/24 -m comment --comment "man generated for LXD bridge" -j MASQUERADE
I modified fail2ban such that it creates a rule for the FORWARD chain.
That now seems all to work as expected.
Unfortunately my ufw firewall now seems to block traffic that it should not block. (I assume this is maybe a question I should not ask here anymore, but I do it in the continuity of this post)
ufw is set to block by default with exceptions on IPv6 and IPv4 for port 80 like:
To Action From
-- ------ ----
80/tcp ALLOW Anywhere
note that this a port that is passing through PREROUTING in order to be passed to lxd.
my iptables rules from ufw are:
CHAIN INPUT:
ufw-before-logging-input all -- anywhere anywhere
ufw-before-input all -- anywhere anywhere
ufw-after-input all -- anywhere anywhere
ufw-after-logging-input all -- anywhere anywhere
ufw-reject-input all -- anywhere anywhere
ufw-track-input all -- anywhere anywhere
CHAIN FORWARD:
ufw-before-logging-forward all -- anywhere anywhere
ufw-before-forward all -- anywhere anywhere
ufw-after-forward all -- anywhere anywhere
ufw-after-logging-forward all -- anywhere anywhere
ufw-reject-forward all -- anywhere anywhere
ufw-track-forward all -- anywhere anywhere
THe new rules for the lxd bridge come after that. In case you see the error quicker than me Iād be happy to learn.
You should also add the interface selector flags to your manual lxd firewall rules otherwise they may allow some external inbound traffic. The output of iptables-save would have had interface selectors on them.
Please can you renable the lxd firewall option, remove your manual ones and leave ufw enabled and show output of āiptables-saveā as ufw just adds iptables rules afaik.
Thank you again for all your insights. It now works the way I expect it to.
After the restart, the lxd_bridge0 rules ended up at the end (just before the firewall) which makes fail2ban work (I modified the action.d configfile such that it added a rule to the chain FORWARD) and my own ipset.
I would close the topic now if you donāt have an opposing view.
Thanks a lot for helping and sharing your knowledge.