Reasoning behind in/out/fwd netfilter rules

[tomp (Thomas Parrott)]

@m1cha

correct me if I’m wrong, but the way I understand it, with the policy of lxd’s chains being accept, all the other rules don’t have any effect and we could just remove them without any change in behavior. That doesn’t mean that we should switch to drop, just that we can basically remove those rules because they don’t do anything.

You are not wrong. Those rules, apart from the SNAT rule, are not really doing anything at this time, by themselves. Really they are a skeleton ruleset used to allow managed bridge services if the user enables the ACL feature (Linux Containers - LXD - Has been moved to Canonical).

There doesn’t appear to be consensus yet on how different applications should interact when creating nftables rules. The table namespace feature is great to simplify rule management for an application, but the behaviour of nftables to apply a drop rule in any table even if the traffic is allowed in another table is awkward as it undoes some of the benefits of table namespace that could have been realised if that was not the behaviour. This is different than when LXD uses xtables (iptables/ip6tables/ebtables) as there are standard chains that we can add rules to to have our traffic policy interact with other application’s rules.

@m1cha

That being said, IMO using LXD on it’s own without an additional firewall is pretty bad because it both enables IP forwarding and sets the forward policy to accept effectively turning your device into a router.

This can be disabled by setting ipv4.routing=false on the managed bridge (see Linux Containers - LXD - Has been moved to Canonical).

@m1cha

For that reason I’d suggest to let LXDs firewall use a default-drop chain to make it secure by default. If the user has their own firewall they can disable LXD’s and copy it’s rules into the firewall configuration.

That would be very disruptive for existing users, and even if we only applied it on newly created networks, it would complicate getting started quickly with LXD (as a novice user would immediately have to think about their outbound internet policy). We wouldn’t be able to add a default drop policy to the entire machine as it would potentially disconnect any remote users or interfere with other applications, so at best we could add a default drop rule to traffic to/from the lxdbr0 interface, which wouldn’t improve security of enabling router mode for the other interfaces on the system anyway.

@m1cha

On top of all that LXD could try to be a better citizen than e.g. docker(which is REALLY bad when it comes to that) and provide integrations with firewalls so you don’t have to hardcode IP addresses hoping that they don’t change. Ideally the interface would be generic enough so we don’t have to support every firewall on earth explicitly, e.g. by calling an external script where the user can react to changes in LXDs network setup.

The alternative would be to do it the other way around and let users subscribe to firewall changes through the LXD server API. That should be made as easy as possible though because most people don’t want to compile code to do scripting on their server.

We don’t go in for external hooks with LXD, as our experiences with LXC indicated that adding hook support meant it was hard to get an understanding of how everyone uses the application and makes changing things risky as it would bound to break someone’s integration work flow. Also in a cluster environment its not always clear where the hook should be run. We do have a REST API you can subscribe to to get events, although im not sure what specific events you would be interested in.
Although one thing comes to mind is something @stgraber wrote as a proof of concept for our BGP integration, which uses the event stream to monitor for instances starting and then integrating the config to get their IP in order to advertise them via BGP.

See GitHub - stgraber/lxd-bgp: A tiny BGP server in Go exposing LXD external routes that may provide some inspiration.

Hopefully in the future we will see some consensus around how applications should coexist when using nftables, or perhaps its behavior will be changed to allow an allow rule to apply irrespective of a drop rule in another table.