I usually use unmanaged bridges on my lxd hosts, but would like to implement IP address filtering (as per Security | LXD - I’m using the 4.0 stable branch, but wasn’t sure what the canonical URL was for the 4.0 docs).
I tried to do this:
lxc config device set mycontainer eth1 security.ipv4_filtering=true
… but this doesn’t work on an unmanaged bridge (although security.mac_filtering=true does, and I can set an ipv4.address for a container nic which is connected to an unmanaged bridge).
I currently use an unmanaged bridge so that I can implement physical machines (both lxd and non-lxd hosts, and containers) on the same Ethernet segment, without NAT.
If instead I set up the network as managed (to allow lxd to create the bridge and assign the local IP address), and add the lxd host’s external NIC to the bridge using bridge.external_interfaces, things more or less work, but I’m unable to set a gateway (ipv4 default route) for the lxd host itself. It doesn’t use the ipv4.dhcp.gateway key to set the gateway IP address for the lxd host (neither does ipv4.routes allow specifying a gateway host).
Also, I’m concerned that an external DHCP server would be able to answer queries from containers - looking at the iptables rules, if the lxd managed bridge dhcp functionality is enabled, I can’t see any rules which would block DHCP traffic on the bridge.external_interfaces.
You would need to manually specify the IP address using lxc config device set mycontainer eth1 ipv4.address=n.n.n.n and then setup static allocations in your external DHCP server for that container’s MAC address so it always allocates the same IP as you specified in the container config (or use static network config inside the container).
An external DHCP server would be able to answer queries from a container, the IP filtering only works to prevent spoofed traffic originating from the container.
I knew that I would need to configure an external DHCP server (or just configure each container with the correct static IPs).
The problem is that I can only set security.ipv4_filtering=true on container interfaces which are attached to managed bridge. Running this command against a container NIC which is attached to an unmanaged bridge fails…
If instead I set up a managed bridge, then I cannot find a way for the LXD host to configure a default route for either IPv4 or IPv6 (i.e. a default route for it to use for traffic originating from the “bare” host i.e. not from within the containers that it hosts) on this interface. Also this limits each container to a single IPv4 and a single IPv6 (unless I add multiple interfaces connected to the same bridge - which feels hacky).
As you say there is currently a restriction on using IP filtering (rather than MAC filtering) with unmanaged bridges (that I had forgotten!).
I’m not really clear on your points about managed bridges though, regarding default routes and multiple IPs. DHCP will provide a default route, and as for multiple IPs, that is a limitation of the IP filtering functionality at the moment I’m afraid (as its designed to be used with DHCP allocations, and as far as I know DHCP can only allocate a single IP per MAC).
My issue with default routes is that this works fine for the containers, but I can’t seem to set a default route for the LXD host. I can do something like this:
lxc network set lxdbr1 ipv4.address=192.168.23.42/24
and I can also then do something like:
lxc network set lxdbr1 ipv4.routes=192.168.20/24
but there is no way to express something like this:
lxc network set lxdbr1 ipv4.routes=0.0.0.0/0 via 192.168.23.1
…to set the default route for the lxd host itself. If I set this:
lxc network set lxdbr1 ipv4.dhcp.gateway=192.168.23.1
… then this gateway value gets given to the containers via DHCP correctly, but the lxd host’s routing table doesn’t get this entry. There isn’t a way to e.g.
lxc network set lxdbr1 ipv4.gateway=192.168.23.1
… anyway I think this is somewhat beside the point, and I think that it would be better to able to do something like this on a container which is attached to an unmanaged bridge:
lxc config device add c2 eth0 nic name=eth0 nictype=bridged parent=br0
lxc config device set c2 eth0 ipv4.address 192.168.42.42
lxc config device set c2 eth0 security.ipv4_filtering=true
or even better, the ability to specify more than one IP address per interface with some variation on this (where the business of actually configuring the IP addresses on this interface would be handled outside of lxd e.g. via static address assignment).
For the basic case (single IP address per interface only) - do you know if that would require any complex code changes?
So LXD will never modify the host’s default gateway, this will be up to you to set via which ever network configuration system you are using on the host (netplan etc).
The ipv4.routes setting is just for adding static routes to the host that route towards the bridge interface (not a specific next-hop address).
Are you trying to connect your external interface to an LXD managed bridge? Can you show me your current network config lxc network show <network name> and ip r on host?
I agree that this is certainly not ideal. If you add a feature request to our github issue tracker then this could be something that may be added in the future. I’ve had a quick scan through the existing code and cannot see any major blockers.
As for filtering packets from external networks, this is outside of the scope of the IP filtering feature, its only designed for filtering traffic from instances on the host, as it can’t know what is/isn’t allowed from the external network.
To (hopefully) clarify, the configuration I’m trying to get to is:
Bridged networking on the lxd host.
External router (the lxd host should not NAT this network itself - the external router could be either NATed, or non-NATed - this varies between containers and NICs).
IP addresses (and gateways, and DNS etc.) statically assigned to the lxd host.
IP addresses (etc. as above) either configured as static IP addresses within each container, or via DHCP (either the lxd host, or an external DHCP server could provide this to the containers).
Filtering configured on the lxd host, so that it’s not possible for root users within containers to reconfigure the container’s network to successfully spoof the IP address or mac address of another device attached to the bridge network.
This works and is straightforward to implement in all respects, except the IP spoofing prevention.
Currently if I try to set this up with an unmanaged bridge (managed by the usual OS means - netplan etc. as you say) I can do this:
lxc config device set c1 eth1 security.mac_filtering=true
but not these:
lxc config device set c1 eth1 security.ipv4_filtering=true lxc config device set c1 eth1 security.ipv6_filtering=true
… so I can’t guard against IP address spoofing (arp spoofing for ipv4 or NDP spoofing for ipv6), unless I set up some mechanism external to lxd to accomplish this - which just feels a bit messy (especially since lxd already supports this for managed bridges).
… as an alternative hack, I tried to create a managed bridge instead, and this nearly works too, but unless I’m missing something, I can’t see a way to make lxd assign a default route (gateway) on bridges (for the host’s routing table) which it manages itself, and I can’t think of any other non-hacky ways to accomplish this.
If you think the original set of requirements sound reasonable, then I’ll create a feature request for this in git, as you suggest…
Yes that (almost) all makes sense, and is clear for adding a feature request.
The only part that I remain unclear on is this:
“as an alternative hack, I tried to create a managed bridge instead, and this nearly works too, but unless I’m missing something, I can’t see a way to make lxd assign a default route (gateway) on bridges (for the host’s routing table) which it manages itself”
Are you connecting the external network interface to to the managed bridge using bridge.external_interfaces?
@stgraber what was the intention for the bridge.external_interfaces option originally? And is there potentially scope for adding a network option to set a gateway for the LXD host?
To me, the “right” way to accomplish my intended configuration would probably be to support container IP spoof protection with unmanaged bridges (assuming that doesn’t violate a load of assumptions elsewhere in LXD which I’m not aware of), and getting LXD to reimplement all aspects of network configuration on the LXD host feels clunky (I just tried it as a workaround) - apart from anything else it’s never going to satisfy all users, since less common network configuration options will always be missing…