LXD container networking isolation suggestions

LXD
, ,
1

Currently I’m using lxdbr0 (the default) which has been a very simple way of using container networking.

However just spotted that the containers running on the same lxdbr0 are visible to each other. Which is undesirable.

What is the simplest way of isolating the network of each container.

Thanks in advance…

2

A PR was merged yesterday (due in LXD 4.10) that allows you to set security.port_isolation=true on a bridged NIC device, which will prevent it communicating with any other instances that also have the same setting enabled.

Keep in mind this will still allow the instances to communicate with the host (and any external interfaces added to the bridge).

3

Perfect… Just have to wait for the release…

4

You can try it now by using snap refresh lxd --channel=latest/edge however be aware that this is an edge release so can have more bugs in it, and if there is a DB schema change you won’t be able to downgrade, so best to use it on a separate test system.

5

LXD4.10 is released I’ve got it installed… How do I enable the option?

root@heata-42050:~# lxc --version
4.10

   root@heata-42050:~# lxc network show lxdbr0
config:
  ipv4.address: 10.220.212.1/24
  ipv4.firewall: "false"
  ipv4.nat: "true"
  ipv6.address: fd42:e920:73f4:96f7::1/64
  ipv6.nat: "true"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/instances/xmrig-0013
- /1.0/profiles/default
managed: true
status: Created
locations:
- none


root@heata-42050:~# lxc network  set lxdbr0 security.port_isolation true
Error: Invalid option for network "lxdbr0" option "security.port_isolation"
6

It is enabled on the NIC device rather than the network (as different NICs can have it enabled or disabled depending on your desired isolation level).

e.g

lxc config device set c1 eth0 security.port_isolation=true

See https://linuxcontainers.org/lxd/docs/master/instances#nic-bridged

7

Thanks but I’m still not able to set it… I’m running current ubuntu 20.04.1 LTE x64

lxc config device set xmrig-0013 eth0 security.port_isolation=true
Error: The device doesn't exist

But when I attempt to change it in the default profile…
root@heata-42050:/home/appliance# lxc profile device set default eth0 security.port_isolation=true
Error: The following instances failed to update (profile change still saved):
- Project: default, Instance: xmrig-0013: Failed to start device “eth0”: Failed to run: bridge link set dev vethb77e7d06 isolated on: Usage: bridge link set dev DEV [ cost COST ] [ priority PRIO ] [ state STATE ]
[ guard {on | off} ]
[ hairpin {on | off} ]
[ fastleave {on | off} ]
[ root_block {on | off} ]
[ learning {on | off} ]
[ learning_sync {on | off} ]
[ flood {on | off} ]
[ mcast_flood {on | off} ]
[ neigh_suppress {on | off} ]
[ vlan_tunnel {on | off} ]
[ hwmode {vepa | veb} ]
[ self ] [ master ]
bridge link show [dev DEV]

8

So the first error you got was because the nic device doesn’t exist in the container, and only in the profile.

Editing the profile is fine, but if you did want to only apply this setting to a specific container then you can copy the device config from the profile into the container and modify it in a single command using the override command, e.g.

lxc config device override c1 eth0 security.port_isolation=true

However whether editing the profile or editing the container’s device config, you’ll get the same error as it appears the port isolation feature isn’t supported on your host currently.

As you’re running Ubuntu Focal, it should work, and indeed does work on my system. However I’ve just now managed to reproduce the issue in a VM running the snap, so I’m wondering if this is some sort of race condition inside the Linux bridging subsystem or packaging issue.

I’ll investigate and let you know.

9

So I can only reproduce this when using the snap package. I think it may be because the LXD snap package still uses the core18 base package, and so the version of the ip tool is likely to be older than in core20 Focal base package.

I’ve confirmed that the man page for bridge command in bionic doesn’t mention “isolated” mode at all.

@stgraber does this sounds feasible, is anything blocking us from moving to core20 yet?

10

@stgraber the new tests confirm that port isolation isn’t available in Bionic:

Possibly a candidate for using netlink if possible if moving to core20 is tricky.

11

Any plans on upgrading from core18?

12

It’s planned but there are a lot of parts in the snap so it’s not an easy thing to do :slight_smile:

13

Any news on getting security.port_isolation working on the snap?

14

Currently our best bet is to re-implement at least that one call to use direct netlink API calls from Go rather than using the CLI.

It’s on the pile of stuff assigned to @brauner but it may be a few months still before he gets around to it.

15

This is now available in LXD 4.14 using the bridged NIC type’s security.port_isolation=true setting.

LXD 4.14 uses the Ubuntu Core 20.04 base image so the ip tool is updated to support it now.

1 Like