Prevent cross-talk

LXD
8

So I’ve come up with a way to do this using a common ACL applied to each managed bridge network:

First create the managed bridge networks and connect some instances to them:

lxc network create lxdbr1
lxc network create lxdbr2
lxc launch images:ubuntu/jammy c1 -n lxdbr1
lxc launch images:ubuntu/jammy c2 -n lxdbr1
lxc launch images:ubuntu/jammy c3 -n lxdbr2
lxc launch images:ubuntu/jammy c4 -n lxdbr2

lxc network ls
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
|      NAME       |   TYPE   | MANAGED |      IPV4       |           IPV6            | DESCRIPTION | USED BY |  STATE  |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| lxdbr1          | bridge   | YES     | 10.6.53.1/24    | fd42:6775:8e08:4b29::1/64 |             | 2       | CREATED |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| lxdbr2          | bridge   | YES     | 10.1.70.1/24    | fd42:59cb:e895:e486::1/64 |             | 2       | CREATED |
+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+

lxc ls
+--------+---------+--------------------+-----------------------------------------------+-----------+-----------+
|  NAME  |  STATE  |        IPV4        |                     IPV6                      |   TYPE    | SNAPSHOTS |
+--------+---------+--------------------+-----------------------------------------------+-----------+-----------+
| c1     | RUNNING | 10.6.53.254 (eth0) | fd42:6775:8e08:4b29:216:3eff:fefa:9454 (eth0) | CONTAINER | 0         |
+--------+---------+--------------------+-----------------------------------------------+-----------+-----------+
| c2     | RUNNING | 10.6.53.178 (eth0) | fd42:6775:8e08:4b29:216:3eff:feae:2263 (eth0) | CONTAINER | 0         |
+--------+---------+--------------------+-----------------------------------------------+-----------+-----------+
| c3     | RUNNING | 10.1.70.10 (eth0)  | fd42:59cb:e895:e486:216:3eff:fecb:b062 (eth0) | CONTAINER | 0         |
+--------+---------+--------------------+-----------------------------------------------+-----------+-----------+
| c4     | RUNNING | 10.1.70.110 (eth0) | fd42:59cb:e895:e486:216:3eff:feed:891a (eth0) | CONTAINER | 0         |
+--------+---------+--------------------+-----------------------------------------------+-----------+-----------+

Now prevent intra-bridge cross-talk using port-isolation:

lxc config device set c1 eth0 security.port_isolation=true
lxc config device set c2 eth0 security.port_isolation=true
lxc config device set c3 eth0 security.port_isolation=true
lxc config device set c4 eth0 security.port_isolation=true

Test intra-bridge cross talk blocked:

lxc exec c1 -- ping c2 -4 -c1
PING  (10.6.53.178) 56(84) bytes of data.
From c1.lxd (10.6.53.254) icmp_seq=1 Destination Host Unreachable

---  ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

Test inter-bridge cross talk still allowed:

lxc exec c1 -- ping 10.1.70.10 -c1
PING 10.1.70.10 (10.1.70.10) 56(84) bytes of data.
64 bytes from 10.1.70.10: icmp_seq=1 ttl=63 time=0.072 ms

--- 10.1.70.10 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.072/0.072/0.072/0.000 ms

Now setup ACL to prevent inter-bridge routed cross-talk:

lxc network set lxdbr1 \
    security.acls.default.egress.action=allow \
    security.acls.default.ingress.action=allow
lxc network set lxdbr2 \
    security.acls.default.egress.action=allow \
    security.acls.default.ingress.action=allow

lxc network acl create external-only
Network ACL external-only created

lxc network acl rule add external-only egress \
    destination=10.6.53.0/24 \
    action=reject
lxc network acl rule add external-only egress \
    destination=10.1.70.0/24 \
    action=reject

lxc network acl show external-only
name: external-only
description: ""
egress:
- action: reject
  destination: 10.6.53.0/24
  state: enabled
- action: reject
  destination: 10.1.70.0/24
  state: enabled
ingress: []
config: {}
used_by: []

Now apply the ACL to each network:

lxc network set lxdbr1 security.acls=external-only
lxc network set lxdbr2 security.acls=external-only

Test inter-bridge routed traffic blocked:

lxc exec c1 -- ping 10.1.70.10 -c1
PING 10.1.70.10 (10.1.70.10) 56(84) bytes of data.
From 10.6.53.1 icmp_seq=1 Destination Port Unreachable

--- 10.1.70.10 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

Test external traffic still allowed (and that DNS is working):

lxc exec c1 -- ping www.linuxcontainers.org -4 -c1
PING rproxy.dcmtl.stgraber.org (45.45.148.7) 56(84) bytes of data.
64 bytes from rproxy.dcmtl.stgraber.org (45.45.148.7): icmp_seq=1 ttl=51 time=91.4 ms

--- rproxy.dcmtl.stgraber.org ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 91.423/91.423/91.423/0.000 ms
5 Likes