LXD forward the same external IP to different Bridges

I have a small VPS with one external IPv4 address (ext_ip1) and would like to forward external ports to several instances. In order to use ACLs between these instances, I am using a separate (non-OVN) network brige for each container, which roughly looks as follows -

lxc list

+------+----------+---------+----------------+---------------------+
| NAME |   TYPE   | MANAGED   |      IPV4              |           |
+------+----------+---------+----------------+---------------------+
| c1   | bridge   | YES       | 10.10.102.102 (eth0)   |           |
+------+----------+---------+----------------+---------------------+
| c2   | bridge   | YES       | 10.10.101.101 (eth0)   |           |
+------+----------+---------+----------------+---------------------+
| c3   | bridge   | YES       | 10.10.100.100 (eth0)   |           |
+------+----------+---------+----------------+---------------------+

lxc network list

+------+----------+---------+----------------+---------------------+
| NAME |   TYPE   | MANAGED   |      IPV4        |               |
+------+----------+---------+----------------+---------------------+
| n1   | bridge   | YES       | 10.10.102.1/24   |               |
+------+----------+---------+----------------+---------------------+
| n2   | bridge   | YES       | 10.10.101.1/24   |               |
+------+----------+---------+----------------+---------------------+
| n3   | bridge   | YES       | 10.10.100.1/24   |               |
+------+----------+---------+----------------+---------------------+

I can easily create a forward to one bridge / instance using lxd like this:

lxc network forward create n1 ext_ip1
Network forward ext1 created

… however, using the command for another network results in the following error:

lxc network forward create n2 ext_ip1
Error: Failed creating forward: Forward listen address "ext_ip1/32" overlaps with another network or NIC

This is probably intentional as the docs state that for regular bridge networks “any non-conflicting listen address is allowed”. It would be quite useful to be able to assign the the same external address to multiple bridge networks and define port forwardings for the instance(s) attached to them though; assuming that different ports are assigned to each bridge / instance of course.

There are several other ways to solve this - I could manually setup some netfilter rules (but I’d rather configure everything using LXD), could attach all the containers to the same bridge (ACL rules won’t work between instances) or switch to an OVN network (which seems overkill for a single host). In short: If possible, I’d rather stick with my current setup -

Which leads to the question: Is forwarding one external IP address to multiple bridges at all possible or is one of the other approaches necessary? Am I perhaps overlooking something else?

You could asd a lxd proxy device with nat=true to each instance rather than use a network forward. They both use dnat under the hood.

1 Like

Thank you very much for your response and that’s certainly another possibility. I’ve read in another thread that forwarding and the proxy device are quite similar when it comes to bridged networks and maybe that is the way to go then; if given any choice in the matter, I’d prefer the centralized approach of network forwarding though.

I’m curious: Is the limitation of not being able to assign the same external IP as a listen address to multiple bridges a design choice or are there technical reasons preventing this? If I added the firewall rules manually, forwarding from the same address to multiple bridges would be no problem at all, so I guess it’s the former? If so, would you consider changing this to make the function more flexible? I’m probably not the only one using multiple bridges on a cheap VPS, or am I? :wink:

Its a design choice for making both bridge and ovn behave the same, but a requirement for ovn network forwarders.

Thank you again for your answer and the proxy device with nat=true it is then. An option to change the behavior (with the default matching OVN) would be nice to have though.

2 Likes