Assign public IP to container for outgoing traffic

1

Hello,

I have a LXD server with multiple IPs (aliases) and a number of containers. I need to set each container to connect to the Internet using a particular IP. Currently, I have no control on which IP is used by each container, as they all use using the “main” interface IP.

I think the reason for this behaviour is that I am masquerading all the traffic that leave the host trought the eth0 interface and I have to use SNAT instead. But, is there any other way to achieve this trought LXD instead of using iptables?

Here there is some info about my setup:

$ lxd --version
4.11

$ sudo ifconfig (edited)
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet publicIp1  netmask 255.255.255.0  broadcast xxx
eth0:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet publicIp2  netmask 255.255.255.0  broadcast xxx
eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet publicIp3  netmask 255.255.255.0  broadcast xxx
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.0.119  netmask 255.255.255.0  broadcast 10.0.0.255
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0

lxdbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.236.142.1  netmask 255.255.255.0  broadcast 0.0.0.0
../..

$ sudo brctl show
bridge name     bridge id               STP enabled     interfaces
lxdbr0          8000.00163e42760c       no              veth1a1899ef
                                                        veth1e58af76
                                                        veth58efcbcc
                                                        veth650d1da6
                                                        vethd29de20f
                                                        vethd7981757
$ sudo iptables -L -n -t nat -v
../..
Chain POSTROUTING (policy ACCEPT 1 packets, 84 bytes)
pkts bytes target     prot opt in     out     source               destination
0     0 TCPMSS     tcp  --  *      eth0    0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 TCPMSS clamp to PMTU
0     0 MASQUERADE  all  --  *      eth0    10.0.0.0/24         !10.0.0.0/24
../..

$ lxc network show lxdbr0
config:
  ipv4.address: 10.236.142.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:63e7:5dca:b480::1/64
  ipv6.nat: "true"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/instances/call
- (more instances here)
managed: true
status: Created
locations:
- none

$ lxc list
+-------------------+---------+-----------------------+------+-----------+-----------+
|       NAME        |  STATE  |         IPV4          | IPV6 |   TYPE    | SNAPSHOTS |
+-------------------+---------+-----------------------+------+-----------+-----------+
| call              | RUNNING | 10.236.142.187 (eth0) |      | CONTAINER | 0         |
+-------------------+---------+-----------------------+------+-----------+-----------+
../..

Currently, the “call” container is using publicIp1 to connect to the Internet, but I want to set it up so that it uses publicIp3:

root@call:~# dig +short myip.opendns.com @resolver1.opendns.com
publicIp1
2

Hi there,

I made it!

I created a new network with the option ipv4.nat.address=publicIp3 so that the containers using that network would use the publicIp3 to connect to the Internet. Then I changed the network for the existing container.

lxc network create lxdbr1 --type=bridge ipv4.address=10.236.143.1/24 ipv4.nat=true ipv4.nat.address=publicIp3
lxc config device add call eth0 nic nictype=bridged parent=lxdbr1 name=eth0

And now the public IP that the container uses to connect to the Internet is publicIp3:

root@call:~# dig +short myip.opendns.com @resolver1.opendns.com
publicIp3

Hope this helps.

Regards,

1 Like
3

You can also do this, https://blog.simos.info/configuring-public-ip-addresses-on-cloud-servers-for-lxd-containers/ which means to give a public IP address to a specific LXD container.

Note that you can get a LXD container to obtain a public IP address with either

  1. public bridge (nictype=bridge)
  2. macvlan (nictype=macvlan)
  3. routed (nictype=routed)
  4. ipvlan (nictype=ipvlan)
4

Hello @simos,

I really appreciate your answer. Indeed, I had read your post before posting my question, but I am finding it hard to get my mind around the different sort of networks (macvlan, sriov, ovn, etc) available in LXD. I haven’t found any good tutorial about this topic. Do you know some link so as to learn more about this kind of networks?

Furthermore, in this case, I would prefer not to give a public IP to the container which is directly accessible from the Internet, but I like to forward just the necessary ports for the service running in the container.

Regards,

5

All these are software-based networking features to enable different types of networking. They are software-based, further based on the Linux kernel. For example, here is a general networking article on ipvlan vs macvlan. Here is the ipvlan documentation for the implementation in the Linux kernel. And then there is a packaging of the ipvlan functionality as provided by LXD that you can use in your containers.

Therefore, when you want to delve deeper in any of these, first look at the general networking concept, then check the Linux kernel implementation documentation, and finally you reach the implementation in LXD.

This narrows down the amount of investigation you need to make. If you are just hosting public-facing websites and web services, then you would probably look into using a reverse proxy. You might be able to make all these work with a single public IP address and the reverse proxy. Have you seen this one, How to Set Up a Reverse Proxy to Host Websites in LXD | Linode Docs ?

If you really need the multiple public IP addresses, then you would create the respective containers to have separate reverse proxies.