How to set public IPs for each container in LXD 3.0.0 & Ubuntu 18.04

(chaines) #1

Hello,
My question is how to setup a public IP address for each container, and preferable how to set it all up from the host machine using the LXD commands (like lxc network set, lxc network attach, etc).

I have my host machine directly on a public IP x.x.x.40. There is no LAN network for my host.
I would like to have each of 5 container have it’s own public IP address x.x.x.41-46. I would like each container to act as it if is a distinct physical server with it’s own public IP.

It looks like the way I need to do this is using macvlan, but most of the tutorials/guides I have read are for setting it up using LAN, DHCP, and manually editing /etc/networking/interfaces file on the container to configure. In addition, many references using older version of LXD referring to /var/lib/lxc/[container]/config files and such.

The closest I got was using this guide, but got stuck when they started editing the container conf file, which does not exist in LXD 3, and I would prefer to use LXC commands to add an IP/netmask/DNS if possible: http://www.bonsaiframework.com/wiki/display/bonsai/5.1+LXC+Advanced+Networking+-+Exposing+Containers+to+the+Network#id-5.1LXCAdvancedNetworking-ExposingContainerstotheNetwork-macvlanwithAdditionalIPAddress

From basically a fresh setup of Ubuntu 18.04 and LXD 3.0.0, and running this container: ubuntu:18.04, can someone please help direct me in the right direction.
If I have to setup a separate macvlan or virtual interface for each container that is fine too.

Thank you!
-C

(Tom) #2

Hi chaines,

Please check: this topic

Use /32.

Host:
lxc network set lxdbr0 ipv4.routes PUBLIC-IP/32

Container:
ip -4 addr add dev eth0 PUBLIC-IP/32 preferred_lft 0

The traffic goes through the host, so a virtual mac is not necessary. I also had problems with this, but this option still works in 18.04 with LXD.

(chaines) #3

Hello,
Yes I have tried that guide. Here are my steps and some info:

ip a
4: lxdbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
    inet 10.226.135.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
    inet6 XXXXXX/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 XXXXXXXX/64 scope link 
       valid_lft forever preferred_lft forever

lxc network set lxdbr0 ipv4.routes X.X.X.41/32

lxc launch ubuntu:18.04 test1
Creating test1
Starting test1


lxc list
+-------+---------+-----------------------+----------------------------------------------+------------+-----------+
| NAME  |  STATE  |         IPV4          |                     IPV6                     |    TYPE    | SNAPSHOTS |
+-------+---------+-----------------------+----------------------------------------------+------------+-----------+
| test1 | RUNNING | 10.226.135.107 (eth0) | fd42:8e36:8eb:b134:216:3eff:fe09:a695 (eth0) | PERSISTENT | 0         |
+-------+---------+-----------------------+----------------------------------------------+------------+-----------+

ip -4 addr add dev eth0 X.X.X.41/32 preferred_lft 0
(^ run inside container)

lxc list
+-------+---------+-----------------------+----------------------------------------------+------------+-----------+
| NAME  |  STATE  |         IPV4          |                     IPV6                     |    TYPE    | SNAPSHOTS |
+-------+---------+-----------------------+----------------------------------------------+------------+-----------+
| test1 | RUNNING | X.X.X.41 (eth0)  | fd42:8e36:8eb:b134:216:3eff:fe09:a695 (eth0) | PERSISTENT | 0         |
|       |         | 10.226.135.107 (eth0) |                                              |            |           |
+-------+---------+-----------------------+----------------------------------------------+------------+-----------+

Now when I go into the container and curl my IP, I get the hosts IP back. Additionally, when I install apache in the container, I can curl it locally in the container and from the host using both IPs, however I cannot curl it from outside the host, or access the apache test page on the internet.

(chaines) #4

I’m really stuck here.

I have even tried manually editing /etc/network/interfaces in ubuntu 16 container and netplan/50-cloud-config in 18 containers. Still cannot get containers to take external static IP. Perhaps incorrect way my bridges are setup?

(chaines) #5

Ok finally figured it out using this guide:
https://blog.simos.info/how-to-make-your-lxd-container-get-ip-addresses-from-your-lan/

Then just manually editing the network config file within the container to set the static IP/subnet/gateway/DNS

Only thing now is I want to be able to set network config for the container from the host. I would think this would work, but doesn’t seem to:

> lxc config device set test1 eth0 ipv4.address X.X.X.41
Error: The device doesn't exist

However, eth0 in the container definitely exists, and I can manually configure it in the container’s OS settings.

Am I doing something wrong?

#6

When a LXD container is configured to get a LAN IP address (compared to LXD managed networking with lxdbr0 private bridge), it gets this IP on its own. The LXD hypervisor does not have a feature to set a static LAN IP address to a container.

Indeed, it would be handy to write something like

lxc config device set test1 eth0 ipv4.address X.X.X.41

but how would LXD be able to go into any type of container and set a static LAN IP address in the networking? If it were to be implemented in LXD, there should be a generic way to make it work for all types of containers.

Having said that, there is cloud-init, and several of the LXD container images support cloud-init. Those from the ubuntu: repository of container images support cloud-init, therefore you can set through there the static LAN IP address from the host, and make it happen in a beautiful and elegant way.
Note that the container images from the images: repository may not support cloud-init.

  1. To see how to use cloud-init with LXD containers, have a look at https://blog.simos.info/how-to-preconfigure-lxd-containers-with-cloud-init/

  2. Then, use a netplan configuration to set the static IP address for Ubuntu 18.04 container images.

(chaines) #7

Thanks. The cloud-init is exactly what I was looking for. Your link is very helpful, however, I am having problems with setting it up a netplan config.
I created a cloud-config file:

#cloud-config
network:
    version: 2
    ethernets:
        eth0:
            dhcp4: false
            addresses: [X.X.X.41/24]

And first tried applying it like this:

lxc launch ubuntu:18.04 test --config=user.user-data="$(cat test.yaml)"

Which worked fine for a test.yaml with:
timezone: America/Denver
in it, however did not touch netplan configs OR /etc/network/*

So then I tried like this with the same file:

lxc launch ubuntu:18.04 test --config=user.network-config="$(cat test.yaml)"

Still no luck. That time the /etc/netplan/ folder was empty (no 50-cloud-init.yaml file anymore). That cloud config file above is basically how my 50-cloud-init file looks like when I manually edit it for static IPs.

Any pointers where to go from here?

#8

Yeah, you just need to specify the network-config in version 1 instead of version 2.

And when you launch the container, you will be able to view the network configuration (in version 1) at /var/lib/cloud/seed/nocloud-net/network-config

Subsequently, that network configuration in version 1 will be auto-converted in version 2, and you will be able to view it in /etc/netplan/50-cloud-init.yaml

And your static networking settings will then work.

(chaines) #9

Awesome @simos! Thank you so much!