Using static IPs with LXD

angristan · February 26, 2018, 11:07pm

Hello,

What’s the proper way to use static IPs with LXD?

With lxc-net, you can define a LXC_DHCP_CONFILE file with the hosts. Is there such a thing with LXD?

vic-t · February 26, 2018, 11:18pm

I think what you’re looking for is the LXD_CONFILE setting in /etc/default/lxd-bridge

This article describes it in detail: https://medium.com/@ali_oguzhan/lxd-assigning-static-ip-to-containers-ecf558982071

angristan · February 26, 2018, 11:48pm

The thing is, I don’t have a /etc/default/lxd-bridge file nor a lxd-bridge service.

Otherwise this is exactly what I want.

I run Debian 9 btw.

simos · February 27, 2018, 10:11am

LXD gives out the same IP address to a container based on their MAC address.
In practice, the a container would get the same IP address.
These associations can be found in

/var/snap/lxd/common/lxd/networks/lxdbr0/dnsmasq.hosts/ (MAC<->container name).
/var/snap/lxd/common/lxd/networks/lxdbr0/dnsmasq.leases (the leases).

I think the issue is, do these leases ever expire?

The configuration file for dnsmasq in LXD (snap) is at
/var/snap/lxd/common/lxd/networks/lxdbr0/dnsmasq.raw

angristan · February 27, 2018, 10:38am

I’m sorry, I meant I wanted to specify the addresses like:

dhcp-host=container_name,ip_address

stgraber · February 27, 2018, 3:22pm

If running a modern LXD with a LXD managed bridge, then you can just set the ipv4.address property on the network interface of the container.

lxc stop c1
lxc network attach lxdbr0 c1 eth0 eth0
lxc config device set c1 eth0 ipv4.address 10.99.10.42
lxc start c1

angristan · February 27, 2018, 6:32pm

Thanks a lot @stgraber! That’s exactly what I was searching for.

Is it the same for IPv6?

root@host ~ # lxc list
+---------+---------+------------------+-------------------------------------------+------------+-----------+
|  NAME   |  STATE  |       IPV4       |                   IPV6                    |    TYPE    | SNAPSHOTS |
+---------+---------+------------------+-------------------------------------------+------------+-----------+
| test    | RUNNING | 10.0.0.4 (eth0)  | fd42:bad:c0de:0:216:3eff:fe58:32a3 (eth0) | PERSISTENT | 0         |
+---------+---------+------------------+-------------------------------------------+------------+-----------+
root@host ~ # lxc config device get test eth0 ipv4.address
10.0.0.4
root@host ~ # lxc config device get test eth0 ipv6.address
fd42:bad:c0de::4

stgraber · February 28, 2018, 3:38am

It’s the same for IPv6 but that only works if you use DHCPv6 in stateful mode, which isn’t the default.

For that, you’d need to set ipv6.dhcp.stateful=true on your bridge and then configure your container to use DHCPv6 to configure its network (that varies between distros).

simos · February 28, 2018, 9:10am

LXD’s dnsmasq does not appear to have a setting to expire the leases.
Do these leases never practically expire?

angristan · February 28, 2018, 10:27am

Great, thanks.

lxc network set lxdbr0 ipv6.dhcp.stateful true

And adding iface eth0 inet6 dhcp to the ct’s /etc/network/interfaces did the job.

angristan · February 28, 2018, 11:10pm

For some reason, I can’t change the IPv6 after setting it up the first time. The CT keeps the first IPv6. Is it an issue on the CT’s side or something about the leases?

eman_no1 · March 1, 2018, 2:29am

I am still stuck at where is /etc/default/lxd-bridge. Using Ubuntu Server 17.10, no snap. Can I install the snap on top of the current system or do In need to remvoe local lxd first?

$ which lxd
/usr/bin/lxd

$ uname -r
4.13.0-36-generic

eman_no1 · March 8, 2018, 1:33am

If the host has multiple static IP addresses, is it possible to assign one to a container?

stgraber · March 8, 2018, 8:10am

/etc/default/lxd-bridge only exists in LXD 2.0.x, all newer releases, starting with LXD 2.1 have a in-LXD network management API which replaces it.

Those are the lxc network commands and you can find out about all the options here:
http://lxd.readthedocs.io/en/latest/networks/

To pass public IPs from the host to the container, you can either have them added to your host and NAT to your container’s IP, or (usually cleaner), route the IPs or subnets to your bridge, using the ipv4.routes option on the bridge and then adding it to the container with something like ip -4 addr add ADDRESS/32 dev eth0 (or the distribution network equivalent of that).

eman_no1 · March 8, 2018, 7:57pm

I am running 17.10 which has lxc/lxd v2.0.11. This does use lxc network. I think my installation got corrupted. I tied to unstall lxd lxc and got this.

$ sudo apt remove lxc lxd
Package ‘lxc’ is not installed, so not removed
The following packages will be REMOVED:
lxd

When I installed lxd from snapcraft
$ lxd --version
2.21
$ lxc --version
2.18
$ lxc start c2
Connection refused; is LXD running?
$ sudo service lxd start
Failed to start lxd.service: Unit lxd.service is masked.

I am going to rebuild the host and try again.

stgraber · March 8, 2018, 11:58pm

You need to remove lxd and lxd-client. It looks like you still have lxd-client installed which would explain your symptoms.

eman_no1 · March 14, 2018, 10:57pm

@stgraber thanks for all of the help, I finally got static ip to work. Here are my steps so that I can find this next time I want to add a static ip.

Add static ip to host
sudo ip -4 addr add 10.0.0.209/24 dev ens2
Add static ip to bridge
lxc network set lxdbr0 ipv4.routes 10.0.0.208/29

6 addresses from 209 to 214, really only need one 10.0.0.209/32
https://www.calculator.net/ip-subnet-calculator.html?cclass=c&csubnet=29&cip=10.0.0.208&ctype=ipv4&printit=0&x=75&y=34

Add static ip to vm
sudo ip -4 addr add 10.0.0.209/32 dev eth0

zrav · August 2, 2018, 2:33pm

For anyone with the same issue:
After upgrading from 2.0.x to 3.0.x (xenial -> bionic) then to 3.3 (snap), I was unable to get externally set static ip addresses to work, until I removed the following settings from the bridge config:
lxc network unset lxdbr0 raw.dnsmasq
lxc network unset lxdbr0 dns.mode

bruce78 · September 17, 2018, 8:25am

Hi all, so going on the 2 posts above from @simos, am I right to assume that the internal IP addresses given to containers by lxdbr0 are static for all practical intents and purposes?

I’m thinking about running commands like this to enable ssh access from my laptop to a specific container

lxc config device add <container-name> sshproxy proxy listen=tcp:<public-ip>:23 connect=tcp:<container-ip>:22 bind=host

or using haproxy to forward http and https to other containers…

Presumably LXD will retain the internal IP of the container in question in perpetuity?

simos · September 17, 2018, 10:37am

Hi!

In practice I have not seen a container get a different IP address when it was restarted.
It is quite important to keep check of IP changes when you use iptables to forward connections from the host to a container (Because from container to container you can just use the hostnames).
In your case I would perform tests such as changing the time of the system when a container is about to start again to see whether it gets a new IP address.

However, if you are going to use the LXD proxy device, you do not need to worry about the IP address of the container. Because instead of specifying the IP address of the container, you can just type localhost.
For the host IP address, if you specify 0.0.0.0, then LXD will bind to all interfaces (as long as the port is free on each interface). If you have many network interfaces, you may want to just be explicit here.

That is, your SSH example would look like:

lxc config device add <container-name> sshproxy proxy listen=tcp:0.0.0.0:2222 connect=tcp:localhost:22 bind=host