SOLVED: Best way to assign static IP addresses to public facing containers?

LXD
1

In a previous LXD deployment I configured containers using the lxdbr0 bridge, and assigned static IP addresses to them like this:

lxc config device set my_container eth0 ipv4.address 10.248.83.3

In my current deployment I’m using a system bridge (br0) to create public facing containers. The previous command no longer works:

[root@gecko lxd]# lxc config device set samba-dc eth0 ipv4.address 192.168.1.80
Error: Device from profile(s) cannot be modified for individual instance. Override device or modify profile instead

Referencing Simos’ excellent blog post, I think one way to solve this is to create a profile for this container that just sets a static IP address? This seems rather inelegant, particularly if I’m going to be spinning up several containers with the same network characteristics. Is there a better way to do this? I’m guessing the answer is to use a cloud-init script of some kind, but I’m not using cloud-init.

This got me curious about how this works when one is using the LXD standard bridge, lxdbr0. grep was able to answer that question:

[root@erap-atx lxd]# cd /var/lib/lxd
[root@erap-atx lxd]# grep -r "10.248.83.3" .
...
./networks/lxdbr0/dnsmasq.hosts/my_container:00:16:3e:14:09:ee,10.248.83.3,my_container
./networks/lxdbr0/dnsmasq.leases:1634733590 00:16:3e:14:09:ee 10.248.83.3 my_container *

So apparently the lxc config device set my_container eth0 ipv4.address 10.248.83.3 command just performs some dnsmasq configuration, setting up a DHCP reservation for this IP/container – can someone confirm this? That could maybe be documented someplace.

2

The error “Override device or modify profile instead” means either add/modify the device in the profile or override the profile device config for this particular instance device using:

lxc config device override <instance> <device> ipv4.address=...

Separately, the static IP address config can only be used with NICs that connect to LXD managed bridge networks, not separately managed bridges (like br0) because we don’t manage the DHCP server for that network.

3

Thanks, Tom! It helps me (and likely others) to understand the architecture of the system. When one exercises the

lxc config device override

command where does this configuration information go? Presumably it’s not generating and applying a custom profile – does this just go into the (I’m presuming here, based on package dependencies) the dqlite LXD database? Part of my issue is a lack of understanding of the persistence of profile information applied to a container; i.e. do the profile directives get inserted into the container image itself somehow or are these always dynamically applied when the container is spun up?

4

The device config is copied (with any modifications specified) from the profile (lxc profile show <profile>) into the instance (lxc config show <instance>) so that it overrides the profile config for that one instance.

Profile and instance config exist in the database. They are separate tables, and the latter can override the former.

Running lxc config show <instance> --expanded shows the resulting config after the instance’s profiles have been applied, and any instance specific config has been applied on top.

5

Perfect. I finally feel like I’m on the downhill slope of understanding how all this works. Very much appreciate your help. Marking this topic as solved.

1 Like