LXD containers w ubuntu images get ipv6 dhcp addresses but not ipv4, alpine does get ipv4

LXD
,
1

TL;DR: With LXD on a non-ubuntu host, what would cause containers using ubuntu images to get an ipv6 dhcp address, but not ipv4, and yet alpine images do get an ipv4 address?

I run LXD-3.3 packaged to build from source on Gentoo Linux ~amd64. Sometime after LXD-2.0.x (on my Gentoo laptop host), containers using ubuntu images stopped getting ipv4 addresses from DHCP, but do get ipv6 addresses. Alpine images do get ipv4 dhcp addresses, so dnsmasq is working properly.

I don’t have this problem on Ubuntu LXD hosts. Is there a post-install configuration step by Ubuntu LXD .deb packaging that enables dhcp ipv4 for ubuntu images from the ubuntu: or images: remotes? Some default cloud-init config perhaps? I haven’t been able to find any relevant configuration when examining ubuntu lxd hosts.

$ lxc list
+------+---------+--------------------+-----------------------------------------------+------------+
| NAME |  STATE  |        IPV4        |                     IPV6                      |   IMAGE    |
+------+---------+--------------------+-----------------------------------------------+------------+
| a001 | RUNNING | 10.81.25.83 (eth0) | fd42:f9b3:165c:1ca3:216:3eff:fe68:f2b6 (eth0) | alpine     |
+------+---------+--------------------+-----------------------------------------------+------------+
| b001 | RUNNING |                    | fd42:f9b3:165c:1ca3:216:3eff:feff:4cb (eth0)  | bionic     |
+------+---------+--------------------+-----------------------------------------------+------------+
| s001 | RUNNING |                    | fd42:f9b3:165c:1ca3:216:3eff:fe7c:df1b (eth0) | suseleap15 |
+------+---------+--------------------+-----------------------------------------------+------------+
| x001 | RUNNING |                    | fd42:f9b3:165c:1ca3:216:3eff:fea0:7c94 (eth0) | xenial     |
+------+---------+--------------------+-----------------------------------------------+------------+

LXD configuration from lxd init is mostly defaults (thanks, LXD-3.3 for the init dump feature!):

$ sudo lxd init --dump
config: {}
networks:
- config:
    ipv4.address: 10.81.25.1/24
    ipv4.nat: "true"
    ipv6.address: fd42:f9b3:165c:1ca3::1/64
  description: ""
  managed: true
  name: lxdbr0
  type: bridge
storage_pools:
- config:
    source: lxd
    volatile.initial_source: /dev/sda2
    zfs.pool_name: lxd
  description: ""
  name: lxd
  driver: zfs
profiles:
- config: {}
  description: Default LXD profile
  devices:
    eth0:
      nictype: bridged
      parent: lxdbr0
      type: nic
    root:
      path: /
      pool: lxd
      type: disk
  name: default

The host laptop has the default network bridge configuration:

$ lxc network list
+--------+----------+---------+-------------+---------+
|  NAME  |   TYPE   | MANAGED | DESCRIPTION | USED BY |
+--------+----------+---------+-------------+---------+
| eth0   | physical | NO      |             | 0       |
+--------+----------+---------+-------------+---------+
| lxdbr0 | bridge   | YES     |             | 4       |
+--------+----------+---------+-------------+---------+
| wlan0  | physical | NO      |             | 0       |
+--------+----------+---------+-------------+---------+

lxd init uses default network configuration:

$ lxc network show lxdbr0 --verbose
config:
  ipv4.address: 10.81.25.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:f9b3:165c:1ca3::1/64
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/containers/a001
- /1.0/containers/b001
- /1.0/containers/s001
- /1.0/containers/x001
managed: true
status: Created
locations:
- none

The lxd-managed dnsmasq instance seems normally configured (via ps listing):

dnsmasq
--strict-order
--bind-interfaces
--pid-file=/var/lib/lxd/networks/lxdbr0/dnsmasq.pid
--except-interface=lo
--interface=lxdbr0
--quiet-dhcp
--quiet-dhcp6
--quiet-ra
--listen-address=10.81.25.1
--dhcp-no-override
--dhcp-authoritative
--dhcp-leasefile=/var/lib/lxd/networks/lxdbr0/dnsmasq.leases
--dhcp-hostsfile=/var/lib/lxd/networks/lxdbr0/dnsmasq.hosts
--dhcp-range 10.81.25.2,10.81.25.254,1h
--listen-address=fd42:f9b3:165c:1ca3::1
--enable-ra
--dhcp-range ::,constructor :lxdbr0,ra-stateless,ra-names
-s lxd
-S /lxd/
--conf-file=/var/lib/lxd/networks/lxdbr0/dnsmasq.raw
-u nobody

The ubuntu images are from remote ubuntu:, alpine and openSUSE are from remote images:, all are set to auto-update:

$ lxc image list
+------------+--------------+--------+---------------------------------------------+--------+----------+------------------------------+
|   ALIAS    | FINGERPRINT  | PUBLIC |                 DESCRIPTION                 |  ARCH  |   SIZE   |         UPLOAD DATE          |
+------------+--------------+--------+---------------------------------------------+--------+----------+------------------------------+
| alpine     | 1bb67d6f1b29 | no     | Alpine 3.8 amd64 (20180727_13:00)           | x86_64 | 2.34MB   | Jul 27, 2018 at 3:08pm (UTC) |
+------------+--------------+--------+---------------------------------------------+--------+----------+------------------------------+
| bionic     | 38219778c2cf | no     | ubuntu 18.04 LTS amd64 (release) (20180724) | x86_64 | 173.75MB | Jul 25, 2018 at 2:50pm (UTC) |
+------------+--------------+--------+---------------------------------------------+--------+----------+------------------------------+
| suseleap15 | d7a2900a93b8 | no     | Opensuse 15.0 amd64 (20180728_00:53)        | x86_64 | 62.11MB  | Jul 28, 2018 at 3:09am (UTC) |
+------------+--------------+--------+---------------------------------------------+--------+----------+------------------------------+
| xenial     | 2a7896bae0f2 | no     | ubuntu 16.04 LTS amd64 (release) (20180724) | x86_64 | 157.79MB | Jul 25, 2018 at 2:51pm (UTC) |
+------------+--------------+--------+---------------------------------------------+--------+----------+------------------------------+

If I execute a dhcp client command on ubuntu containers, they do get an ipv4 dhcp address:

$ lxc exec x001 -- dhclient eth0
$ lxc exec b001 -- dhclient eth0
$ lxc list
+------+---------+--------------------+-----------------------------------------------+------------+-----------+
| NAME |  STATE  |        IPV4        |                     IPV6                      |    TYPE    | SNAPSHOTS |
+------+---------+--------------------+-----------------------------------------------+------------+-----------+
| a001 | RUNNING | 10.81.25.83 (eth0) | fd42:f9b3:165c:1ca3:216:3eff:fe68:f2b6 (eth0) | PERSISTENT | 0         |
+------+---------+--------------------+-----------------------------------------------+------------+-----------+
| b001 | RUNNING | 10.81.25.87 (eth0) | fd42:f9b3:165c:1ca3:216:3eff:feff:4cb (eth0)  | PERSISTENT | 0         |
+------+---------+--------------------+-----------------------------------------------+------------+-----------+
| s001 | RUNNING |                    | fd42:f9b3:165c:1ca3:216:3eff:fe7c:df1b (eth0) | PERSISTENT | 0         |
+------+---------+--------------------+-----------------------------------------------+------------+-----------+
| x001 | RUNNING | 10.81.25.33 (eth0) | fd42:f9b3:165c:1ca3:216:3eff:fea0:7c94 (eth0) | PERSISTENT | 0         |
+------+---------+--------------------+-----------------------------------------------+------------+-----------+

Containers using the openSUSE image exhibit the same symptom, but the image doesn’t have dhclient. I’m not clear how it is getting an ipv6 dhcp address, for the same reasons as ubuntu images. Until I investigate that further on openSUSE, manually setting the ipv4 address works:

$ lxc exec s001 -- ip link set eth0 up
$ lxc exec s001 -- ip address add 10.81.25.128/24 dev eth0
$ lxc list
+------+---------+---------------------+-----------------------------------------------+------------+-----------+
| NAME |  STATE  |        IPV4         |                     IPV6                      |    TYPE    | SNAPSHOTS |
+------+---------+---------------------+-----------------------------------------------+------------+-----------+
| a001 | RUNNING | 10.81.25.83 (eth0)  | fd42:f9b3:165c:1ca3:216:3eff:fe68:f2b6 (eth0) | PERSISTENT | 0         |
+------+---------+---------------------+-----------------------------------------------+------------+-----------+
| b001 | RUNNING | 10.81.25.87 (eth0)  | fd42:f9b3:165c:1ca3:216:3eff:feff:4cb (eth0)  | PERSISTENT | 0         |
+------+---------+---------------------+-----------------------------------------------+------------+-----------+
| s001 | RUNNING | 10.81.25.128 (eth0) | fd42:f9b3:165c:1ca3:216:3eff:fe7c:df1b (eth0) | PERSISTENT | 0         |
+------+---------+---------------------+-----------------------------------------------+------------+-----------+
| x001 | RUNNING | 10.81.25.33 (eth0)  | fd42:f9b3:165c:1ca3:216:3eff:fea0:7c94 (eth0) | PERSISTENT | 0         |
+------+---------+---------------------+-----------------------------------------------+------------+-----------+

Thanks for any insights you might be able to provide. Feel free to ask for more information about this particular system configuration.

2

Sounds like systemd failing to start. Usual culprit on Gentoo is you not having the right cgroup setup.

You likely want to follow the Gentoo FAQ here: https://wiki.gentoo.org/wiki/LXD#Running_systemd_based_containers_on_OpenRC_hosts

1 Like
3

Thanks Stéphane, those steps fixed the problem completely. Ubuntu xenial and bionic and openSUSE containers now start and get both ipv4 and ipv6 addresses.

For my edification what is different about the mechanism of ipv6 dhcp assignment that it worked when systemd was not starting, but ipv4 assignment would not happen?

4

It seems you haven’t enabled stateful DHCPv6 which is needed for assigning addresses. Your containers use SLAAC which is based on the prefix(es) received in RA messages and the MAC address.

5

Correct, IPv6 was working because SLAAC is entirely done by the kernel with no userspace involvement, so even though systemd crashed and no processes were running, the kernel still managed to get you an IPv6 address.

1 Like
6

This is really useful information, thanks @stgraber and @mikma for the perspective.

It might be useful to provide a tip for an ipv4-only (disable/supress SLAAC) use case in the documentation. I note that the properties for ipv4 and ipv6 differ, it might help to describe the difference and rationale:

| ipv4.dhcp          | true  | Whether to allocate addresses using DHCP                      |
| ipv6.dhcp          | true  | Whether to provide additional network configuration over DHCP |
| ipv6.dhcp.stateful | false | Whether to allocate addresses using DHCP                      |
7

It’s quite simple, there’s no such thing as stateless assignment of IPv4, so only IPv6 has such a thing as stateless or stateful DHCP, IPv4 only has stateful.

To turn off IPv6 entirely, you just unset ipv6.address (or set it to none).