IPv6 static address with stateful dhcp

Hi, How can i use IPv6 static address with stateful dhcp?

I saw Github issue #3519 and topic ‘Using static IPs with LXD’ .
They said we need to set the ipv6.dhcp.stateful property on the network to true to have LXD run dnsmasq in stateful mode.

I tried it as below, but it didn’t work well.

ubuntu@ubuntu18:~$ lxc network create lxd0719 ipv6.dhcp.stateful=true
Network lxd0719 created

ubuntu@ubuntu18:~$ lxc init images:ubuntu/xenial test0719
Creating test0719

ubuntu@ubuntu18:~$ lxc config device add test0719 eth0 nic nictype=bridged parent=lxd0719 ipv4.address=10.228.181.123 ipv6.address=fd42:7e3e:264b:5dca::123
Device eth0 added to test0719

ubuntu@ubuntu18:~$ lxc start test0719
ubuntu@ubuntu18:~$ lxc exec test0719 bash

root@test0719:~# vim /etc/network/interfaces
root@test0719:~# cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
iface eth0 inet6 dhcp

root@test0719:~# reboot

ubuntu@ubuntu18:~$ lxc list test0719
+----------+---------+-----------------------+------+------------+-----------+
|   NAME   |  STATE  |         IPV4          | IPV6 |    TYPE    | SNAPSHOTS |
+----------+---------+-----------------------+------+------------+-----------+
| test0719 | RUNNING | 10.228.181.129 (eth0) |      | PERSISTENT | 0         |
+----------+---------+-----------------------+------+------------+-----------+
ubuntu@ubuntu18:~$ lxc config show test0719
architecture: x86_64
config:
  image.architecture: amd64
  image.description: Ubuntu xenial amd64 (20180718_07:42)
  image.os: Ubuntu
  image.release: xenial
  image.serial: "20180718_07:42"
  volatile.base_image: bf0b1111f90a161a5e905c76a3021cb09e3668dc8c7d371535cd9492a0d8dd50
  volatile.eth0.hwaddr: 00:16:3e:33:1c:cd
  volatile.eth0.name: eth0
  volatile.idmap.base: "0"
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    ipv4.address: 10.228.181.123
    ipv6.address: fd42:7e3e:264b:5dca::123
    nictype: bridged
    parent: lxd0719
    type: nic
ephemeral: false
profiles:
- default
stateful: false
description: ""

ubuntu@ubuntu18:~$ lxc network show lxd0719
config:
  ipv4.address: 10.228.181.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:7e3e:264b:5dca::1/64
  ipv6.dhcp.stateful: "true"
  ipv6.nat: "true"
description: ""
name: lxd0719
type: bridge
used_by:
- /1.0/containers/test0719
managed: true
status: Created
locations:
- none

IPv4 and IPv6 didn’t works well, so i did as below and only IPv4 work well. Ipv6 didn’t.

ubuntu@ubuntu18:~$ lxc network set lxd0719 ipv4.dhcp true
ubuntu@ubuntu18:~$ lxc network set lxd0719 ipv6.dhcp true

ubuntu@ubuntu18:~$ lxc restart test0719
ubuntu@ubuntu18:~$ lxc list test0719
+----------+---------+-----------------------+------+------------+-----------+
|   NAME   |  STATE  |         IPV4          | IPV6 |    TYPE    | SNAPSHOTS |
+----------+---------+-----------------------+------+------------+-----------+
| test0719 | RUNNING | 10.228.181.123 (eth0) |      | PERSISTENT | 0         |
+----------+---------+-----------------------+------+------------+-----------+

ubuntu@ubuntu18:~$ lxc network show lxd0719
config:
  ipv4.address: 10.228.181.1/24
  ipv4.dhcp: "true"
  ipv4.nat: "true"
  ipv6.address: fd42:7e3e:264b:5dca::1/64
  ipv6.dhcp: "true"
  ipv6.dhcp.stateful: "true"
  ipv6.nat: "true"
description: ""
name: lxd0719
type: bridge
used_by:
- /1.0/containers/test0719
managed: true
status: Created
locations:
- none

My LXD and kernel version are below.

ubuntu@ubuntu18:~$ uname -a
Linux ubuntu18 4.15.0-23-generic #25-Ubuntu SMP Wed May 23 18:02:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

ubuntu@ubuntu18:~$ dpkg -l lxd*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                          Version                     Architecture                Description
+++-=============================================-===========================-===========================-================================================================================================
ii  lxd                                           3.0.1-0ubuntu1~18.04.1      amd64                       Container hypervisor based on LXC - daemon
ii  lxd-client                                    3.0.1-0ubuntu1~18.04.1      amd64                       Container hypervisor based on LXC - client
un  lxd-tools                                     <none>                      <none>                      (no description available)

What happens if you manually run dhclient -6 eth0 after the container is online (to effectively skip any potential issue with ifupdown)?

I did as below but nothing to happen.

ubuntu@ubuntu18:~$ lxc list test0719
+----------+---------+-----------------------+------+------------+-----------+
|   NAME   |  STATE  |         IPV4          | IPV6 |    TYPE    | SNAPSHOTS |
+----------+---------+-----------------------+------+------------+-----------+
| test0719 | RUNNING | 10.228.181.123 (eth0) |      | PERSISTENT | 0         |
+----------+---------+-----------------------+------+------------+-----------+

ubuntu@ubuntu18:~$ lxc exec test0719 bash

root@test0719:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
23: eth0@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:33:1c:cd brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.228.181.123/24 brd 10.228.181.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe33:1ccd/64 scope link 
       valid_lft forever preferred_lft forever

root@test0719:~# dhclient -6 eth0

When I ran dhclient -6 eth0, I ran tcpdump -n -nn -i lxd0719 ip6 on the host.

root@ubuntu18:~# tcpdump -n -nn -i lxd0719 ip6
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lxd0719, link-type EN10MB (Ethernet), capture size 262144 bytes
14:33:56.371910 IP6 fe80::216:3eff:fe33:1ccd.546 > ff02::1:2.547: dhcp6 solicit
14:33:57.413213 IP6 fe80::216:3eff:fe33:1ccd.546 > ff02::1:2.547: dhcp6 solicit
14:33:59.395800 IP6 fe80::216:3eff:fe33:1ccd.546 > ff02::1:2.547: dhcp6 solicit
14:34:03.457035 IP6 fe80::216:3eff:fe33:1ccd.546 > ff02::1:2.547: dhcp6 solicit

It seems the DHCPv6 request was sent but there was no reply.

I also noticed the problem with DHCPv4. On my system running the command above triggered creation of the file /var/lib/lxd/networks/lxd0719/dnsmasq.hosts/test0719, which didn’t exist before that.
After a restart it got the correct IPv4 address, but to get the correct IPv6 address I also needed to first remove /var/lib/dhcp/dhclient6.eth0.leases.

Update: An instance with bionic seems to work better since you don’t need to remove any file on the instance. Running “ip l set down eth0; ip l set up eth0” or restarting is enough.

I removed /var/lib/dhcp/dhclient6.eth0.leases and restart container. The container didn’t get IPv6 address with DHCPv6.

ubuntu@ubuntu18:~$ lxc exec test0719 -- ls -l /var/lib/dhcp/dhclient6.eth0.leases
 -rw-r--r-- 1 root root 62 Jul 24 01:21 /var/lib/dhcp/dhclient6.eth0.leases
ubuntu@ubuntu18:~$ lxc exec test0719 -- rm /var/lib/dhcp/dhclient6.eth0.leases
ubuntu@ubuntu18:~$ lxc restart test0719
ubuntu@ubuntu18:~$ lxc list test0719
 +----------+---------+-----------------------+------+------------+-----------+
 |   NAME   |  STATE  |         IPV4          | IPV6 |    TYPE    | SNAPSHOTS |
 +----------+---------+-----------------------+------+------------+-----------+
 | test0719 | RUNNING | 10.228.181.123 (eth0) |      | PERSISTENT | 0         |
 +----------+---------+-----------------------+------+------------+-----------+

DHCPv6 request was sent by container but there was no reply…

root@ubuntu18:~# tcpdump -n -nn -i lxd0719 ip6
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lxd0719, link-type EN10MB (Ethernet), capture size 262144 bytes
15:58:11.059600 IP6 fe80::216:3eff:fe33:1ccd.546 > ff02::1:2.547: dhcp6 solicit
15:59:14.664293 IP6 fe80::216:3eff:fe33:1ccd.546 > ff02::1:2.547: dhcp6 solicit

Of course, dnsmasq for lxd0719 is running.

ubuntu@ubuntu18:~$ ps auxf
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

  .....

lxd      22920  0.0  0.0  52872   384 ?        S    Jul24   0:00 dnsmasq --strict-order --bind-interfaces --pid-file=/var/lib/lxd/networks/lxd0719/dnsmasq.pid --except-interface=lo --interface=lxd0719 --quiet-dhcp --quiet-dhcp6 --quiet-ra --listen-address
root     25961  0.0  0.0 605600  7336 ?        Ss   15:57   0:00 [lxc monitor] /var/lib/lxd/containers test0719
100000   25983  0.1  0.0  37048  5184 ?        Ss   15:57   0:00  \_ /sbin/init
100000   26052  0.0  0.0  35272  5000 ?        Ss   15:57   0:00      \_ /lib/systemd/systemd-journald
100000   26057  0.0  0.0  41588  3316 ?        Ss   15:57   0:00      \_ /lib/systemd/systemd-udevd
100000   26078  0.0  0.0   4412  1764 ?        Ss   15:57   0:00      \_ /sbin/ifup -a --read-environment
100000   26207  0.0  0.0   4504   796 ?        S    15:57   0:00      |   \_ /bin/sh -c /sbin/dhclient -1 -6 -pf /run/dhclient6.eth0.pid -lf /var/lib/dhcp/dhclient6.eth0.leases -I -df /var/lib/dhcp/dhclient.eth0.leases eth0
100000   26208  0.0  0.0  16120  3704 ?        S    15:57   0:00      |       \_ /sbin/dhclient -1 -6 -pf /run/dhclient6.eth0.pid -lf /var/lib/dhcp/dhclient6.eth0.leases -I -df /var/lib/dhcp/dhclient.eth0.leases eth0
100104   26111  0.0  0.0 182660  3172 ?        Ssl  15:57   0:00      \_ /usr/sbin/rsyslogd -n
100000   26112  0.0  0.0  28980  2824 ?        Ss   15:57   0:00      \_ /usr/sbin/cron -f
100000   26126  0.7  0.0   4504  1768 ?        S    15:57   0:00      \_ /bin/sh /etc/init.d/ondemand background
100000   26197  0.0  0.0   7288   648 ?        S    15:57   0:00      |   \_ sleep 60
100000   26160  0.0  0.0  16120   852 ?        Ss   15:57   0:00      \_ /sbin/dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -I -df /var/lib/dhcp/dhclient6.eth0.leases eth0

ubuntu@ubuntu18:~# cat /proc/22920/cmdline 
dnsmasq--strict-order--bind-interfaces--pid-file=/var/lib/lxd/networks/lxd0719/dnsmasq.pid--except-interface=lo--interface=lxd0719--quiet-dhcp--quiet-dhcp6--quiet-ra--listen-address=10.228.181.1--dhcp-no-override--dhcp-authoritative--dhcp-leasefile=/var/lib/lxd/networks/lxd0719/dnsmasq.leases--dhcp-hostsfile=/var/lib/lxd/networks/lxd0719/dnsmasq.hosts--dhcp-range10.228.181.2,10.228.181.254,1h--listen-address=fd42:7e3e:264b:5dca::1--enable-ra--dhcp-rangefd42:7e3e:264b:5dca::2,fd42:7e3e:264b:5dca:ffff:ffff:ffff:ffff,1h-slxd-S/lxd/--conf-file=/var/lib/lxd/networks/lxd0719/dnsmasq.raw-ulxd

Weird, anything showing up in /var/log/syslog on the host when those DHCP requests are sent through? Also, anything in ip6tables -L -n -v?

No message was showing up in host’s /var/log/syslog when I ran dhclient -6 eth0 in container.
The results I ran ip6tables -L -n -v on LXD host is below.

root@ubuntu18:~# ip6tables -L -n -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp      lxd0719 *       ::/0                 ::/0                 tcp dpt:53 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     udp      lxd0719 *       ::/0                 ::/0                 udp dpt:53 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     udp      lxd0719 *       ::/0                 ::/0                 udp dpt:546 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     tcp      lxdbr0 *       ::/0                 ::/0                 tcp dpt:53 /* generated for LXD network lxdbr0 */
    0     0 ACCEPT     udp      lxdbr0 *       ::/0                 ::/0                 udp dpt:53 /* generated for LXD network lxdbr0 */
    0     0 ACCEPT     udp      lxdbr0 *       ::/0                 ::/0                 udp dpt:546 /* generated for LXD network lxdbr0 */
96849  501M ACCEPT     all      *      *       ::/0                 ::/0                 state RELATED,ESTABLISHED
 4010  409K ACCEPT     icmpv6    *      *       ::/0                 ::/0                
    0     0 ACCEPT     all      lo     *       ::/0                 ::/0                
    0     0 ACCEPT     all      ens4   *       ::/0                 ::/0                
    0     0 ACCEPT     tcp      *      *       ::/0                 ::/0                 tcp dpt:2000 state NEW
    0     0 ACCEPT     udp      *      *       ::/0                 fe80::/64            udp dpt:546 state NEW
  322 35766 REJECT     all      *      *       ::/0                 ::/0                 reject-with icmp6-adm-prohibited

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all      *      lxd0719  ::/0                 ::/0                 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     all      lxd0719 *       ::/0                 ::/0                 /* generated for LXD network lxd0719 */
    7   728 ACCEPT     all      *      lxdbr0  ::/0                 ::/0                 /* generated for LXD network lxdbr0 */
    7   728 ACCEPT     all      lxdbr0 *       ::/0                 ::/0                 /* generated for LXD network lxdbr0 */
    0     0 REJECT     all      *      *       ::/0                 ::/0                 reject-with icmp6-adm-prohibited

Chain OUTPUT (policy ACCEPT 25790 packets, 1935K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp      *      lxd0719  ::/0                 ::/0                 tcp spt:53 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     udp      *      lxd0719  ::/0                 ::/0                 udp spt:53 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     udp      *      lxd0719  ::/0                 ::/0                 udp spt:546 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     tcp      *      lxdbr0  ::/0                 ::/0                 tcp spt:53 /* generated for LXD network lxdbr0 */
    0     0 ACCEPT     udp      *      lxdbr0  ::/0                 ::/0                 udp spt:53 /* generated for LXD network lxdbr0 */
    0     0 ACCEPT     udp      *      lxdbr0  ::/0                 ::/0                 udp spt:546 /* generated for LXD network lxdbr0 */

I tried on other LXD host with Ubuntu 18.04 which has GUI and run on VirtualBox. Unfortunately, it didn’t work well, too.

Hmm, I wonder if your problems are caused by some traffic getting rejected (traffic that’s not already allowed by LXD’s default rules) as you have a blanket REJECT rule in your INPUT chain that’s seeing quite a few hits.

Could you try ip6tables -I INPUT -i lxd0719 -j ACCEPT see if allowing all incoming traffic from that bridge fixes your issue?

Oh! Thank you very much :bowing_man:
It works well! :smile:

I finally run ip6tables -I INPUT -i lxd0719 -p udp -m udp --dport 547 -j ACCEPT.

root@ubuntu18:~# ip6tables -L INPUT -vn
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    2   264 ACCEPT     udp      lxd0719 *       ::/0                 ::/0                 udp dpt:547
    0     0 ACCEPT     tcp      lxd0719 *       ::/0                 ::/0                 tcp dpt:53 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     udp      lxd0719 *       ::/0                 ::/0                 udp dpt:53 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     udp      lxd0719 *       ::/0                 ::/0                 udp dpt:546 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     tcp      lxdbr0 *       ::/0                 ::/0                 tcp dpt:53 /* generated for LXD network lxdbr0 */
    0     0 ACCEPT     udp      lxdbr0 *       ::/0                 ::/0                 udp dpt:53 /* generated for LXD network lxdbr0 */
    0     0 ACCEPT     udp      lxdbr0 *       ::/0                 ::/0                 udp dpt:546 /* generated for LXD network lxdbr0 */
97305  502M ACCEPT     all      *      *       ::/0                 ::/0                 state RELATED,ESTABLISHED
 4095  418K ACCEPT     icmpv6    *      *       ::/0                 ::/0                
    0     0 ACCEPT     all      lo     *       ::/0                 ::/0                
    0     0 ACCEPT     all      ens4   *       ::/0                 ::/0                
    0     0 ACCEPT     all      *      *       2401:2500:10a:1032::/64  ::/0                
    0     0 ACCEPT     tcp      *      *       ::/0                 ::/0                 tcp dpt:2000 state NEW
    0     0 ACCEPT     udp      *      *       ::/0                 fe80::/64            udp dpt:546 state NEW
  334 37014 REJECT     all      *      *       ::/0                 ::/0                 reject-with icmp6-adm-prohibited

In IPv4, LXD automatically add rule to allow DHCP server port(67/udp) to INPUT chain as below.

root@ubuntu18:~# iptables -L INPUT -vn
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  lxd0719 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     udp  --  lxd0719 *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 /* generated for LXD network lxd0719 */
    6  1968 ACCEPT     udp  --  lxd0719 *       0.0.0.0/0            0.0.0.0/0            udp dpt:67 /* generated for LXD network lxd0719 */
    0     0 ACCEPT     tcp  --  lxdbr0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53 /* generated for LXD network lxdbr0 */
    9   888 ACCEPT     udp  --  lxdbr0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 /* generated for LXD network lxdbr0 */
22990 7541K ACCEPT     udp  --  lxdbr0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:67 /* generated for LXD network lxdbr0 */
48826   14M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
  127  6812 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           
  818 57965 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  ens4   *       0.0.0.0/0            0.0.0.0/0           
   16   832 ACCEPT     all  --  *      *       133.242.73.240/28    0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       210.152.9.91         0.0.0.0/0           
   12   604 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:2000 state NEW
 4937  207K REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited

In IPv6, It seems LXD add rule to allow DHCPv6 client port(546/udp). Is it correct?
According to https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?&page=10 , DHCPv6 server port is 547/udp.

1 Like

Yeah, I think we’re just a bit lazy and allow both server and client ports in both FORWARD and INPUT, in this case we’re missing the server port (547), I’ll send a patch.

Ah, actually, we only do INPUT, so yeah, should be 547 instead of 546.

@stgraber Thank you :smiley: