I have purchased a server from Heztner, and I am trying to setup LXD to use the IPv6 block that they gave me.
My understanding is that:
I can use the free public IPv6 addresses they gave me for the containers without having to buy any IPv4 addresses.
I can do this without any complicated routed or bridge setups if i use the IPv6 addresses. Simply by setting up the IPv6 CIDR notation during init then LXD will handle the rest for me.
Here is how I setup LXD
root@baremetal01 ~ # lxd init
Would you like to use LXD clustering? (yes/no) [default=no]:
Do you want to configure a new storage pool? (yes/no) [default=yes]:
Name of the new storage pool [default=default]:
Name of the storage backend to use (dir, lvm, zfs, ceph, btrfs) [default=zfs]:
Create a new ZFS pool? (yes/no) [default=yes]:
Would you like to use an existing empty block device (e.g. a disk or partition)? (yes/no) [default=no]:
Size in GB of the new loop device (1GB minimum) [default=30GB]: 100GB
Would you like to connect to a MAAS server? (yes/no) [default=no]:
Would you like to create a new local network bridge? (yes/no) [default=yes]:
What should the new bridge be called? [default=lxdbr0]:
What IPv4 address should be used? (CIDR subnet notation, âautoâ or ânoneâ) [default=auto]:
What IPv6 address should be used? (CIDR subnet notation, âautoâ or ânoneâ) [default=auto]: cafe1:1234:6789:abcd::2/64
Would you like LXD to NAT IPv6 traffic on your bridge? [default=yes]:
Would you like the LXD server to be available over the network? (yes/no) [default=no]:
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:
Here you go, I changed the public IP address with 123 and abc so if it something does not make sense let me know.
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp41s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether a8:a1:59:8b:35:a5 brd ff:ff:ff:ff:ff:ff
inet 123.123.123.72/32 scope global enp41s0
valid_lft forever preferred_lft forever
inet6 2a01:abcd:abcd:abcd::2/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::aaa1:abcd:abcd:abcd/64 scope link
valid_lft forever preferred_lft forever
3: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:16:3e:c4:57:41 brd ff:ff:ff:ff:ff:ff
inet 10.56.132.1/24 scope global lxdbr0
valid_lft forever preferred_lft forever
inet6 2a01:abcd:abcd:abcd::2/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::216:abcd:abcd:abcd/64 scope link
valid_lft forever preferred_lft forever
7: veth15f0dc10@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxdbr0 state UP group default qlen 1000
link/ether 1a:0c:d1:ae:55:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0
$ ip r
default via 123.123.123.72 dev enp41s0 proto static onlink
10.56.132.0/24 dev lxdbr0 proto kernel scope link src 10.56.132.1
Additional IP addresses talks about forwarding IPv6 addresses. In this case I have not bought IP addresses etc. They give me one IPv4 address, and a block of Ipv6 addreses.
Ah can you also provide ip -6 r from the host and container please.
Although I think I can already see the issue. You cannot have 2a01:abcd:abcd:abcd::2/64 defined on both enp41s0 and lxdbr0. As that will create 2 routes for 2a01:abcd:abcd:abcd::/64 one going out of enp41s0 and the other going out of lxdbr0. When you ping an IP in the subnet which interface will the host use? (Hint: thereâs no happy answer here :))
This has come up in the past and youâve got a couple of options:
Remove the IPs from enp41s0 and move them to an unmanaged bridge such as br0 (e.g. using Netplan | Backend-agnostic network configuration in YAML) and then get your containers to directly attach to the external network using lxc config device add <instance> <eth0> nic nictype=bridged parent=br0. This will then rely on the external networkâs DHCP/SLAAC and DNS services (if they exist). It will also mean that each instance will get its own MAC address, which may be restricted by Hetznerâs network.
Use a routed approach. As it sounds like you have the whole /64 routed to your LXD host directly without the need for NDP proxying. You could just take a single IP from the /64 subnet and assign it to enp41s0 with a /128 subnet (so it doesnât add any routes to the host). Then pick a different IP for the lxdbr0 interfaceâs IP and use the /64 subnet. This way youâll only have one /64 route on the LXD host for your subnet (going down lxdbr0) and the host should still respond to its own IP on enp41s0. That way LXD will provide DHCP/SLAAC and DNS services for lxdbr0 as it is solely responsible for the subnet. And all packets leaving the host will use the hostâs external interface MAC address.
$ ip -6 r
::1 dev lo proto kernel metric 256 pref medium
2a01:4f9:abcd:abcd::/64 dev enp41s0 proto kernel metric 256 pref medium
2a01:4f9:abcd:abcd::/64 dev lxdbr0 proto kernel metric 256 pref medium
fe80::/64 dev enp41s0 proto kernel metric 256 pref medium
fe80::/64 dev lxdbr0 proto kernel metric 256 pref medium
default via fe80::1 dev enp41s0 proto static metric 1024 pref medium
I had asked Heztner for additional mac addresses (as per there docs) for the IPv6 addresses, and they told me The IPv6 subnet must use the MAC address of the primary IP address .
It sounds like to me I am trying to setup number 2, so when you say âyou could just take a single IP from the /64 subnet and assign it to enp41s0 with a /128 subnet (so it doesnât add any routes to the host).â, do you mean adjust the netplan below so the IPv6 address is written /128 ?
Yeah so enp41s0 could have an address of 2a01:abcd:abcd:abcd::1/128 and then lxdbr0 could have an address of 2a01:abcd:abcd:abcd::2/64 (as it does now).
You already have the default gateway set to fe80::1 statically, so it wonât affect how that is reached.
Youâre ip -6 r output should then only show 1 line for 1 2a01:abcd:abcd:abcd::/64
Thereâs a certain symmetry to your IPv4 config to, as youâve already got the equivalent setup for IPv4 using a /32 (single IP) and a out of subnet on-link default route.
I did that and now I get internet inside the container, but still cant reach the container from the outside.
ip -6 r
::1 dev lo proto kernel metric 256 pref medium
2a01:abcd:abcd:abcd::2 dev enp41s0 proto kernel metric 256 pref medium
2a01:abcd:abcd:abcd::/64 dev lxdbr0 proto kernel metric 256 pref medium
fe80::/64 dev enp41s0 proto kernel metric 256 pref medium
fe80::/64 dev lxdbr0 proto kernel metric 256 pref medium
default via fe80::1 dev enp41s0 proto static metric 1024 pref medium