I have been following this very useful thread on routable IPV6 allocation. This seems to work well for a single node lxd install.
I am using 3 nodes in a LXD cluster and followed the OVN LXD Cluster tutorial and everything work well. The only issue is how to assign routable IPV6 addresses that can be accessed externally for the instances running in each LXD host.
Each LXD host has its own /64 IPV6 subnet that is routed directly to the host (using Hetzner). My question, Is it possible to have LXD setup such that the instances that are placed on different hosts gets different IPV6 addresses and those IPV6 addresses are from the /64 subnet assigned to the host.
@tomp Either OVN or Normal bridge will work for me.
Trying to figure out if VM1 is placed on host1, it gets a routable IPV6 address from IPV6_Subnet1 that is tied to the host1. If VM2 is placed on host2, it then gets a routable IPV6 address from IPV6_Subnet2 that is tied to the host2.
Not sure how I would achieve this with OVN or plain bridge network. Since it is Hetzner, I cannot have the Mac addresses of the instances appear on the Hetzner network.
If you have for example routable /64 to your host1 what would you do is to assign single IPv6 to host1 ie. your:ipv6:subnet**::1/128** and than durring ‘lxd init’ you would provide your:ipv6:subnet**::2/64**
Pay attention to bold parts. This is with bridge network type.
This works for me on couple of dedicated/VPS servers providers that supply servers with routable subnets
@amarc Were you using standalone lxd cluster? I am trying this on 3 node lxd cluster, if an instance is placed on host1 it should get a ipv6_subnet1/64 address and if it placed on host2, it should get ipv6_subnet2/64 and so on.
The IPV6 subnet assigned to the instance should change based on which host it is placed on.
As you’re not trying to span the same network over multiple LXD servers I would suggest not using OVN as that is for doing overlay networking.
Instead I would suggest you setup a bridge on each LXD server using the IP addressing schema that @amarc suggested.
For standalone servers you could use lxc network create, but as this is a cluster it won’t work as it will not allow you to create a bridge with a different subnet on each server.
As @amarc suggested, you can then use the the /64 subnet on your bridge (you will need to a pick an address in the subnet for the bridge’s IP address, e.g. <prefix>::FFFF/64 and then use <PREFIX>::1/128 on the server’s external interface.
However because this isn’t a LXD managed bridge you’ll have to also potentially worry about setting up SNAT, DHCPv4 and IPv6 SLAAC (if you want these) using something like dnsmasq (as LXD does).
An alternative if you don’t need automatic IP setup is to use the routed NIC as this avoids the need for a bridge and you just specify the IPv6 addresses you want the container to have.