I have BGP coming to my core router and i’m able to handle iBGP but a limited number of IPv4 (a /28) . I’m thinking to do the following:
have nodes connected each others using vxlan : Fan Network
Use BGP to advertise a /64 per nodes and agive a public /32 from the + /28 to some nodes.
2 iinstances would work as a router with BGP failover for IPv4
DO you think it could be setup with current LXD? ANything I can optimize for it?
Separately I may want to connect a fan network with another network outside of these servers. Is there a way to know which VID is used so i can possibly connect this vxlan to a normal network iusing a physicla switch?
How do you manage/ get access to nodes inside the Fan network that don’t have a public IP? VPN? A Jump Proxy host?
I’ve never combined our BGP support with the Fan.
Instead, I either use OVN or I use a simple bridge (not fan) and then use ipv4.routes.external or ipv6.routes.external to advertise public addresses routed to a specific instance.
So you could have a specific /32 IPv4 for one of your instances, put that in ipv4.routes.external and assuming it’s on a LXD network that has BGP configured, that address will then be advertised to your router with a next-hop set to go through the LXD machine it’s running on.
In the instance itself, you’ll need to manually put the address on the interface through a static network configuration.
OK I see. One thing I just checked though is that lxd config can only happen on a managed (created by lxd) bridge and not a physical bridge. Which means I’m stuck for now. How to connect different nodes LXD machines in a cluster when we only have 2 NICs. can we set the ip on an interface (so the machine can be joined) and put that interface in a bridge? Sounds odd for me.
If you’re advertising routes via BGP then you shouldn’t need to have the instances be connected to the external parent bridge. Instead LXD would advertise the managed bridge’s IP address space with a next hop address of the LXD host’s IP on the external network.