As you’re not trying to span the same network over multiple LXD servers I would suggest not using OVN as that is for doing overlay networking.
Instead I would suggest you setup a bridge on each LXD server using the IP addressing schema that @amarc suggested.
For standalone servers you could use lxc network create, but as this is a cluster it won’t work as it will not allow you to create a bridge with a different subnet on each server.
So you will need to create the bridge manually, take a look at netplan/examples at main · canonical/netplan · GitHub
As @amarc suggested, you can then use the the /64 subnet on your bridge (you will need to a pick an address in the subnet for the bridge’s IP address, e.g. <prefix>::FFFF/64 and then use <PREFIX>::1/128 on the server’s external interface.
However because this isn’t a LXD managed bridge you’ll have to also potentially worry about setting up SNAT, DHCPv4 and IPv6 SLAAC (if you want these) using something like dnsmasq (as LXD does).
An alternative if you don’t need automatic IP setup is to use the routed NIC as this avoids the need for a bridge and you just specify the IPv6 addresses you want the container to have.
See