Apparently I was missing ipv6.routing: "true".
Found out thanks to OP in Lxd ipv6 interface and bridge in same ipv6 network allowed or not? .
EDIT: Apparently something else is also at play because after a full reboot it stopped working, now to figure what else it is that I did from shell history…
EDIT2: So actually what helped was enabling NDP Proxy on host and manually registering the container, I have no clue what is going on or how to automate this. This works even after removing the above ipv6.routing config.
sysctl net.ipv6.conf.ens2.proxy_ndp=1
ip -6 neigh add proxy 2001:bc8:1640:1c45:216:3eff:fe42:2947 dev ens2
Additionally I changed the bridge to have a different IPv6 from the ens2 interface.
ipv6.address: 2001:bc8:1640:1c45::1/64
EDIT 3: So I just added the proxy adds for each container to a shell script that runs on boot as a lazy solution, one can also use ndppd as per How to Assign IPv6 Addresses to LXD Containers on a VPS | Ryan Young but I do not get why I have to do this as my /64 block is supposedly routed to the host according to Scaleway?