Gobetween supports LXD out the box for the use case your asking for GitHub - GoBetween project load balancer for the Сloud supports both local & remote LXD servers.
Gobetween is now in maintenance mode though so someone should probably just write some generic program for haproxy / nginx / apache / whatever else is in these days.