As you can see getting the various firewall implementations to play nicely together is a non-trivial task.
In the meantime these commands seems to suffice to allow traffic from lxdbr0 interface to the LXD host and for traffic from lxdbr0 to be routed to the external network without allowing all external inbound traffic:
sudo ufw allow in on lxdbr0
sudo ufw route allow in on lxdbr0
sudo ufw route allow out on lxdbr0
See our page in the docs about this also: