You'd typically do this with:
lxc network set lxdbr0 ipv4.routes 188.8.131.52/32
Which will have a route added on the host to send traffic to your container's IP to the right bridge.
Then you need to make sure your container has that IP on its main network interface.
For testing you can just do it with:
ip -4 route add dev eth0 184.108.40.206/32
At which point you should be able to access the container using that IP, but as you mentioned, container traffic may still show up with the host IP.
You can avoid that by completely statically configuring your container with:
iface eth0 inet static
pre-up ip -4 route add dev eth0 10.0.3.1/32
The MASQUERADE rule that LXD maintains is scoped so that only traffic using the bridge's subnet is NATed, so if your container sends traffic out using its public IP, it won't get NATed by the rules that LXD addded to iptables.