Hi. Yes that approach is probably a lot simpler than having a vlan per container.
And as you say, by using firewall on the LXD host to enforce your network policy with regard to source address that works too.
I made some changes to the bridge driver recently that should make allowing IP filtering on unmanaged bridges easier to implement, so its still something that is on my radar.