I applied the solution of this other thread Lxd uses iptables-legacy even if iptables-nft is default and that solved my problem 
which is basically to flush all the iptable rules. This is all I did -
# for ipt in iptables iptables-legacy ip6tables ip6tables-legacy; do $ipt --flush; $ipt --flush -t nat; $ipt --delete-chain; $ipt --delete-chain -t nat; $ipt -P FORWARD ACCEPT; $ipt -P INPUT ACCEPT; $ipt -P OUTPUT ACCEPT; done
# systemctl reload snap.lxd.daemon
And that was it