If your non-root user account is a member of the lxd group, then this non-root user account is able to use the lxc client and perform all sorts of management tasks on the LXD server. Including the launching of containers with the security.privileged=true flag.
It is a known issue that the non-root user account that is a member of the lxd group, should be considered an administrator. The reason is that LXD is so versatile as a hypervisor that there should be many ways to get root in such ways. The security.privileged flag is the most straightforward way.
LXD supports Role-based Access Control (RBAC), although there is no (AFAIK) free RBAC service currently available. There is the Canonical RBAC, which you may be able to use for free for a limited number of LXD installations. See https://landscape.canonical.com/
All in all, if the above sound too complicated, you can just avoid using a non-root user account with the lxd group membership. If you do not have such an account, then you need to use sudo for every lxc command. Hence, you will be asked for a password.