I must have misconfigured something. I created a fresh container, setup UFW, setup the forwarding policy, and used the very same container config as well as wireguard config and everything works as expected.
To recap: install the wireguard kernel module and on the host, only the tools are needed in the container/run the interface in the container.