Gpu passthrough failed for lxc/lxd virtual machine

I enabled iommu, sr-iov and svm in bios then also added GRUB_CMDLINE_LINUX_DEFAULT=“amd_iommu=on iommu=pt” to /etc/default/grub.

Next, I launched lxc launch images:ubuntu/20.04/cloud v1 --vm successfully.

Finally, I executed lxc config device add v1 gpu0 gpu gputype=physical pci=0000:xx:00.0.

When I tried to perform lxc start v1, it just got stuck there with no error message and I have to reset the server.

Can anyone advice where I did wrong? Is there something I need to setup for VFIO? Thank you.

Specs:
AMD Milan Processor
Asrock Rack ROMED8-2T Motherboard
Nvidia RTX A4000 GPU

Anything in dmesg? Also, just making sure that this GPU is dedicated to this and it’s not currently tied to host NVIDIA GPU drivers or the like?

I saw https://github.com/bryansteiner/gpu-passthrough-tutorial tutorial that can dynamically unbind the nvidia/amd drivers and bind the vfio drivers right before the VM starts and subsequently reversing these actions when the VM stops. That way, whenever the VM isn’t in use, the GPU is available to the host machine to do work on its native drivers. How can this be achieved for lxc/lxd?