Install NVIDIA KVM driver on the host machine, how to use CUDA in MIG instance

As mentioned for some reason the latest NVidia driver are acting different during system boot to initialize the GPU support. I stumbled across this during my research on why my card didn’t fully work in Incus after system reboot. Some people mentioned their X-Server didn’t come up or similar. Not sure if NVidia will fix it anytime soon. DOn’t have that link handy but you will find it.

There are quite a few topics around this issue and so far I haven’t found the golden solution that works out of the box. For example Container with nvidia.runtime=true refuse to start after reboot of the host is more or less doing the same like I mentioned “run a small proc to intialize”. I haven’t tried the solution mentioned in this topic and if nvidia.runtime=true will work. In my case I run into some permissions issues following this path during testing, so I decided to install drivers locally. May revisit it again if time permits…

For now there is a solution / workaround that keeps my container applications happy.

1 Like