My box does have an ipmi, so I don’t need that to run. I would rather spread the GPU across the containers than dumping it into a single VM. Good suggestion though.
I also use Immich, Kasm, and a few others that take advantage for the GPU.
My understanding is that NVIDIA is dropping support for the 1080ti from their driver in the next couple of months, so you’re likely to run into some more problems soon.
What’s the state of nouveau’s support for the 1080ti?
Let me check. It’s too good of a card to drop to the wayside lol
Not sure how that would work with the nvidia-container-toolkit either.
This is why it may be better to do a standard build and just run the last version of the driver supported.
Hardware deprecation stinks…
I do have a Vega64 in my Linux desktop I could switch for the 1080ti I suppose. I’ve also looked at the Intel ARC devices as well.
Yeah, from what I can tell nouveau won’t work. That would be fine for basic desktop usage.
It looks like I have probably another year or so before I’ll start having issues with the 1080ti.
I am looking at ordering an Intel Arc A770, that will make it easy to do GPU passthrough without having to jump through all the Nvidia hoops with drivers, toolkits, etc.
From what I can tell, this shouldn’t be an issue. Kind of frustrating that I need to do this, but it is what it is.
But the good news is that there’s recently been a lot of progress on both NVIDIA and AMD GPU virtualization for the mainline kernel, so we’re hoping to be able to provide good support for GPU passthrough and GPU virtualization on those platforms without having to include binary drivers directly.
This is probably getting quite off topic so I apologize in advance, but could you elaborate on this? It’s to my understanding that - barring physical issues like AMD 7000 series PCIe reset bug - VFIO has been quite mature in PCIe passthrough to VMs; or are you referring to things like Virtio GPU (GL, Venus, and Native Context).
On the Nvidia portion, do you mean this Nvidia repository? GitHub - NVIDIA/open-gpu-kernel-modules: NVIDIA Linux open GPU kernel module source
Direct VFIO works pretty well so long as you have a suitable IOMMU environment.
What I was referring to was the datacenter grade stuff, not consumer grade.
That is, AMD GPUs with SR-IOV support to give you 64 VFs per GPU that can be handed over to VMs, or the equivalent on the NVIDIA side with GRID.
Currently both of them (AMD mxgpu/gim and NVIDIA GRID/vGPU) require separate license agreements with binary drivers that require a contract and subscription just to even be downloaded (they’re not using the standard binary NVIDIA or AMD drivers at all).
But both of them are now making moves to have that bit of infrastructure be available in an open source driver (not mainline, but at least open source), so we should in time be able to take DC grade GPUs and hand over slices to a bunch of VMs without all the paperwork that’s currently needed.
1 Like