Having an issue getting vm’s to boot when a nic is configured to a sriov network. Containers work fine with same sriov network.
popup reads “Failed setting up device via monitor: Failed setting up device “eth1”: Failed adding NIC device: GenericError: vfio 0000:02:10.5: group 2 is not viable”
Not real sure what to look at and didn’t have any luck finding similar issues. Would appreciate any help. thanks
You’ll want to check the usual things:
iommu=pt typically)If that’s the case and you still have a problem, then look at incus info --resources as that will show you what devices are in each IOMMU group.
For passthrough, all devices in a particular group must be passed together, so if your NIC is somehow mixed with another device, then they both need to be passed to the VM. Usually server platforms have better IOMMU group handling effectively guaranteeing distinct group for each device. Desktop boards have more of a tendency to do lane sharing and have groups containing multiple devices.
ah i see all the vf’s are in the same iommu group. so vm’s need the whole group but containers don’t?
Yeah, containers don’t use the PCI passthrough logic, instead the VF is handled like a PF by the host and the resulting OS level network interface is then passed into the container.
Obviously SR-IOV is really meant for the VM case, so it’s pretty unusual that those can’t be passed through. You really should go take a look in your BIOS, on top of the aforementioned VT-d or similar setting, there often is a dedicated SR-IOV setting which may also need to be enabled for things to work optimally.
Did some more research and it seems i would need to recompile the kernel with a acs-override patch to get it to split the interfaces up since the bios doesn’t really support it on this pc… not sure about that so looking to at other options.
Ah yeah, the acs-override stuff is quite involved 