No matching virtual function for SRIOV enabled Intel i350 network card

,
1

I’m getting the lxd error “no matching virtual function” which leads me to this:

I have Intel i350 cards on two different servers running void linux kernel 5.6.2 and lxd 3.21. I can use the sriov virtual functions just fine with libvirtd, e.g., I assign a PF to a network connection and let it automatically assign available VFs.

I’d like to do the same in lxd, but I get this:

sudo lxc config device add void-test eth0 nic nictype=sriov parent=enp4s0f1
Error: Failed to start device "eth0": no matching virtual function found

ip link show enp4s0f1
3: enp4s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
   link/ether a0:36:9f:4f:42:71 brd ff:ff:ff:ff:ff:ff
   vf 0     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
   vf 1     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
   vf 2     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
   vf 3     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
   vf 4     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
   vf 5     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off
   vf 6     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off

Is there something else lxd expects in order to use the 7 available VFs?

2

Looks like the output of the ip link show command is different than we are expecting.

Here’s an example on my ubuntu 18.04 machine:

10: enp4s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 6c:b3:11:1c:09:52 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
    vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off

Yours is missing the “MAC” prefix before the MAC address.

Can you give me the output of ip -V please?

3

What OS distro/version are you using?

4

I’m using fully updated voidlinux.

ip -V
ip utility, iproute2-ss200330

And here’s the xbps package info for iproute2 in void:

architecture: x86_64
conf_files:
        /etc/iproute2/ematch_map
        /etc/iproute2/rt_dsfield
        /etc/iproute2/rt_protos
        /etc/iproute2/rt_realms
        /etc/iproute2/rt_scopes
        /etc/iproute2/rt_tables
filename-sha256: 3381634985dcbded0736a5e6f5823f36b32b7fd8cea04430df70ac8b56c34082
filename-size: 889KB
homepage: https://wiki.linuxfoundation.org/networking/iproute2
installed_size: 2950KB
license: GPL-2.0-only
maintainer: Enno Boland <gottox@voidlinux.org>
pkgname: iproute2
pkgver: iproute2-5.6.0_1
repository: https://mirrors.servercentral.com/voidlinux/current
run_depends:
        libmnl>=1.0.3_1
        glibc>=2.29_1
        iptables>=1.6.1_1
        libdb>=5.3.21_1
shlib-requires:
        libmnl.so.0
        libdl.so.2
        libc.so.6
        libm.so.6
        libxtables.so.12
        libdb-5.3.so
short_desc: IP Routing Utilities
source-revisions: iproute2:c44ab9a2e2
5

That information isn’t available in sysfs I take it?

Otherwise, that’d be far more reliable than parsing iproute output :slight_smile:

6

/sys/class/net has all the interface to PCI mappings. I assume with that you could use something like:

https://www.kernel.org/doc/html/v5.6/driver-api/pci/pci.html?highlight=sriov#c.pci_sriov_get_totalvfs

That said, this is beyond my skill level, so I could be way off. :slight_smile:

7

Yeah, that’s what we use for the data at /1.0/resources, the question is whether we can extract the special features like MAC pinning, VID, … through it too.

8

I’ll take a look to confirm. I suspect I struggled to get that info when I added this as it wouldn’t have been my first preference to use ip output.

9

Also note that on Ubuntu 20.04 and 18.04.4 LTS I see “link/ether” and not “MAC” in the output of ip link.

10

Interesting, maybe its a driver level output.

11

See this iproute2 commit: https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/commit/?id=0f48f9f46ae83c042d36c1208b0f79966f92a951

(Thanks to user tds on ##networking on freenode for helping me find this.)

1 Like
12

Also, some suggestions were to use ip -j link as a more reliable output as a stopgap if it will take longer to figure out how to interface with netlink or sysfs (or whatever the most direct but lower level approach may be).

13

Thanks for digging out that info, that really helps.

Can you show me the output of ip -j link show <nic> please for reference.

Although I need to check when the -j flag as introduced.

14

I think the -j option is no good for us as it was only present from 18.04 onwards.

15

@stgraber are you happy with trying ip -j link show and if that fails falling back to the old logic. Parsing JSON would be more reliable.

@markmcb if you can get me an example json output from your system then I can update the parser to support both the older and current json output formats (the field names for MAC are still different between versions I think).

16

Here you go:

ip -j link show enp4s0f1
[{"ifindex":3,"ifname":"enp4s0f1","flags":["BROADCAST","MULTICAST","UP","LOWER_UP"],"mtu":1500,"qdisc":"mq","operstate":"UP","linkmode":"DEFAULT","group":"default","txqlen":1000,"link_type":"ether","address":"a0:36:9f:4f:42:71","broadcast":"ff:ff:ff:ff:ff:ff","vfinfo_list":[{"vf":0,"link_type":"ether","address":"00:00:00:00:00:00","broadcast":"ff:ff:ff:ff:ff:ff","vlan_list":[{}],"rate":{"max_tx":0,"min_tx":0},"spoofchk":true,"link_state":"auto","trust":false},{"vf":1,"link_type":"ether","address":"00:00:00:00:00:00","broadcast":"ff:ff:ff:ff:ff:ff","vlan_list":[{}],"rate":{"max_tx":0,"min_tx":0},"spoofchk":true,"link_state":"auto","trust":false},{"vf":2,"link_type":"ether","address":"00:00:00:00:00:00","broadcast":"ff:ff:ff:ff:ff:ff","vlan_list":[{}],"rate":{"max_tx":0,"min_tx":0},"spoofchk":true,"link_state":"auto","trust":false},{"vf":3,"link_type":"ether","address":"00:00:00:00:00:00","broadcast":"ff:ff:ff:ff:ff:ff","vlan_list":[{}],"rate":{"max_tx":0,"min_tx":0},"spoofchk":true,"link_state":"auto","trust":false},{"vf":4,"link_type":"ether","address":"00:00:00:00:00:00","broadcast":"ff:ff:ff:ff:ff:ff","vlan_list":[{}],"rate":{"max_tx":0,"min_tx":0},"spoofchk":true,"link_state":"auto","trust":false},{"vf":5,"link_type":"ether","address":"00:00:00:00:00:00","broadcast":"ff:ff:ff:ff:ff:ff","vlan_list":[{}],"rate":{"max_tx":0,"min_tx":0},"spoofchk":true,"link_state":"auto","trust":false},{"vf":6,"link_type":"ether","address":"00:00:00:00:00:00","broadcast":"ff:ff:ff:ff:ff:ff","vlan_list":[{}],"rate":{"max_tx":0,"min_tx":0},"spoofchk":true,"link_state":"auto","trust":false}]}]
1 Like
17

Fine with me.

18

Please can you advise whether a command like this works for you:

sudo ip link set enp4s0f1 vf 0 mac 00:00:00:00:00:00

or whether it needs to be something like:

sudo ip link set enp4s0f1 vf 0 address 00:00:00:00:00:00

Thanks

19

I’ve started work on adding support for this in PR:

However I would like to know whether the ip tool has also changed the way it supports setting a VF’s MAC address too.

Thanks
Tom

20

This works:

sudo ip link set enp4s0f1 vf 0 mac 00:00:00:00:00:00

Using address instead of mac results in an error.

1 Like