Instance fails to start at system boot as parent virtual SR-IOV nic isn't ready

Incus
1

Hello,

I have an instance where nictype=physical and parent set to a host nic which is SR-IOV virtual function:

devices:
  eth0:
    nictype: physical
    parent: enp1s0f0v0
    type: nic

The problem with SR-IOV nic in my case that during a system boot it takes quite a time to appear. Up to ≈30 secs since the start of a boot sequence whereas a login prompt is ready at ≈10 seconds. Thus, when incus begins starting the instance, a parent nic isn’t there yet. Incus does couple of attempts and eventually fails as nic is not created yet.

Jun 03 23:15:53 incusd[770]: time="2025-06-03T23:15:53+02:00" level=warning msg="Failed auto start instance attempt" attempt=1 err="Failed to start device \"eth0\": Parent device 'enp1s0f0v0' doesn't exist" instance=openwrt maxAttemp>
Jun 03 23:15:58 incusd[770]: time="2025-06-03T23:15:58+02:00" level=warning msg="Failed auto start instance attempt" attempt=2 err="Failed to start device \"eth0\": Parent device 'enp1s0f0v0' doesn't exist" instance=openwrt maxAttemp>
Jun 03 23:16:03 incusd[770]: time="2025-06-03T23:16:03+02:00" level=warning msg="Failed auto start instance attempt" attempt=3 err="Failed to start device \"eth0\": Parent device 'enp1s0f0v0' doesn't exist" instance=openwrt maxAttemp>
Jun 03 23:16:03 incusd[770]: time="2025-06-03T23:16:03+02:00" level=error msg="Failed to auto start instance" err="Failed to start device \"eth0\": Parent device 'enp1s0f0v0' doesn't exist" instance=openwrt project=default

I would increase number of attempts but I can’t find such an option.
What options do I have to fix that?

Thanks.

2

The better option would be to use a systemd unit override on the incus.service unit to have it only start after whatever unit sets up your virtual function.

3

Thank you for replying.

I had hoped there could be something in Incus that I missed, but as there’s no other option, so I am managing this now with a target template unit incus-wait-nic@.target:

[Unit]
Before=incus.service
After=sys-subsystem-net-devices-%i.device
Requires=sys-subsystem-net-devices-%i.device

[Install]
RequiredBy=incus.service

This allows me to make Incus waiting as many VFs as I need with systemctl enable incus-wait-nic@enp1s0f0v0.target command once VFs are added to a system.

While it’s working, it seems to me like a workaround, because it requires manual interaction in managing extra stuff every time an interface is added or removed. I still think it would be nice if the number of start retries could be controlled through the instance or profile options to cover such cases. Incus already has implemented retries for the exact same reason, so why not make this configurable?

4

Are you managing your VFs with systemd?
Incus starts after network-online.target, so I’m not having this problem on my cluster heavily relying on SR-IOV interfaces, although your setup may make this not work.

5

Yes, I’m using solely ‘link’ units to create VFs. No custom scripts in service units are used to bring them up.

I don’t know why you think my setup may make network-online.target not work. I just added an additional ‘requires’ target to Incus, which doesn’t replace existing ones.

6

Nah I mean, if you were mixing systemd with other stuff, that could have happened. The thing is, network-online.target is triggered when you have a link up. If your VF takes a lot of time to get up (which is often the case if you configure your card with plenty of VFs), this part can also be messed up.

I’m lucky enough (I guess) to do SR-IOV on all my links, so that network-online.target is really up when my card is fully configured. If you have 2 cards (or a plugged integrated interface), this will obviously not be true.