IncusOS installing new network interface card

IncusOS
1

Hi all, happy new year!

I’ve been playing around with IncusOS over the holidays with a new machine, but just today received a new NIC to install (upgrading from the built-in 1GbE interface to a 2.5GbE interface).

But how can I use it? The new interface doesn’t work when installed, and also breaks the old interface.

I think the problem is that initially, the working configuration was that my 1G NIC is given the name enp4s0. Indeed on a live USB I have laying around, I can see with only the 1G NIC connected, lspci reports 04:00.0 for this NIC (which I think corresponds to enp4s0 with the predictable naming scheme).

But when I install the 2.5G NIC alongside (unable to remove the 1G NIC), I can see in the live USB that the new card is now on 04:00.0, with the old 1G NIC pushed down to 05:00.0 (so much for a predictability!)

As a consequence, my understanding is the old NIC would now be found on enp5s0 and the new NIC replaces enp4s0.

But this causes problems for IncusOS that I can see is still trying to bring the network up on the originally installed enp4s0. I get a timeout error, I assume due to the fact that the network configuration (specficially the hwaddr field) now doesn’t match the new card.

Assuming I’m not barking up the wrong tree with the above, how can I proceed? The only way I can access IncusOS is by unplugging the new NIC, reverting to the original hardware state at install time.

I can then use incus admin os system network edit to change settings. But I’m not sure what to do here. I have tried changing to the future expected hardware state after shutting down and reinstalling the new NIC, by setting the new hwaddr on enp4s0, but that doesn’t work, it gives me some errors. After a hopeful reboot, the network still doesn’t come up (as presumably the configuration didn’t stick). Instead I need to remove the new NIC, back to the originally installed state, to have the network come up correctly.

So my attempts at diagnosis above might be way off. Regardless, I’m not sure how to proceed. I’m tempted to reinstall IncusOS with the hope it correctly configures things with the new hardware state, but I still think it useful to know if this is a recoverable situation.

Cheers in advance!

2

IncusOS only cares about MAC addresses, so the name / PCI address should not matter.

It will not be happy if a network card disappears though, so swapping a card for another won’t really work, but adding the card should let the existing card still work, then lets you edit the network config to use the new card and then remove the old card.

If not possible and you have to swap cards, you may be able to use a USB NIC as a temporary option, adding that, configuring it, removing the config for the old NIC, reboot with the new NIC, configure that and remove the USB NIC config.

3

That’s the bit I’m struggling with, that isn’t my observation. Adding the new NIC alongside the old one seems to prevent either from connecting.

I hadn’t thought of trying a separate USB NIC so I’ll give that a try now. Thanks!

4

Ok I got things working.

There was definitely some issue with adding the new PCI NIC (alongside the original, at no point did I remove the original as its fixed on the motherboard). As mentioned previously, this didn’t work. The network configuration at the bottom just showed an empty enp4s0( ) and ultimately timed out.

The necessary steps were to go back to the initial configuration (that is, remove the new NIC), and let the OS successfully boot. Then with a spare USB NIC I had lying around, plug it in and use incus admin os system network edit to add it as an additional interface. Unlike the new PCI NIC, I had no problem adding the USB NIC.

This let me power off, reinstall the new PCI NIC, and then bring up the machine with connectivity (through the USB NIC, as having the new PCI NIC prevented the original NIC from connecting).

That then finally let me see all my interfaces with incus admin os system network edit which I hadn’t been able to do yet with Incus.

And according to the state section, this confirmed my suspicion that my original NIC had been moved to enp5s0 and the new NIC stole the name enp4s0.

I had to then explicitly remove the old interfaceenp4s0from the interfaces section (which still was using the old hwaddr, apply the new network config, then re-add enp4s0this time with the hwaddr for the new NIC (same as it was listed in the state section).

That was a long winded bunch of steps, but point is it is working now. Thanks for the suggestion on trying the USB NIC.

What this does imply is there might be some unexpected behaviour around adding a new NIC. You state the name is not important, but could there maybe be some unanticipated conflict around names colliding with old interfaces when hardware changes? Does this need a bug report? It might be tricky to repro if its based on my very particular set of hard peculiarities.