IncusOS Network Bridges

IncusOS
, ,
22

Okay and you brought the eth1 interface up in the container and ran tcpdump -ni eth1 or similar on it and you’re not seeing anything at all?

23

That is correct
I do see some broadcast but nothing related to the fact that the mirror should span ALL infra traffic

24

What’s span connected to? Is that all physical networking, is there any kind of MAC address filtering type stuff going on?

25

It is a USB NIC directly connected to a Netgear switchport configured as mirror destination (probe)

26

Ah, I think the issue here is that IncusOS creates another bridge, so the bridge then filters the port mirroring traffic and that never makes it to the container.

27

That is what I was thinking indeed.
Is there any way “around” this ?

28

So that’s going to need a bit of hackery as Incus doesn’t have configuration to force effectively promiscuous mode on one of its NICs.

Can you maybe try this:

incus config device remove NAME-OF-INSTANCE eth-1
incus config device add NAME-OF-INSTANCE nic nictype=macvlan name=eth1 parent=_pMAC-OF-SPAN

That should get you a macvlan device directly attached to the physical NIC, bypassing the bridge and its filtering.

29

Sure, give me a sec

30 
derek@dmbr incus % ./incus config device add debian-container nic nictype=macvlan name=eth1 parent=<redacted>
Error: Invalid devices: Device validation failed for "nic": Failed loading device "nic": Unsupported device type
31

Oh, oops, sorry, typo:

incus config device add NAME-OF-INSTANCE eth-1 nic nictype=macvlan name=eth1 parent=_pMAC-OF-SPAN
32

Hmm

Error: Failed to start device "eth-1": Parent device 'MAC-OF-SPAN' doesn't exist

Of course I put the MAC address of the SPAN interface instead of MAC-OF-SPAN :slight_smile:

33

Format for parent is _p112233445566 so the MAC address with now separator and I believe all lower case (or matching the case you’ve used in the config)

1 Like
34

Error: Failed to start device “eth-1”: device or resource busy

35

Ah, that’s different :wink:

Could you try removing that interface from your IncusOS network configuration completely, then reboot your IncusOS system?

That should then make the interface show up as unmanaged (in the state but not in config). At which point you can do incus admin os system show network to see what name it’s using as an unconfigured NIC (probably enpXYZ) and can adjust your macvlan NIC to use that as the parent.

Basically the goal is to not have the span device be put into a bridge, just keep it standalone as an interface that can be used by a container directly.

1 Like
36

Actually by doing that ^ you’ll have two options:

  1. Use macvlan as described above which will hopefully work correctly this time and would allow multiple instances to be attached to the span device.
  2. Use the physical nictype instead which will have the device be moved from the host to the container instead. This is basically guaranteed to work, but comes with the limitation of having only one container be able to use it and it causing some confusion at the host level since the interface will disappear from it while in use by the container.
1 Like
37

Lovely !
Indeed, the macvlan type does not work but having the physical does. It is OK since I only need a single instance using it !.

Thanks !

38

Excellent, great to hear that did the trick!

Getting access to a port mirroring device is definitely an interesting edge case that we didn’t think of :wink:

We do have customers who need some port mirroring stuff for traffic analysis (IDS) but most of those actually get an encapsulated stream from the switches instead (GRE/VXLAN) instead of a physical mirror.

39

It is indeed for an IDS :wink: but since this is my home setup, I KISS using physical mirror :wink:

40

And yes, I do run my prod home setup in Incus (well, IncusOS now ;))