Trying to migrate libvirt/qemu VM to incus, but vlan trunking not working

Incus
1

Hi All,

I am trying to migrate from libvirt/qemu to incus, but one particular appliance (F5 BIG-IP VE) fails with trunking vlans.

I have created a bridge via openvswitch with ‘ovs-vsctl add-br trunk’
I attached this bridge to several other virtual machines/containers in incus (debian 13) and even an Infoblox DDI appliance.
All above mentioned vm’s and containers can happily add vlan tagged interfaces on their interface and talk happily with each other (when in the same vlan offcourse)
The only one that won’t work is the F5. It looks like traffic (tagged correctly) does arrive on the F5, and the F5 says it’s responding, but a capture on the host on the trunk doesn’t see the return traffic from the F5. It doesn’t look like a firewall/stp issue as running the F5 outside of incus (just via libvirt/qemu) it all works fine (same host on same bridge).

I have tried all kinds of different network scenario’s in incus, but could not found a working one.
Would anyone have a clue/hint/tip where to look further?

The only solution so far is to stick to libvirt/qemu for the VM’s and use incus only for the containers, but all in one would be preferrable offcourse.

I am using incus 6.18 (from the zabbly repo) on a debian 13 server.

2

Was the libvirt VM also using virtio-net for networking?

If you have other instances that can interact with those VLAN just fine, then the issue is almost certainly within the VM itself.

Can you somewhat easily try a clean installation of this appliance in a new VM, see if there’s maybe some config/state that got messed up with the move?

3

Hi Stephane,

Yes also virtio-net for networking. Also tried with a fresh installation of the F5 Virtual edition, but same result. It’s pulling my hair out cause have no clue what it can be :frowning: . Using a systemd-networked bridge instead of openvswitch, didn’t change the outcome either (did expect that already)..

4

I take it you’ve been dumping the tap device directly on the host using something like tcpdump -eni tapXYZ to get to see the VLAN tags going in and out?

5

actually I captured on the trunk itself: tcpdump -ni trunk

So I see all traffic on the trunk

6

Try dumping the tap device, that should let you see if anything is going on with the bridge that’s messing with the tags.

7

capturing on the host on the tap device gives the same result: I see packets tagged correctly in the correct vlan going to the F5, but no return traffic. Looks like it is not allowed to speak somehow.

I’ve booted the working F5 again and noticed that it does get ports registered in ovs as vnet instead of tap. So I thought I should try defining the nic in incus to be of network, but that only seems to work for incus managed bridges. Also using an incus bridge and attaching all devices to it doesn’t give joy for the F5 :frowning:

8

How easy is it to get one’s hands on the F5 Big-IP VE image?
I’d ideally like to try this here and see if we can figure out what’s going on exactly.

I’ve had customers run it before but never had to get a copy of it for myself.

9

Not sure if following this would work? https://www.f5.com/trials/big-ip-virtual-edition

Looks like you would be able to get a free trial version. I am using the bigip ve LTM (for load balancing) version.

Your customers ran it via Incus i.s.o. libvirt/qemu?

10

Yeah, they were running it in Incus, not sure about the VLAN config though, I vaguely remember them passing in a bunch of different network interfaces in, so maybe everything was passed as untagged that way.

11

yeah untagged is no problem indeed. My instance has 4 interfaces (as described in the deplyment guide): first one for managment, which runs on an untagged interface also bridged to the trunk, but then with my mgmt vlan assigned. The others are just bridged to the trunk, which should do the vlan trunking (1 vlan for syncing the cluster F5’s and 1 for the actual loadbalancing frontend with BGP).

NB: just to be sure the 2 non used interfaces would not give problems, I deleted them from incus.

12

Hi Stephane,

Interesting…I thought lets make a complete setup procedure, so I requested some trial keys via the link I send earlier and downloaded the v17.5.1.3 qcow2, converted it to raw and imported it in incus (with incus-migrate), deleted the default profile and assigned 2 nics, 8G RAM and 4 cores. Booted it up, licensed the box and created a tagged vlan on the second interface and it all works just fine??

Since I got 2 trial licenses, I used the other one to download and install the same version as my failing box (v.16.1.6.1-0.0.11). Same procedure as above and this times it DOES fail (as in same result as my original box)

Upgrading this box to 17.5.1.3 results in : working…hmmm

So.. I guess there’s something different in the F5 software versions…not sure what the difference is between libvirt/qemu and incus networking then as my original box still doesn’t work in Incus. Can’t upgrade it to v17 though as my license is a bit old…Have to renew them anyway, so good time to do that soon..

13

Interesting…..I now configured the trunk bridge with networkmanager and now all works on my original F5 too?? So systemd-networked didn’t work and openvswitch not, but Networkmanager it does…weird stuff..

14

That’s pretty weird indeed…