InucusOS & LACP

After a lot of research and testing I am reaching out to the community to find if anyone has LACP experience setting up LACP with paramatures and working.

We are utilizing a Cisco switch, cisco-sw-sx350x-52, and port channels. We have solid working LACP connection to our hardware with Ubuntu 2024.04, netplan and Incus.

Example of Ubuntu 2024.04 netplan LACP:

  bonds:
    bond0:
      dhcp4: no
      dhcp6: no
      interfaces:
      - eno5
      - eno6
      parameters:
        mode: 802.3ad
        lacp-rate: fast
        transmit-hash-policy: layer2+3
        mii-monitor-interval: 100
        min-links: 1

IncusOS Testing

bonds:
- name: "instances"
  mode: "802.3ad"
  lldp: true
  lacp_rate: fast
  min_links: "1"
  xmit_hash_policy: layer3+4
  members:
  - "eno5"
  - "eno6"
  - "eno7"
  - "eno8"

The vlan we have based on the bond “instances” is setup but we are finding the packets are not reaching the switch, we are believing it has to do with the LACP is not getting set up properly. I did not add the mii config as I am not sure how it is spelled out yet. I still need to learn how to convert the API into workable yaml settings.

Has others been able to setup a proper LACP yet with IncusOS with additional parameters?

Here are the resources I have used to get us where are today with our latest test this morning

file:///opt/incus/doc/api-extensions/index.html#network-state-bond-bridge

Hmm, I’m not sure where you’re getting some of the keys you’re setting.

^ Those are the fields available when configuring a bond.

We default to a fast LACP rate and to 3+4 balancing internally.
I’m assuming you’re setting this all up through a network.yaml seed as otherwise you’d need to specify MAC addresses rather than interface names.

Here is what I’m running on one of my clusters:

config:
  bonds:
  - addresses:
    - 2602:fc62:a:101::101/64
    lldp: true
    members:
    - 00:1b:21:d4:8f:0c
    - 00:1b:21:d4:8f:0d
    mode: 802.3ad
    mtu: 9000
    name: uplink
    required_for_online: ipv6
    roles:
    - management
    - cluster
    - storage
    - instances
    routes:
    - to: ::/0
      via: 2602:fc62:a:101::1
    vlan_tags:
    - 50
    - 51
    - 52
    - 100
    - 102
    - 200
    - 300
  dns:
    domain: hosts.dcmtl.stgraber.net
    hostname: langara
    nameservers:
    - 2602:fc62:a:10::1
    search_domains:
    - stgraber.net
  time:
    timezone: UTC
  vlans:
  - id: 100
    mtu: 1500
    name: ovn-uplink
    parent: uplink
  - addresses:
    - 2602:fc62:a:300::101/64
    id: 300
    mtu: 9000
    name: cluster
    parent: uplink

It’s probably a bit different from your case as I have an IP on the untagged VLAN on top of having a couple of VLAN interfaces. Also I’m in an IPv6-only environment.

I was trying to refrain from using MAC address as much as we could for enterprise roll out but for testing, why not. The current “install” completes with “successful”, including the new incusos interface and the proper IP; but not able to connect for NTP or Secure Boot updates.

I have to look again where I received the wrong parms. I had looked at the API options but I was not sure if it were all inclusive. Good to know. It does make sense why all the examples do not add the options since they are default.

Thank you for sharing your networking. I can test first thing early tomorrow morning and let you know.