IncusOS Network Bridges

IncusOS
, ,
1

Hallo :waving_hand:

Switching to IncusOS from Ubuntu and trying to figure out how to configure bridges with physical interfaces as well as VLANs .

I followed the documentation and looked closely at the Network - IncusOS documentation page but I can’t seems to find how to configure the bridges.

Here is the configuration that I have on the Ubuntu host that I would like to replicate.

derek@hive:/home/derek# cat /etc/netplan/90-derek-custom.yaml 
# CUSTOM: Derek
network:
  version: 2
  ethernets:
    enp1s0:
      dhcp4: false
      dhcp6: false
      accept-ra: false
    enxc033deadbeef:
      dhcp4: false
      dhcp6: false
      accept-ra: false
  vlans:
    enp1s0.2:
      id: 2
      link: enp1s0
      dhcp4: false
      dhcp6: false
      accept-ra: false
  bridges:
    br-vlans:
      interfaces:
        - enp1s0
      dhcp4: false
      dhcp6: false
      accept-ra: false
    br-vlan2:
      interfaces:
        - enp1s0.2
      addresses: [ 10.10.10.10/26 ]
      routes:
        - to: default
          via: 10.10.10.1
      nameservers:
        addresses: [ 10.10.10.1 ]
    br-span:
      interfaces:
        - enxc033deadbeef
      parameters:
        aging-time: 0
      dhcp4: false
      dhcp6: false
      accept-ra: false
# \CUSTOM

And here’s my current IncusOS Network Configuration

{
    "interfaces": [
        {"name": "enp1s0",
         "hwaddr": "enp1s0"},
        {"name": "enxc033deadbeef",
         "hwaddr": "enxc033deadbeef"}
    ],
    "vlans": [
        {"name": "enp1s0.2",
         "parent": "enp1s0",
         "id": 2,
         "addresses": ["dhcp4", "slaac"]
        }
    ]
}

Any hints ?

Cheers !

2 
dns:
  nameservers:
  - 10.10.10.1

interfaces:
  - name: vlans
    hwaddr: enp1s0
    roles:
    - instances
  - name: span
    hwaddr: enxc033deadbeef
    roles:
    - instances

vlans:
  - name vlans-2
    id: 2
    parent: vlans
    addresses:
    - 10.10.10.10/26
    routes:
    - to: 0.0.0.0/0
      via: 10.10.10.1

Everything in IncusOS is a bridge, so you don’t need to do anything about bridging.

The config above will make incus network list contain two entries:

  • vlans
  • span

Both of which will be bridges that can directly be used to put instances on top of them.

The config above will make the vlans interface accept VID 2 on top of the untagged VLAN. If there are other VLANs that you want to allow in for use by Incus, you’ll need to set the vlan_tags interface to a list of VLAN IDs that need to be allowed, for example:

vlan_ids:
- 2
- 10
- 50
1 Like
3

Thanks @stgraber !

Awesome support as usual !

4

What would be the most IncosOS-esque way to then also attach instances to a defined VLAN?

I’ve noticed the roles prop can be placed on the vlans entries. When that’s set incus network list shows an entry with type vlan rather than bridge. Is that still a bridge under the hood, or should a seperate bridge be created and linked to that?

Use case here is to keep any VIDs defined in the host config, rather than downstream on instances / profiles.

5

What you really want to do at the Incus level is:

incus config device add MY-INSTANCE eth0 nic nictype=bridged parent=vlans vlan=10 name=eth0

Assuming a physical interface named vlans with 10 in the vlan_ids.

We do allow roles on VLAN devices, but there’s basically no reason to ever put the instances role on one of those as that would then make Incus use macvlan on top of that device, just adding overhead where it’s not needed.

6

a man is sitting in a car with the words `` like a glove '' written on the screen .|833x467.40555555555557

7

It’s the vlan=10 property that I was hoping to avoid specifying at the instance level.

Ideally something similar to this in the incus admin os system network config:

  interfaces:
  - addresses:
    - dhcp4
    hwaddr: <redacted>
    name: enp5s0
    required_for_online: "no"
    roles:
    - instances
  vlans:
  - id: 10
    name: example_vlan
    parent: enp5s0
    roles:
    - instances

Followed by the ability to incus network attach example_vlan my_host eth1 when required.

In my specific case ending up with macvlan interface on the host this is attaching to is desirable, but also just trying to get a general feel for what best practice is here.

8

I think the way I’d like to see what you’re describing work would be:

  interfaces:
  - addresses:
    - dhcp4
    hwaddr: <redacted>
    name: enp5s0
    required_for_online: "no"
    roles:
    - instances

Followed by:

incus network create example_vlan --type=physical parent=enp5s0 vlan=10
incus launch images:debian/13 foo --network example_vlan

But I’m not 100% sure that this would work as intended today, if it doesn’t, we’ll need a bug against Incus to fix it (as per documentation, it should be working).

9

That works here, with the prerequisite that vlan_tags: [10] (or others as appropriate) also needs to be set on the IncusOS parent interface.

1 Like
10

Following up on this, I can’t seems to get an IP address on VLAN 3 from inside a container.

{
  "interfaces": [
    {
      "name": "vlans",
      "hwaddr": "enp1s0",
      "vlantags": [
        2,
        3,
        4
      ],
      "roles": [
        "instances"
      ]
    },
    {
      "name": "span",
      "hwaddr": "<redacted>",
      "roles": [
        "instances"
      ]
    }
  ],
  "vlans": [
    {
      "name": "<redacted>",
      "parent": "vlans",
      "id": 2,
      "addresses": [
        "dhcp4",
        "slaac"
      ]
    }
  ]
}

And then

incus init images:debian/12 debian-container
incus config device add debian-container eth0 nic nictype=bridged parent=vlans vlan=3 name=eth0

Am I missing something ?

11

I hit this too and spent far to long than I care to admit trying to figure it out.

Default config is dhcp on eth0 only. Additional interfaces need to be configured either by cloud-unit or on the instance directly.

12

Yeah, that I know :slight_smile:
This is why I added the interface as eth0.
Also, even if forcing it through dhclient eth0, no success…

13

You show vlantags when it should be vlan_tags.

Is the above what IncusOS reported back when reading from the API or was that the data you pushed?

Typoes are (annoyingly) allowed by the YAML parser but will not be saved, so reading back the config should then have stripped that vlantags entry.

14

This is what was pushed since I was not able to get a client to read it (as per my other thread ;))

Stay tuned

16

Indeed, I read incus-os/incus-osd/api/system_network.go at main · lxc/incus-os · GitHub but used the name instead of the attribute…

Is there a way that I can change that without having to reinstall ?

17

Sure, you usually want to push a minimal initial config just to get the server to respond on the network, then you can use incus admin os system edit network to change the config further.

18

That was it :slightly_smiling_face:

Cheers !

19

Hello again :wave:

Last remaining thing that does not seems to work as expected, the span interface.
Is there anything that should be configured for this to work properly ?
When adding the interface as a nic on an instance and then putting it in promisc inside the instance, there is nothing coming up (yes, the mirroring on the network equipment is properly configured)

20

What does incus admin os system show network and incus config show --expanded NAME-OF-INSTANCE show you?

21 
incus admin os system show network

derek@dmbr incus % ./incus admin os system show network
WARNING: The IncusOS API and configuration is subject to change

config:
  interfaces:
  - hwaddr: <redacted>
    lldp: false
    name: vlans
    roles:
    - instances
    vlan_tags:
    - 2
    - 3
    - 4
    - 42
  - hwaddr: <redacted>
    lldp: false
    name: span
    roles:
    - instances
  vlans:
  - addresses:
    - dhcp4
    - slaac
    id: 2
    name: HAUS
    parent: vlans
state:
  interfaces:
    HAUS:
      addresses:
      - <ipv4_redacted>
      - <ipv6_redacted>
      hwaddr: <redacted>
      mtu: 1500
      roles:
      - management
      - cluster
      routes:
      - to: default
        via: <ipv4_redacted>
      speed: "1000"
      state: routable
      stats:
        rx_bytes: 1.11666101e+08
        rx_errors: 0
        tx_bytes: 6.895292e+06
        tx_errors: 0
      type: vlan
    span:
      hwaddr: <redacted>
      mtu: 1500
      roles:
      - instances
      speed: "1000"
      state: carrier
      stats:
        rx_bytes: 108918
        rx_errors: 0
        tx_bytes: 220
        tx_errors: 0
      type: interface
    vlans:
      hwaddr: <redacted>
      mtu: 1500
      roles:
      - instances
      speed: "1000"
      state: carrier
      stats:
        rx_bytes: 1.13365632e+08
        rx_errors: 0
        tx_bytes: 6.895512e+06
        tx_errors: 0
      type: interface

incus config show --expanded NAME-OF-INSTANCE

derek@dmbr incus % ./incus config show --expanded debian-container
architecture: x86_64
config:
  image.architecture: amd64
  image.description: Debian bookworm amd64 (20251116_05:24)
  image.os: Debian
  image.release: bookworm
  image.serial: "20251116_05:24"
  image.type: squashfs
  image.variant: default
  volatile.base_image: 31360b27eada7442ee1f9f6609ac6ee54a888022046d65e0b2caffb7f7fc2036
  volatile.cloud-init.instance-id: fc157726-9cb5-49ff-b6bf-566b1df1a481
  volatile.eth-1.host_name: veth24d3876d
  volatile.eth-1.hwaddr: <redacted>
  volatile.eth-1.name: eth1
  volatile.eth0.host_name: veth18505687
  volatile.eth0.hwaddr: <redacted>
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
  volatile.uuid: 728ea964-dd89-4284-892e-258932a6ae66
  volatile.uuid.generation: 728ea964-dd89-4284-892e-258932a6ae66
devices:
  eth-1:
    nictype: bridged
    parent: span
    type: nic
  eth0:
    name: eth0
    network: incusbr0
    type: nic
  root:
    path: /
    pool: local
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""