Add second network interface to OCI instance

, , ,
1

In some cases you need more than one network interface attached to an instance and if you have a standard OS container the procedure is pretty much straight forward. However, deploying OCI images and assigning a second network interface comes with a challenge. It doesn’t work the same automatic way like eth0, except I’m missing something.

You can perform all the required incus low level steps like you do with normal OS containers like:

incus config device add <name> eth1 nic nictype=<type> parent=<parent> name=eth1

The interface is added to the config and initialised after successful instance start:

    eth1:
      Type: broadcast
      State: DOWN
      Host interface: eth1
      MAC address: 00:16:3e:6a:03:18
      MTU: 1500
      Bytes received: 0B
      Bytes sent: 0B
      Packets received: 0
      Packets sent: 0
      IP addresses:

Now the challenge is to bring it up in the container. In most cases OCI images don’t have the required tools installed as you don’t need them in docker. So far I have been lucky that I could run:

apt update && apt -qq install  isc-dhcp-client

echo "/sbin/dhclient -q eth1" >>/init

Or add a service to s6-overlay or similar which finally brings the interface up and an IP is assigned.

    eth1:
      Type: broadcast
      State: UP
      Host interface: vlan4
      MAC address: 00:16:3e:a2:13:e6
      MTU: 1500
      Bytes received: 119.78kB
      Bytes sent: 2.18kB
      Packets received: 1832
      Packets sent: 18
      IP addresses:
        inet:  192.168.225.185/24 (global)
        inet6: fd33:36c1:983b:225:216:3eff:fea2:13e6/64 (global)
        inet6: fe80::216:3eff:fea2:13e6/64 (link)

The procedure pretty much depends on the base image OS and init path chosen.

As mentioned above nothing needs to be done for eth0 (default interface) but when it comes to a second one you need to add some kind of logic into the image and this can sometimes be a challenge.

Adding OCI support is definitely a great feature and I love it already. I’m impressed what the first release has to offer. This is more or less about awareness how you can do it with the current 6.3 release and I’m looking forward what comes next.

2

Yeah, this is something that’s basically intentional.
We currently only do DHCP4 on eth0. Any additional interface is ignored on our side.

We could in theory have the DHCP client fallback to eth1, eth2, … if eth0 doesn’t provide a valid configuration, but you should generally avoid running the DHCP client on multiple interfaces at the same time as you may then get conflicting default routes and cause issues that are hard to diagnose.

3

You are right Stephan, automatically add eth1 with DHCP runs into trouble of default route etc but it also depends on how you setup your DHCP for the second network.

In my case I use static assignments and disable default route and DNS settings on my DHCP server, so it works as expected :wink: On the other hand even if I want to assign a static IP on Incus level you properly hit this issue Troubleshooting Static IP on unmanaged bridge. Back to square one…

I’m not sure how many will run into this or have the requirement it just makes migrating OCI images tricky as most of them have no notion of TCP/IP.

Just thought to bring this to your attention for future enhancement.

1 Like