I am a bit confused about how to set up networking on IncusOS and have hit a wall a few times. The problem with networking is if you screw up, you can lose access to the server completely
First question is that my server has 2 NICs, but only 1 seems to be picked up by IncusOS as far as I can tell. But that isn’t essential to the network configuration I’m trying to set up.
I’d like IncusOS to have multiple static IP addresses. One should be used for the IncusOS management only. I’d like to attach other static IP addresses directly to certain containers or VMs.
I’m looking for an example config, or some assistance walking through how to set this up.
OK one issue I see is that you define dhcp4 and a static address to IncusOS. That might work but it is better to stick with one definition. If you have control on the DHCP server I would define a static dhcp lease for IncusOS there. This way you always get an IP assigned via DHCP and don’t loose control by accident if you misconfigure IncusOS.
After your solved the static IP approach you need to add the bridge as explained before. Which can be used by Incus. All you need to do is to create an instance and assign the correct network to it.
Right. I thought that would be the case, but IncusOS actually seems to accept this config, although the static IPs don’t seem to work/route (even when I try adding routes).
I want to assign multiple static IPs to IncusOS, to be assigned to different instances (contrary to the single static IP in my example above). I don’t this can be accomplished with DHCP.
Setting the networking config with static IPs through IncusOS I have tried to set what I assumed I would be need for the static IP to work, but I’ve missed something every time and gotten locked out. So I’m asking for an example/a bit of hand holding.
Understand, IncusOS validates configuration definitions but doesn’t perform any network related checks at this state. Hence you can configure an invalid network configuration. However, as long as you can access IncusOS you should see all configuration settings.
In general there should be only one IP configured for IncusOS and the rest will be handled by Incus. It is by design to improve security and configuration options. The IP for IncusOS is a management IP and will be in general higher protected to avoid for example un-allowed access.
Everything else should be managed by Incus itself. Requires to add the bridge as mentioned and you can assign any static IP to your instance from Incus.
This follows a simple design pattern where you configure your OS (IncusOS) once and don’t need to change it afterwords. Everything else is managed by Incus itself. Different layers where different permissions are required.
Your steps are:
configure your IncusOS netowrk stack with a single IP
create the managed network interface for Incus
create an instance
assign the new network interface and an static IP to the instance
start your instance
You even can use DCHP for this instances and assign a static IP later. The guide linked should contain all the required steps.
It just needs a few iterations to understand the concept of IncusOS
I’m about two dozen iterations deep waiting 10 minutes for “IncusOS is starting…” to stall out so I can see some debug information to understand the install process. But I’ll keep trying. I understand your post about 60%, I’ll give it a try and see if it clicks. Thanks.
But it’s not clear how you intend to then give those IP addresses to the instances.
As was pointed out by @osch, if you want the instances to be on the same physical network as the IncusOS system, then this is no the way to do it. We have a tutorial covering how to put instances on the physical network instead.
If the extra addresses are to be used for proxy devices or network forwards, then the config above should work for that.
Perhaps it will make more sense if I explain my approach from Incus on Debian. In that case, I would set up /etc/networking/interfaces like this:
auto eth0
iface eth0 inet static
address 192.168.1.10
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 1.1.1.1
# Secondary IP address
auto eth0:1
iface eth0:1 inet static
address 192.168.1.11
netmask 255.255.255.0
# Another secondary IP address
auto eth0:2
iface eth0:2 inet static
address 192.168.1.12
netmask 255.255.255.0
I would then be able to attach/passthrough eth0:1 to VM-1 and eth0:2 to VM-2
Each VM will have it’s own static IP address on the physical network.
What is the best-practice approach to achieve a similar set up with IncusOS? I’m happy to admit I don’t understand deep networking that well!
Is there a way to do that in IncusOS without attaching the instances directly to the physical network? I would like Incus to manage the static IP the instance gets, rather than allow the instances to configure it themselves, if that makes sense.
You have a few main options to do something like that:
Just bridge the instances to the physical network. They can then either DHCP or be configured with a static IP config themselves, just as you would a physical system on the network.
Run the instances on a private bridge like incusbr0 and then route the external IP addresses to them by using ipv4.routes.external and matching configuration in the instance.
Run the instances on a private bridge like incusbr0 and have the external IP be used within a network forward which will then NAT to the instance’s private address.
