AWS-like IP addresses allocation in managed network

Incus
1

Hello !

For an homelab project, I would like to know if it’s technically possible to have something similar on how AWS is doing IP address allocation (IPv4 for now) in a VPC with Incus virtual machines and a managed bridge network.

My goal is to bootstrap a Kubernetes cluster but I don’t want to manage IP addressing of the node as they will be mid-short lived (few weeks or months, but also want to experiment rollover of node by replacing them instead of updating in-place). However, not managind the IP addressing doesn’t mean I don’t want my nodes having their IP addresses changed during their lifecycle. For example, my clusters may be offline for few days as they are not used and I want that the IP assigned by the DHCP is still the same until the nodes are deleted (this can trouble with etcd otherwise).

In the link above, the last sentence of the Private IPv4 section is what I want to achieve:

A private IPv4 address, regardless of whether it is a primary or secondary address, remains associated with the network interface when the instance is stopped and started, or hibernated and started, and is released when the instance is terminated.

From what I know, a managed Incus network is using DHCP through dnsmasq with lease of a limited time. I’ve done some research in this forum and found this for containers:

Having said that, when you launch a container with managed networking (LXD manages the networking, default settings), then the container gets a random but fixed MAC address.
This MAC address is used by LXD’s dnsmasq for the DHCP lease. As long as your container has the same MAC address, you get the same lease. This is how it’s done in practice, something might change in the future, but it is the current case.

https://discuss.linuxcontainers.org/t/best-practices-for-assigning-static-ip-address/4012

For containers, this is what I’m looking for, but I won’t use LXC containers for my clusters.

Since you don’t manage MAC addresses for virtual machines (QEMU does I think ?), how can I be sure that the MAC address will not change between start/stop and so, does this means that work like a classic DHCP setup for virtual machines ?

Thanks !

2

The MAC address is controlled by Incus for VMs, the same as is done for containers.
You can make things safer by bumping the ipv4.dhcp.expiry to say 2w to have it be valid for two weeks.

There’s also the option to set it to infinite which would then have the lease never expire and only be removed when the instance gets removed.

1 Like
3

That’s perfect then ! Will try this and see how it goes. Thanks @stgraber :wink: