Hi,
I’m getting the following error when starting a Virtual Machine:
$ incus start [vm name] --project=[project name]
Error: Failed to start device “eth1”: Failed setting up OVN port: Insufficient dynamic addresses allocated
I think the error message here might be misleading in my case.
We have just over 50 virtual machines in the cluster. Most of these VM’s are attached to 2 networks - the egress network described below and 1 of the 15/16 private networks.
The one egress network that most of the VM’s are connected to is a /16 which provides internet uplink and is working fine.
We also have 16 private networks, and the above error message only began occurring after creating the 16th network, and attempting to attach it to a VM. Each of the private networks is 10.20.X.1/24, with X being 0-15.
All was working fine before adding the 16th network and trying to attach VM’s to it. Also, once the error is occurring, I cannot attach VM’s to any of the other 15 private networks either.
I don’t think the real error under the hood is specifically that there are not enough addresses available per CIDR, as each of the private networks has from 2-10 VM’s attached from an available /24.
In order to test the theory that its something to do with the CIDR/Network count, I deleted the 16th network, and tried to create a VM attached to another private network. This failed with the same error. I then deleted two virtual machines attached to that other private network. I was then able to continue creating VMs (only two though) and attaching them to any of the first 15 networks created. These VM’s will create and start as expected.
It would seem that a total number of available IPs being calculated from all networks under the 10.20.X.X/16 cidr, and that incus is seeing this total as less than is actually available.
50 seems to be the limit of VMs that can be attached to all networks under that prefix of 10.20.X.X . There are 52 VMs attached to the egress network.
First issue I found with a similar error is this one:
I dont think the ovn-nb database is down, as removing VM’s from private networks seems to free up space
Second issue I found with a similar error is this one:
I could change all the private network CIDRs to /16s instead of /24s, but that would mean redeploying everything currently deployed - so I would really rather not if it can be avoided.
EDIT: I tried this, creating a 10.100.X.1/16, and get the same error that instances wont start
Any help would be very greatly appreciated
Kind Regards