Incus UI doesn't correctly handle network creation in clusters

oddjobz · June 29, 2025, 8:34pm

Hi, I have a two-node cluster that just updated itself … I noticed when I started getting errors in my backup (!) … it would seem that if I create a new Network (called say “public”) in the GUI, it creates a network in an ERRORED state. No errors in the log file I can see.

Not only that, the GUI then failes to delete the network when asked.

Dropping to the CLI it also shows the network in the errored state, but allows me to delete it. If I then do;

incus network create public --target=rpi1
incus network create public --target=rad
incus network create public

It works quite happily.

I’ve repeated a couple of times with different network names, same issue.

stgraber · June 30, 2025, 2:35am

@presztak can you take a look?

It’s about creation of non-OVN networks in a cluster environment. It looks like it’s not attempting to do the per-target creation and just goes straight to creating the global item triggering some kind of failure.

oddjobz · June 30, 2025, 3:59am

It’s about creation of non-OVN networks in a cluster environment. It looks like it’s not attempting to do the per-target creation and just goes straight to creating the global item triggering some kind of failure.

Yes, I’m no longer using OVN anywhere. Regular networks, default settings.

Piotr_Resztak · June 30, 2025, 4:06pm

Sure!

Piotr_Resztak · July 3, 2025, 12:40pm

@oddjobz I wasn’t able to reproduce the issue on my end, it doesn’t seem to be a general UI problem. That said, could you please provide a bit more detail?

It would be helpful if you could open your browser’s Developer Tools and check the Console tab for any errors. Also, take a look at the Network tab, in particular, check the following requests:

/1.0/networks?project=<project>&target=rpi1
/1.0/networks?project=<project>&target=rad
/1.0/networks?project=<project>

Do any of these requests fail or return something other than a 2xx status? If so, could you share the response details?

oddjobz · July 3, 2025, 12:50pm

Ok, so trying it again now, I get this in the JS console;

index-CfMUciCt.js:93 
 GET https://rpi1:8443/1.0/networks/t?project=default 404 (Not Found)
index-CfMUciCt.js:93 
 GET https://rpi1:8443/1.0/networks/te?project=default 404 (Not Found)
index-CfMUciCt.js:93 
 GET https://rpi1:8443/1.0/networks/tes?project=default 404 (Not Found)
index-CfMUciCt.js:93 
 GET https://rpi1:8443/1.0/networks/test?project=default 404 (Not Found)
index-CfMUciCt.js:93 
 GET https://rpi1:8443/1.0/networks/test?project=default 404 (Not Found)
index-CfMUciCt.js:93 
 GET https://rpi1:8443/1.0/networks/test?project=default 404 (Not Found)
useNetworks-4mRAJCTt.js:1 
 POST https://rpi1:8443/1.0/networks?project=default net::ERR_NETWORK_CHANGED
useNetworks-4mRAJCTt.js:1 
 GET https://rpi1:8443/1.0/networks/test?project=default net::ERR_NETWORK_CHANGED
useNetworks-4mRAJCTt.js:1 
 Uncaught (in promise) TypeError: Failed to fetch
    at f (useNetworks-4mRAJCTt.js:1:692)
    at useNetworks-4mRAJCTt.js:1:1769

Leaves me with a green spinner on the button, i.e. doesn’t complete.
If I reload the GUI I see;

Connections
test
General
Type
Bridge
Description
-
IPv4 address
10.99.223.1/24
IPv6 address
fd42:7156:3344:41f9::1/64
ACLs
-
RX
0 B (0 packets)
TX
0 B (0 packets)
Status
Errored

In the GUI. Created yaml looks like this;

project: default
name: test
description: ''
type: bridge
config:
  ipv4.address: 10.99.223.1/24
  ipv4.nat: 'true'
  ipv6.address: fd42:7156:3344:41f9::1/64
  ipv6.nat: 'true'

CLI looks like this;

# incus network ls test
+------+--------+---------+----------------+---------------------------+-------------+---------+---------+
| NAME |  TYPE  | MANAGED |      IPV4      |           IPV6            | DESCRIPTION | USED BY |  STATE  |
+------+--------+---------+----------------+---------------------------+-------------+---------+---------+
| test | bridge | YES     | 10.99.223.1/24 | fd42:7156:3344:41f9::1/64 |             | 0       | ERRORED |
+------+--------+---------+----------------+---------------------------+-------------+---------+---------+

Nothing in incus.log.

Piotr_Resztak · July 8, 2025, 12:04pm

@oddjobz What’s strange is that I don’t see a request with the target parameter from you. But even so, even if such a request wasn’t sent, you should still get:
Error: Network not pending on any node (use --target <node> first)
or, in case the request was sent to only one server:
Error: Network not defined on nodes: <node1>, <node2>

so, I’m still having trouble reproducing this error. Could you also provide the version of the UI you’re using?

oddjobz · July 8, 2025, 12:14pm

Hi, sure, version in the bottom of the screen;

Version 6.14-ui-0.17.1

So just to update, this issue is coming from my original cluster that inadvertently upgraded to 6.14. I have a “new” production cluster, all machines installed on 6.14 from scratch which seems to work Ok.

Here’s what happens on the old UI:

I save, and it seems to go ok, but then when I look at the networks;

Piotr_Resztak · July 8, 2025, 12:15pm

I also see that you’re receiving net::ERR_NETWORK_CHANGED, which can be caused by a broken DNS configuration. Can you try connecting using the IP address?

oddjobz · July 8, 2025, 12:23pm

So in this particular instance the URL I’m using is;

https://rpi1:8443/ui/settings

Where “rpi1” is an /etc/hosts entry. I just moved to the address;

https://192.168.1.204:8443/ui/project

And deleted the network created above. Had to do it twice, first attempt I got a red error box on the screen and this in the console;

DELETE https://192.168.1.204:8443/1.0/networks/dummy?project=default net::ERR_NETWORK_CHANGED
(anonymous) @ useNetworks-4mRAJCTt.js:1
T @ useNetworks-4mRAJCTt.js:1
o @ NetworkDetail-DS4aaMmw.js:1
D @ index-CfMUciCt.js:76
...

Second attempt worked.
Tried to create network again;

POST https://192.168.1.204:8443/1.0/networks?project=default net::ERR_NETWORK_CHANGED

Same thing, shows errored state in networks.