Undeletable Project

I smell a bug, but I wanna double check. I posted this in chat, but I suspect this is going to be more involved.

I have an Incus test cluster, 3 nodes, each node has 1 drive (btrfs), 1 NIC. I’m not using ceph or ovn.

I’m playing around with Incus and TerraformOpenTofu, and it seems that twiddling features.networks on a project can make it undeletable.

I don’t see any resources in the UI, but if I try to delete it, it says Failed loading network: Failed to connect to OVN: Failed to connect to OVS: Failed to connect to OVS: Failed to connect to OVS: failed to connect to unix:/run/openvswitch/db.sock: failed to open connection: dial unix /run/openvswitch/db.sock: connect: no such file or directory

$ incus project show forge
config:
  features.images: "false"
  features.networks: "true"
  features.networks.zones: "true"
  features.profiles: "false"
  features.storage.buckets: "true"
  features.storage.volumes: "true"
  restricted: "false"
description: ""
name: forge
used_by:
- /1.0/networks/default?project=forge
$ incus --project=forge network list
+------+------+---------+------+------+-------------+---------+-------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY | STATE |
+------+------+---------+------+------+-------------+---------+-------+

I do have a network default in my default project (as opposed to my default network, which is incusbr0). But deleting that network doesn’t change either of the above commands or allow the project to be deleted.

At this point, I have:

$ incus network list --all-projects
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| PROJECT |      NAME       |   TYPE   | MANAGED |      IPV4       |           IPV6            | DESCRIPTION | USED BY |  STATE  |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| default | enx9cbf0d0016f8 | physical | NO      |                 |                           |             | 1       |         |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| default | incusbr0        | bridge   | YES     | 10.187.122.1/24 | fd42:a4e7:10cd:d9a9::1/64 |             | 3       | CREATED |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| default | ingress         | bridge   | YES     | 10.187.164.1/24 | fd42:596:cb79:eebd::1/64  |             | 0       | CREATED |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| default | lo              | loopback | NO      |                 |                           |             | 0       |         |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| default | public          | macvlan  | YES     |                 |                           |             | 0       | CREATED |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
$ incus project show forge
config:
  features.images: "false"
  features.networks: "true"
  features.networks.zones: "true"
  features.profiles: "false"
  features.storage.buckets: "true"
  features.storage.volumes: "true"
  restricted: "false"
description: ""
name: forge
used_by:
- /1.0/networks/default?project=forge

features.networks=true is only supported with OVN which then requires OVS.
I think the bug is in allowing you to set it to true in the first place here :slight_smile:

You should be able to modify properties on an empty project though, so can probably flip it back to false and then delete.

I’ll send a fix to prevent features.networks=true from being set in an environment that doesn’t have OVN configured.

That error message makes sense given that enabling network zones requires OVN and an OVS socket, even though the project itself has no visible resources. It is interesting that the project still retains a project-scoped reference to the default network after the network was removed. Could clearing features.networks.zones and features.networks first make the project deletable in this state?

I can’t clear features.network from the UI

And the CLI doesn’t like it either:

$ incus project set forge features.networks=false
Error: Project feature "features.networks" cannot be disabled on non-empty projects

Oh, this is weird because you’re also showing a network in there…

Can you show incus admin sql global "SELECT * FROM projects" and incus admin sql global "SELECT * FROM networks"?

$ incus admin sql global "SELECT * FROM projects"
+----+---------+-----------------------+
| id |  name   |      description      |
+----+---------+-----------------------+
| 1  | default | Default Incus project |
+----+---------+-----------------------+
| 5  | forge   |                       |
+----+---------+-----------------------+
$ incus admin sql global "SELECT * FROM networks"
+----+------------+----------+-------------+-------+------+
| id | project_id |   name   | description | state | type |
+----+------------+----------+-------------+-------+------+
| 1  | 1          | incusbr0 |             | 1     | 0    |
+----+------------+----------+-------------+-------+------+
| 2  | 1          | public   |             | 1     | 1    |
+----+------------+----------+-------------+-------+------+
| 3  | 1          | ingress  |             | 1     | 0    |
+----+------------+----------+-------------+-------+------+
| 5  | 5          | default  |             | 2     | 3    |
+----+------------+----------+-------------+-------+------+

and just to emphasize

$ incus network list --all-projects
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| PROJECT |      NAME       |   TYPE   | MANAGED |      IPV4       |           IPV6            | DESCRIPTION | USED BY |  STATE  |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| default | enx9cbf0d0016f8 | physical | NO      |                 |                           |             | 1       |         |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| default | incusbr0        | bridge   | YES     | 10.187.122.1/24 | fd42:a4e7:10cd:d9a9::1/64 |             | 3       | CREATED |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| default | ingress         | bridge   | YES     | 10.187.164.1/24 | fd42:596:cb79:eebd::1/64  |             | 0       | CREATED |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| default | lo              | loopback | NO      |                 |                           |             | 0       |         |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| default | public          | macvlan  | YES     |                 |                           |             | 0       | CREATED |
+---------+-----------------+----------+---------+-----------------+---------------------------+-------------+---------+---------+

incus admin sql global "DELETE FROM networks WHERE id=5" will clean things up enough that you’ll be able to clear the project.

yes it did, thank you