What kinds of network can be project-specific?

Incus
1

Suppose I want to give a project its own networks - in particular so it can have its own NAT network without seeing any others.

Creating a “bridge” network doesn’t appear to be allowed:

$ incus project create testproj -c features.networks=true
Project testproj created
$ incus network create testbr0 --type bridge --project testproj
Error: Network type does not support non-default projects

So my question is, what kinds of network can be project-specific?

1 Like
2

Never mind, I found a tip here:

Unlike the other network types, you can create and manage an OVN network inside a project.

3

For my use case, implementing OVN just for project network isolation seemed like overkill when I just wanted all project instances to be on a different subnet. So, I set features.networks: "false" and create a bridge network per-project for isolation (it’s then allowed). I’m not sure if there are still ways for the instances in different projects to reach each other but I believe not. Just wanted to share my approach so far.

4

Yep, that’s fine. That’s actually what we do for the incus-user daemon where a project is generated for every user. We also generate a bridge per project and restrict each project to only using that one bridge through a project restriction.

2 Likes
5

Oh that’s excellent, thanks for sharing! I just had a peek in incus-user and can also use this “only this network” configuration💃🏻 source: incus/cmd/incus-user/server.go at main · lxc/incus · GitHub

6

Ah right. So you create a global network, and then restrict the project in question so that it can only access that network and no others.

It’s not quite as good as a project-specific network (since other projects can access that network), but good enough.

7

My understanding is that you can still restrict the project to a single global network using "restricted.networks.access": "my-network-name" in the project configuration.

8

Indeed, but you can’t restrict other projects from accessing that network, unless you apply restrictions individually to every other project.

1 Like
9

Actually, doing some further debugging, my assumptions were maybe not correct?

In Network Isolation by Project on Single Server Incus Host we see that instances on different project and on different bridges do have connectivity between each other?

I am not quite sure what restricted.networks.access: incusbr-foo is actually doing?

10

So, If I’m not mistakenly reading this through: we need to additionally create and assign ACLs to the per-project bridges (as demonstrated in Network Isolation by Project on Single Server Incus Host - #4 by stgraber) to stop instances on different projects/bridges from communicating with each other.

Does this mean that incus-user instances of different users on a single host are not network isolated from each other by default?

1 Like
11

My understanding is: if you are in project “bar” (where this restriction is applied), then you can only make use of the incusbr-foo network.

However, it does not affect:

  1. Whether traffic can flow between the incusbr-foo network and other networks (incus or otherwise) - that requires network ACLs to be applied
  2. Whether projects other than “bar” can make use of the the incusbr-foo network
1 Like