Incus 0.7 has been released

News
,
1

Introduction

The Incus team is pleased to announce the release of Incus 0.7!

This is going to be our last release before Incus 6.0 LTS which is now scheduled to be released next week. As releases go, this is quite a busy one, which is how we like it before releasing an LTS, trying to keep the amount of new features in the LTS release itself to a minimum.


image

As usual, you can try it for yourself online: Linux Containers - Incus - Try it online

Enjoy!

New features

Network integrations

A new top-level concept, network integrations are a way to connect an Incus deployment to networks outside of its own control.

Currently the only implementation of the concept is OVN interconnection which makes it possible for an Incus cluster to directly peer its OVN networks with equivalent networks running on other Incus clusters or even other OVN users like OpenStack or Kubernetes.

Here is an example of creating a new network integration using an OVN interconnection gateway, then peering an existing network through it:

root@az01-server01:~# incus network integration create ovn-region ovn
Network integration ovn-region created
root@az01-server01:~# incus network integration set ovn-region ovn.northbound_connection tcp:[10.50.1.12]:6645,tcp:[10.50.2.13]:6645,tcp:[10.50.3.19]:6645
root@az01-server01:~# incus network integration set ovn-region ovn.southbound_connection tcp:[10.50.1.12]:6646,tcp:[10.50.2.13]:6646,tcp:[10.50.3.19]:6646
root@az01-server01:~# incus network peer create default region ovn-region --type=remote
Network peer region created

Documentation: How to configure network integrations - Incus documentation

Image server management tool

A common way to run an Incus image server, be it for some internal servers or as a publicly available image server is through a static web server providing Incus images using simplestreams.

To make this easier to set up, we’re now introducing a new tool, incus-simplestreams which can easily manage a simple image server, listing the images available, adding and removing images as well as generating the needed metadata files.

stgraber@dakara:~$ mkdir image-server
stgraber@dakara:~$ cd image-server/
stgraber@dakara:~/image-server$ incus-simplestreams generate-metadata ~/Downloads/incus.tar.xz
Operating system name: Red Hat Enterprise Linux
Release name: 9
Variant name [default="default"]:
Architecture name: x86_64
Description [default="Red Hat Enterprise Linux 9 (default) (x86_64) (202403260239)"]:·
stgraber@dakara:~/image-server$ incus-simplestreams add ~/Downloads/incus.tar.xz ~/Downloads/rhel9.qcow2·
stgraber@dakara:~/image-server$ incus-simplestreams list
+------------------------------------------------------------------+--------------------------------------------------+--------------------------+---------+---------+--------------+-----------------+----------------------+
|                           FINGERPRINT                            |                   DESCRIPTION                    |            OS            | RELEASE | VARIANT | ARCHITECTURE |      TYPE       |       CREATED        |
+------------------------------------------------------------------+--------------------------------------------------+--------------------------+---------+---------+--------------+-----------------+----------------------+
| 7d256e4fac6fc63fb47bc1e07e1c6ee234281cdf1ed21788c920d763b7bd93ba | Red Hat Enterprise Linux 9 x86_64 (202403252239) | Red Hat Enterprise Linux | 9       | default | x86_64       | virtual-machine | 2024/03/25 00:00 UTC |
+------------------------------------------------------------------+--------------------------------------------------+--------------------------+---------+---------+--------------+-----------------+----------------------+
stgraber@dakara:~/image-server$ find . | sort
.
./images
./images/ef6cf538776b05a64c789f16f235a757522724f2c490c7e118645be2eb920d30.incus.tar.xz
./images/ef6cf538776b05a64c789f16f235a757522724f2c490c7e118645be2eb920d30.qcow2
./streams
./streams/v1
./streams/v1/images.json
./streams/v1/index.json

Put that on an HTTPS capable web server and then add it with:

incus remote add my-server https://xyz.example.net --protocol=simplestreams

Documentation: Default image server - Incus documentation

JSON Web Token authentication

Incus basically supports two mechanisms for remote authentication:

  • TLS client certificates (added to the local trust store with or without restrictions)
  • OpenID Connect external authentication (with or without OpenFGA for authorization)

The former is the most common for simple interactions with a remote Incus server.
Our own CLI tool and most 3rd party tools don’t have any problems using a TLS keypair to establish the HTTPS connection and get authenticated that way.

But there are some situations, like running Incus behind a reverse HTTP(S) proxy where TLS client certificates can become a bit problematic.

To address that, we now support using a JSON Web Token (JWT) bearer token through the HTTP Authorization field. That token can be generated by any user with a valid TLS client certificate by setting the Subject field to the certificate fingerprint, setting applicable NotBefore/NotAfter values and signing the JWT with their private key.

Incus will treat any such connections as equivalent to using the TLS client certificate.

stgraber@dakara:~$ openssl req -x509 -newkey rsa:4096 -sha384 -keyout client.key -nodes -out client.crt -days 1 -subj "/CN=test.local"
.+.........+...+...+..+....+......+........+.+.....+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+.+......+...+......+......+..+...+...+...............+.......+...+......+...+.....+......+....+......+..+...+......+....+............+.....+....+.....+.+............+..+.........+......+....+......+...........+....+........+...+...+.+...+..+..........+.....+...+......+............+...+.......+........+....+.....+.+..+.......+......+..+....+........+..........+...+..+.+.....+.+......+..+.......+.....+.+..+..........+..+....+..............+.+..+...........................+...+....+......+...+..............+.+..+....+.....+.+.........+...+..+....+..+.............+.........+.....+...+..........+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.....+...+...+..+...+.........+.+...+............+..+............+.+.......................................+.....+...+......+.........+......+.+.....+...+.+...........+......+.......+.....+.......+......+.....+..........+...+..+.........+....+.........+...........+......+.+..................+..+....+...........+.............+.....+....+..+......+............+..........+......+......+......+..+.............+.....+...+.+........+............+....+.................+.........+......+.......+...+.........+.....+....+......+........+.+..+....+......+........+....+...+.................+.+..+.........+....+............+.....+.........+....+.....+.......+.....+...............+..............................+....+........+.......+...........+.......+.....+......+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
...+...............+.+...+...+...+.........+..+.+...+..+.......+.....+.+..+...+...................+...........+......+....+..+............+...+.......+..+.........+....+...+.........+.....+...+...+....+...........+...+.+.........+.....+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+...+......+..+...+......+.+..+.............+..+.......+........+...+...+....+...............+..+....+..+....+...+.....+.+.....+..........+.....+.+.....+....+............+.........+...+.....+......+......+.............+..+.+..+.......+...+........+...+.......+.........+......+..+.+......+...........+...+.........+...+...+....+..+.........+....+.....+.+......+.........+..+..........+..+...+...+....+..+...+.+.....+......+.+...+......+.....+.+.....+.......+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*........+..+.......+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
stgraber@dakara:~$ incus config trust add-certificate client.crt --restricted --projects demo
stgraber@dakara:~$ tls2jwt client.key client.crt now 120
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2MzI3Y2Q5YmIxYTFmN2ExMWM3ODBkZjc4YjVkNjg5YzhkMGQ5YzcwZGQxOGQ1YTMyYzI1M2ZiODA0N2U2M2E0IiwiZXhwIjoxNzExNDcyMjE2LCJuYmYiOjE3MTE0NzIwOTYsImlhdCI6MTcxMTQ3MjA5Nn0.pNQ4AcgoymxWHROXVjcYX8QMKdf9QgRH3zex7qc16avX7_Ax1q_WFWzQWfP48Fh-ooeh9hBQKCQkZxjVxYx8Sy-cNqmkf1AI9KGh5uemHh3FYAbvebCTaIXan0B6glWHVnDSwLZKBWTDDai2VXOmUfntyV9yPJdTqxt1J0j8PNuIWzNVdFlcTxzpggcJMhbcqtf4GRwSMKx69HU5sP4AQ7GJ2cBvN7Im-nkRXTc7xiyYnIsFx0vIWJzojC4zwg0-C1LHKQD4DyEKhqOVISIKUSa3GhD6ajcDuGDS8af4Iz19sNPsSoSULBUG-a7E5lXx2vk802vOFFWV68ZHugsJHpdSpLFwTVixipQ1-QdKRozlMjNPguu-5CYxhZVR1p32lbN9D879xGbFXUgPJVwK25NILvbEMcrqnGPgKcRUjJlHtVljGOgXrjmG7dMiW5QOsyy1eIvJ1D1sNsG02fDTbchTzXHmIybxQTK0FXCyNDLOAl6xgW0Jundg7AN1uJU2cLEWy1x3TusqC7lyeTeF3WYT-G8xE2CU4GpLBeYWyLwuJgxRkaWcg9IXiivguPbWpcT0RMl1bmpn0TJ2VgEPCuSG0mJxMBp8HbAgxwgar8AHdpoZ43dCCwZnB0a0O_kmGkBE2xGKKvgTx_U6eSixZzyyNmHDC1KH1Vy1WW1ZcF0stgraber@dakara:~$·
stgraber@dakara:~$ curl -s -k -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2MzI3Y2Q5YmIxYTFmN2ExMWM3ODBkZjc4YjVkNjg5YzhkMGQ5YzcwZGQxOGQ1YTMyYzI1M2ZiODA0N2U2M2E0IiwiZXhwIjoxNzExNDcyMjE2LCJuYmYiOjE3MTE0NzIwOTYsImlhdCI6MTcxMTQ3MjA5Nn0.pNQ4AcgoymxWHROXVjcYX8QMKdf9QgRH3zex7qc16avX7_Ax1q_WFWzQWfP48Fh-ooeh9hBQKCQkZxjVxYx8Sy-cNqmkf1AI9KGh5uemHh3FYAbvebCTaIXan0B6glWHVnDSwLZKBWTDDai2VXOmUfntyV9yPJdTqxt1J0j8PNuIWzNVdFlcTxzpggcJMhbcqtf4GRwSMKx69HU5sP4AQ7GJ2cBvN7Im-nkRXTc7xiyYnIsFx0vIWJzojC4zwg0-C1LHKQD4DyEKhqOVISIKUSa3GhD6ajcDuGDS8af4Iz19sNPsSoSULBUG-a7E5lXx2vk802vOFFWV68ZHugsJHpdSpLFwTVixipQ1-QdKRozlMjNPguu-5CYxhZVR1p32lbN9D879xGbFXUgPJVwK25NILvbEMcrqnGPgKcRUjJlHtVljGOgXrjmG7dMiW5QOsyy1eIvJ1D1sNsG02fDTbchTzXHmIybxQTK0FXCyNDLOAl6xgW0Jundg7AN1uJU2cLEWy1x3TusqC7lyeTeF3WYT-G8xE2CU4GpLBeYWyLwuJgxRkaWcg9IXiivguPbWpcT0RMl1bmpn0TJ2VgEPCuSG0mJxMBp8HbAgxwgar8AHdpoZ43dCCwZnB0a0O_kmGkBE2xGKKvgTx_U6eSixZzyyNmHDC1KH1Vy1WW1ZcF0' https://localhost:8443/1.0/projects | jq
{
  "type": "sync",
  "status": "Success",
  "status_code": 200,
  "operation": "",
  "error_code": 0,
  "error": "",
  "metadata": [
    "/1.0/projects/demo"
  ]
}

Documentation: Remote API authentication - Incus documentation

Configurable OIDC username field

For those using OpenID Connect, you may have noticed that Incus will use the e-mail claim when available as the user’s identifier. Then if missing, it will rely on the Subject.

As different deployments may make different information available through OIDC claims, it’s now possible to set oidc.claim to the claim to use as the user identifier.

stgraber@dakara:~$ incus query s-dakara:/1.0 | jq -r .auth_user_name
stgraber@stgraber.org
stgraber@dakara:~$ incus config set oidc.claim=name
stgraber@dakara:~$ incus query s-dakara:/1.0 | jq -r .auth_user_name
Stéphane Graber
stgraber@dakara:~$ incus config set oidc.claim=sub
stgraber@dakara:~$ incus query s-dakara:/1.0 | jq -r .auth_user_name
99cb8caa-3640-45b9-b87a-55266366aaf3
stgraber@dakara:~$ incus config set oidc.claim=email
stgraber@dakara:~$ incus query s-dakara:/1.0 | jq -r .auth_user_name
stgraber@stgraber.org

Improved NUMA handling

With this release, we spent a fair amount of time trying to improve both the container and virtual-machine performance on large systems. This obviously includes multi-socket systems but also AMD systems running in NPS4 or similar mode where each CPU is exposed as multiple NUMA nodes.

In general, our goal has been to make it easy to distribute workloads across NUMA nodes while keeping their CPU and memory properly pinned and also selecting PCIe resources that are closest to their NUMA node(s).

As part of that, a few things were done:

  • limits.cpu.nodes is now supported for virtual-machines too
  • A new balanced value has been added to limits.cpu.nodes which will have Incus pick the NUMA node with the least instances configured to use it
  • SR-IOV GPU selection now also considers NUMA nodes as part of the selection logic and when no match is found, will prefer PCIe devices that are attached to the same CPU socket

For example:

stgraber@gputest:~$ incus list stgraber-gpu -cns4,limits.cpu.nodes,volatile.cpu.nodes,volatile.gpu.last_state.pci.parent,volatile.gpu.last_state.vf.id
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
|      NAME      |  STATE  |         IPV4          | LIMITS CPU NODES | VOLATILE CPU NODES | VOLATILE GPU LAST STATE PCI PARENT | VOLATILE GPU LAST STATE VF ID |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu01 | RUNNING | 10.232.44.8 (enp5s0)  | balanced         | 0                  | 0000:63:00.0                       | 1                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu02 | RUNNING | 10.232.44.9 (enp5s0)  | balanced         | 2                  | 0000:03:00.0                       | 1                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu03 | RUNNING | 10.232.44.10 (enp5s0) | balanced         | 4                  | 0000:e3:00.0                       | 1                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu04 | RUNNING | 10.232.44.11 (enp5s0) | balanced         | 5                  | 0000:c3:00.0                       | 2                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu05 | RUNNING | 10.232.44.12 (enp5s0) | balanced         | 6                  | 0000:c3:00.0                       | 1                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu06 | RUNNING | 10.232.44.13 (enp5s0) | balanced         | 7                  | 0000:83:00.0                       | 0                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu07 | RUNNING | 10.232.44.15 (enp5s0) | balanced         | 1                  | 0000:43:00.0                       | 3                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu08 | RUNNING | 10.232.44.16 (enp5s0) | balanced         | 2                  | 0000:03:00.0                       | 0                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu09 | RUNNING | 10.232.44.17 (enp5s0) | balanced         | 3                  | 0000:03:00.0                       | 2                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu10 | RUNNING | 10.232.44.18 (enp5s0) | balanced         | 4                  | 0000:e3:00.0                       | 0                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu11 | RUNNING | 10.232.44.19 (enp5s0) | balanced         | 5                  | 0000:c3:00.0                       | 0                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu12 | RUNNING | 10.232.44.20 (enp5s0) | balanced         | 6                  | 0000:83:00.0                       | 1                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu13 | RUNNING | 10.232.44.21 (enp5s0) | balanced         | 7                  | 0000:83:00.0                       | 2                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu14 | RUNNING | 10.232.44.22 (enp5s0) | balanced         | 1                  | 0000:43:00.0                       | 1                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu15 | RUNNING | 10.232.44.23 (enp5s0) | balanced         | 2                  | 0000:43:00.0                       | 2                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+
| stgraber-gpu16 | RUNNING | 10.232.44.24 (enp5s0) | balanced         | 3                  | 0000:03:00.0                       | 3                             |
+----------------+---------+-----------------------+------------------+--------------------+------------------------------------+-------------------------------+

In this case we can see 16 VMs each using the new balanced option for NUMA nodes and getting scheduled across 8 NUMA nodes (2 sockets AMD NPS4) with GPUs being selected to match.

More options to select USB devices

USB device passhtrough for both containers and virtual-machines has so far been using the vendorid and productid fields. This works fine as long as there is only one USB device of any one type connected to the system.

When multiple identical devices are present, the inability to distinguish them has been a problem.

To address this, three new fields have now been added to usb devices:

  • busnum referring to the USB bus number
  • devnum referring to the USB device number (on its bus)
  • serial referring to the USB device serial number (not present on all devices)

The same fields can be found in the full Incus resources list through:

incus query /1.0/resources

Disk I/O throttling for VMs

One more feature gap between containers and virtual-machines is now gone.

The limits.write and limits.read properties on disk devices are now properly enforced on virtual-machines by having Incus setup an I/O throttle in QEMU.

Both bytes per second and I/O per second type limits are supported.

Per-remote client certificates

It’s now possible to put a <remote>.crt and <remote>.key file in a new clientcerts folder within the Incus command line client config directory (typically ~/.config/incus/) and have those certificates be used when interacting with that particular remote.

While this may be useful on its own, it becomes a lot more useful when combined with global remotes, which can be added in /etc/incus/config.yml. Now with this feature, those global remotes can also have a client certificate made available in /etc/incus/clientcerts/ which will then be used by all users on the system.

Manual generation of the client certificate keypair

A new command to manually trigger the generation of the main client.crt and client.key keypair is now available.

This is done by running incus remote generate-certificate

Improvements to lxd-to-incus

The lxd-to-incus tool keeps evolving with every release.

In this one, it gains support for migrating users from the newly released LXD 5.21 LTS as well as handling Alpine installations.

Additionally, a static binary version of the tool is now available on Github, making it easier for users to fetch the latest version of the tool, useful as bugs get fixed in between Incus releases.

Improvements to incus-migrate

The workload migration tool incus-migrate has also seen a couple of small improvements.

It can now use the local Incus system as the target of the migration, useful when importing virtual-machine images from another virtualization tool.

And it’s also now prompting for whether the imported virtual machine should be using a UEFI firmware or instead use a legacy BIOS.

Additional image restrictions

A bit of an internal detail, or at least only relevant to public image server operators, but two new image restrictions have been added:

  • requirements.nesting which will require the container have security.nesting=true
  • requirements.cdrom_agent which will require that a source=agent:config disk device be added to the virtual-machine

Those two can be used to flag specific images that need extra user interaction to work properly, resulting in a clear client-side error rather than starting a potentially broken instance.

Complete changelog

Here is a complete list of all changes in this release:

Full commit list
  • Translated using Weblate (Japanese)
  • Translated using Weblate (Japanese)
  • incus/image: Fix column handling with --all-projects
  • Replace util.ValueInSlice with slices.Contains
  • shared/util: Delete ValueInSlice function
  • incus/image: Fix column handling with --all-projects
  • incusd/instance/qemu: Relocate image requirement checks
  • doc/images: Add requirements.cdrom_agent
  • incusd/instance/qemu: Add support for requirements.cdrom_agent
  • incusd/device/disk: Fix incorrect block volume usage
  • Translated using Weblate (Japanese)
  • incusd/network/ovn: Use ParseIPToNet instead of manual IPToNet and net.ParseIP
  • incusd/network/ovn: Use listenAddressNet in family check
  • incusd/instance/drivers: Disable architecture check on incus cp with snapshots
  • Translated using Weblate (French)
  • incusd/network/bridge: Set local address on all VXLAN tunnels
  • incus/instance/qemu: Fix RecordOutput
  • incus: add completions for instance actions and snapshots
  • incus: add completions for profiles
  • incusd/network/ovn: Introduce get helper
  • incusd/network/ovn: Add some missing indices
  • incusd/network/ovn: Use get helper
  • incusd/network/ovn: Fix LogicalSwitchPortIPs logic
  • incusd/network/bridge: Fix gofmt
  • incusd/network/ovn: Fix gofmt
  • cmd/incus: Use proper timestamp check
  • cmd/incus: Use consistent date format and timezone
  • client: Rename network_peer for consistency
  • cmd/incusd: Rename network_peer to network_peers
  • shared/api: Rename network_allocation for consistency
  • incusd/db: Fix comment typoes
  • incusd/db/generate: Fix bad camel case handling
  • incusd/db/network_peers: Fix duplicate type definitions
  • incusd/auth: Drop Permission type
  • incusd/auth: Add boilerplate doc strings
  • incusd/images: Properly handle null creation and expiry dates
  • incus: add completions for remotes
  • incus: add completions for projects
  • incusd/images: Fix reporting of images in multiple projects
  • github: Add static build of lxd-to-incus
  • lxd-to-incus: Add support for Alpine service name
  • lxd-to-incus: Re-organize target list
  • lxd-to-incus: Add support for APK
  • Makefile: Add OVN IC to update-ovsdb
  • incusd/network: Update OVS/OVN schemas
  • incusd/network/ovn: Add IC clients
  • incusd/network/ovn: Add GetName to NB client
  • incusd/network/ovn: Add GetGateways to ICSB
  • incusd/network/ovn: Introduce new errors
  • incusd/network/ovn: Add CreateTransitSwitch and DeleteTransitSwitch to ICNB
  • incusd/device/gpu_sriov: Add locking
  • incusd/device/gpu_sriov: Re-locate vfio-pci loading
  • incusd/device/gpu_sriov: Rework VF allocation logic
  • incus/remote: Add a generate-certificate sub-command
  • i18n: Update translation templates
  • incusd/drivers/qmp: Add SetBlockThrottle
  • incusd/device/disk/config: Add DiskLimits
  • incusd/device/disk: Re-shuffle limit parsing
  • incusd/device/disk: Add disk limits on VMs
  • incusd/device/disk: Support live limits update for VMs
  • incusd/instance/qemu: Support disk I/O limits
  • incus/remote: Add missing docstrings
  • incusd/certificates: Improve token handling when clustered
  • cmd/incusd/api_1.0: Update context
  • cmd/incusd/api_cluster: Update context
  • cmd/incusd/api_internal: Update context
  • cmd/incusd/daemon: Update context
  • cmd/incusd/api_project: Update context
  • cmd/incusd/certificates: Update context
  • cmd/incusd/images: Update context
  • cmd/incusd/instance: Update context
  • cmd/incusd/network: Update context
  • cmd/incusd/operations: Update context
  • cmd/incusd/profiles: Update context
  • cmd/incusd/storage: Update context
  • cmd/incusd/warnings: Update context
  • incusd/devices: Skip isolated threads from NUMA CPUs
  • incusd/devices: Restrict CPU threads by NUMA node
  • incusd/instance/qemu: Add support for limits.cpu.nodes
  • incusd/device/gpu: Add support for limits.cpu.nodes for VF selection
  • incusd: Fix import shadowing
  • incusd/images: Fix potential race condition
  • incusd/instance/qemu: Add support for NUMA node restrictions for memory
  • incusd/apparmor/qemu: Silence apparmor failures
  • incusd/network/ovs: Introduce new errors
  • incusd/network/ovn/nb: Move SetChassisGroupPriority to new function signature
  • incusd/network/ovn/sb: Move GetLogicalRouterPortActiveChassisHostname to new function signature
  • incusd/network/ovs: Move GetBridge to new function signature
  • incusd/network/ovs: Move CreateBridge to new function signature
  • incusd/network/ovs: Move DeleteBridge to new function signature
  • incusd/network/ovs: Move CreateBridgePort to new function signature
  • incusd/network/ovs: Move GetChassisID to new function signature
  • incusd/network/ovs: Move GetOVNBridgeMappings to new function signature
  • incusd/network: Update for function changes
  • incusd/device/nic: Update for function changes
  • incusd: Update for function changes
  • doc: Fix bad snapshot syntax
  • Translated using Weblate (French)
  • doc: Fix token creation procedure
  • incusd/network/ovn/nb: Add GetLogicalSwitch
  • incusd/network/ovn/nb: Replace ChassisGroupChassisDelete with SetChassisGroupPriority
  • incusd/network/ovn/nb: Port CreateLogicalRouterPort to OVSDB
  • incusd/network/ovn/nb: Replace LogicalRouterPortLinkChassisGroup with CreateLogicalRouterPort
  • incusd/network/ovn/nb: Port CreateChassisGroup to OVSDB
  • incusd/network/ovn/nb: Port CreateLogicalSwitch to OVSDB
  • incusd/network/ovn: Update for function changes
  • incusd/network/ovn: Remove state references
  • incusd/state: Add OVNNB and OVNSB handles
  • incusd: Update to use state for OVN
  • incusd/device: Make init function return error
  • incusd/device: Add OVN check on nicOVN
  • client: Still return response on RawQuery error
  • incus/query: Respect --raw for errors
  • incusd/network/acl: Add OVN check
  • incusd/network: Make init function return error
  • incusd/network: Add OVN check on ovn driver
  • incusd/api: Re-order config checks
  • incusd: Add OVN loader
  • Translated using Weblate (French)
  • incusd/network/ovn/nb: Port CreateLogicalSwitchPort to OVSDB
  • incusd/network/ovn/nb: Port DeleteLogicalSwitchPort to OVSDB
  • incusd/network/ovn/nb: Port DeleteLogicalRouterPort to OVSDB
  • incusd/network/ovn: Update for function changes
  • incusd/network/ovs: Port GetOVNSouthboundDBRemoteAddress to OVSDB
  • incusd/network/ovs: Port DeleteBridgePort to OVSDB
  • incusd/network/ovs: Port GetInterfaceAssociatedOVNSwitchPort to OVSDB
  • incusd/network/ovs: Align GetChassisID with other functions
  • incusd: Update for OVS function changes
  • incusd/network/ovn/icsb: Fix bad DB schema
  • incusd/network/ovn/nb: Introduce GetLogicalRouterPort
  • incusd/network/ovn/nb: Extend OVNSwitchPortOpts to handle router ports
  • incusd/network/ovn/nb: Change type of RouterPort field to OVNRouterPort
  • incusd/network/ovn/nb: Port DeleteChassisGroup to OVSDB
  • incusd/network/ovn/icnb: Update DeleteTransitSwitch to handle missing switches
  • incusd/network/ovn: Update for function changes
  • Translated using Weblate (French)
  • incus/completion: do not add a space after remote names completion
  • incusd/device/disk: Disable virtiofsd caching
  • incus-agent: Cleanup mount logic
  • Translated using Weblate (French)
  • incus: expose parseVolume to entire package
  • incus: add completions for storage pools and volumes
  • incusd/device/gpu_sriov: Fix default handling
  • doc/packaging: Add mention of documentation
  • incusd/auth: Fix --all-projects for restricted users
  • doc: Add third party tools page
  • gomod: Update dependencies
  • incusd/auth/tls: Prevent project modifications
  • doc: Update wordlist
  • internal/usbid: allow path override of usb.ids path
  • incus/completion: fix image names completion
  • doc/environment: document INCUS_USBIDS_PATH
  • incusd/instance/qemu/agent: Check for semanage
  • incusd/project: Fix config name in ImageProjectFromRecord
  • incus/restart: Fix long description
  • i18n: Update translations
  • lxd-to-incus: Handle common existing bridges
  • shared/simplestreams: Remove defaultOS
  • shared/simplestreams: Add NewLocalClient
  • incus-simplestreams: Introduce new command
  • incus-simplestreams: Simplify delete logic
  • doc: Re-organize image server doc
  • doc: Add section for incus-simplestreams
  • incusd/seccomp: Add support for pidfd threads
  • incus: add completions for clusters
  • incus: add completions for cluster groups
  • incus: add completions for cluster roles
  • incus: add completions for config devices
  • incus: add completions for config templates
  • update translations
  • doc: Update references to mage docs
  • doc/backup: Remove bad reference
  • incus: add completions for network acls
  • shared/api: Add new structs to support configuration metadata
  • client: Add GetMetadataConfiguration
  • incusd: Rename documentation.go → metadata.go
  • doc/rest-api: Refresh swagger YAML
  • shared/api/metadata: Add GetKeys to simplify usage
  • incusd: Add support for JWT authentication
  • gomod: Update dependencies
  • tests: Add tls2jwt tool
  • tests: Add JWT authentication test
  • api: auth_tls_jwt
  • doc/authentication: Add section on JWT
  • doc/instances: Remove size.state requirement for live migration
  • incusd/instance/qemu: Allow live migration without size.state
  • shared/idmap: Support uid/gid in subuid/subgid
  • shared/cliconfig: Copy clientcerts on remote copy
  • shared/cliconfig: Add HasRemoteClientCertificate
  • shared/cliconfig: Support per-remote client certificates
  • doc: Add clientcerts
  • incusd/cluster/config: Add oidc.claim
  • incusd/auth/oidc: Add support for using a specific claim as username
  • incusd: Pass OIDC claim to verifier
  • api: oidc_claim
  • doc: Update configs
  • doc/howto/instances: Mention extra resources in ISO guidea
  • doc/installing: Add Debian backport
  • doc: Add backported to dictionary
  • lxd-to-incus: Add support for LXD 5.21
  • shared/cliconfig: Ensure client certificate key is 0600
  • api: device_usb_serial
  • doc: Add busnum, devnum and serial to USB devices
  • shared/api: Add Serial to ResourcesUSBDevice
  • incusd/resources: Add USB Serial
  • incusd/devices/usb: Add serial, busnum and devnum options
  • doc/rest-api: Refresh swagger YAML
  • incusd/instance/qemu: Fix handling of > 64 limits.cpu
  • incusd/device/gpu_sriov: Implement NUMA fallback
  • incus: add completions for network forwards
  • incus: add completions for network load balancers
  • shared/validate: Remove stringInSlice
  • shared/validate: Add And and Or functions
  • shared/util: Move ParseUint32Range
  • incusd/project: Update for ParseUint32Range
  • doc/instance_options: Remove mention of limits.cpu.nodes from container-only section
  • incusd/devices: Better handle bad config
  • api: numa_cpu_balanced
  • internal/instance: Add support for balanced NUMA nodes
  • doc: Update configs
  • incusd/instance/common: Add NUMA balancing
  • incusd/instance/lxc: Add support for balanced NUMA allocation
  • incusd/instance/qemu: Add support for balanced NUMA allocation
  • incusd/devices: Add support for balanced NUMA allocation
  • incusd/device/gpu_sriov: Simplify NUMA logic
  • doc/cloud-init: Don’t mention non-existing remotes
  • doc/howto/images_remote: Fix wording around image servers
  • doc/benchmark: Fix install command
  • incusd/instance/common: Fix CanMigrate mutating devices
  • incusd/instance/qemu: Reduce agent queries
  • incusd/metrics: Don’t filter out all server metrics
  • incusd/auth/tls: Include project restrictions for metrics certificates
  • incusd/auth/tls: Return project-aware checker for metrics
  • incusd/metrics: Use project-specific checker if no global access
  • internal/server/instance/lxd: add support for image.requirments.nesting
  • api: add image_restriction_nesting
  • doc/images: introduce requirements.nesting
  • Show the count values in snapshot count mismatch error
  • incus/admin/init: Use btrfs subvol in --auto
  • incus-migrate: Clarify that disk image files must be raw
  • incusd/network/ovn/icnb: Fix comment
  • incusd/project: Re-format the comments
  • incusd/project: Fix bad default value
  • doc: Update configs
  • incus/migrate: Add CSM support
  • incusd/storage/backend: Better handle name conflicts
  • incus-migrate: Support using the local server
  • api: network_integrations
  • shared/api: Add type and target_integration fields to NetworkPeersPost
  • incusd/db/cluster: Add networks_integrations
  • incusd/db/cluster: Re-generate schema
  • incusd/db/cluster: Add generated DB code for network integrations
  • incusd/db: Update network peer DB query functions
  • client: Add check for network_integrations in CreateNetworkPeer
  • incus/network/peer: Add support for network peer types
  • shared/api: Add network integrations
  • client: Add network integration functions
  • incus/network: Introduce support for integrations
  • incusd/auth: Add network integration functions
  • shared/api: Add lifecycle events for network integrations
  • incusd/lifecycle: Add network integration events
  • incusd: Add network integration API
  • incusd/db: Add GetNetworkPeersURLByIntegration
  • incusd/network_integration: Add UsedBy field
  • incusd/network_integrations: Add validator
  • incusd/network/ovn: Add support for peering with OVN IC
  • incusd/project: Add restricted.networks.integrations
  • incusd/project: Add NetworkIntegrationAllowed
  • incusd/network/integrations: Respect project restrictions
  • incusd/network/ovn: Add support for integration restrictions
  • incusd/auth/openfga: Update the model
  • incusd/auth/openfga: Update the generated model
  • incusd/auth/openfga: Handle model updates
  • incusd: Remove openfga.store.model_id
  • incusd/db/cluster: Remove openfga.store.model_id
  • doc/ovn_peers: Add remote peering
  • doc: Add documentation for network integrations
  • doc/rest-api: Refresh swagger YAML
  • i18n: Update translation templates
  • doc: Update configs
  • gomod: Update dependencies

Documentation

The Incus documentation can be found at:

Packages

There are no official Incus packages as Incus upstream only releases regular release tarballs. Below are some available options to get Incus up and running.

Installing the Incus server on Linux

Incus is available for most common Linux distributions. You’ll find detailed installation instructions in our documentation.

Homebrew package for the Incus client

The client tool is available through HomeBrew for both Linux and MacOS.

Chocolatey package for the Incus client

The client tool is available through Chocolatey for Windows users.

Winget package for the Incus client

The client tool is also available through Winget for Windows users.

Support

At this early stage, each Incus release will only be supported up until the next release comes out. This will change in a few months as we are planning an LTS release to coincide with the LTS releases of LXC and LXCFS.

Community support is provided at: https://discuss.linuxcontainers.org
Commercial support is available through: Zabbly - Incus services
Bugs can be reported at: Issues · lxc/incus · GitHub

9 Likes
3

typo:
Replace Incus basically supports to mechanisms for remote authentication
with Incus basically supports two mechanisms for remote authentication

2 Likes
4

Video overview of Incus 0.7:

6 Likes
5

Thank you for your work to every contributer.

3 Likes
6

Incredible ! The simplestreams management tool looks really useful !

3 Likes
7

Great
OVN interconnection and disk limit is very useful

8

Thanks for the work on incus-simplestreams tool, that will simplify (no pun intended) simplestreams setup!

I also like the JWT function, as it’ll allow me to more easily expose my incus install behind the free tier cloudflare tunnel option (which doesnt allow client certs to be passed to the backend) without having to go into an OIDC provider in my homelab. I cant seem to find the tls2jwt binary mentioned in the notes however.
Edit: ah, here it is: incus/test/tls2jwt/tls2jwt.go at main · lxc/incus · GitHub would be nice if this could be built and added to a package at some point

9

Yeah, tls2jwt is very rough right now, just supports the bare minimal for testing. For example it doesn’t really handle anything but RSA based certs now.

A more complete version of it could be included as a incus remote command down the line.
At which point it would also make sense for the Incus CLI and Go package to also start supporting JWT for authentication.

Not that it would work for the initial setup of a remote from a token as for that we do still need a full TLS client certificate connection so we can extract the public certificate from it. But it would work for any already trusted certificate.

1 Like
10

Is this something that would enable direct authentication to Incus via JWT tokens,if you put Incus behind some sort of Auth proxy (e.g. like Teleport), which can inject JWT tokens?

11

Yeah, you could generate a long lasting JWT and then have a proxy inject it so the actual user talking to the proxy doesn’t need to authenticate.

12

Incus includes a tool called incus-simplestreams which can be used to manage a file system tree using the Simple streams format.

But it doesn’t seem to be included in the zabbly Ubuntu packages, nor available as a binary from Releases · lxc/incus · GitHub. You have to build it yourself?

13

It’s now available in an incus-extra package for those using the Zabbly repository.

3 Likes