Incus 0.6 has been released


The Incus team is pleased to announce the release of Incus 0.6!

This second release of 2024 features a number of improvements, both large and small.

It comes with a completely new storage driver for cluster users, import/export support for storage buckets, a number of improvements for OVN users, support for new container kernel features, quite a few improvements to the migration tooling and more!


As usual, you can try it for yourself online: Linux Containers - Incus - Try it online


New features

Clustered LVM storage driver

A new storage driver named lvmcluster has been added.

As the name implies, this is for clustered use of the LVM storage driver.
This relies on llvmlockd and a lock manager like sanlock to provide concurrent access to a shared storage device across a cluster.

This is aimed at anyone who wants to run an Incus cluster and use a shared storage device like a FiberChannel SAN, iSCSI export, NVMEoF/NVMEoTCP disk, …

With the ability of having the same storage available on all servers comes support for much faster live migrations, server evacuations and the ability to automatically recover should a server suddenly die.

Documentation: LVM - lvm - Incus documentation

Storage bucket backup and import

It is now possible to backup a full storage bucket and later re-import back into Incus.

stgraber@dakara:~$ incus storage bucket create default foo
Storage bucket foo created
Admin access key: HD2GWC1KX693MFVT3P30
Admin secret key: tX9+G9G5UlcuT21yatKIUImSGvcXzPyA7ONgkjUv

stgraber@dakara:~$ dd if=/dev/random of=out.img bs=4M count=10
10+0 records in
10+0 records out
41943040 bytes (42 MB, 40 MiB) copied, 0.0975758 s, 430 MB/s
stgraber@dakara:~$ s3cmd --host= --host-bucket= --no-check-certificate --access_key=HD2GWC1KX693MFVT3P30 --secret_key=tX9+G9G5UlcuT21yatKIUImSGvcXzPyA7ONgkjUv put out.img s3://foo
upload: 'out.img' -> 's3://foo/out.img'  [part 1 of 3, 15MB] [1 of 1]
 15728640 of 15728640   100% in    0s   262.42 MB/s  done
upload: 'out.img' -> 's3://foo/out.img'  [part 2 of 3, 15MB] [1 of 1]
 15728640 of 15728640   100% in    0s   241.54 MB/s  done
upload: 'out.img' -> 's3://foo/out.img'  [part 3 of 3, 10MB] [1 of 1]
 10485760 of 10485760   100% in    0s   241.99 MB/s  done
stgraber@dakara:~$ s3cmd --host= --host-bucket= --no-check-certificate --access_key=HD2GWC1KX693MFVT3P30 --secret_key=tX9+G9G5UlcuT21yatKIUImSGvcXzPyA7ONgkjUv ls s3://foo
2024-02-23 03:26     41943040  s3://foo/out.img

stgraber@dakara:~$ incus storage bucket export default foo
Backup exported successfully!
stgraber@dakara:~$ incus storage bucket delete default foo
Storage bucket foo deleted

stgraber@dakara:~$ incus storage bucket import default backup.tar.gz
stgraber@dakara:~$ s3cmd --host= --host-bucket= --no-check-certificate --access_key=HD2GWC1KX693MFVT3P30 --secret_key=tX9+G9G5UlcuT21yatKIUImSGvcXzPyA7ONgkjUv ls s3://foo
2024-02-23 03:27     41943040  s3://foo/out.img

API: Main API specification - Incus documentation

Listing image across all projects

Just as it’s possible to list instances across all projects using the --all-projects, it’s now possible to do the same but for images.

stgraber@dakara:~$ incus image list --all-projects
| PROJECT | ALIAS | FINGERPRINT  | PUBLIC |               DESCRIPTION                | ARCHITECTURE |      TYPE       |   SIZE    |          UPLOAD DATE          |
| default |       | 256f59a72af5 | no     | Ubuntu jammy amd64 (20240222_07:42)      | x86_64       | VIRTUAL-MACHINE | 267.19MiB | Feb 23, 2024 at 12:27am (UTC) |
| default |       | 0941e441dbb9 | no     | Alpine edge amd64 (20240222_13:00)       | x86_64       | CONTAINER       | 2.93MiB   | Feb 23, 2024 at 12:27am (UTC) |
| default |       | d5fc6024f0fa | no     | Openwrt snapshot amd64 (20240222_11:57)  | x86_64       | CONTAINER       | 3.50MiB   | Feb 23, 2024 at 2:55am (UTC)  |
| demo    |       | f44a6b4e56f4 | no     | Archlinux current amd64 (20240222_04:18) | x86_64       | CONTAINER       | 188.86MiB | Feb 23, 2024 at 3:32am (UTC)  |

Over the API, this is done by passing ?all-projects=true.

binfmt_misc in unprivileged containers

Linux 6.7 added support for mounting of binfmt_misc inside of unprivileged containers.

Incus 0.6 will detect kernels that support this feature and when they do, it will no longer bind-mount binfmt_misc from the host system but instead allow it to be mounted from within the container.

stgraber@castiana:~$ incus launch images:ubuntu/22.04 foo
Launching foo
stgraber@castiana:~$ incus exec foo bash
root@foo:~# uname -a
Linux foo 6.7.4-zabbly+ #debian12 SMP PREEMPT_DYNAMIC Mon Feb  5 23:37:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
root@foo:~# mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc/
root@foo:~# ls -lh /proc/sys/fs/binfmt_misc/
total 0
--w------- 1 root root 0 Feb 23 03:46 register
-rw-r--r-- 1 root root 0 Feb 23 03:46 status

The binfmt_misc filesystem is used to define custom binary formats for emulation.
It’s most notably used by qemu-user-static as a way to run binaries of other architectures.

Control over shared block storage volumes

A new security.shared volume storage option has been introduced to control whether a custom block volume should be allowed to be used by multiple instances.

It is now required to set that configuration key prior to adding a custom block volume to a profile or add it directly to multiple virtual machines.

stgraber@castiana:~$ incus config device add lvm01 shared disk pool=default source=shared-lvm io.bus=nvme
Device shared added to lvm01

stgraber@castiana:~$ incus config device add lvm02 shared disk pool=default source=shared-lvm io.bus=nvme
Error: Failed add validation for device "shared": Cannot add un-shared custom storage block volume to more than one instance

stgraber@castiana:~$ incus storage volume set default shared-lvm security.shared=true

stgraber@castiana:~$ incus config device add lvm02 shared disk pool=default source=shared-lvm io.bus=nvme
Device shared added to lvm02
stgraber@castiana:~$ incus config device add lvm03 shared disk pool=default source=shared-lvm io.bus=nvme
Device shared added to lvm03
stgraber@castiana:~$ incus start lvm01 lvm02 lvm03

OVN logical router name in network info

To make it easier to see what’s going on within OVN, incus network info now gives you the name of the logical router for a particular network.

root@abydos:~# incus network info default
Name: default
MAC address: 00:16:3e:38:dd:28
MTU: 1500
State: up
Type: broadcast

IP addresses:
  inet (link)
  inet6	2602:fc62:a:1004::1/64 (link)

Network usage:
  Bytes received: 0B
  Bytes sent: 0B
  Packets received: 0
  Packets sent: 0

  Chassis: abydos
  Logical router: incus-net6-lr

File ownership and permissions in image templates

Template files can now have a uid, gid and mode set on them.
This can be particularly useful if a template is meant to be an executable shell script.

stgraber@castiana:~$ incus config metadata show foo
architecture: amd64
creation_date: 1708588077
expiry_date: 1711180077
  architecture: amd64
  description: Ubuntu jammy amd64 (20240222_07:42)
  name: ubuntu-jammy-amd64-default-20240222_07:42
  os: ubuntu
  release: jammy
  serial: "20240222_07:42"
  variant: default
    - create
    - copy
    create_only: false
    template: hostname.tpl
    properties: {}
    - create
    - copy
    create_only: false
    template: hosts.tpl
    properties: {}
    - start
    create_only: false
    template: hello.tpl
    properties: {}
    uid: "1000"
    gid: "2000"
    mode: "0755"

stgraber@castiana:~$ incus config template show foo hello.tpl
echo "Hello world!"

stgraber@castiana:~$ incus start foo
stgraber@castiana:~$ incus exec foo bash
root@foo:~# ls -lh /root/
-rwxr-xr-x 1 ubuntu 2000 30 Feb 23 04:07 /root/
root@foo:~# /root/
Hello world!

Documentation: Image format - Incus documentation

Encrypted EC client certificate keys

For those interacting with remote Incus servers, you may not know that it’s possible to protect your Incus private key with a password.

Up until now, this was only possible for RSA keys, but with Incus 0.6, we’re now adding support for EC keys too. That’s particularly relevant as EC keys have been the default for a while now.

Documentation: Remote API authentication - Incus documentation

Worth noting that if using this feature, you’ll likely also want to make use of the recently introduced “keepalive mode” as this will then significantly reduce the number of password prompts you’ll get while using Incus.

lxd-to-incus improvements

Lastly, lxd-to-incus saw a number of improvements:

  • Support for Void Linux
  • Detection of the boot.debug_edk2 configuration key
  • Handling of OVN SSL database connections
  • Automatic clearing of the simplestreams cache during migration

Complete changelog

Here is a complete list of all changes in this release:

Full commit list
  • incusd/instance/qemu/qmp: Don’t risk blocking QMP on eject
  • github: Switch to minio deb and pin working version
  • github: Use stable MicroCeph
  • incusd/db: Fix error handling in CreateNetworkPeer
  • incusd/apparmor: Allow more mounts in unprivileged containers
  • Translated using Weblate (Portuguese (Brazil))
  • incusd/instance/qemu/agent-loader: Handle legacy lxd-agent-loader
  • incusd/device/disk: Use genisoimage when mkisofs can’t be found
  • shared/api: Add example of Chassis in NetworkStateOVN
  • doc/rest-api: Refresh swagger YAML
  • api: network_state_ovn_lr
  • shared/api: Add LogicalRouter field to NetworkStateOVN
  • incusd/network/ovn: Expose LogicalRouter name
  • incus: Add OVN logical router name
  • i18n: Update translation templates
  • doc/rest-api: Refresh swagger YAML
  • scripts/bash: Don’t follow aliases
  • Translated using Weblate (Japanese)
  • test: Adapt test_database_no_disk_space() to newer libraft versions
  • incus/launch: Fix --console
  • api: image_template_permissions
  • doc: Add uid/gid/mode to image template metadata
  • shared/api: Add owner/permission fields to ImageMetadataTemplate
  • doc/rest-api: Refresh swagger YAML
  • incusd/instance/lxc: Add support for template ownership/mode
  • incus-agent: Add support for template ownership/mode
  • lxd-to-incus: Add comment
  • lxd-to-incus: Add cache cleanup logic
  • .github: Bump minimum Go version to 1.21
  • Makefile: Bump minimum Go version to 1.21
  • gomod: Update dependencies
  • doc: Bump minimum Go version to 1.21
  • Translated using Weblate (French)
  • incusd/daemon: Fix log format
  • incusd/sys: Sort kernel features
  • incusd/sys: Add doc strings
  • incusd/checkfeature: Add unprivileged binfmt detection
  • incusd/sys: Add UnprivBinfmt
  • incusd/daemon: Detect unprivileged binfmt_misc
  • incusd/api_1.0: Add unpriv_binfmt kernel feature
  • incusd/instance/lxc: Support unpriv binfmt_misc
  • incusd/apparmor: Support unpriv binfmt_misc
  • Translated using Weblate (Italian)
  • completion: support returning non-incus remotes
  • incus/image: add dynamic command line completions
  • shared/cliconfig: Support SSH encrypted keys
  • tests: Add crypto/ssh
  • doc/remotes: Add mention of keepalive
  • doc: Add Ansible to wordlist
  • doc/authentication: Hints on encrypting client key
  • api: images_all_projects
  • lxd-to-incus: Indicate what existing configuration was found
  • shared/api: Add project property to Image
  • client: Add GetImagesAllProjects
  • incusd/images: Add support for all_projects
  • doc/rest-api: Refresh swagger YAML
  • incus/image: Add --all-projects flag to list
  • i18n: Update translation templates
  • shared/cliconfig: Fix static analysis
  • gomod: Update dependencies
  • Makefile: Pin rpc2 version due to OVN issue
  • incusd/instance/qemu: Re-shuffle agent NIC handling
  • incusd/instance/qemu/agent-loader: Handle SELinux
  • incus/config: add completions
  • incus-agent: Load virtio_net before configuring NICs
  • incus: Run gofmt
  • doc/storage/lvm: Cleanup tables
  • incusd/patches: Move to be server-specific
  • incusd/db: Mark as server-specific
  • doc/faq: Add mention of the -mtu network interfaces
  • incusd/device: Bump base VM filesystem volume to 500MiB
  • incusd/storage: Create rootfs dir on empty instances
  • incusd/storage/quota: Don’t fail on missing project
  • incusd/instances: Use correct project on cross-project copy
  • incusd/patches: Fix bad SQL query
  • incusd/instances: Don’t bypass instance limit check
  • Added translation using Weblate (Portuguese)
  • incus-agent: Re-order imports
  • api: Add storage_bucket_backup extension
  • shared/api: Add storage bucket backup
  • incusd/db: Add storage bucket backup functions
  • incusd/db/operation: Add storage volume backup types
  • incusd/lifecycle: Add storage bucket backup events
  • incusd/project: Add StorageBucket function
  • incusd/storage/s3: Add transfer manager
  • incusd: Add storage bucket backup
  • client: Add storage bucket backup
  • incus: Add storage bucket import/export
  • doc/rest-api: Refresh swagger YAML
  • i18n: Update translation templates
  • test: Add storage bucket backup
  • client: Remove ceph-specific logic
  • incusd/storage/s3: Fix typo
  • incus: Fix import shadowing
  • incus: Fix comments on exported functions
  • client: Fix comments on exported functions
  • incusd: Fix import shadowing
  • incusd: Fix comments on exported functions
  • incusd/cluster: Make remote storage volume logic generic
  • incusd/db: Don’t hardcode remote storage drivers
  • incusd/storage_volumes: Don’t hardcode ceph
  • incusd/instances: Update ceph-specific comment
  • incusd/cluster: Update ceph-specific comment
  • incusd/instances: Generalize Ceph logic
  • lxd-to-incus: Support SSL authentication in OVN
  • lxd-to-incus: Don’t fail on missing OVN bridge mapping
  • lxd-to-incus: Ignore OVN chassis external_ids
  • lxd-to-incus: Add boot.debug_edk2 to deprecated keys
  • lxd-to-incus: Support Void Linux
  • cmd/incusd: Disable the architecture check on incus cp/mv
  • incusd/instance: Disable the architecture check on incus cp/mv
  • api: storage_lvm_cluster
  • incus/admin/init: Add LVM cluster logic
  • i18n: Update translation templates
  • lxd-to-incus: Add LVM cluster
  • doc: Add LVM cluster
  • shared/api: Add cephobject
  • shared/api: Add lvmcluster
  • doc/rest-api: Refresh swagger YAML
  • incusd/storage/lvm: Add clustered LVM
  • incusd/storage/lvm: Fix import shadowing
  • incusd/storage/lvm: Tweak locking in cluster
  • internal/server/device/config: Don’t include empty values
  • internal/server/device/config: Fix return values of Update function
  • api: Add shared_custom_block_volumes API extension
  • doc/reference: Add security.shared config key
  • instance/server/device: Validate shared block devices
  • internal/server/storage: Handle security.shared update
  • internal/server/storage: Allow security.shared key for custom block volumes
  • internal/server/storage/drivers: Handle security.shared in fillVolumeConfig
  • incusd/instance/qemu: Cap hotplug CPU slots to 64
  • incusd/storage/lvm: Make gofmt happy
  • shared/subprocess: Improve error handling
  • incusd/instance/qemu: Improve error handling
  • incusd/network/ovs: Wait for bridge interface to appear
  • incusd/storage/zfs: Fix refresh of VM volumes
  • internal/instance: Fix volatile key definitions
  • doc: Update configs
  • incusd/network/ovn: Extend validateExternalSubnet to allow uplink subnets
  • incusd/network/ovn: Add static routes to load-balancer and forwards
  • gomod: Update dependencies


The Incus documentation can be found at:


There are no official Incus packages as Incus upstream only releases regular release tarballs. Below are some available options to get Incus up and running.

Installing the Incus server on Linux

Incus is available for most common Linux distributions. You’ll find detailed installation instructions in our documentation.

Homebrew package for the Incus client

The client tool is available through HomeBrew for both Linux and MacOS.

Chocolatey package for the Incus client

The client tool is available through Chocolatey for Windows users.

Winget package for the Incus client

The client tool is also available through Winget for Windows users.


At this early stage, each Incus release will only be supported up until the next release comes out. This will change in a few months as we are planning an LTS release to coincide with the LTS releases of LXC and LXCFS.

Community support is provided at:
Commercial support is available through: Zabbly - Incus services
Bugs can be reported at: Issues · lxc/incus · GitHub


Does this release resolve the workaround for MicroOVN?

Not quite yet, there are hundreds of different places that currently directly call the OVN tools so it will take some time until all of them are ported.

Using MicroOVN with Incus is definitely possible but you need to jump through the few hoops I described before on this forum. Basically installing the tools and OVS on the host system, making sure that OVS doesn’t actually start there (to avoid conflicting with the one in MicroOVN) and then expose the OVS socket and OVN certs at their standard locations on the system.

This release does include the needed changes to lxd-to-incus to handle OVN deployments that use SSL such as the MicroOVN ones. That logic has been used to migrate a few MicroCloud deployments over to Incus while keeping the existing MicroOVN and MicroCeph in place.

1 Like

Release video is out now!


One great feature that landed, I think from a community member, is the ability to copy containers with different architectures. The use case here is to use the copy container feature to backup containers.

I am happy I can do this now. My laptop is amd64 and my backup server is arm64. :smiley:

It only took a few lines of code to do, but those few lines made a big difference to my current workflow.

Zabbly repo for Ubuntu reports an error when trying to install on Ubuntu 22.04 (actually, the equivalent Mint).

Had to replace the command with jammy and download that way.

The docs regarding migration from LXC are a bit lacking.

$ sudo lxc-to-incus --all
$ sudo incus list
If this is your first time running Incus on this machine, you should also run: incus admin init
To start your first container, try: incus launch images:ubuntu/22.04
Or for a virtual machine: incus launch images:ubuntu/22.04 --vm


Are they all gone! :sweat_smile:
Thankfully after reinstalling lxc I can see them (still) with lxc list. Whew!

How can we tell if sudo lxc-to-incus --all executed okay? My incus service is running but nothing gets listed even after I stop lxd.

There’s lxc-to-incus and lxd-to-incus. You should have used the latter.

LXC is when you use commads that look like lxc-create.
LXD is when you use commands like lxc launch ubuntu:....

1 Like

Keepalive sounds cool. I will encrypt my keys now!

How is it implemented? My understanding is that the incus client is not a daemon. Does it require a system keychain?

The keepalive mode has the client tool spawn a sub-process called incus remote proxy which will stick around and handle the backend connection until the keepalive timeout is reached.

That daemon is what will hold the private key material so further calls to the CLI will not need to open the key but instead will just interact with that proxy.

It’s pretty similar to how SSH’s control socket mechanism works except that in our case the initial command is allowed to exit without impacting any other commands being run at the time.

Whoa… I didn’t expect there may be two similar commands. lxd-to-incus worked.

I was surprised that lxc-to-incus didn’t provide any output, but the docs don’t mention lxd- in top level menu items so even as I read the docs I didn’t realize there’s another command and decided to continue.
I was wondering why there was nothing about VMs, but I attributed that to the brevity of that page…

[1] How to migrate containers from LXC to Incus - Incus documentation

Indeed, the top page for Migration does not mention the migration from LXD to Incus, Migration - Incus documentation

It’s in a different location, Migrating from LXD - Incus documentation

I also posted something on the migration to Incus from LXD, Migrating to Incus from LXD – Mi blog lah!

If you Google “migrate LXD Incus”, it shows both those links above at the top of the results.

1 Like