Easy way to try Incus on macOS with Colima

[abiosoft (Abiola Ibrahim)]

Incus support has been added to Colima, a tool that provides easy access to container runtimes on macOS.

Feedbacks would be appreciated.

Getting started with Incus on macOS with Colima

Install dependencies

# install incus client
brew install incus

# incus development version of Colima
brew install --head colima 

Start Colima using Incus runtime

colima start --runtime=incus

Start using Incus

$ incus version
Client version: 6.3
Server version: 6.3

$ incus list 
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+

$ incus info --resources | head -n5
System:
  Vendor: QEMU
  Product: QEMU Virtual Machine
  Version: virt-9.0
  Type: virtual-machine

It feels native, how does it work?

• Colima starts an Ubuntu VM with Incus pre-installed using QEMU (or macOS Virtualization.Framework).
• Incus unix socket is forwarded to the macOS host
• Incus client (on macOS) is configured to utilise the forwarded unix socket. Automatically configured and removed on Colima startup and shutdown.

Other features

• Incus UI available at https://localhost:8443
• Volume mounts - $HOME is mounted into the Incus VM
• Port forwarding - all ports exposed to the VM is forwarded to macOS. e.g. a proxy device listening on 8080 is accessible on localhost:8080 on macOS.
• Docker remote is pre-configured for OCI images. e.g. incus launch docker:caddy should just work.

Limitations

• Only containers are supported, no virtual machines.
• Containers are not accessible on the host via the bridge IP address, unless it is further exposed to the host e.g. via proxy device.

[stgraber (Stéphane Graber)]

Great to see!

[abiosoft (Abiola Ibrahim)]

While testing I realised that it is possible to run x86_64 binaries utilising either QEMU or Rosetta (even in a nested container i.e. Docker in Incus)

root@aca2def12571:/# ps faux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           1  1.8  0.2 179432  5352 pts/0    Ss   10:01   0:00 /mnt/lima-rosetta/rosetta /bin/bash
root           9 83.3  0.2 182476  4608 pts/0    R+   10:01   0:00 /usr/bin/ps faux

[abiosoft (Abiola Ibrahim)]

Nested Virtualization is now supported on m3 Macs running macOS 15 or later.
Colima may soon be able to support virtual machines for Incus runtime.

This is Incus running on a Debian VM on macOS, able to launch both containers and virtual machines.

[stgraber (Stéphane Graber)]

Nice! That’s great to see, will make it possible to use all of Incus on MacOS then!

How’s the performance for those VMs? Is there any known amount of overhead on the nested virtualization?

[abiosoft (Abiola Ibrahim)]

Looks like performance is impressive and the overhead is minimal.

Geekbench kept freezing in the multi-core tests, so I had to limit the VM to single CPU to get it to complete successfully. I am mainly interested in the single-core performance anyways.

You can suggest other tests or benchmarking tools to use.

The left window is the VM, right is the nested VM.

Link to the VM result: Apple Inc. Apple Virtualization Generic Platform - Geekbench
Link to the nested VM result: QEMU KVM Virtual Machine - Geekbench

[abiosoft (Abiola Ibrahim)]

After having tested it successfully, PRs have been created in upstream projects.

When they both get merged, Incus virtual machines would be supported in Colima.

add support for nested virtualization

vz: add support for nested virtualization

[stgraber (Stéphane Graber)]

Great to see! Very impressive indeed!