Easy way to try Incus on macOS with Colima

[jeremiah (Jeremiah C. Foster)]

@abiosoft - I appreciate your work and am happy to test if you’d like me to on my M1 before (or after release).

[abiosoft (Abiola Ibrahim)]

@jeremiah

To me this seems logical. I perhaps was confused by the discussion in this thread on the patch bringing in “nested virtualization”. If I’m running a Linux VM, perhaps via QEMU since that is installed, then don’t we already have nested virtualization?

Nope. It is not nested and you can only run Incus containers. For most people, this is more than sufficient.

To launch Incus virtual machines, the Linux virtual machine running Incus needs to be able to launch virtual machines, and that is where nested virtualization comes in.

[candlerb (Brian Candler)]

@abiosoft

There are no surprises there as the container (debian) is using the host’s kernel (ubuntu).

Ah right: I hadn’t realised you were looking from inside the container, rather than the outer VM running incus. It’s clear now.

[abiosoft (Abiola Ibrahim)]

Upstream PRs for nested virtualization support have gotten merged.

MacOS users with m3 devices would be able to use full feature set of Incus from the next release.

[abiosoft (Abiola Ibrahim)]

In the new Colima v0.7.6 release, you can now keep up with Incus updates by running the colima update command.

[brightzheng100 (Bright Zheng)]

@abiosoft

Nested Virtualization is now supported on m3 Macs running macOS 15 or later.
Colima may soon be able to support virtual machines for Incus runtime.

Just curious how to make it work in my MBP with M1 chip.

$ colima version 0.8.1
git commit: 96598cc5b64e5e9e1e64891642b91edc8ac49d16

runtime: incus
arch: aarch64
Client version: 6.9
Server version: 6.8

$ incus ls
+-------+---------+----------------------+----------------------------------------------+-----------+-----------+
| NAME  |  STATE  |         IPV4         |                     IPV6                     |   TYPE    | SNAPSHOTS |
+-------+---------+----------------------+----------------------------------------------+-----------+-----------+
| first | RUNNING | 192.100.0.104 (eth0) | fd42:7dec:40c5:4d6:216:3eff:febc:f5aa (eth0) | CONTAINER | 0         |
+-------+---------+----------------------+----------------------------------------------+-----------+-----------+

$ incus launch images:ubuntu/22.04 ubuntu-vm --vm
Launching ubuntu-vm
Error: Failed instance creation: Failed creating instance record: Instance type "virtual-machine" is not supported on this server: KVM support is missing (no /dev/kvm)

[abiosoft (Abiola Ibrahim)]

@brightzheng100

Just curious how to make it work in my MBP with M1 chip.

It is an hardware limitation and thereby not supported on M1 and M2 devices. You need an M3 or newer device for nested virtualization.

[brightzheng100 (Bright Zheng)]

I see, thanks for the clarification – time to find an excuse for a new MBP with M3 chip

[neodino]

Getting error:

uname -r
24.3.0

colima start --runtime=incus

INFO[0000] starting colima
INFO[0000] runtime: incus
INFO[0002] creating and starting ...                     context=vm
INFO[0002] downloading disk image ...                    context=vm
FATA[0003] error starting vm: error at 'creating and starting': error getting qcow image: error during image download: error downloading 'https://github.com/abiosoft/colima-core/releases/download/v0.8.1/ubuntu-24.04-minimal-cloudimg-arm64-incus.qcow2': error retrieving redirect url: error running [curl -ILs -o /dev/null -w %{url_effective} https://github.com/abiosoft/colima-core/releases/download/v0.8.1/ubuntu-24.04-minimal-cloudimg-arm64-incus.qcow2], output: "", err: "exit status 6"

Any idea?

[stgraber (Stéphane Graber)]

curl exist status 6 is DNS resolution failure.

You may want to try running the curl command manually to see if it works then, or just try accessing the URL it’s trying to download in a web browser to see if that works.

[abiosoft (Abiola Ibrahim)]

You can also manually download the disk image and specify it via the --disk-image flag.

[cemzafer]

Hi,
Is there any plan to publish, arm Linux desktop images for Mac?
Regards.

[stgraber (Stéphane Graber)]

Not currently as those images are rather heavy and seeing very low use in general (on x86 at least). For now, I’d recommend grabbing a normal image and then installing the relevant packages. For Ubuntu that would be ubuntu-desktop-minimal for example.

[stgraber (Stéphane Graber)]

Running Incus on a Mac

[abiosoft (Abiola Ibrahim)]

Improving the network experience

To bring more feature parity to the native Linux experience, I am looking at adding to Colima, the ability for instance IP addresses to be accessible from the macOS host.

Because the incus server is running in a virtual machine, the instances are only reachable within the virtual machine. Some familiarity with network forwards or proxy devices is needed to make the instances accessible from the macOS host.

Right now, this can actually be achieved via an undocumented feature
Starting Colima with the --network-address flag and assigning the col0 network to Incus instances will make them accessible from the macOS host.

However, because it leverages the macvlan network, it has a number of limitations that prevent it from being an ideal solution.

It will take some time and efforts to be able to provide a similar experience to Linux out of the box. But before I embark on this, I would like to get the perspective of other users.

Is your workflow or Incus experience being hindered due to this?

• I only run Incus client on macOS, I am indifferent.
• Incus network forwarding options suffices for me.
• The col0 workaround suffices for me.
• I will appreciate feature parity with Linux.

[stgraber (Stéphane Graber)]

@abiosoft if by macvlan limitation you refer to the fact that the VM itself can’t talk to the containers or nested VMs. This could be worked around either by using a bridge inside of the VM (shouldn’t be too difficult with a bit of netplan) or you can use the trick of making the VM’s own network also be a macvlan, leaving the parent device unconfigured, at which point everything are siblings and you don’t run into the limitation anymore.

[abiosoft (Abiola Ibrahim)]

Thanks for the suggestions, it didn’t occur to me to use macvlan for the VM itself. I would play around with both suggestions.

Would it be possible to leverage Incus dns so that <instance> and <instance>.incus still resolve within all instances.

[stgraber (Stéphane Graber)]

Hmm, no. The XYZ.incus and XYZ resolving is a side effect of Incus using dnsmasq for that network. So instances self-register into that when they do DHCP.

When you’re not on the Incus managed dnsmasq, Incus doesn’t track the instance IPs and so can’t really serve a DNS zone. It will show the IPs in incus list but it does that by interrogating the instances directly during that API call.

Your best bet would be to handle the DNS part in whatever provides DHCP to the colima VM. I don’t know if that’s also dnsmasq, just running on MacOS instead of Linux. If it is, it should be pretty straightforward to have it handle that for you.

[cemzafer]

Hi,
I have recently tried a vm on M3 but get this error message,

Error: Failed instance creation: Failed creating instance record: Instance type "virtual-machine" is not supported on this server: KVM support is missing (no /dev/kvm)

Here is my configuration details.

○ → colima version
colima version 0.8.1
git commit: 96598cc5b64e5e9e1e64891642b91edc8ac49d16

runtime: incus
arch: aarch64
Client version: 6.12
Server version: 6.12

Do I need to do something different?
Regards.

[abiosoft (Abiola Ibrahim)]

@cemzafer

Hi,
I have recently tried a vm on M3 but get this error message,

Error: Failed instance creation: Failed creating instance record: Instance type "virtual-machine" is not supported on this server: KVM support is missing (no /dev/kvm)

You may need to specify --nested-virtualization flag.

Can you share the output of colima status?