How to launch a NixOS VM in Incus

Incus
1

Hi folks,

After couple of hours install/re-install a finally find a working state:

  • incus version (6.12)
  • images:nixos/25.05
  • vm

I hope this tutorial provides some visibility and guidance to make testing and installing NixOS in Incus easier.

  • incus-agent refuse to start properly by default, none tutorial work (read only system/ config )
  • but I found a fix ā€¦ see last part.

List image

To find the right image.

incus image list images:nixos

Create

Force memory limit + fix for qemu (a small bug with slot)
Address space limit 0xfffffffff < 0x147fffffff phys-bits too low (36)

Adjust your memory settings:

incus create images:nixos/25.05 --vm  nixos << EOF
config:
  limits.memory: 4GiB
  raw.qemu.conf: |-
    [memory]
    maxmem = "16384M"
    size = "4096M"
    slots=8
  security.secureboot: "false"
EOF

First Start

Check you network configuration.

incus start nixos --console
----- wait boot
nix-channel --update

Setup Config

note that incus.agent is disabled.


echo {} > /etc/nixos/incus.nix
## add patch.nix into configuration.nix

cat <<EOF > /etc/nixos/patch.nix
{ modulesPath, pkgs, ... }:

{
environment.systemPackages = with pkgs; [ vim curl wget zsh ];
services.openssh.enable = true;
virtualisation.incus.agent.enable= false;


users.users.whoo = {
    description = "first user";
    password= "changeme";
    isNormalUser= true;
    extraGroups = [ "wheel" ];
    group = "users";
};

}
EOF

nixos-rebuild switch

Tada

Last minute check

Regarding incus-agent ā€¦ It seems that glibc is not correctly configured in my configuration/incus.nix


/run/incus_agent/incus-agent
bash: /run/incus_agent/incus-agent: cannot execute: required file not found

# works
/nix/store/[xxxxxx]-glibc-2.40-66/lib64/ld-linux-x86-64.so.2 /run/incus_agent/incus-agent

So I find a way to add lib64 with system.activationScripts.ldso and nix-ld patch.

echo {} > /etc/nixos/incus.nix
## add patch.nix into configuration.nix

cat <<EOF > /etc/nixos/patch.nix
{ modulesPath, pkgs, lib, ... }:

{
environment.systemPackages = with pkgs; [ vim curl wget zsh strace file glibc];
services.openssh.enable = true;
virtualisation.incus.agent.enable = true;

programs.nix-ld.enable = true;

system.activationScripts.ldso = lib.stringAfter [ "usrbinenv" ] ''
    mkdir -m 0755 -p /lib64
    ln -sfn \${pkgs.glibc.out}/lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2.tmp
    mv -f /lib64/ld-linux-x86-64.so.2.tmp /lib64/ld-linux-x86-64.so.2 # atomically replace
  '';

users.users.whoo = {
    description = "first user";
    password= "changeme";
    isNormalUser= true;
    extraGroups = [ "wheel" ];
    group = "users";
};

}
EOF

nixos-rebuild switch

And Tadaaaaamm.

1 Like
2

This could go into the Tutorials section of the forum.

The title of the post could change to something like How to launch a NixOS VM in Incus.
As far as I understand, a NixOS VM has those peculiarities that need to be addressed before it can become useful.

Is that the case as well with NixOS system containers?

3

Hi,
Iā€™m not sure if I can move that to the tutorial section, but it might be a better place.

For NixOS on a container with Incus, my experimentation works with:

incus launch images:nixos/25.05 -c security.nesting=true
## for sandboxing

## I removed firewall in configuration.nix
networking.firewall.enable=false;

nixos-rebuild --upgrade-all switch
4

I tried to simplify the instructions, I got into this. That is,

  1. The Incus Agent managed to start properly (because I could incus shell nixos and I can see the process with ps in the VM).
  2. SecureBoot is off.

The default 1GiB RAM and the 10GiB disk space is enough to boot the VM image. Upon boot, the VM uses 330MiB memory and 1.7GiB disk space.

$ incus launch --vm images:nixos/25.05 nixos -c security.secureboot=false
Launching nixos
$ incus shell nixos

[root@nixos:~]# ps ax | grep incus
    454 ?        Ssl    0:00 /run/incus_agent/incus-agent
    821 pts/0    S+     0:00 grep incus

[root@nixos:~]#

Would these instructions be sufficient for a new user of Nixos in Incus? Note that I do not know much about Nixos.