Create empty vm with two root disks for mirroring

Hi,

I’m trying to create an empty vm with two disks in order to test FreeBSD (installed via ISO) and ZFS mirroring for boot system.

When the vm is created using the incus init freebsd --empty --vm command it is associated with the default profile which has / as the path.

I tried adding another disk with path=/ and as expected I had an error.

Is there an easy way to have an empty vm, with a NIC, two root disks which will be configured as an ZFS mirror for the OS ?

Thanks.

A VM can only have a single root disk in Incus but you can attach additional disks if you’d like, they just won’t be custom block storage volumes instead of a primary instance disk.

You mean I can :

  1. create 2 volumes of type block in the zfs pool
  2. attach these volumes to the vm using the source option and without the path one ?

Will the root disk with path=/from the default profile not interfear with them ?

You can have your root disk be 50GiB and then attach one custom volume that’s also 50GiB to give you both drives. Or you can make the root drive tiny and keep it empty, then use two custom volumes for the two drives.

I tried the second solution as follows :

# incus init freebsd-15 --vm --empty -c limits.cpu=4 -c limits.memory=4GiB -d root,size=1MiB
# incus storage volume create incus-zfs vdev1 --type=block
# incus storage volume create incus-zfs vdev2 --type=block
# incus storage volume attach incus-zfs vdev1 freebsd-15
# incus storage volume attach incus-zfs vdev2 freebsd-15
# incus config set freebsd-15 security.secureboot=false
# incus config device add freebsd-15 install disk source=/home/shared/Isos/BSD/FreeBSD-15.0-RELEASE-amd64-bootonly.iso boot.priority=10
# incus start freebsd-15 --console=vga
[..] ## install FreeBSD then poweroff
# incus config device remove freebsd-15 install
# incus config device remove freebsd-15 root
# incus start freebsd-15 --console=vga

Unfortunately the boot disks were not found as you can see in the following screenshot

incus - boot disk not found

Did I miss something ?

Set boot.priority=10 on those disks so they’re picked up for boot ahead of the built-in one.

Many thanks @stgraber for your support :ok_hand:

With the following I acheived it :

# incus init freebsd-mirrored-boot --vm --empty -c limits.cpu=4 -c limits.memory=4GiB -d root,size=1MiB
# incus storage volume create incus-zfs vdev1 --type=block
# incus storage volume create incus-zfs vdev2 --type=block
# incus storage volume attach incus-zfs vdev1 freebsd-mirrored-boot
# incus config device set freebsd-mirrored-boot vdev1 boot.priority=5
# incus storage volume attach incus-zfs vdev2 freebsd-mirrored-boot
# incus config device set freebsd-mirrored-boot vdev2 boot.priority=5
# incus config set freebsd-mirrored-boot security.secureboot=false
# incus config device add freebsd-mirrored-boot install disk source=/home/shared/Isos/BSD/FreeBSD-15.0-RELEASE-amd64-bootonly.iso boot.priority=10
# incus start freebsd-mirrored-boot --console=vga
### install FreeBSD then poweroff
# incus config device remove freebsd-mirrored-boot install
# incus config device remove freebsd-mirrored-boot root
# incus start freebsd-mirrored-boot --console=vga

There is stil a remaining disk of 1 MiB attached to the vm but it works as expected :

$ lsblk
DEVICE         MAJ:MIN SIZE TYPE                                    LABEL MOUNT
da0              0:94  1.0M -                                           - -
da1              0:95   10G GPT                                         - -
  da1p1          0:97  260M efi                              gpt/efiboot0 /boot/efi
  da1p2          0:98  512K freebsd-boot                     gpt/gptboot0 -
  <FREE>         -:-   492K -                                           - -
  da1p3          0:99  2.0G freebsd-swap                 gpt/freebsd-swap SWAP
  da1p4          0:100 7.7G freebsd-zfs                   gpt/freebsd-zfs <ZFS>
  <FREE>         -:-   1.0M -                                           - -
da2              0:96   10G GPT                                         - -
  da2p1          0:101 260M efi                              gpt/efiboot1 -
  da2p2          0:102 512K freebsd-boot                     gpt/gptboot1 -
  <FREE>         -:-   492K -                                           - -
  da2p3          0:103 2.0G freebsd-swap                 gpt/freebsd-swap SWAP
  da2p4          0:104 7.7G freebsd-zfs                   gpt/freebsd-zfs <ZFS>
  <FREE>         -:-   1.0M -                                           - -