I built a ParticleOS image with mkosi, which has many output formats. It seems designed for a setup where the entire image is uploaded, then a VM is created by cloning the uploaded image itself (ex. if uploaded as virtio-blk file).
I couldn’t find a way to handle this in Incus(OS). While I could upload the image as a volume, creating a new VM requires selecting an iso.
Attempting to use the scsi-cd output format + selecting it as a custom iso didn’t work either.
Also, I’m also unsure of what the recommended output format is ( virtio-blk, virtio-scsi, nvme, or scsi-cd).
What I do for manipulating VM root disks (although it’s slow and messy) is to boot from an Ubuntu 24.04 installer ISO image, and then once the agent is running, ignore the installer and instead “incus exec” into it to do whatever you like to the VM’s root disk.
As far as I can see, a VM root disk and a block “storage volume” are two different, non-interchangeable concepts in incus (*). Hence you can’t just attach a storage volume, prepare it the way you want, and then get a VM to boot from it.
Hmm, that’s kind of a shame. It would be nice if I could store my immutable root filesystem as a volume, and just mount it readonly into all of the VMs.
Well, if you reformat it as an ISO image, that will work. You import --type iso and then attach it to all of your VMs with a higher boot.priority. You can then either ignore the “root disk” that incus provides, or treat it as a private, tied storage volume.
This dichotomy doesn’t arise in other virtualization systems I’ve used. In EC2 for example, an EBS volume is just an EBS volume: whether it’s the first, second or third one attached to a virtual machine makes no difference.