Specifics of running Windows VM in Incus

Hi,
I am little confused by Incus requirements for running Windows VM.
I read How to run a Windows virtual machine on Incus on Linux – Mi blog lah!
I also tried searching through docs.

Look like before using Windows installer iso you need to prepare it with distrobuilder.

  1. Is this hard requirement or just recommended? For example Proxmox also recommends installing virtio drivers but it works even without them (at least if I remember it correctly). Since Proxmox uses the same backend QEMU/KVM as Incus I thought it would behave similarly. So can Incus install (and run) Windows from the original iso installer like Proxmox?

  2. What exactly distrobuilder does? I read it adds virtio drivers and maybe some other config files? Let’s say I wanted to prepare my Windows iso manually without distrobuilder. Is that possible? Can I find anywhere what would I need to do?

Thanks.

Thanks for providing feedback to the tutorial!

  1. When you use distrobuilder to prepare the ISO so that you can easily install Windows in an Incus VM, distrobuilder makes changes to the ISO and adds drivers. Of course you can use directly the vanilla Windows ISO but you would need to add manually any VM drivers. It would be great if you could attempt an installation of the vanilla Windows ISO and give some feedback. Note that if the VM works without drivers, you must check whether the performance has also deteriorated.
  2. The source code of distrobuilder is here, GitHub - lxc/distrobuilder: System container image builder for LXC and Incus It is used to create Linux Incus images for containers and virtual machines with these configuration files. In the case of Windows, it takes an existing ISO and repacks it so that it’s easier to install in Incus. You can either read the source code (Go language) or when you perform the repacking, add the --debug flag so that you can see even more progress messages, and have a good understanding on what is happening to the ISO.

Hi,
thanks for explanation.

  1. I will try it myself with vanilla ISO. But I am not sure I will be able to test performance. But it wouldnt matter even if the performance is worse. The basic use case why I was asking was so I can install Windows with vanilla ISO, start it and then install drivers after that. At least that how it’s done in Proxmox. Virtio drivers are either added from 2nd CD during installation or installed additionally after that. But I will take a look at it.
  2. Thanks for tips. I will try that. Lets see if Golang is really as easy to read for newbies as they say :smiley:

Ok, first thing I found out during my testing.
If you use Media Creation Tool to create Windows ISO it will NOT contain install.wim file and distrobuilder will fail with error: Unable to find install.wim.

Looks like someone already had this problem. Trying to create an LXD Windows 11 VM and getting errors about "source write-protected, mounted read-only" · Issue #708 · lxc/distrobuilder · GitHub

Windows ISO created by Media Creation Tool instead has inside (in sources folder) install.esd file instead of install.wim.

So you need to download the Windows ISO directly or from another source so it contains install.wim

Another solution is that install.esd can be converted to install.wim following this guide. How to Convert Install.ESD to Install.WIM on Windows – TheITBros
If I understand correctly ESD is basically just better compressed WIM.

And final solution would be to make distrobuilder capable of handling install.esd. But I dont know for hard that would be.

This should be mentioned somewhere because it looks like ESD is the format Microsoft prefers for newer builds so this problem could be common.

Edit: Distrobuilder added ESD handling.

In my next tests I tried converting ESD to WIM and repack with distrobuilder. This worked flawlessly. I even installed Windows as VM inside VM. I ran Ubuntu with Incus inside VM and the repacked Windows installed without any problems.

Next I tried without repacking. So my next tries are for native ISO. With Incus inside VM the Windows installer VM just froze on first install screen. I tried multiple times. I guess not enough performance.

Then I tried booting Ubuntu with Incus from USB on my notebook. This time it was on metal so better but still too slow to do anything and it crashed often.

Finally I tried booting from USB on my desktop. This time it had enough performance (or maybe because CPU had enabled virtualization settings in bios). Sometimes it crashed and sometimes my whole PC froze so still not ideal.
I could click Install in the Windows VM but then I got stuck on A media driver your computer needs is missing

I tried to solve it by mounting Virtio drivers iso as additional disk/cdrom so the installer could get it from there. Similar as per Proxmox guide Windows 10 guest best practices - Proxmox VE

But the Windows installer didnt see the mounted iso. I thought it would maybe see it if it was mounted as IDE in io.bus option but Incus doesnt support this.

Then I tried looking into directly editing QEMU config but that was above my capability.

Since software like Proxmox which also use QEMU can perform Windows install even without virtio drivers this should be also possible for Incus. Or at least it’s capable of loading drivers from additional iso.
But as of now I am not able to find what configuration enables this.

After some more testing I tried editing raw.qemu and add virtio.iso manually.
I tried both: -cdrom /path/to/virtio.iso and -drive file=/path/to/virtio.iso,media=cdrom,if=ide,index=2

When testing in VM it had problem with permissions (I guess because of nested virtualisation) so I had to boot Linux from USB.

But that also didnt work as it always threw error qemu-system-x86_64: -cdrom /home/ondrej/virtio.iso: Failed to lock byte 100

I would very much like for Incus to be able to mount virtio drivers ISO as IDE device so Windows can get those drivers during installation like Proxmox does it. It would be useful for people that have problem rebuilding Windows with distrobuilder. Its also more noob friendly approach.

But I am stuck. If anyone can help I would be glad.
Thanks.

Is that a common issue? Distrobuilder supports many versions of Windows. If there’s a version that is not supported, one option is to add support to distrobuilder for that new version.

If you have trouble adding ISOs to the Windows VM setup, perhaps add them as storage volumes? (How to run a Windows virtual machine on Incus on Linux – Mi blog lah!)

No, I dont think it’s something common.
I just thought loading drivers from additional ISO like Proxmox does could be useful because it would allow Incus to also use non-rebuild Windows ISOs.

But I understand this is not priority. That’s why I tried to find some workaround that would enable this while not needing to spend dev time on this.

Thanks for suggestion. To explain the problem. I am trying to use Windows install iso without included virtio drivers. I can mount additional virtio drivers iso to Incus instance but Windows dont see it. Likely because Incus uses virtio-scsi based block devices when configuring QEMU and vanilla Windows dont have virtio drivers.

Basically, I need to somehow add iso to QEMU instance that uses some device that vanilla Windows can see. Likely IDE.

So I finally figured it out. I had to move virtio.iso to /run/incus/instance for QEMU to be able to use it.

So here is guide how you can install Windows in Incus without repacking it with Distrobuilder. I tried it only with Windows 10. In this guide I will call my Windows VM instance win10vm.

TLDR version:

  1. Set up Incus and Windows VM instance as per Simos guide
  2. Start and stop the Windows instance so folders in /run/incus/ are created
  3. Download Virtio ISO and copy it into /run/incus/win10vm
  4. Add cdrom to you VM with incus config set win10vm raw.qemu -- "-cdrom /run/incus/win10vm/virtio.iso"
  5. Start VM and run Windows installer.
  6. Install vioscsi.inf driver from your Virtio ISO. Path: /vioscsi/w10/amd64
  7. Finish installation.
  8. On desktop run virtio-win-guest-tools.exe from your Virtio ISO which will install all other missing Virtio drivers.
  9. Finally, remove the Virtio ISO cdrom from Incus config to prevent potential future problems. First stop the VM and then you can use incus config set win10vm raw.qemu -- ""

Detailed version:

  1. Download your Windows ISO. Install and prepare Incus. Configure your Windows VM instance according to Simos guide. Dont forget to add your Windows ISO into the instance.
  2. Start the instance and then stop it so Incus creates folder in /run/incus/
    incus start win10vm
    incus stop win10vm --force
  3. Download recent Virtio ISO. Link are in this Github repo or you can download it here.
  4. Move your Virtio ISO into directory /run/incus/win10vm/ so QEMU can use it. If you dont you will get Failed to lock byte 100 error. You will need sudo to get there.
  5. Add Virtio ISO to VM as cdrom using QEMU flag.
    Just run incus config set win10vm raw.qemu -- "-cdrom /run/incus/win10vm/virtio.iso"
    Alternatively you can edit Incus config file with incus config edit win10vm
    This mounts Virtio ISO as cdrom IDE device into your QEMU instance. We need to mount it as IDE (or SATA) device so Windows installer can read it.
  6. Now you can start the VM with vga console: incus start win10vm --console=vga and boot into cdrom by pressing any button during boot sequence.
  7. When I tried this in another VM (nested VM) mouse didnt work (likely because of missing virtio drivers) and I had to use keyboard unitil I installed all drivers at the end. On bare metal mouse worked.
  8. After clicking Install now you will see message A media driver your computer needs is missing. This is because without repacking our Windows ISO dont have Virtio drivers which are needed to see drives which are virtio-based in Incus instances. But we have those drivers in our Virtio ISO. So select Browse and navigate to CD Drive named virtio-win.
  9. Go to /vioscsi/w10/amd64 and confirm. You should see vioscsi.inf driver available. Select it and press next.
  10. If everything is OK we continue as with normal Windows install. Activate Windows. Select edition. Accept terms. Custom install. Select drive.
  11. In Select drive screen you can optionally load additional drivers with Load driver option. But it’s easier to just continue with the installation and install missing virtio drivers all at once using its own installer.
  12. Then Installing Windows will proceed.
  13. After that Windows will restart and your VGA console will close. Reconnect with incus console win10vm --type=vga Maybe it will restart/close multiple times.
  14. When we get into Windows configuration it’s possible network will not work because of missing drivers. This happened to me in nested VM. I dont know about bare metal install.
  15. Once configuration is finished go to your Virtio ISO (This PC → CD drive virtio-win) and install all other virtio drivers by running virtio-win-guest-tools.exe
  16. After all drivers are installed you just need to restart Windows.
  17. Thats all. Now everything should work correctly.
  18. After you’re done remove the Virtio ISO cdrom from Incus config to prevent potential future problems. First stop the VM and then you can use incus config set win10vm raw.qemu -- "" which will delete raw.qemu config.

Notes:
This is based on Proxmox guide
Maybe there are other ways to make virtio.iso available to QEMU but I dont know.
Hope this helps someone.
If someone can test this I will be grateful. I hope I didnt make any mistakes.

1 Like

Thank you so much for this, this was extremely helpful after distrobuilder failed on me… Much appreciated!