Windows Server 2022 VM from published image fails UEFI boot (BdsDxe: Not Found, no filesystem visible in Boot From File)

[wajatmaka (Dwiyan)]

Windows Server 2022 VM from published image fails UEFI boot (BdsDxe: Not Found, no filesystem visible in Boot From File)

Environment

• Incus version: 7.2 (Zabbly packages)
• Host OS: Debian 13
• Topology: 4-node cluster (Lenovo ThinkAgile HX630)
• Storage backend: LVM (lvmcluster with sanlock/lvmlockd) on shared QNAP iSCSI LUNs, multipath via multipathd
• Guest: Windows Server 2022, UEFI (default security.secureboot settings, no CSM)
• Projects: instance launched in a non-default project (Staging), image published from a VM in the same cluster

What I did

1. Installed Windows Server 2022 in an Incus VM (UEFI). Installation and boot worked fine on the original VM.
2. Published the VM as an image:

   incus stop <source-vm>
   incus publish <source-vm> --alias win2022-template
   

3. Launched a new instance from the published image:

   incus launch win2022-template trim-vm-stg-t1-s21-h2h-permata --vm ...
   

Problem

The new instance fails to boot. EDK2 output:

BdsDxe: failed to load Boot0002 "UEFI QEMU QEMU HARDDISK " from PciRoot(0x0)/Pci(0x1,0x1)/Pci(0x0,0x0)/Scsi(0x0,0x1): Not Found

>>Start PXE over IPv4.

The disk itself is detected at SCSI level (it appears in the Boot0002 device path), but the firmware cannot find a bootable EFI application on it and falls through to PXE.

What I’ve tried / observed

1. Expected NVRAM behavior: I understand that UEFI NVRAM variables are per-instance and not carried in a published image, so the original “Windows Boot Manager” entry pointing to \EFI\Microsoft\Boot\bootmgfw.efi is gone on the new instance, and the firmware falls back to the removable-media path \EFI\BOOT\BOOTX64.EFI, which Windows does not install on fixed disks. My plan was to boot manually once via the firmware menu and then install the fallback loader.

2. Firmware menu → Boot Maintenance Manager → Boot From File: here it gets worse — the File Explorer is completely empty. No volumes are listed at all, so I cannot even manually browse to bootmgfw.efi. This suggests the firmware sees no readable FAT filesystem (i.e. no ESP) on the disk, not just a missing boot entry.

3. Inspecting the volume from the host: the instance’s block volume exists as expected:

   # lvs | grep -i permata
   virtual-machines_Staging_trim--vm--stg--t1--s21--h2h--permata          vg-qnap-hdd-01  -wi------k  500.00m
   virtual-machines_Staging_trim--vm--stg--t1--s21--h2h--permata.block    vg-qnap-hdd-01  -wi------k   66.01g
   

   Partition table of the .block LV (after lvchange -aey -K):

   <PASTE: fdisk -l /dev/vg-qnap-hdd-01/virtual-machines_Staging_trim--vm--stg--t1--s21--h2h--permata.block>
   

   Contents of the first partition (via kpartx), if mountable:

   <PASTE: ls -laR of mounted ESP, or mount error>
   

4. Instance config:

   <PASTE: incus config show trim-vm-stg-t1-s21-h2h-permata --expanded>
   

5. Source VM: the original VM the image was published from booted normally before publishing. (If still available: same fdisk -l output of the source VM’s .block LV for comparison.)

   <PASTE if available>
   

Questions

1. Is incus publish expected to preserve the full GPT layout including the EFI System Partition for VM images, or are there known cases (e.g. with LVM block volumes / clustered LVM) where the image ends up without a valid partition table or ESP?
2. Is there a supported way to have Windows instances from a published image boot without manual firmware intervention — i.e. does Incus regenerate any default boot entries, or is installing a fallback \EFI\BOOT\BOOTX64.EFI inside the template the recommended approach?
3. Could volume.size / rounding on the target pool truncate the image on unpack (original disk size vs. 66.01g on the new instance), cutting off the backup GPT or data?
4. Any recommended way to debug what incus publish actually captured (e.g. inspecting the unified tarball / rootfs.img of the published image)?

Config Yaml VM

 incus config show vm_name --expanded --project Staging
architecture: x86_64
config:
  raw.qemu: -boot menu=on,splash-time=10000
  volatile.base_image: 2783afba8794a00a7ebe3b0ef7c4122dce6579767c2654f651cdc438e5cb2e6f
  volatile.cloud-init.instance-id: 34e7c79f-a2bf-480d-84cb-a75cd42ba81d
  volatile.eth809.hwaddr: 10:66:6a:d2:44:33
  volatile.last_state.power: STOPPED
  volatile.last_state.ready: "false"
  volatile.uuid: ad838e63-0b29-4977-85e8-71537a6f50c8
  volatile.uuid.generation: ad838e63-0b29-4977-85e8-71537a6f50c8
  volatile.vm.rtc_adjustment: "-2"
  volatile.vm.rtc_offset: "0"
  volatile.vsock_id: "822980815"
devices:
  eth809:
    nictype: bridged
    parent: br-vm
    type: nic
    vlan: "xxx"
  root:
    path: /
    pool: pool-qnap-hdd-01
    size: 66GiB
    type: disk
ephemeral: false
profiles:
  - os-trim-win2022
  - net-vlanxxxx-stg-thirdparty
stateful: false
description: ""

Happy to provide any additional logs (incus info --show-log, qemu.log, image metadata). Thanks!

[stgraber (Stéphane Graber)]

@wajatmaka

Is incus publish expected to preserve the full GPT layout including the EFI System Partition for VM images, or are there known cases (e.g. with LVM block volumes / clustered LVM) where the image ends up without a valid partition table or ESP?

It will preserve the disk as it is. The only thing that happens to the GPT table is a resize if the VM created from the image is larger than the original one.

@wajatmaka

Is there a supported way to have Windows instances from a published image boot without manual firmware intervention — i.e. does Incus regenerate any default boot entries, or is installing a fallback \EFI\BOOT\BOOTX64.EFI inside the template the recommended approach?

Incus doesn’t generate any UEFI boot entry on its own. I’m a bit confused by you hitting issues with creating Windows images though as I’ve done that before without much of an issue.
Did you use sysprep prior to publishing your image? If not, that may well be your problem.

@wajatmaka

Could volume.size / rounding on the target pool truncate the image on unpack (original disk size vs. 66.01g on the new instance), cutting off the backup GPT or data?

No, any rounding of partition/disk sizes will round up, never down.

@wajatmaka

Any recommended way to debug what incus publish actually captured (e.g. inspecting the unified tarball / rootfs.img of the published image)?

You can incus image export, then unpack the image and inspect it.

It looks like your LLM was trying to have you do more debugging in step 3), 4) and 5) but you forgot to include that information which could indeed be useful.

We have found a few issues with LVM cluster and its internal QCOW2 layer recently which had to do with disk size handling, so it’s not impossible that there is another issue affecting you here. The information in those steps 3), 4) and 5) would likely help with that.

Also make sure your Incus 7.2 packages are up to date as we’ve pushed a bunch of fixes last weekend.

[stgraber (Stéphane Graber)]

I found the issue and sent a fix for it upstream.

Various bugfixes

[wajatmaka (Dwiyan)]

Ok many thanks @stgraber

[wajatmaka (Dwiyan)]

With incus 7.3 issue already resolved