VM image converted from qcow2 does not boot

Incus
1

I have converted an Ubuntu 20.04 ova image to qcow2 and from there to raw. The qcow2 image boots up nicely with qemu:

qemu-system-x86_64 -enable-kvm -m 32G -drive file=casper-tooflow-2021.qcow2,if=virtio -boot c

The .raw image is imported with incus-migrate but booting ends with an error message

BdsDxe: failed to load Boot0001 "UEFI QEMU QEMU HARDDISK " from PciRoot(0x0)/Pci(0x1,0x1)/Pci(0x0,0x0)/Scsi(0x0,0x1): Not Found

>>Start PXE over IPv4.

and finally drops to a shell

UEFI Interactive Shell v2.2
EDK II
UEFI v2.70 (EDK II, 0x00010000)
Mapping table
      FS0: Alias(s):HD0a1b:;BLK1:
          PciRoot(0x0)/Pci(0x1,0x1)/Pci(0x0,0x0)/Scsi(0x0,0x1)/HD(1,MBR,0x8FF35EEC,0x800,0x100000)
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Pci(0x0,0x0)/Scsi(0x0,0x1)
     BLK2: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Pci(0x0,0x0)/Scsi(0x0,0x1)/HD(2,MBR,0x8FF35EEC,0x100FFE,0x1FEFF002)
     BLK3: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Pci(0x0,0x0)/Scsi(0x0,0x1)/HD(2,MBR,0x8FF35EEC,0x100FFE,0x1FEFF002)/HD(1,MBR,0x00000000,0x101000,0x1FEFE800)


Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell>

Launching vm’s from repository works fine. Here’s information about storage setup:

/mnt # incus storage list
+------+--------+--------+-------------+---------+---------+
| NAME | DRIVER | SOURCE | DESCRIPTION | USED BY |  STATE  |
+------+--------+--------+-------------+---------+---------+
| pool | zfs    | rpool  |             | 7       | CREATED |
+------+--------+--------+-------------+---------+---------+
/mnt # incus storage show pool
config:
  source: rpool
  volatile.initial_source: rpool
  zfs.pool_name: rpool
description: ""
name: pool
driver: zfs
used_by:
- /1.0/images/1cfbc6e7a4d547b1fd8580d247c45492034e2978128688dd411c558ab2e3ee1e
- /1.0/images/b2d6c85fbfc7555328d24c7e132d314f95e391ef133639b750b9e957c298f441
- /1.0/images/e11d95790cc9513e205ee0dc26f55959783e7e49a8e46e47c7e279f22aa420b6
- /1.0/instances/caspervm
- /1.0/instances/clear-chicken
- /1.0/instances/debian-base
- /1.0/instances/first
status: Created
locations:
- none

The host is running Alpine linux on an old 12 core 96 GB RAM HP Gen 7 (no UEFI) server.

Any suggestions?

2

Try setting security.csm=true and security.secureboot=false as your manual QEMU was running in BIOS mode but Incus defaults to UEFI.

3

Many thanks for the suggestion, however when I

/home/yll # incus config set caspervm  security.csm=true
/home/yll # incus config set caspervm  security.secureboot=false

I get an error

qemu-system-x86_64:/run/incus/caspervm/qemu.conf:46: Could not open '/usr/share/OVMF/OVMF_CODE.csm.fd': No such file or directory

There is a file /usr/share/OVMF/OVMF_CODE.fd but it seems that there is no Alpine package containing /usr/share/OVMF/OVMF_CODE.csm.fd.

Unfortunately I am not familiar with OVMF, as other distros have this file, could I just copy it there or are there dependencies to the distribution?

Edit:
The command line starting qemu is

/usr/bin/qemu-system-x86_64 -S -name caspervm -uuid 154196d1-3597-4d1a-8501-e4a18b83effe -daemonize -cpu host,hv_passthrough -nographic -serial chardev:console -nodefaults -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=allow,resourcecontrol=deny -readconfig /run/incus/caspervm/qemu.conf -spice unix=on,disable-ticketing=on,addr=/run/incus/caspervm/qemu.spice -pidfile /run/incus/caspervm/qemu.pid -D /var/log/incus/caspervm/qemu.log -smbios type=2,manufacturer=LinuxContainers,product=Incus -runas nobody

From a quick look -smbios type=2,manufacturer=LinuxContainers,product=Incus causes the problem, can it be disabled somehow? I tried to run it separately and it seemed to start OK.

4

The smbios isn’t going to be the problem, the problem is the other 500 or so options we feed through the config file :slight_smile:

You probably don’t want to use “OVMF_CODE.csm.fd”, instead you’d want “seabios.bin” which we also look for in /usr/share/OVMF. It may be at a different location on your system so you may need to copy it over.

5

OK, I

# ln -s /usr/share/seabios/bios.bin /usr/share/OVMF/seabios.bin

Still I get

qemu-system-x86_64:/run/incus/caspervm/qemu.conf:46: Could not open '/usr/share/OVMF/OVMF_CODE.csm.fd': No such file or directory

There are

# ls /usr/share/seabios
bios-256k.bin      bios-coreboot.bin  bios-csm.bin       bios.bin

I tried to link bios-csm.bin but got the same missing /usr/share/OVMF/OVMF_CODE.csm.fd error.

It seems strange that even if qemu is run under the hood, additional files are needed for incus… Of course I could run plain qemu but incus is a cleaner solution.

6

Can you try with a new VM?

incus launch --empty --vm foo -c security.csm=true -c security.secureboot=false

7 
# incus launch --empty --vm foo -c security.csm=true -c security.secureboot=false
Launching foo
Error: Failed creating instance record: Failed initializing instance: Invalid devices: Failed detecting root disk device: No root device could be found

Maybe I misunderstood?

8

Ah, you don’t appear to have a storage pool in your default profile.
Add --storage pool to that command then.

9

OK, of course, :slight_smile:

/mnt # incus info --show-log foo
Name: foo
Status: ERROR
Type: virtual-machine
Architecture: x86_64
PID: 32664
Created: 2024/08/05 23:30 EEST
Last Used: 2024/08/05 23:31 EEST

Resources:
  Processes: 0
  Disk usage:
    root: 56.00KiB

Log:

KVM internal error. Suberror: 1
extra data[0]: 0x0000000000000000
extra data[1]: 0x0000000000000030
extra data[2]: 0x0000000000000184
extra data[3]: 0x0000000000000000
extra data[4]: 0x0000000000000000
extra data[5]: 0x0000000000000000
emulation failure
EAX=00000000 EBX=00000000 ECX=00000010 EDX=000f113e
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00007004
EIP=000a0000 EFL=00010046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
SS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     000f6740 00000037
IDT=     000f677e 00000000
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
10

That looks like it didn’t like your firmware file :slight_smile:

Can you try with the one from https://dl.stgraber.org/seabios.bin ?
That’s the one I build in my Incus packages here GitHub - zabbly/incus: Incus package repository

11

With your seabios.bin foo starts:

~ # incus launch --empty --vm foo -c security.csm=true -c security.secureboot=false -s pool
Launching foo

~ # 
~ # incus info --show-log foo
Name: foo
Status: RUNNING
Type: virtual-machine
Architecture: x86_64
PID: 25591
Created: 2024/08/06 09:28 EEST
Last Used: 2024/08/06 09:28 EEST

Resources:
  Processes: -1
  Disk usage:
    root: 56.00KiB
  Network usage:
    eth0:
      Type: broadcast
      State: UP
      Host interface: tapfce4f053
      MAC address: 00:16:3e:9b:c0:65
      MTU: 1500
      Bytes received: 1.92kB
      Bytes sent: 1.37kB
      Packets received: 12
      Packets sent: 4
      IP addresses:
        inet:  10.136.206.113/24 (global)
        inet6: fd42:d292:1527:1e77:216:3eff:fe9b:c065/64 (global)

Log:

But when starting the VM, it still seems to be wanting to use OVMF_CODE.csm.fd and I get the same error:

~ # incus start caspervm --console
Error: Failed to run: forklimits limit=memlock:unlimited:unlimited fd=3 fd=4 -- /usr/bin/qemu-system-x86_64 -S -name caspervm -uuid 154196d1-3597-4d1a-8501-e4a18b83effe -daemonize -cpu host,hv_passthrough -nographic -serial chardev:console -nodefaults -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=allow,resourcecontrol=deny -readconfig /run/incus/caspervm/qemu.conf -spice unix=on,disable-ticketing=on,addr=/run/incus/caspervm/qemu.spice -pidfile /run/incus/caspervm/qemu.pid -D /var/log/incus/caspervm/qemu.log -smbios type=2,manufacturer=LinuxContainers,product=Incus -runas nobody: : exit status 1
Try `incus info --show-log caspervm` for more info
~ # incus info --show-log caspervm
Name: caspervm
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2024/08/03 21:01 EEST
Last Used: 2024/08/05 11:05 EEST

Log:

qemu-system-x86_64:/run/incus/caspervm/qemu.conf:46: Could not open '/usr/share/OVMF/OVMF_CODE.csm.fd': No such file or directory
12

For the existing VM, toggle security.csm back to false and then to true again, that should have it reset the firmware record to point to seabios.

  • incus config set caspervm security.csm=false
  • incus config set caspervm security.csm=true
13

Did that and the VM booted!

eaBIOS (version rel-1.16.3-0-ga6ed6b7)
Machine UUID 154196d1-3597-4d1a-8501-e4a18b83effe

iPXE (http://ipxe.org) 05:00.0 CA00 PCI2.10 PnP PMM+3EFD3360+3EF33360 CA00
                                                                               
Booting from Hard Disk...

However, I can’t start a shell:

/home/yll # incus shell caspervm
Error: VM agent isn't currently running

And the log is not promising (-1 processes):

/home/yll # incus info --show-log caspervm
Name: caspervm
Status: RUNNING
Type: virtual-machine
Architecture: x86_64
PID: 26101
Created: 2024/08/03 21:01 EEST
Last Used: 2024/08/06 19:42 EEST

Resources:
  Processes: -1
  Disk usage:
    root: 122.12GiB
  Network usage:
    eth0:
      Type: broadcast
      State: UP
      Host interface: tap76941b48
      MAC address: 00:16:3e:92:37:c4
      MTU: 1500
      Bytes received: 76.75MB
      Bytes sent: 327.02kB
      Packets received: 8732
      Packets sent: 4147
      IP addresses:
        inet:  10.136.206.77/24 (global)
        inet6: fd42:d292:1527:1e77:216:3eff:fe92:37c4/64 (global)
        inet6: fe80::ac4b:5524:fa2c:4b8d/64 (link)

Log:
14

That’s normal, both the extended info in incus info and things like incus exec and incus file rely on our agent running in the VM.

As you just converted that VM from another platform, it’s not going to have the agent in there.
Hopefully you can connect to it through the text console or over the network and then go through the steps to add the agent.

15

Aha, OK, makes sense.

However it seems that ssh connection is refused and console does not respond. What the number of processes of -1 mean? Just that incus cannot figure it out because the agent is not running?

So there is a chicken-and-egg problem, I have to think about how to solve this. Maybe trying to add the agent in a qemu session, is the file system containing the agent
available outside of incus?

Should the bootup messages show up in incus console, only Booting from Hard Disk... is shown?

16

incus console caspervm --type vga should get you the graphical console if the text one isn’t working.

And yeah, -1 is just because we can’t tell the number of processes inside the VM without the agent. The rest we can mostly guess by looking at the host data.

17

Indeed, I installed xfce4, vncserver and virt-viewer and got the desktop!

I went to install the agent but it did not start. In the debugging process I accidentally shut down the server and have to drive to the computer center tomorrow :-D. But there is progress and it is very relieving to know that the VM is really running!

Many thanks!