Excuse me if these are dumb questions, but I may be overlooking something obvious.
1. Importing existing VM images
What’s the simplest way to take a raw VM image file, say foo.qcow2 or foo.img (raw), and run it under Incus as a VM?
(1) Thanks to another user’s posting, I found the convoluted path of going via an incus “image”: create image, launch from image, delete image.
cat <<EOS >metadata.yaml
architecture: x86_64
creation_date: $(date +%s)
properties:
description: custom virtual machine
os: custom
release: $(date +%Y%m%d)
EOS
tar -czf metadata.tar.gz metadata.yaml
incus image import metadata.tar.gz foo.qcow2 --alias custom1
incus create custom1 myvm -s default
incus image delete custom1
incus start --console myvm
Am I missing something simpler? Things I’ve tried:
(2) I can create an empty VM:
$ incus create myvm --empty --vm -s default --device root,size=30GiB
Creating myvm
…but I don’t see an official way to initialize the storage from an existing VM file. I guess depending on the storage type I could overwrite it in-place:
# ls -ls /var/lib/incus/storage-pools/default/virtual-machines/myvm/root.img
0 -rw------- 1 root root 32212254720 Mar 25 12:35 /var/lib/incus/storage-pools/default/virtual-machines/myvm/root.img
# qemu-img convert -O raw foo.qcow2 /var/lib/incus/storage-pools/default/virtual-machines/myvm/root.img
# incus start --console myvm
(Although the size of this root.img may now no longer match what incus thinks it is)
(3) I see that it’s possible to export a VM instance and reimport it. However the export gives a tarball with a large bundle of files. It seems rather hard to replicate this structure to allow a simple image file to be imported via incus import
(4) Storage volumes can also be exported and imported. However, volume export/import only applies to “custom” volumes on VM (which represent additional drives attached to an instance), rather than the primary drive.
(5) There’s incus-migrate which is not supplied with incus by default. It’s a very manual process:
-
It refuses to run unless it’s run as root (this should not be necessary as I’m just reading a VM file)
$ ./bin.linux.incus-migrate.x86_64 Error: This tool must be run as root -
It won’t talk to the local incus server over a Unix socket:
$ sudo ./bin.linux.incus-migrate.x86_64 Please provide Incus server URL: unix:// Error: Failed to get remote certificate: Get "unix://:8443": unsupported protocol scheme "unix"As a result you have to jump through the trust setup hoops.
However, in the end I did manage to transfer an image this way, although it was not straightforward. This was the result:
# file /var/lib/incus/storage-pools/default/virtual-machines/myvm/root.img
/var/lib/incus/storage-pools/default/virtual-machines/myvm/root.img: QEMU QCOW2 Image (v3), 42949672960 bytes
(I note this has been imported as qcow2 rather than raw)
2. Booting non-UEFI images
Having imported an image via incus-migrate, I’m having trouble booting it:
BdsDxe: failed to load Boot0001 "UEFI QEMU QEMU HARDDISK " from PciRoot(0x0)/Pci(0x1,0x1)/Pci(0x0,0x0)/Scsi(0x0,0x1): Not Found
>>Start PXE over IPv4.
...
Tthis image does boot under libvirt just fine, and also under GNS3. I’ve checked it has security.secureboot=no.
# incus config show -e myvm
architecture: x86_64
config:
security.secureboot: "false"
volatile.cloud-init.instance-id: c865bb8b-dae4-461c-9e3b-19c9b4061f6a
volatile.eth0.host_name: tap182b5fca
volatile.eth0.hwaddr: 00:16:3e:38:6c:71
volatile.last_state.power: RUNNING
volatile.uuid: f1c132af-8ef5-4f44-81de-bbe15eff94d9
volatile.uuid.generation: f1c132af-8ef5-4f44-81de-bbe15eff94d9
volatile.vsock_id: "3686203789"
devices:
eth0:
name: eth0
network: incusbr0
type: nic
root:
path: /
pool: default
type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""
I’m wondering if I have to override some settings in qemu.conf, but I don’t know what those might be.
Oddly, I can’t see any reference to the /var/lib/incus/... storage directory in the qemu process. It’s not on the qemu command line:
incus 168039 27.6 0.1 1686292 109816 ? Sl 13:26 5:37 /opt/incus/bin/qemu-system-x86_64 -S -name myvm -uuid f1c132af-8ef5-4f44-81de-bbe15eff94d9 -daemonize -cpu host,hv_passthrough -nographic -serial chardev:console -nodefaults -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=allow,resourcecontrol=deny -readconfig /run/incus/myvm/qemu.conf -spice unix=on,disable-ticketing=on,addr=/run/incus/myvm/qemu.spice -pidfile /run/incus/myvm/qemu.pid -D /var/log/incus/myvm/qemu.log -smbios type=2,manufacturer=LinuxContainers,product=Incus -runas incus
And it’s not in qemu.conf:
# grep /var/lib /run/incus/myvm/qemu.conf
path = "/var/lib/incus/devices/myvm/config.mount"
# grep img /run/incus/myvm/qemu.conf
#
3. Converting a custom volume to an incus image or a VM
Related to question 1: I am trying to build VM images, using an incus VM as the builder, with a second disk attached (i.e. custom volume which appears as /dev/sdb) that I mount and build the image into.
Once I’ve done this, I’m trying to find what’s the simplest way to:
- Convert this custom volume into either an incus image or an incus VM instance, so I can boot it for testing?
- Convert this custom volume into a qcow2 file that I can take elsewhere to run on a different VM platform?
I can’t see a direct way (from the incus command line) to publish a custom storage volume as an image; nor to launch a VM with a custom storage volume as its root.
option 1: Given dir storage, I guess I could pick up /var/lib/incus/storage-pools/default/custom/POOL_VOL/root.img directly. That’s very unofficial. It appears to be a plain image, rather than qcow2:
# file /var/lib/incus/storage-pools/default/custom/default_testzfs/root.img
/var/lib/incus/storage-pools/default/custom/default_testzfs/root.img: DOS/MBR boot sector; partition 1 : ID=0xee, start-CHS (0x0,0,2), end-CHS (0x3ff,255,63), startsector 1, 41943039 sectors, extended partition table (last)
option 2: I can do incus storage volume export, which gives me:
$ tar -tvzf default_testzfs.tgz
-rw-r--r-- root/root 325 2024-03-25 10:06 backup/index.yaml
-rw------- root/root 21474836480 2024-03-25 10:06 backup/volume.img
But when I untar it, it loses its sparseness:
$ tar -S -xvzf default_testzfs.tgz
backup/index.yaml
backup/volume.img
$ ls -ls backup/
total 20971548
4 -rw-r--r-- 1 nsrc nsrc 325 Mar 25 10:06 index.yaml
20971544 -rw------- 1 nsrc nsrc 21474836480 Mar 25 10:06 volume.img
$ du -sch backup
21G backup
21G total
I can recover that by sending it through qemu-img convert, but it does require a potentially very large temporary file first. (tar is supposed to handle sparse files, so I wonder if that’s a bug?)
Thanks,
Brian.