I’ve encountered a strange problem with Incus 6.10.1 after upgrade. I think it’s probably related to my setup because I cannot find anything similar here, in issues or via search services. Basically I can’t create VM instances like this:
Launching testthin0
Error: Failed instance creation: Failed creating instance from image: Failed to run: /usr/sbin/sgdisk --move-second-header /dev/vg0/images_6dc6e0522da81db5c254e14e038860636e1e26edd3e6748d4a5f1d32df7c16aa.block: exit status 4 (Warning! Main and backup partition tables differ! Use the ‘c’ and ‘e’ options
on the recovery & transformation menu to examine the two tables.
Warning! One or more CRCs don’t match. You should repair the disk!
Main header: OK
Backup header: OK
Main partition table: ERROR
Backup partition table: ERROR
Aborting write operation!)
I have four different storage pools:
thin0 - thinpool storage on 2xNVMe drives in RAID 1
thin1 - thinpool storage on 4xSATA SSD in RAID10
pool0 - btrfs storage on the NVMe drives
pool1 - btrfs storage on the SATA drives
I can create VM just fine on both btrfs pools. The error is returned for both thinpools. I tested multiple images with the same result. Image storage is on pool0. I tested pool1 to check if the drives are affected but I get the same problem.
I have no idea how to debug this. It seems like the image is somehow broken but I have no idea why.
The issue is on my home server running Fedora Server. I have my notebook with similar setup (thinpool storage, same version of Incus, Fedora Desktop) and everything works just fine there. That leads me to conclusion that the problem is in my setup on the home server
Can I please ask you to help me to debug this issue?
Just an update. I tested my notebook again with, as I mentioned, almost the same software setup. The VM instance can be created but it won’t boot. I think the issue is either related to Fedora or latest kernel 6.13.
Before I answer I tried to return to kernel 6.12.7 and the issue remained.
I also tried to create a new thinpool (in remaining space of one of the RAID arrays) and it’s the same result.
Then I tried to image.os set to windows to disable IOMMU (not sure it was even enabled), same result.
And one note, one of the existing VMs, that was perfectly fine before, didn’t boot up after the issue occurred - the same behaviour as on my notebook - just stuck during boot.
Today, without any change in the config, hw or system it started behaving the same as on my notebook. The VM instance is created but refuses to boot. The boot process is stuck here:
I switched to Incus from Fedora’s main repo which is version 6.8. It still didn’t work but I noticed that if I delete all images from all projects and let Incus to download them again it starts working as before.
So I think Incus 6.10 treats images differently during importing into the local storage and as result those images are not compatible with whatever is specific for Fedora.
The deleted images and the newly downloaded images had the same fingerprints.
What I also noticed is that incus export successfully exported an instance and when I imported it on another machine the disk image was unbootable and data damaged. I would say it’s hardware problem but I experience the first or the other issue on five different machines with Fedora and Debian. Both issues always occurs on LVM thinpool.
I think Incus somehow damages data in the disk image during importing on LVM thinpools. I have TBs of data on instances created in past and there is no issue with it. It always gets damaged when Incus imports data into a fresh thinpool volume. Either with import command or launch command. I am pretty sure that if I install the system from ISO it will be ok.
Update: I installed Debian on a new VM instance with no issue.
I think I found a solution. I created those thinpools manually and it seems that zeroing needs to be enabled. I guess Incus enables this for its own thinpools.
I haven’t tested it much but this did the trick for my initial runs:
