BootC + Incus Image issues

Hey All, I’ve ran into an issue and I’m stumped.

I’m building qcow2 images using BootC with bootc-image-builder and I keep running into the issue of:

Error: Failed instance creation: Failed creating instance from image: Failed to run: /usr/sbin/sgdisk --move-second-header /var/lib/incus/storage-pools/default/images/ffdd2ab56591b3febdf70aea12e2bb8fd7ba0668f2dec5d8e7291f6f1f9d0b26/root.img: exit status 4

I don’t know what’s special about my setup that could be causing this, I’ve ruled out that it’s a version incompatibility issue between incus 6.19.1 (where image is created) and 6.21 (where I’m trying to launch it). I’ve mounted the qcow2 image and it doesn’t give me any errors in fdisk when verified. I’ve compared it to similar images I create for VyOS and those work while these don’t.

What other information would you like to figure this out?

Seems like it has some similarities to Can’t create VM instance on a thinpool (Main and backup partition tables differ!) but I’m not using thinpools, I’m attempting to launch it on btrfs.

it could be that the GPT partition table generated by bootc is a bit odd or that it’s partitioned in a way that doesn’t leave space for the secondary header?

Here’s the output from a gdisk verify:

Warning: There is a gap between the main partition table (ending sector 33)
and the first usable sector (2048). This is helpful in some exotic configurations,
but is unusual. The util-linux fdisk program often creates disks like this.
Using 'j' on the experts' menu can adjust this gap.

Caution: Partition 4 doesn't end on a 2048-sector boundary. This may
result in problems with some disk encryption tools.

No problems found. 0 free sectors (0 bytes) available in 0
segments, the largest of which is 0 (0 bytes) in size.

Sounds like you’re probably correct. I could probably attach it directly to an instance manually and it’d work just fine. Just can’t bake it into an image the way I’d hoped. I’ll keep trying on it every now and again but I’ve more or less just given up and went to use the ISO installer that it can generate.

Is there a way to tell Incus to not try expand the disk for root and just use the image that is provided?

Maybe try importing the image directly as an instance by using incus-migrate instead?

I could do that, but I’m trying to get this to where I can just import it as an image and run it on another host by pulling it from the build server I’ve put together. I haven’t yet found command line arguments for incus-migrate that let it run in a non user-interactive way leaving my build script in an awkward spot. I’ll see if I can take this to the bootc-image-builder project and see if there’s a way to make it build a more standard image.

I did some more testing. I manually extended a raw version of the image with qemu-img and ran the sgdisk –move-second-header disk.raw and it ran without issue. I’m thinking it has to be something it doesn’t like once it tries to push it onto the block device Incus creates in the btrfs pool.

Created a bug for bootc-image-builder: QCOW2 and RAW Disk Image Iregularties (That break in Incus) · Issue #1195 · osbuild/bootc-image-builder · GitHub

It doesn’t help that all error code 4 for sgdisk means “An error prevented saving changes” which is amazingly useful since it doesn’t always tell you what error caused the issue. When I googled for my problem initially I kept finding people who would have this issue and it was saying that the partition table data area was overlapping with the last partition. That isn’t what this one is saying.

I found the code, how hard would it be to set a flag on an image that it won’t run sgdisk on it’s own?