How to convert incus container image to incus vm

Incus
1

Hi everyone,
Is there a simple task to do it when publish the container to image? Or another way.
Regards.

2

Not a simple one that I know of, because the container image will be missing a kernel, a bootloader, and the incus agent - all the things needed to make it boot as a VM.

For Ubuntu images you can see the distrobuilder differences here. Look for all the steps that are qualified with

  types:
  - vm

and you’ll see all the steps which are missing from container images. Also, a VM image is a block device image containing partition table and filesystems, not a tarball of files.

Therefore, if you have a scripted build of a container, I’d suggest just rerunning the same scripts but starting from a --vm image.

In principle, it is possible to convert one to the other, and indeed I have managed it, but it was a very tricky process. Effectively it’s installing from scratch onto an empty disk.

First I imported an Ubuntu ISO image:

URL="https://releases.ubuntu.com/noble"
ISO="ubuntu-24.04-live-server-amd64.iso"
curl -fsSL "$URL/$ISO" -o "/var/tmp/$ISO"
incus storage volume import "$POOL" "/var/tmp/$ISO" ubuntu2404-server --type=iso

Then I created an empty VM, and booted it from the ISO:

incus create "$VM_NAME" --vm --empty -s "$POOL" -d root,size=10GiB -c limits.memory=8GiB -c limits.cpu=4
incus config device add "$VM_NAME" iso-volume disk pool="$POOL" source=ubuntu2404-server boot.priority=10
incus start "$VM_NAME"

Once booted, I could control it using incus exec or incus shell.

Then it got more complex: I had to partition the drive, format the root filesystem, mount it under /mnt, import the published container image and untar it under /mnt, bind-mount /{dev,sys,proc}, chroot to /mnt and perform the extra steps for installing the kernel, bootloader and incus-agent.

The reason I did it was because I wanted an image with ZFS root, and distrobuilder doesn’t publish one AFAICS, so I essentially followed the Ubuntu Root on ZFS guide with some changes, importing an existing container image instead of using debootstrap. I had to refer to the partitions by UUID rather than device path in order for it to be bootable. Finally, when it’s done, you remove the iso-volume device and cross your fingers that it boots.

It’s not an easy rabbit hole to go down, and I don’t know of any shortcuts or tools to automate this process. Therefore, if you want to publish a VM image, you’d most likely be better off creating a VM in the first place rather than a container.

EDIT: I just found that distrobuilder has a pack-incus subcommand. This might be able to do the work for you if you have the container image unpacked somewhere (but untested and I don’t know if it does the kernel/bootstrap installation steps)

distrobuilder pack-incus def.yaml /path/to/rootfs /path/to/output --vm
2 Likes
3

This is the script I use when I build my custom Debian images. It uses pack-incus twice. One for the container and the other for the VM. The first command builds the rootfs directory.

It might be possible to untar an existing container then use the third command.

#!/bin/bash

set -e

distrobuilder \
    --cache-dir=cache \
    --cleanup=false \
    build-dir debian-custom.yaml rootfs \
    --keep-sources=true \
    --sources-dir=sources \
    -o image.variant=cloud \
    -o image.architecture=amd64 \
    -o image.release=bookworm

distrobuilder \
    --cache-dir=cache \
    --cleanup=false \
    pack-incus debian-custom.yaml rootfs tarballs \
    -o image.variant=cloud \
    -o image.architecture=amd64 \
    -o image.release=bookworm \
    -o image.serial=container \
    --compression=gzip \
    --type=unified

distrobuilder \
    --cache-dir=cache \
    --cleanup=false \
    pack-incus debian-custom.yaml rootfs tarballs \
    -o image.variant=cloud \
    -o image.architecture=amd64 \
    -o image.release=bookworm \
    -o image.serial=virtual-machine \
    --compression=gzip \
    --type=unified \
    --vm
2 Likes
4

Thanks for the valuable informations both Brian and Jarrod.
Regards.

5

I tested the idea and it worked.

The only thing that might not work in your case is that I converted a system container to a VM using the same distrobuilder configuration I used to create the container in the first place.

Since I don’t know how your container was built, I can’t guarantee that the method will work for you. If you used a base image from images remote, then you can grab the distrobuilder configuration from the Jenkins build server. It should work.

You might need to adjust the raw image size used when creating the image. I think it is measured in bits.

-o targets.incus.vm.size=8589934592 \
1 Like
6

I havent tested yet but eventually it will work. I will try it later, thanks Jarrod.
Regards.

7

@jarrodu Seems like converting existing container to VM worked…but I see the BIOS screen in the console, hitting continue takes me to boot screen but it fails.

I used the Ubuntu YAML from here: lxc-ci/images/ubuntu.yaml at main · lxc/lxc-ci · GitHub

Not the correct one? I created the container (Ubuntu Noble) from the images: remote

Thanks for any input you have here.

8

I tested Debian Bookworm. I use Debian for nearly everything.

Are you trying to use the desktop variant?

9

No. I’m trying the cloud variant.

Can you share your Debian YAML (if it is okay for you)? That way I can refer to it when building.

This is what I see in the console, so somehow boot fails.

BdsDxe: loading Boot0006 "Ubuntu" from HD(1,GPT,459737F2-51C6-4F4C-B39B-75785F4E6840,0x800,0x32000)/\EFI\ubuntu\shimx64.efi
BdsDxe: starting Boot0006 "Ubuntu" from HD(1,GPT,459737F2-51C6-4F4C-B39B-75785F4E6840,0x800,0x32000)/\EFI\ubuntu\shimx64.efi
Error: write unix:@->/run/incus/neofetch-vm/qemu.console: broken pipe

I had to set a environment variable for the distrobuilder to start building

this one:

export DISTROBUILDER_ROOT_UUID=$(uuidgen)

Since this looks like somehow a fstab parameter, I think maybe fstab content is not generated correctly but, in that case, the error should be something different in the console and not the broken pipe thing.

10

I just tested the default Debian YAML.

It worked in the same way as I described earlier. :slight_smile:

My custom one is similar but contains a lot of personal customization.

You might want to create a new thread for your Ubuntu issue. I bet there are some people here that have more experience with those images.

11

Thanks @jarrodu

I managed to get it work - had to install distrobuilder from the edge channel to get the latest one.

Thanks again!

1 Like
12

Okay. That could explain it. I remember having to compile the binary myself because of some error. I don’t remember what it was.

I commented on a related issue and asked for the binaries to be added to the GitHub release page. Feel free to add your experiences there. It might help anyone trying to solve the same problem.