How to use incus-simplestreams with images published from containers

[votsalo]

I tried to use incus-simplestreams to make an images website from images that I create from containers.

When I publish a container into an image and then export the image, I get a single .tar.gz file containing rootfs/, metadata.yaml, and templates/.

incus-simplestreams seems to require a squashfs file with rootfs, and a tar.xz file with metadata.yaml and template/. So I had to extract rootfs from the .tar.gz file and convert it to a squashfs file, using mksquashfs. I also had to unpack metadata.yaml and templates/ from the .tar.gz file and repack them into a tar.xz file.

Is there a way to simplify this process? Can I publish/export the image from incus in the format required by incus-simplestreams?

[toby63]

Not so helpful maybe, but Image format - Incus documentation mentions two types of image formats (a unified tarball and split tarballs (what you seem to need)), but I can’t find any information how to tell the incus image export command to use a different format.

Maybe a missing feature.

[toby63]

Interesting:
Distrobuilder supports defining the type:

The --type flag can take one of the
following values:

• split (default)
• unified

As an alternative you could use an incus server:

Default image server

How to expose Incus to the network

[toby63]

Found this:

Best practice create/share simplestreams from lxd image export

Yeah, we’ve considered adding support for publishing to split images for a while but it’s never really made it to the top of the list of features to implement Scripting the repacking process should be reasonably straightforward though: tar xvf unified.tar.gz mksquashfs rootfs/ rootfs.squashfs -noappend -comp xz -b 1M tar Jcf metadata.tar.xz metadata.yaml templates/ You’ll then get your metadata.tar.xz and rootfs.squashfs for a normal split image.

It is indeed a missing feature, but as stgraber said there, you should be able to automate it.

[votsalo]

Using Distrobuilder is a whole different procedure than publishing images from containers. The images in images.linuxcontainers.org are probably all built using Distrobuilder.

Incus can import multiple image formats but simplestreams only supports the split format?

I’ve avoided sharing images via an incus server, because:

• I don’t want to dedicate an incus host just for serving images
• I don’t think it’s a good idea to use a production server with private instances to also serve public images. Even if it is technically feasible, I would worry about misconfiguring it.
• I assume an incus server manages images of a single architecture, but I use both x86 and arm images.

I’ve been wanting to have my own simplestreams server for a long time. incus-simplestreams makes this feasible.

[toby63]

I added a feature request ([Feature Request] add --type flag to incus image export · Issue #871 · lxc/incus · GitHub), I assume that stgraber will add the request to “easy issues” so it will probably be low-priority, but if some volunteers and maintainers see it, who knows maybe someone will implement it.

[candlerb (Brian Candler)]

Aside: although incus publish --format split was added, it also looks like incus-simplestreams now supports publishing a unified image, in which case it’s not needed for this purpose.

$ incus-simplestreams add --help
...
  If one argument is specified, it is assumed to be a unified image,
  with both the metadata and rootfs in a single tarball.

  Otherwise, it is a split image (separate files for metadata and rootfs/disk).

Usage:
  incus-simplestreams add <metadata tarball> [<data file>] [flags]

Ah here:

commit 71c463f8b9981b1e8c8615387aa84d813be26059
Merge: d57a1ecac a3da639ef
Author: Stéphane Graber <stgraber@stgraber.org>
Date:   Tue Jul 9 04:41:36 2024 +0200

    Merge pull request #963 from melato/incus-simplestreams-add

    incus-simplestreams add: can use unified or split image files