Just out of curiosity, would it be possible for lxd images to be published to an image stream that you operate?
Itâs certainly technically feasible, though not something Iâd really recommend.
They certainly wouldnât qualify for publishing on our own image server as all images on our server are built and tested on our infrastructure, which those obviously arenât.
We also wouldnât be very likely to recommend folks use a random 3rd party re-published tree as that 3rd party would be able to inject whatever they want in the images or image metadata.
But Canonical isnât a random third party, is it? Why would we inject whatever into the image streams users are requesting?
I also donât quite understand what assurance you would have about the lxd metadata format even if you could read the code â couldnât it still change? And even if you coudlnât read the code, and it changed, and your users asked for it, wouldnât that still be a thing to enable?
What Iâm saying is that a customer doing incus remote add ubuntu-images https://cloud-images.ubuntu.com will obviously expect those images to have been built by Canonical and will therefore have a trusted source for those. Thatâs ideally what weâd like to see work here.
But if the Canonical simplestreams server canât publish the needed Incus metadata and we need to instead have a 3rd party web server pull the images from https://cloud-images.ubuntu.com and re-generate the simplestreams data for them. That would then have our users need to do incus remote add ubuntu-images https://some-random-url.com instead, which wouldnât really be too trustworthy as whoever runs that server would now be able to easily ship modified artifacts.
On the LXD image format side of things. Back when LXD was still Apache 2.0, we had automation to look at new commits in the LXD codebase and when applicable, cherry-pick those into Incus so our APIs would generally remain compatible, usually letting us deal with potential breaking changes even before either side releases a new version.
None of that is possible now, so we only support importing data from LXD from as recently as LXD 5.21 since thatâs the last release of LXD under Apache 2.0 and so the last release we were able to look at for both its external (REST image format, backup format, âŠ) and internal (database, filesystem structures) APIs.
If we simply publish a bit-for-bit reproduction of our lxd images with a new name, isnât that tantamount to the scenario youâre describing with the images we currently do publish at cloud-images.ubuntu.com? how does renaming the file solve the problem? Maybe Iâm missing that there are additional structural or content requirements for the incus.tar.xz?
I guess I donât quite understand what the available infra for third party image streams looks like. I was asking about / suggesting publishing our images to a stream that you host and/or control, I also donât think it would be a good idea to host them under some stream controlled by someone other than you or us.
You donât even need to publish anything again or rename any file, itâs really purely metadata, you add one chunk of 10 lines or so in the JSON and youâre done 
Example for the most recent Ubuntu 24.04 amd64 build:
--- orig.json 2026-01-06 16:15:43.280149573 -0500
+++ com.ubuntu.cloud:released:download.json 2026-01-06 16:15:25.330301638 -0500
@@ -298535,6 +298535,17 @@
"sha256": "2b5f90ffe8180def601c021c874e55d8303e8bcbfc66fee2b94414f43ac5eb1f",
"size": 626655744
},
+ "incus.tar.xz": {
+ "combined_disk1-img_sha256": "32a5c526db8fa14cd237dd4e716d816a21a11fbc1339b2340fb1705e756c34dc",
+ "combined_rootxz_sha256": "5df7b199707ce5329e61879e6fef6e77109b48889324495a037ea548f44f4e48",
+ "combined_sha256": "5df7b199707ce5329e61879e6fef6e77109b48889324495a037ea548f44f4e48",
+ "combined_squashfs_sha256": "ad33d28f277ca00cf032b72c4af83f0be1f731c5dce01c91af58bada4b0d0bba",
+ "ftype": "incus.tar.xz",
+ "md5": "217afcf031fae37a40306d034e810d2f",
+ "path": "server/releases/noble/release-20251213/ubuntu-24.04-server-cloudimg-amd64-lxd.tar.xz",
+ "sha256": "9a85890e52683a6cf7f845ed1ad9e6a26df558ce71467f8b2bbfb2044b79eb80",
+ "size": 412
+ },
"lxd.tar.xz": {
"combined_disk1-img_sha256": "32a5c526db8fa14cd237dd4e716d816a21a11fbc1339b2340fb1705e756c34dc",
"combined_rootxz_sha256": "5df7b199707ce5329e61879e6fef6e77109b48889324495a037ea548f44f4e48",
Extra work would only be needed if for some reason the LXD metadata image format has diverged or will diverge in the future. Given that LXD 5.21 is still supported and works with images from cloud-images.ubuntu.com, as it stands today, weâd expect things to just work.
It would certainly be our strong preference that the stream data on cloud-images.u.c just include the extra chunk of JSON, making this work for everyone without really costing anymore more than the few bytes of extra storage for it.
I, as the person running our image servers and infrastructure, personally donât care at all about the official Ubuntu Cloud images. Weâve got our own Ubuntu images which we build and test, including some with cloud-init enabled, so Iâm quite happy with those and donât have any intention to run a separate web server with our own mirror of the official Ubuntu Cloud images on it.
Thatâs why Iâve been saying that the alternative to you adding the metadata on your end would be a community member running their own server (3rd party) at which point folks wanting to use the official Ubuntu Cloud images would need to trust a very much not official mirror by an Incus community member.
I still donât quite see why publishing identical metadata under a different json key solves the problem you described initially. If the lxd metadata changes, the incus metadata changes too. What am I missing?
It shifts whoâs responsible for fixing it.
Basically, if you publish the content of your lxd.tar.xz as incus.tar.xz and a change to the file format ends up breaking things when run under Incus, then you have a bug on your end which you can resolve by at that point by turning them into distinct files and using the old logic to generate incus.tar.xz.
Whereas if we have Incus start accepting lxd.tar.xz as equivalent to incus.tar.xz, then now a change to the file format would require a change to Incus to make it work again, turning this into an Incus bug and one which we may not be able to trivially fix given that we canât legally go look at the LXD source code.
I see. Thanks for the clarity. I think I understand the feature request now. Iâll bring it back to the team.
For the time being, since this is just a matter of using a specific json key, a local workaround for your users would be something like the following for a streams file (e.g. com.ubuntu.cloud:daily:download.json pulled from Index of /daily/streams/v1 ):
$ jq 'walk(if type=="object" and has("lxd.tar.xz") then .["incus.tar.xz"]=.["lxd.tar.xz"] | .["incus.tar.xz"].ftype="incus.tar.xz" end)' com.ubuntu.cloud:daily:download.json
1 Like
That workaround is less than ideal, because we need to download the .json file frequently to keep it updated.
But nice you will keep the ball rolling. Thanks.