I recently have been updating code for private image server.
There are numerous changes in upstream “https://uk.lxd.images.canonical.com” , with which I try to catch up.
There are root.squashfs and root.tar.xz files.
Which is by default fetched when creating a container? Is there an optional choice?
There are multiple versions of item sets maintained, Is this is an insurance policy?
I guess, by launching a container, the latest version is fetched.
In order to save space and migration, I maintain only one version. In this case, can I skip the createion of ‘root.date…vcdiff’: ftype:squashfs.vcdiff diff delta?
lxc image list has a field “type” : container/virtual-machine. Is this based on whether “disk.qcow2” is given in i.versions.tems []l?
Is there a way to publish a unified tarball instead of split (meta+root)?
how would that unified item look like? Compared to this split context:
“items”: {
“lxd.tar.xz”: {
“ftype”: “lxd.tar.xz”,
“sha256”: “fc”,
“size”: 1356,
“path”: “images/almalinux/8/amd64/cloud/20220209_23:09/lxd.tar.xz”,
“combined_sha256”: “ed”,
“combined_rootxz_sha256”: “ed”,
“combined_squashfs_sha256”: “b3”,
“combined_disk-kvm-img_sha256”: “bc”
},
“root.tar.xz”: {
“ftype”: “root.tar.xz”,
“sha256”: “d3128e914e2a4c8f4539f3fdb5a8c4b802a132c9c9fb8c53fa128476419d3d58”,
“size”: 128108152,
“path”: “images/almalinux/8/amd64/cloud/20220209_23:09/rootfs.tar.xz”
},
“root.squashfs”: {
“ftype”: “squashfs”,
“sha256”: “eaf930fa7c057fdd737272f23f6346cec61b3b6eb1d7198db44152728523c938”,
“size”: 153804800,
“path”: “images/almalinux/8/amd64/cloud/20220209_23:09/rootfs.squashfs”
},
Generally maintaining an image server consumes a lot of resources and time, I dont run all images, but certain os are chosen with extra options to create hundreds ready applications.
The fresh fetching os and creation of those applications is taking hours/day with distrobuilder.
Now sha256 once single once combined is quite challenge. I used to assign attr to those tarballs or get the timestamp of the files, in order to prevent a nonsense re-calculating of sha256s when only selected images where newly created.
Also selectively reduce the images.json inplace by selecting “amd64” or "default’ or by the lastest set of items.
LXD supports squashfs, xz and gz and will attempt to use them in that order.
It detects their presence by looking at the LXD hash properties.
Right, we keep the past 3 images which can be useful when an issue happens in a newer image and lets us compute delta (vcdiff) to reduce the overall bandwidth use as clients download the refreshed images. If you don’t care about that, then having a single image is fine.
Yes, if we have a qcow2 image and matching hash key, then we list the image as available for VM usage.
I don’t believe so. It’d certainly be possible but that would need deciding on a new ftype for this and having LXD detect such a case and only download the one file instead of looking for the split layout.
On the production image server, we have hidden files to keep that data and avoid having to re-calculate it when just re-generating the index.
root@shell01:~# ls -lah /data/images.linuxcontainers.org/images/ubuntu/focal/amd64/default/20220213_07:42/
total 462M
drwxrwxr-x 2 lxc lxc 21 Feb 13 10:14 .
drwxrwxr-x 5 lxc lxc 3 Feb 13 10:24 ..
-rw-rw-r-- 1 lxc lxc 65 Feb 13 10:14 .lxd_fingerprint
-rw-rw-r-- 1 lxc lxc 65 Feb 13 10:14 .lxd_fingerprint_qcow2
-rw-rw-r-- 1 lxc lxc 65 Feb 13 10:13 .lxd_fingerprint_squashfs
-rw-rw-r-- 1 lxc lxc 633 Feb 13 10:14 SHA256SUMS
-rw-rw-r-- 1 lxc lxc 833 Feb 13 10:14 SHA256SUMS.asc
-rw-rw-r-- 1 lxc lxc 111K Feb 13 10:13 build.log
-rw-rw-r-- 1 lxc lxc 833 Feb 13 10:14 build.log.asc
-rw-rw-r-- 1 lxc lxc 3.2M Feb 13 10:14 delta-20220212_07:42.vcdiff
-rw-rw-r-- 1 lxc lxc 833 Feb 13 10:14 delta-20220212_07:42.vcdiff.asc
-rw-rw-r-- 1 lxc lxc 252M Feb 13 10:13 disk.qcow2
-rw-rw-r-- 1 lxc lxc 833 Feb 13 10:14 disk.qcow2.asc
-rw-rw-r-- 1 lxc lxc 26K Feb 13 10:13 image.yaml
-rw-rw-r-- 1 lxc lxc 833 Feb 13 10:14 image.yaml.asc
-rw-rw-r-- 1 lxc lxc 680 Feb 13 10:13 lxd.tar.xz
-rw-rw-r-- 1 lxc lxc 833 Feb 13 10:14 lxd.tar.xz.asc
-rw-rw-r-- 1 lxc lxc 952 Feb 13 10:13 meta.tar.xz
-rw-rw-r-- 1 lxc lxc 833 Feb 13 10:14 meta.tar.xz.asc
-rw-rw-r-- 1 lxc lxc 110M Feb 13 10:13 rootfs.squashfs
-rw-rw-r-- 1 lxc lxc 833 Feb 13 10:14 rootfs.squashfs.asc
-rw-rw-r-- 1 lxc lxc 98M Feb 13 10:13 rootfs.tar.xz
-rw-rw-r-- 1 lxc lxc 833 Feb 13 10:14 rootfs.tar.xz.asc
Thanks for elaboration.
The index.json is requested when lxd pulls the image server, but it doesnt make real use of it, as long as format convention is kept, it accepts it without caring about products listed there.
When I create new images, without diff delta and disabled images auto update, there still is the function of caching simplestreams remotes. By launching a container it throws an error message of hash mismatch.
Any workaround for that?