The future of LXC/Incus images on ChromeOS

Incus
1

ChromeOS users utilizing Crostini employ LXD containers behind the scenes.

ChromeOS reportedly acquires its LXD images from https://storage.googleapis.com/cros-containers, as indicated in the deprecated file run_container.sh.
I’m not sure if this is still the case as the latest LastModified date is from 2022.

Also, this remote image URL isn’t specified in crosh:

(termina) chronos@localhost ~ $ lxc remote list   
+-----------------+------------------------------------------+---------------+-------------+--------+--------+--------+
|      NAME       |                   URL                    |   PROTOCOL    |  AUTH TYPE  | PUBLIC | STATIC | GLOBAL |
+-----------------+------------------------------------------+---------------+-------------+--------+--------+--------+
| images          | https://images.linuxcontainers.org       | simplestreams | none        | YES    | NO     | NO     |
+-----------------+------------------------------------------+---------------+-------------+--------+--------+--------+
| local (current) | unix://                                  | lxd           | file access | NO     | YES    | NO     |
+-----------------+------------------------------------------+---------------+-------------+--------+--------+--------+
| ubuntu          | https://cloud-images.ubuntu.com/releases | simplestreams | none        | YES    | YES    | NO     |
+-----------------+------------------------------------------+---------------+-------------+--------+--------+--------+
| ubuntu-daily    | https://cloud-images.ubuntu.com/daily    | simplestreams | none        | YES    | YES    | NO     |
+-----------------+------------------------------------------+---------------+-------------+--------+--------+--------+

The Canonical image mirror sites (UK, US) were last updated in August 2023.

If I’m not wrong, it is not possible to install Incus in crosh without placing the ChromeOS device in developer mode. Sadly, migrating from LXD to Incus doesn’t seem to be possible.

As a ChromeOS user, I have two questions:

  1. Where can I obtain additional LXD images for distributions other than Ubuntu (e.g., Alpine or Busybox) in the future?
  2. Is it possible to use Incus on ChromeOS without nesting it inside of LXD?
2

The current images: remote will keep fully functioning until mid-April at least. After that, access for the LXD version used on Chromebooks is scheduled to get phased out.

It’s pretty hard to know what will happen after that. Our plan is to fully phase out LXD support on our image servers, but we are prepared to make some extensions to those deadlines where they make sense. Chromebooks may be one of those if no alternatives are ready yet.

It’s also a pretty well known fact that Google has a policy not to use AGPL license software, which LXD is now being released under. When and how this will end up affecting Chromebook users isn’t known at this point, but I’d expect something to happen sooner or later.

I don’t believe so. The termina VM that runs LXD is effectively read-only so doesn’t allow for LXD to easily get swapped out for Incus.

1 Like
3

As far as I know, all Chromebooks download their Debian-based images from Google’s CDN, they don’t directly consume anything from our image servers out of the box.

Our image server only gets used by those few Chromebooks users that directly interact with LXD within the VM.

1 Like
4

Speaking as one of the few people that want to use something other than debian for their chrombook linux environment, I have to admit that I find this situation a bit confusing. I’m a lifelong user of unix/linux, but I haven’t had a reason to use containers until now. Below is the understanding that I’ve come to after looking at this for a day or so, but I’m undoubtedly missing at least a few key points. Any comments, clarifications, or suggestions would be appreciated.

Thanks in advance, -Jeff

LXC itself is a system for locally creating, running, and managing containers that run on the computer you are using. You use LXC from the command line via a small plethora of lxc‑foo commands, such as lxc‑create, lxc‑start, lxc‑stop, etc. Afaict, none of lxc- commands connect to the net, so they can’t download a container image from images.linuxcontainers.com or elsewhere.

LXD is a network‑aware daemon process for managing LXC containers. It seems to me that the main advantage is that LXD lets you manage containers on remote systems in exactly the same way as local ones. There are other advantages as well. You interact with the LXD daemon from the command line using a single command that is, very confusingly, named ‘lxc’ (undoubtedly for bizarre historical reasons). You can use ‘lxc start’, ‘lxc stop’, etc, for either local or remote containers. There are additional commands such as ‘lxc remote’ for managing remote systems and images.

Incus is linuxcontainers’ fork of LXD. It is better (of course) and also not under the control of cannonical.

–

The headline at images.linuxcontainers.org says that it “hosts a public image server for use by Incus and LXC,” but that support for LXD users is being phased out. It’s not clear to me exactly what this means. I haven’t been able to find an explanation for how to download an image to use directly with LXC (without using LXD or Incus). Does dropping LXD really mean that only Incus will be supported?

LXD can download images in two ways: from a remote LXD via the ‘lxd protocol’ or via the simpler http‑based ‘simplestreams protocol’. I dont’ think that dropping lxd protocol support will effect chromebooks, since (as the original post shows) they seem to use simplestreams to retrieve remote images.

Incus seems to use a new metadata format, but is there any difference between image formats for LXC or LXD? Looking at the current build for devuan, for example, shows 6 files:

image.yaml      16.22 KiB
incus.tar.xz      680 B
meta.tar.xz       936 B
rootfs.squashfs 87.37 Mib
rootfs.tar.xz   77.62 Mib
serial  	       15 B

There are two root filesystem images, but I believe that they are for VMs vs Containers, not LXC vs LXD vs Incus. Is that correct? The contents of the images are identical except for the hostname file. The incus.tar.xz file is metadata for Incus, and I think that meta.tar.xz is for LXD. Will meta.tar.xz files no longer be created when LXD support is dropped? If so, then it should, in principal, still be possible to use linuxcontainers.org for a chromebook by downloading the rootfs file and re‑using (perhaps with sight modification) an old meta.tar.xz file. Would that be particularly hard to do?

5

FWIW, I note that the incus-base package includes minio, which is AGPL. That might be an argument for splitting this off into a separate incus-minio package.

Aside: I also note that incus-base includes a ton of documentation:

$ dpkg-query -L incus-base | grep /opt/incus/doc | wc -l
1530

Maybe that could also be factored out.

6

Not really, Google doesn’t use a Debian/Ubuntu base for their LXD host, instead it’s a custom Gentoo based distro with their own package. They build their own LXD so should they switch to Incus, they simply wouldn’t build/include minio.