Is there a LXD hub in the pipleline anywhere?

just curious … Is there any plan (or maybe existing project) to get something similar to docker hub started in the LXC/LXD realm?

I mean a location (a ‘remote’ in LXD terms I guess) where pre-made container templates/images that are prepared to deliver certain functionality out-of-the box?

3 Likes

There isn’t. Operating a public image repository where people can push their own images is pretty costly and also potentially difficult legally (as you’d be distributing your user’s binaries without being able to provide the sources, so violating the GPL).

One thing we may do at some point is providing a standalone image server binary, that’d implement a LXD-compatible simplestreams server, doing both HTTP and HTTPs and would support enough of the LXD image protocol for you to be able to copy images to it.

But it’s not on our roadmap right now and so not very much of a priority. It could be a very fun side project for someone who has a bit of spare time on their hands.

ok, thanks for clarifying …

how if it wasn’t completely public but only allowed ‘official’ images (like the official images on the docker hub) provided by originator’s of any software. So those ‘vendors’ had the opportunity to provide an easy way to deploy their solution in a container and the hub deals only with trusted parties who will be responsible for maintenance and also legally?

That would work and I had something like that designed in the original LXD spec where images: would actually act as some kind of registry, letting us delegate some images directly to the vendors who would then operate their images servers.

That’d keep us in control of deciding who we trust and who we don’t but the day to day image publishing could then be done by the vendors.

We decided against that model back then because of the added complexity and the fact that we didn’t really have any such vendors lined up anyway. Even now, we’ve not seen much interest from other distributions in providing official tested images for LXC/LXD. Looks like most are happy with our daily images even though they’re not being tested at all.

I could imagine ‘vendors’ who provide complex stacks utilizing a lot of services (like an ERP system i.e.) might be the ones to be interested being able to provide/maintain their own container image.

Maybe the way to go for such would be to provide the infrastructure required (with as little resources as possible spent on that) for such a ‘hub’ by the LXD team and then just see whether ‘vendors’ (which may be other distributions as well) may make use of it.

Last year I’d exchanged messages on the TurnKey Linux website (https://www.turnkeylinux.org/) with multiple people on the TurnKey Linux project about this same topic.

TurnKey Linux pre-builds perhaps 100 or so top linux applications which people can download & run ranging from things like Nagios to ERM like Redmine etc etc. Their Apps list is here: https://www.turnkeylinux.org/all

They were quite interested in if/how they might implement their own LXC/LXD hub which could contain the pre-built linux applications that TurnKey Linux provides…

Here was that message thread if you are interested in reading it…

https://www.turnkeylinux.org/forum/general/20160708/lxd-image-server

Brian

interesting read. I guess it would be great wether both projects could pull this off and likewise benefit equally from such an effort

I plan to create a generic image build server using Packer, anyone up for that? It would let you add a Git repositories with Packer image definitions that would be automatically built on push and then made available via a integrated LXD SimpleStreams API. Would be awesome if LXD provided some library to assist with that @stgraber :).

This is basically the automated build feature of Docker Hub, I’m not interested in image push support. Also at least initially it will only feature a API and no web interface, but that could certainly be added down the line.

Since this will be quite a bit of work, I’m looking for people who want to join me. My preferred language is Go, since that is what LXD uses, and I’d like to make it available under the MIT or a similar license.

Running a public instance would require funding of some sort, but I’m definitely interested in making this happen at a later stage.

1 Like

[quote=“jgillich, post:8, topic:103”]
I plan to create a generic image build server using Packer, anyone up for that? …
… Since this will be quite a bit of work, I’m looking for people who want to join me [/quote]

not understanding any of the technical mechanics of such an endeavor but …

wouldn’t it make sense to work this out in collaboration with the people from Turn Key Linux who have expressed interest in such a thing in the thread from the Turn Key forum quoted here earlier by @bmullan ?

They seem primarily interested in running a public image host, not a a build system (in Docker terms, a registry and not a hub). But I will contact them.

Hi there,

Sorry I’m a bit late to the party…

Firstly I should state that I use LXC on a fairly regular basis, but have never used LXD. Other than insights shared by others and my brief reading online, I know next to nothing about it. So please excuse me if I say something dumb! :slight_smile:

We’d certainly be interested in being involved in some sort of LXD repository. Ideally, we’d probably prefer to just have our images included in someone else’s repository (although we’d be happy to host the actual downloadable images on our mirror if that helps).

Alternatively, if it’s not too much overhead and could be installed safely and securely on one of our existing public servers, we’d consider hosting our own LXD image repo. However, the actual images would need to reside on our mirror (rather than with the server software).

FWIW we don’t currently provide any LXD images. We do provide an LXC template though and a ton of other build types (ISO, OVA, Xen, Docker etc) so adding another supported build type seems like a no-brainer…

We already have a build system! :slight_smile: It’s called TKLDev. It uses build code (e.g. our LAMP appliance) to build the software appliance from scratch. Unlike Packer it only builds ISOs. However, if you team TKLDev with our BuildTasks scripts, then you can convert an ISO to any of the other build types we support )OVA, LXC, etc).

I know it’s a bit sucky to have to learn a new infrastructure to be able to get involved, but once you get your head around it, it’s pretty straight forward. It simply uses filesystem overlays and (bash) conf scripts to configure and install everything. Probably the trickiest part is making sure the whole install is unattended and creating the inithooks to set passwords and stuff like that. We aim to be supportive of contributors and open to anyone who wants to contribute.

Having said that, we’ve been doing a pretty poor job of publishing new community developed appliances lately. As part of the current release (which I’m working on) I’m aiming to remove as much of the stuff that blocks that as possible. One of our aims is to improve our responsiveness to community…

I’ll try to keep an eye on this thread in case there is anything else I can add, but please feel free to reach out. Best way to get hold of me is post on the forums; I do spam cleanups most days and try to respond to new posts within a day or 2 (although at really busy times that can grow up to a week). If you have something you’d rather discuss privately, then you can get me at jeremy AT turnkeylinux.org

1 Like

FYI but JedMeister is one of the developers for:

+1
let me know if you started something, I am also happy to contribute something where people could build up their own lxd ‘registry’ with https and something better than hard-coded (.htaccess) server password…

It should be as straight forward as starting it’s own docker registry. Probably using an official canonical lxc image…

Hi Stephane,

I am coming back to the idea of building a LXDHub. I did some investigation using a lxd server instance as specified in the github: https://github.com/lxc/lxd/issues/1581

I dug a bit into the simplestreams topic but was not really successful to use the juju stack to generate already lxd hosted images (https://jujucharms.com/docs/1.24/howto-privatecloud)

The first approach would be to have a lxd instance as a server where one can interract with lxc directly with the command:

lxc remote add mylxd lxdimages.hub --accept-certificate --password=unsecret

then one can push images to this remote, unfortunately the authentication is done with a static password.

The idea of using simplestreams would complement this mechanism but using the lxd server image store and generate the appropriate simplestreams to be hosted as a flat directory structure via Nginx of Apache.

Here is some quick drawing of the idea: lxd-server/docs/LXD-hub.svg at master · erickeller/lxd-server · GitHub

Is there is a possibility to take an existing instance of lxd images and convert them to the propper simplestreams format to be hosted?

LXDHub is now opensourced:


7 Likes

If you can pull off not requiring Docker for a system to manage LXD containers you might have a winner!

1 Like

You mean packaging it within a lxd image :slight_smile: or do you have another idea in mind like snap, Debian packages?

LXDHub now has a public live demo: https://lxdhub.xyz
It is also possible to run LXDHub as linux container using the commands:


lxc remote add lxdhub https://lxdhub.xyz:8443 --accept-certificate --public
lxc launch lxdhub:lxdhub mylxdhub
lxc info mylxdhub | grep "eth0:.*inet" | head -n1 | awk '{print "Open http://"$3":3000 in your browser"}'

We will not host your LXC images on our public remote, so it is not possible to publish your own images on there. But if you run LXDHub on your own server, you can configure your remote so it is not readonly.

The public live demo at the moment visualizes three remotes:

  • images: A public readonly remote (https://us.images.linuxcontainers.org)
  • local: You can clone images from the images: or lxdhub: remote using the webinterface to the local: remote. Gets constantly flushed on the public demo, since we do not want to host your images. Just for testing, so you see how it behaves.
  • lxdhub: Is used to publish the newest LXDHub images so you can download them from there.

The image size of LXDHub itself is currently huge (~555MB), because the build dependencies are currently included. We’re working on it, so the size should be smaller in future releases.

cc
@mgaboury
@Eric_Keller

Cheers
Livio

3 Likes

wondering how this may be useful for a scenario as per the initial usecase of this thread

If I am not mistaken (which could easily be the case) it is already by default possible to create a ‘private’ remote and provide an image for such an application through that, isn’t it?
So I am wondering what benefit the hub adds up to that for such a usecase.

The is probably the main reasons why Docker surpassed easily Linux Containerization.

The concept of a containerization/dockerization is actually the fast deploying without having the pain to setup everything manually.

Being said, i tried to find some repo image for LXC, as PVE use it and ended up launch a new independent instance to use Docker.

In my own personal point of view, i’m not able to spots any advantages to use LXC these days while the you have Docker, as for me, this is literally a VM limited to run the same kernel as his host.

This is bad as LXD was promising.

2 Likes