Is there a LXD hub in the pipleline anywhere?


(Gunnar) #1

just curious ... Is there any plan (or maybe existing project) to get something similar to docker hub started in the LXC/LXD realm?

I mean a location (a 'remote' in LXD terms I guess) where pre-made container templates/images that are prepared to deliver certain functionality out-of-the box?


(Stéphane Graber) #2

There isn't. Operating a public image repository where people can push their own images is pretty costly and also potentially difficult legally (as you'd be distributing your user's binaries without being able to provide the sources, so violating the GPL).

One thing we may do at some point is providing a standalone image server binary, that'd implement a LXD-compatible simplestreams server, doing both HTTP and HTTPs and would support enough of the LXD image protocol for you to be able to copy images to it.

But it's not on our roadmap right now and so not very much of a priority. It could be a very fun side project for someone who has a bit of spare time on their hands.


(Gunnar) #3

ok, thanks for clarifying ....

how if it wasn't completely public but only allowed 'official' images (like the official images on the docker hub) provided by originator's of any software. So those 'vendors' had the opportunity to provide an easy way to deploy their solution in a container and the hub deals only with trusted parties who will be responsible for maintenance and also legally?


(Stéphane Graber) #4

That would work and I had something like that designed in the original LXD spec where images: would actually act as some kind of registry, letting us delegate some images directly to the vendors who would then operate their images servers.

That'd keep us in control of deciding who we trust and who we don't but the day to day image publishing could then be done by the vendors.

We decided against that model back then because of the added complexity and the fact that we didn't really have any such vendors lined up anyway. Even now, we've not seen much interest from other distributions in providing official tested images for LXC/LXD. Looks like most are happy with our daily images even though they're not being tested at all.


(Gunnar) #5

I could imagine 'vendors' who provide complex stacks utilizing a lot of services (like an ERP system i.e.) might be the ones to be interested being able to provide/maintain their own container image.

Maybe the way to go for such would be to provide the infrastructure required (with as little resources as possible spent on that) for such a 'hub' by the LXD team and then just see whether 'vendors' (which may be other distributions as well) may make use of it.


(Brian Mullan) #6

Last year I'd exchanged messages on the TurnKey Linux website (https://www.turnkeylinux.org/) with multiple people on the TurnKey Linux project about this same topic.

TurnKey Linux pre-builds perhaps 100 or so top linux applications which people can download & run ranging from things like Nagios to ERM like Redmine etc etc. Their Apps list is here: https://www.turnkeylinux.org/all

They were quite interested in if/how they might implement their own LXC/LXD hub which could contain the pre-built linux applications that TurnKey Linux provides...

Here was that message thread if you are interested in reading it..

https://www.turnkeylinux.org/forum/general/20160708/lxd-image-server

Brian


(Gunnar) #7

interesting read. I guess it would be great wether both projects could pull this off and likewise benefit equally from such an effort


(Jakob Gillich) #8

I plan to create a generic image build server using Packer, anyone up for that? It would let you add a Git repositories with Packer image definitions that would be automatically built on push and then made available via a integrated LXD SimpleStreams API. Would be awesome if LXD provided some library to assist with that @stgraber :).

This is basically the automated build feature of Docker Hub, I'm not interested in image push support. Also at least initially it will only feature a API and no web interface, but that could certainly be added down the line.

Since this will be quite a bit of work, I'm looking for people who want to join me. My preferred language is Go, since that is what LXD uses, and I'd like to make it available under the MIT or a similar license.

Running a public instance would require funding of some sort, but I'm definitely interested in making this happen at a later stage.


(Gunnar) #9

not understanding any of the technical mechanics of such an endeavor but ...

wouldn't it make sense to work this out in collaboration with the people from Turn Key Linux who have expressed interest in such a thing in the thread from the Turn Key forum quoted here earlier by @bmullan ?


(Jakob Gillich) #10

They seem primarily interested in running a public image host, not a a build system (in Docker terms, a registry and not a hub). But I will contact them.


(Jeremy Davis) #11

Hi there,

Sorry I'm a bit late to the party...

Firstly I should state that I use LXC on a fairly regular basis, but have never used LXD. Other than insights shared by others and my brief reading online, I know next to nothing about it. So please excuse me if I say something dumb! :slight_smile:

We'd certainly be interested in being involved in some sort of LXD repository. Ideally, we'd probably prefer to just have our images included in someone else's repository (although we'd be happy to host the actual downloadable images on our mirror if that helps).

Alternatively, if it's not too much overhead and could be installed safely and securely on one of our existing public servers, we'd consider hosting our own LXD image repo. However, the actual images would need to reside on our mirror (rather than with the server software).

FWIW we don't currently provide any LXD images. We do provide an LXC template though and a ton of other build types (ISO, OVA, Xen, Docker etc) so adding another supported build type seems like a no-brainer...

We already have a build system! :slight_smile: It's called TKLDev. It uses build code (e.g. our LAMP appliance) to build the software appliance from scratch. Unlike Packer it only builds ISOs. However, if you team TKLDev with our BuildTasks scripts, then you can convert an ISO to any of the other build types we support )OVA, LXC, etc).

I know it's a bit sucky to have to learn a new infrastructure to be able to get involved, but once you get your head around it, it's pretty straight forward. It simply uses filesystem overlays and (bash) conf scripts to configure and install everything. Probably the trickiest part is making sure the whole install is unattended and creating the inithooks to set passwords and stuff like that. We aim to be supportive of contributors and open to anyone who wants to contribute.

Having said that, we've been doing a pretty poor job of publishing new community developed appliances lately. As part of the current release (which I'm working on) I'm aiming to remove as much of the stuff that blocks that as possible. One of our aims is to improve our responsiveness to community...

I'll try to keep an eye on this thread in case there is anything else I can add, but please feel free to reach out. Best way to get hold of me is post on the forums; I do spam cleanups most days and try to respond to new posts within a day or 2 (although at really busy times that can grow up to a week). If you have something you'd rather discuss privately, then you can get me at jeremy AT turnkeylinux.org


(Brian Mullan) #12

FYI but JedMeister is one of the developers for:


(Eric Keller) #13

+1
let me know if you started something, I am also happy to contribute something where people could build up their own lxd 'registry' with https and something better than hard-coded (.htaccess) server password...

It should be as straight forward as starting it's own docker registry. Probably using an official canonical lxc image...


(Eric Keller) #14

Hi Stephane,

I am coming back to the idea of building a LXDHub. I did some investigation using a lxd server instance as specified in the github: https://github.com/lxc/lxd/issues/1581

I dug a bit into the simplestreams topic but was not really successful to use the juju stack to generate already lxd hosted images (https://jujucharms.com/docs/1.24/howto-privatecloud)

The first approach would be to have a lxd instance as a server where one can interract with lxc directly with the command:

lxc remote add mylxd lxdimages.hub --accept-certificate --password=unsecret

then one can push images to this remote, unfortunately the authentication is done with a static password.

The idea of using simplestreams would complement this mechanism but using the lxd server image store and generate the appropriate simplestreams to be hosted as a flat directory structure via Nginx of Apache.

Here is some quick drawing of the idea: https://github.com/erickeller/lxd-server/blob/master/docs/LXD-hub.svg

Is there is a possibility to take an existing instance of lxd images and convert them to the propper simplestreams format to be hosted?


(Livio Brunner) #15

LXDHub is now opensourced:



(Mike Gaboury) #16

If you can pull off not requiring Docker for a system to manage LXD containers you might have a winner!


(Eric Keller) #17

You mean packaging it within a lxd image :slight_smile: or do you have another idea in mind like snap, Debian packages?