Continuing the discussion from LXC 3.0.0 has been released:
First of all, congrats on the release. Great work, guys!
Is there any info about “daemonized app containers”? I’ve found only this commit and nothing else. Could you, please, share more details or a link?
Thanks.
The general structure for how we run application container has been the same since 2008: Use minimal-init system and run the actual binary as the second process.
The idea to extend this was that you can have application containers that run any command daemonized. Even a bash shell. So you should now be able to do:
chb@conventiont|~
> lxc-execute xenial bash -d
chb@conventiont|~
> lxc-info xenial
Name: xenial
State: RUNNING
PID: 2870
Memory use: 7.66 MiB
KMem use: 1.40 MiB
Link: vethLYN8MT
TX bytes: 586 bytes
RX bytes: 5.82 KiB
Total bytes: 6.39 KiB
chb@conventiont|~
> lxc-attach xenial
root@xenial:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 1296 4 ? Ss 04:23 0:00 init
root 22 0.0 0.0 18208 3096 ? Ss+ 04:23 0:00 bash
root 32 0.0 0.0 18208 3224 ? Ss 04:24 0:00 /bin/bash
root 42 0.0 0.0 34424 2772 ? R+ 04:24 0:00 ps aux
root@xenial:/#
or
> lxc-execute xenial -d -- sleep 100
chb@conventiont|~
> lxc-info xenial
Name: xenial
State: RUNNING
PID: 6556
Memory use: 1.36 MiB
KMem use: 1.20 MiB
Link: vethOJS2KE
TX bytes: 516 bytes
RX bytes: 2.98 KiB
Total bytes: 3.48 KiB
chb@conventiont|~
> lxc-attach xenial
root@xenial:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 1296 4 ? Ss 04:25 0:00 init
root 22 0.0 0.0 4376 660 ? Ss+ 04:25 0:00 sleep 100
root 23 0.0 0.0 18208 3188 ? Ss 04:25 0:00 /bin/bash
root 33 0.0 0.0 34424 2872 ? R+ 04:25 0:00 ps aux
root@xenial:/#
Thanks for answer, but looks like i wasn’t clear enough. I know how containers work in general and i have some experience with application containers (i.e. docker & co). But i was initially asking about technical details of this new feature.
For example, is there some docs or any other info how to get it working in case of LXC. At least where to get this new one “minimal-init”. Does it come with LXC 3.0 or …?
And most important - is there any known issues or limitations on using this feature within containers under LXD management? As i understood, it mostly about building your own “custom” image. But nonetheless, i would appreciate any details on this from the first source (someone from dev team or docs on this feature).
Thanks.
Well, today’s your lucky day then since I’m the main developer/tech lead for liblxc and a core-developer and maintainer of LXD. Also, you’re referencing my patch. 
But that’s what I showed you. You can use it via lxc-execute or by directly using the API if you build your own tool.
This minimal init has been shipping with LXC since 2008. It will be installed when you install LXC unless your distro of choice has explicitly refrained from packaging it.
LXD currently does not support application containers run through this minimal init. If you only care about running a different binary as init withou the minimal-init then you can make it work with LXD right now:
lxc config set <container-name> raw.lxc "lxc.init.cmd= /bin/bash"
I don’t understand what you mean by this? LXD is a container-manager daemon sitting on top of liblxc/LXC which is a shared C library.
Hmm … Let me try to explain my previous question. For now LXD by default uses “pre-built” images (with slight customization for running in containers). Except that, you can also build your own image, import it and launch container based on that image. So, now you can use what you given already or you can built your own containers for your specific requirements.
In previous questions I was wondering about second scenario. For now LXD doesn’t have any layered filesystems (aufs, overlay, etc) to offer and there is huge question about containers size on disk. Partially it could be solved by using of BTRFS or ZFS, but system-in-container still has some big weight. And we’re trying to minimize it by removing some extra software from default images. It lead us to ~300Mb per image and only 4-5 running extra processes in containers (mostly upstart/systemd and their dependecies).
So, that’s about our goal. We’re trying to minimize LXC images and run them under LXD to reduce overhead in resources (import/export images, CPU on compress/uncompress, disk usage, etc). And also to reduce number of running “extra” processes inside of same containers.
By reading changelog for LXC 3.0 i saw an opporutunity to reduce images+containers even more by replacing upstart/systemd with your minimal-init. But, i guess, there could be some mandatory behavior or strict requirements, that LXD expects from underlying container and its content.
For example LXD currently expects, that init-process will handle SIGINT and SIGPWR (found here incus/doc/container-environment.md at main · lxc/incus · GitHub).
In other words - did someone already tried to do something like that? I’m just trying to estimate “does it worth to move in this direction” or someone already tried and there was a dead-end “because you can’t solve this problem/limitation/etc …”.
I hope, i explained my question well enough this time.
PS: And yes, i know, that all of that looks like we’re trying to build Docker from LXC/LXD 
That wouldn’t be a problem with our minimal init. It is supposed to behave like a regular init just in a very dumb manner, i.e. just handling signals and reaping children and nothing else.
It is possible, yes. To enable this one would just need a very minimal API extension in LXD which I had at one point. It all depends on whether we as core team (/cc @stgraber, @hallyn) agree that it would make sense.
Quoting Christian Brauner (discuss@linuxcontainers.org):
It is possible, yes. To enable this one would just need a very minimal API extension in LXD which I had at one point. It all depends on whether we as core team (/cc @stgraber, @hallyn) agree that it would make sense.
I’m not sure @stgraber will agree :), but I think it makes sense.
It’s mostly about optimization in favor of “density” of containers on the server. And yes, i know, that it looks like re-inventing Docker (or similar approach), but with LXD. But in general, we (our team) prefer to use LXC+LXD instead of overhyped Docker & Co. And may be, just may be, there are more teams with similar preferences or requirements. Could you and your colleagues, please, at least consider such usage case?
PS: Or just announce your position on such matter (i.e. “We won’t do …”, “Not now, but may be …”, etc).
I think the current position is not now but maybe. It’s certainly not particularly difficult for us to add but I fear the deluge of requests we’d then get to try to get to full feature parity with tools like Docker and then keeping up as new features are added to those tools.
This would make it much harder for us to keep focusing on system containers and keep innovating in this field.
As things stabilize in the container space, this may change, but for now, it’s probably best to consider app containers and system containers as two separate worlds with their own set of tools.
Stéphane,
I understand your concerns about supporting the app container use case, but I hope you reconsider the two use cases as separate worlds with no potential for overlap in tools. For the container space to stabilize, there needs to be ubiquitous standardization. Adopting OCI interoperability (not necessarily full spec compliance) in distrobuilder and LXD could have enormous benefits.