Not quite, here at least. Running on a debian stretch system snap allows for an initial level of container but it is not possible to nest containers.
Dunno if that’s lxd or its snap or something else entirely or some kind of confabulation somewhere between any or all of the packages mentioned.
Issue has been posted as:
Nesting LXD in LXD using a Debian stretch container
on this forum.
The LXD snap doesn’t prevent nesting, but nesting snapd daemon doesn’t work so well unless you’re on Ubuntu and even then with some limitations.
So I’d totally expect this to work:
And this will work too:
The first works because snapd is only used outside of containers.
The second works because while snapd is used inside the container, the host kernel is the Ubuntu kernel and so allows for unprivileged FUSE filesystems.
To make things even more confusing, only a single level of nesting can work in that second case, as doing something like:
Would fail due to AppArmor only allowing for one level of namespacing, causing the nested nested snapd to fail.
Running the LXD deb instead of the snap, works around both the FUSE and AppArmor kernel limitations.
There’s ongoing work to upstream the unprivileged FUSE bits (discussion was updated earlier today on LKML) and the AppArmor team has been thinking about how to remove the nesting limitation in AppArmor.
Anyway, this is a long convoluted way of saying that the snap itself isn’t at fault here, it’s really just a bunch of kernel restrictions that are being hit.
Is there any chance of a wider solution being part of some future lxd?
Its looking like lxd is designed to really work with ubuntu - - - period.
As ubuntu is a debian derivative one would think that things should work there too - - - but they don’t.
I’m looking for something like:
so even if I’m 10 levels deep - - - well its no issue.
I get that I can’t today - - - but what about say - - - tomorrow? (As this somewhere out there time frame.)
The FUSE patchset needed to run snapd in containers is being reviewed upstream so should be in a Linux kernel in the next few months.
With that in place, nesting as you describe it will be fine so long as apparmor isn’t involved.
With AppArmor involved, we’ll need AppArmor namespacing and stacking to be reworked to allow multiple levels of nesting.
This isn’t really an Ubuntu vs other distro problem, both of those limitations are kernel limitations. Ubuntu happens to have a kernel change that fixes one of those while the other affects everyone.
Hmmmmmmmmmmmmmmmmmm - - - does that mean that if I compile the kernel myself with specific options either enabled or disabled that it might be possible to change this situation earlier than waiting for all of these changes to happen?
It’s actual code that’s missing, not just options. So you’d need to manually apply the unprivileged fuse patch series.
https://github.com/kinvolk/linux/commits/dongsu/fuse-userns-v5-1
Has the most up to date copy I think.
Thanks but I think I had better wait for others to affect those changes - - - grin!
Don’t want to make that much extra work for you!!!