shiftfs is a kernel filesystem which was initially developed by James Bottomley, then stabilized and extended by @sforshee and @brauner.
That filesystem acts like an overlay on top of an existing mount on the host and then allows mounting inside the container, shifting uid/gid on all filesystem operations. This allows for instant creation and startup of unprivileged containers as no costly filesystem remapping is needed on creation or startup.
In tests, on fast systems backed by NVME storage, we’ve seen the typical
lxc launch time for an Ubuntu 18.04 image go from around 2.5s down to just 500ms. For systems using hard disks, the difference should be far more noticeable, possibly saving tens of seconds.
shiftfs will also allow for multiple containers that use non-overlapping maps (security.idmap.isolated=true) to share custom storage volumes, which is currently impossible.
Where can it be found
At this time, shiftfs isn’t mainline. It’s included in the Ubuntu kernel starting with 5.0 and some other Linux distributions may pick it up. It should also be reasonably easy to package as a DKMS package for distributions that ship a suitably recent kernel.
Ubuntu users have it in their kernel out of the box, this can be confirmed with
As we had to fix a few early issues, you should make sure that your kernel is fully up to date and you are booted onto it.
The only limitation we’re currently aware of is that shiftfs will prevent the use of overlayfs inside the container. This may break Docker users. If this applies to you, you should stay away from shiftfs for now.
How can it be used
The easiest is with the LXD snap on version 3.12 or higher. We currently have it disabled out of the box because of the above limitations but you can opt into it if those don’t apply to you.
To turn it on, do:
- sudo snap set lxd shiftfs.enable=true
- sudo systemctl reload snap.lxd.daemon
You can then run:
- lxc info
And should see:
kernel_features header. If you see it, then you’ve got it enabled.
After that, any newly created container will use shiftfs.
To convert an existing container, the easiest is to temporarily convert it to privileged:
- lxc config set NAME security.privileged true
- lxc restart NAME
- lxc config unset NAME security.privileged
- lxc restart NAME
Any feedback on shiftfs would be appreciated and we will be updating the
limitations section above as we become aware of new issues.
Once all of them are addressed, we will slowly start rolling it out by default to our users as this is a significant performance improvement and completely avoids the rather complex and risky remapping logic that we have to use today.