Share folders and volumes between host and containers

Note:
In recent kernels (>5.12) a new approach called “idmapped mounts” is directly included in the Linux kernel.
It acts as a replacement/sucessor of shiftfs and uses all the below mentioned commands as well.
It only supports ext4, xfs, vfat and btrfs (>5.15) for the underlying filesystems of LXD for now, but ZFS and cephfs are planned to be included as well.

If you use unsupported filesystems or older kernels, stick with shiftfs for now.


Usecases:

  1. General Advantages:
  • faster startup of containers
  • easier and less risky setup of uid/gid-shifting
  1. sharing disk-devices:
    If you want to share e.g. a folder between host & container or between containers.

  2. sharing volumes:
    If you want to share volumes between isolated containers.

For instructions, see further below.


In case you want to use shiftfs, follow the below steps:

What is shiftfs?
See @stgraber’s post.

How to get shiftfs:

  • For Ubuntu Users: It is already included in the standard Ubuntu Kernel.
  • For other Distros: It is not included in the mainline kernel, but you can add it via dkms.
    I created a github repo for this: https://github.com/toby63/shiftfs-dkms/

Instructions:
1. General use:
“Idmapped mounts” should be enabled by default, if a kernel that supports it is in use.

For the alternative with shiftfs, see @stgraber’s post on how to enable shiftfs in LXD.

2. Sharing disk-devices:

If you want to share a disk device (for example a folder) between host/container or between containers, so that both parties can have full access (rwx) to it.

You only need to add this key to your device-configuration in the container/profile-config:
shift: true

For example:

folder1:
path: /home/user1/folder1
source: /home/hostuser1/folder1
shift: true
type: disk

Note:
This will match the hosts uid/gid (of the folder owner) with the container uid/gid.
So if the hosts uid is 1000, the user in the container also needs to have the uid 1000 to be able to access it.
See forum post by stgraber.

3. Sharing volumes:

If you want to share a volume between isolated containers.

First add this key to your volume-configuration:
security.shifted=true

Then attach the volume to both containers:
lxc storage volume attach POOL-NAME VOLUME-NAME container1 DEVICENAME /PATH

lxc storage volume attach POOL-NAME VOLUME-NAME container2 DEVICENAME /PATH


Recommendations:

  • If you don’t want LXD to remap (the UIDs/GIDs of) your container when shiftfs is not available (for example because of a failed dkms update), you can apply the following config key to your container (profile):
    security.protection.shift: "true"
    "Prevents the instance's filesystem from being uid/gid shifted on startup" (See: Instances | LXD )

    Related error report: Container error after changing shiftfs (false/true)


Security Notes:


Sources:

Tags:
share folder
share volume
:rw

4 Likes

Thanks, have moved this to the tutorials category.

1 Like

@tomp @stgraber
I would like to change the title to something like:
“[Howto] Share folders and volumes between host<->containers resp. between containers”

So it can be found easier and also because the new approach with “idmapped mounts” is now natively available.
I will rework the article further once shiftfs is completely replaced.

@stgraber @brauner @tomp
Some additional questions regarding “idmapped mounts”:

  1. I assume this is true?
  1. Is this still true?
  1. Is the shift protection option still available?
  1. Are these security implications still valid?

This post is already in the Tutorial category, so prefixing the title actually makes things a bit confusing. I just fixed a few more posts in that category which had that duplication.

I’ve otherwise updated the title as suggested.

1 Like

This is notably not true for Focal kernel linux-image-oem-20.04c (version 5.13.0.1020.24 as of this post), which is the only way to get the 5.13 package using Focal repositories (that I know of). This kernel does not package the shiftfs module (ubuntu bug).

The shiftfs module can be built via DKMS as suggested for other distros. But this has associated limitations (such as not being able to use overlayfs on top).

Update: It seems that OEM has a different meaning for Ubuntu: Kernel/OEMKernel - Ubuntu Wiki
So I was wrong I guess.
Though I still don’t understand what it is to be honest.

I updated the article above, it now states that shiftfs is included in the “standard Ubuntu kernel”.