Share folders and volumes between host and containers

Tutorials
1

Note:
In recent kernels (>5.12) a new approach called “idmapped mounts” is directly included in the Linux kernel, so no need for additional dkms modules.
It acts as a replacement/sucessor of shiftfs and uses all the below mentioned commands as well.
(Update 2024) It now supports most filesystems, including ext4, xfs, vfat, btrfs (>5.15), ZFS and cephfs.

If you use older kernels, you can stick with shiftfs for now (but it might become outdated, check for updates in the ubuntu repos).

Usecases:

  1. General Advantages:
  • faster startup of containers
  • easier and less risky setup of uid/gid-shifting

  1. sharing disk-devices:
    If you want to share e.g. a folder between host & container or between containers.

  2. sharing volumes:
    If you want to share volumes between isolated containers.

For instructions, see further below.

In case you have to use shiftfs, follow the below steps:

What is shiftfs?
See @stgraber’s post.

How to get shiftfs:

  • For Ubuntu Users: It is already included in the standard Ubuntu Kernel.
  • For other Distros: It is not included in the mainline kernel, but you can add it via dkms.
    I created a github repo with scripts to install shiftfs as kernel module (but my repo is not actively maintained at the moment): GitHub - toby63/shiftfs-dkms: shiftfs kernel module via dkms . Upstream still seems to develop and support shiftfs though (Status: May 2024).

Instructions:
1. General use:
Notes:

  • “Idmapped mounts” should be enabled by default, if a kernel that supports it is in use.
  • For the alternative with shiftfs, see @stgraber’s post and my github wiki on how to enable and use shiftfs in Incus and LXD.

2. Sharing disk-devices:

If you want to share a disk device (for example a folder) between host/container or between containers, so that both parties can have full access (rwx) to it.

You only need to add this key to your device-configuration in the container/profile-config:
shift: true

For example:

folder1:
path: /home/user1/folder1
source: /home/hostuser1/folder1
shift: true
type: disk

Note:
This will match the hosts uid/gid (of the folder owner) with the container uid/gid.
So if the hosts uid is 1000, the user in the container also needs to have the uid 1000 to be able to access it.
See forum post by stgraber.

3. Sharing volumes:

If you want to share a volume between isolated containers.

First add this key to your volume-configuration:
security.shifted=true

Then attach the volume to both containers:
incus storage volume attach POOL-NAME VOLUME-NAME container1 DEVICENAME /PATH

incus storage volume attach POOL-NAME VOLUME-NAME container2 DEVICENAME /PATH

Recommendations:

  • If you don’t want Incus to remap (the UIDs/GIDs of) your container when shiftfs is not available (for example because of a failed dkms update), you can apply the following config key to your container (profile):
    security.protection.shift: "true"
    "Prevents the instance's filesystem from being uid/gid shifted on startup"

    Related error report: Container error after changing shiftfs (false/true)

Security Notes:

Sources:

Updates:

  • 13.05.24: Update for incus, should still work the same way

Tags:
share folder
share volume
:rw

7 Likes
2

Thanks, have moved this to the tutorials category.

1 Like
3

@tomp @stgraber
I would like to change the title to something like:
“[Howto] Share folders and volumes between host<->containers resp. between containers”

So it can be found easier and also because the new approach with “idmapped mounts” is now natively available.
I will rework the article further once shiftfs is completely replaced.

4

@stgraber @brauner @tomp
Some additional questions regarding “idmapped mounts”:

  1. I assume this is true?
  1. Is this still true?
  1. Is the shift protection option still available?
  1. Are these security implications still valid?
5

This post is already in the Tutorial category, so prefixing the title actually makes things a bit confusing. I just fixed a few more posts in that category which had that duplication.

I’ve otherwise updated the title as suggested.

1 Like
6

This is notably not true for Focal kernel linux-image-oem-20.04c (version 5.13.0.1020.24 as of this post), which is the only way to get the 5.13 package using Focal repositories (that I know of). This kernel does not package the shiftfs module (ubuntu bug).

The shiftfs module can be built via DKMS as suggested for other distros. But this has associated limitations (such as not being able to use overlayfs on top).

7

Update: It seems that OEM has a different meaning for Ubuntu: Kernel/OEMKernel - Ubuntu Wiki
So I was wrong I guess.
Though I still don’t understand what it is to be honest.

I updated the article above, it now states that shiftfs is included in the “standard Ubuntu kernel”.

8

@stgraber: Do idmapped mounts now support ZFS and cephfs?

9

They do, yes

1 Like