LXD snap version: Share folder with host

Hi, I just started to use LXD and have version 4.4 installed through snap on Ubuntu 20.04.

I have tried to make a shared folder by adding a disk device to the container, add ID mappings to raw.idmap, and edit /etc/subXid to allow LXD to use those uids/gids. However, the shared files show up as with uid:gid nobody:nogroup. Then I read somewhere that my snap-installed LXD cannot access the /etc/subXid files.

Is there another way of sharing a directory between the host and a few containers?

raw.idmap with a suitable both entry + restart of the container should do the trick.

You don’t need to touch subuid/subgid when using the snap.

This is what I tried. Did I miss something?

gustav@fridolf:~$ lxc launch ubuntu:20.04 test
Creating test
Starting test
gustav@fridolf:~$ lxc config device add test share disk source=/tank/video path=video
Device share added to test
gustav@fridolf:~$ ls -l /tank/video
total 51
drwxr-xr-x  4 media media 34 Aug  6 13:04 dvd
drwxr-xr-x  4 media media 48 Aug  5 15:45 film
drwxr-xr-x 19 media media 19 Aug  5 21:28 tv
gustav@fridolf:~$ id media
uid=1001(media) gid=1001(media) groups=1001(media)
gustav@fridolf:~$ lxc config set test raw.idmap='both 1001 1000'
gustav@fridolf:~$ lxc restart test
gustav@fridolf:~$ lxc exec test -- ls -l /video
total 3
drwxr-xr-x 2 nobody nogroup 2 Aug  5 09:19 dvd
drwxr-xr-x 2 nobody nogroup 2 Aug  5 09:18 film
drwxr-xr-x 2 nobody nogroup 2 Aug  5 09:19 tv

Can you show stat /tank/video/dvd on the host and cat /proc/self/uidmap in the container?

gustav@fridolf:~$ stat /tank/video/dvd
  File: /tank/video/dvd
  Size: 34        	Blocks: 18         IO Block: 2560   directory
Device: 4ah/74d	Inode: 34          Links: 4
Access: (0755/drwxr-xr-x)  Uid: ( 1001/   media)   Gid: ( 1001/   media)
Access: 2020-08-17 20:02:50.509113816 +0200
Modify: 2020-08-06 13:04:14.284090791 +0200
Change: 2020-08-17 20:02:50.509113816 +0200
 Birth: -
gustav@fridolf:~$ lxc exec test -- cat /proc/self/uid_map
         0    1000000       1000
      1000       1001          1
      1001    1001001  999998999

Hmm, that looks correct, so I’m a bit confused as to what’s going on here.
Can you show cat /proc/self/mountinfo inside the container too?

Here is a cut out of the output:

gustav@fridolf:~$ lxc exec test -- cat /proc/self/mountinfo
[...]
2300 2283 0:69 / /video rw,relatime master:188 - zfs tank/video rw,xattr,posixacl
[...]

Yeah, that’s not answering what I was trying to figure out :slight_smile:

Anyway, one option for you given you’re on 20.04 is to use shiftfs instead:

  • snap set lxd shiftfs.enable=true
  • systemctl reload snap.lxd.daemon
  • lxc config unset test raw.idmap
  • lxc config device remove test share
  • lxc restart test
  • lxc config device add test share disk source=/tank/video path=/video shift=true

If that works, it actually will work for all uid/gid in the container and not just for uid/gid 1000.

I get an error when adding the device.

gustav@fridolf:~$ lxc config device add test share disk source=/tank/video path=/video shift=true
Error: Failed to start device “share”: Failed to add mount for device inside container: Failed to run: /snap/lxd/current/bin/lxd forkmount lxd-mount – 66374 3 /dev/.lxd-mounts/lxdmount_794817290 video true: Failed to mkdir video: Permission denied

Hmm, sounds like the container didn’t properly re-shift during its restart?
Try restarting it again and then check that ls -lh / in the container looks okay?

gustav@fridolf:~$ lxc restart test
gustav@fridolf:~$ lxc exec test -- ls -lh /
total 82K
lrwxrwxrwx   1 root   root      7 Aug  4 21:39 bin -> usr/bin
drwxr-xr-x   2 root   root      2 Aug  4 21:47 boot
drwxr-xr-x   8 root   root    500 Aug 19 20:07 dev
drwxr-xr-x  89 root   root    179 Aug 19 19:16 etc
drwxr-xr-x   3 root   root      3 Aug 19 19:16 home
lrwxrwxrwx   1 root   root      7 Aug  4 21:39 lib -> usr/lib
lrwxrwxrwx   1 root   root      9 Aug  4 21:39 lib32 -> usr/lib32
lrwxrwxrwx   1 root   root      9 Aug  4 21:39 lib64 -> usr/lib64
lrwxrwxrwx   1 root   root     10 Aug  4 21:39 libx32 -> usr/libx32
drwxr-xr-x   2 root   root      2 Aug  4 21:39 media
drwxr-xr-x   2 root   root      2 Aug  4 21:39 mnt
drwxr-xr-x   2 root   root      2 Aug  4 21:39 opt
dr-xr-xr-x 462 nobody nogroup   0 Aug 19 20:07 proc
drwx------   4 root   root      6 Aug 19 19:16 root
drwxr-xr-x  21 root   root    680 Aug 19 20:07 run
lrwxrwxrwx   1 root   root      8 Aug  4 21:39 sbin -> usr/sbin
drwxr-xr-x   6 root   root      7 Aug 19 19:16 snap
drwxr-xr-x   2 root   root      2 Aug  4 21:39 srv
dr-xr-xr-x  13 nobody nogroup   0 Aug 19 20:07 sys
drwxrwxrwt  11 root   root     11 Aug 19 20:07 tmp
drwxr-xr-x  14 root   root     14 Aug  4 21:40 usr
drwxr-xr-x  13 root   root     15 Aug  4 21:42 var

Ok, that looks correct.
What does grep shiftfs /proc/self/mountinfo get you in the container?

gustav@fridolf:~$ lxc exec test -- grep shiftfs /proc/self/mountinfo
2258 623 0:128 / / rw,relatime - shiftfs /var/snap/lxd/common/lxd/storage-pools/default/containers/test/rootfs rw,passthrough=3
2295 2258 0:128 /snap /snap rw,relatime shared:1015 - shiftfs /var/snap/lxd/common/lxd/storage-pools/default/containers/test/rootfs rw,passthrough=3

Okay, so that looks good and the device add still fails because of permissions?

It actually works! I must have missed the restart command. Thank’s so much for the help!

Excellent!
shiftfs is definitely the direction we’re going with on this. It’s currently opt-in (as you’ve noticed) because there are still some rough edges but the only issue I’m aware of right now affects btrfs, I have many systems using it on zfs, so far without issues.

What are the security implications for shiftfs compared to remapping only the uids/gids you need?

Well, it didn’t quite work out as I expected. The /video directory is owned by 1001:1001, as on the host, but the subdirectories (dvd, film and tv) are owned by root:root and are empty.

shiftfs effectively makes all access to the particular mount get unshifted before hitting the real mount on the host.

The main advantage is that every uid/gid in the container can be represented on it and so can use the mount as they normally would.

A potential security concern depending on visibility and access to the path on the host is that root in your container can create setuid files in the path which are then real setuid root binaries on the host. This can be prevented by having the mount on the host be nosuid or be hidden away from untrusted users.

A security advantage of shiftfs is that the container map does not need to be changed, so uid 1001 in your container is still 101001 at the kernel level and not 1001 as would be with a raw.idmap entry, so this prevents the container from doing resource denial of service attacks on a uid.