Container mount with user 'nobody' and group 'nogroup'

LXD
1

Hello,

I am currently in ZFS on Linux for my host server and I am running with snapd for LXD.
I had a problem today where I lost the whole user and group in my container. They all went to nobody:nogroup.

root@box:/# id nobody
uid=65534(nobody) gid=65534(nogroup) groups=65534(nogroup)

Example:

ls -lha /
drwxr-xr-x  22 nobody nogroup  22 Apr  5  2018 .
drwxr-xr-x  22 nobody nogroup  22 Apr  5  2018 ..
drwxr-xr-x   2 nobody nogroup 172 Nov  7 12:42 bin
drwxr-xr-x   3 nobody nogroup   3 Apr  5  2018 boot
drwxr-xr-x   8 root   root    500 Nov 19 12:02 dev
drwxr-xr-x 105 nobody nogroup 199 Nov 16 06:33 etc
drwxr-xr-x   3 nobody nogroup   3 Apr 10  2018 home
drwxr-xr-x  22 nobody nogroup  25 Dec 16  2018 lib
drwxr-xr-x   2 nobody nogroup   3 Dec 16  2018 lib64
drwxr-xr-x   7 nobody nogroup   7 Oct  3  2018 media
drwxr-xr-x   2 nobody nogroup   2 Apr  5  2018 mnt
drwxr-xr-x   2 nobody nogroup   2 Apr  5  2018 opt
dr-xr-xr-x 984 nobody nogroup   0 Nov 19 12:02 proc
drwx------   8 nobody nogroup  17 Nov 14 23:35 root
drwxr-xr-x  17 root   root    620 Nov 19 12:02 run
drwxr-xr-x   2 nobody nogroup 229 Oct 10 10:41 sbin
drwxr-xr-x   2 nobody nogroup   3 May 23  2018 snap
drwxr-xr-x   3 nobody nogroup   3 Apr 16  2018 srv
dr-xr-xr-x  13 nobody nogroup   0 Nov 19 11:56 sys
drwxrwxrwt   9 nobody nogroup   9 Nov 19 12:09 tmp
drwxr-xr-x  10 nobody nogroup  10 Apr  5  2018 usr
drwxr-xr-x  14 nobody nogroup  16 Apr 11  2018 var

OS: Ubuntu 18.04.3 LTS
LXD Version: 3.18

I tried to rollback the container but the owner/group has not changed and the problem is still there. Let me know if you need more information.

2

Looks like the uid/gid map isnā€™t applied or the filesystem is wrongly shifted.

Can you show (need to run as root):

  • lxc config show --expanded NAME
  • lxc exec NAME ā€“ grep shiftfs /proc/mounts
  • ls -lh /var/snap/lxd/common/mntns/var/snap/lxd/common/lxd/storage-pools/default/containers/NAME/rootfs/

That may give us some clues as to whatā€™s going on here.

3 
# lxc config show --expanded box
architecture: x86_64
config:
  boot.autostart: "1"
  boot.autostart.priority: "0"
  image.architecture: amd64
  image.description: ubuntu 16.04 LTS amd64 (release) (20180405)
  image.label: release
  image.os: ubuntu
  image.release: xenial
  image.serial: "20180405"
  image.version: "16.04"
  limits.memory: 1GB
  security.privileged: "false"
  volatile.base_image: be7cec7c948958adfbb9bc7dbd292762d2388cc883466815fc2b6bc06bf06f5a
  volatile.eth0.host_name: vethd877a234
  volatile.eth0.hwaddr: 00:16:3e:c7:b6:04
  volatile.eth0.name: eth0
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    nictype: bridged
    parent: br0
    type: nic
  root:
    path: /
    pool: default
    type: disk
  storage1:
    path: /media/storage1
    source: /media/storage1
    type: disk
  storage2:
    path: /media/storage2
    source: /media/storage2
    type: disk
  storage3:
    path: /media/storage3
    source: /media/storage3
    type: disk
  storage4:
    path: /media/storage4
    source: /media/storage4
    type: disk
ephemeral: false
profiles:
- default
- eth0
- mem-1GB
stateful: false
description: ""

# lxc exec box ā€“ grep shiftfs /proc/mounts
#

# ls -lh /var/snap/lxd/common/mntns/var/snap/lxd/common/lxd/storage-pools/default/containers/box/rootfs/
total 106K
drwxr-xr-x   2 165536 165536 172 Nov  7 12:42 bin
drwxr-xr-x   3 165536 165536   3 Apr  5  2018 boot
drwxr-xr-x   4 165536 165536  90 Apr  5  2018 dev
drwxr-xr-x 105 165536 165536 199 Nov 16 06:33 etc
drwxr-xr-x   3 165536 165536   3 Apr 10  2018 home
drwxr-xr-x  22 165536 165536  25 Dec 16  2018 lib
drwxr-xr-x   2 165536 165536   3 Dec 16  2018 lib64
drwxr-xr-x   7 165536 165536   7 Oct  3  2018 media
drwxr-xr-x   2 165536 165536   2 Apr  5  2018 mnt
drwxr-xr-x   2 165536 165536   2 Apr  5  2018 opt
drwxr-xr-x   2 165536 165536   2 Apr 12  2016 proc
drwx------   8 165536 165536  17 Nov 14 23:35 root
drwxr-xr-x   4 165536 165536   4 Apr  5  2018 run
drwxr-xr-x   2 165536 165536 229 Oct 10 10:41 sbin
drwxr-xr-x   2 165536 165536   3 May 23  2018 snap
drwxr-xr-x   3 165536 165536   3 Apr 16  2018 srv
drwxr-xr-x   2 165536 165536   2 Feb  5  2016 sys
drwxrwxrwt   9 165536 165536   9 Nov 20 18:39 tmp
drwxr-xr-x  10 165536 165536  10 Apr  5  2018 usr
drwxr-xr-x  14 165536 165536  16 Apr 11  2018 var
4

Yeah, so the map doesnā€™t look correct.
You could try:

  • lxc stop box
  • lxc config set box volatile.idmap.current ā€˜[{ā€œIsuidā€:true,ā€œIsgidā€:false,ā€œHostidā€:165536,ā€œNsidā€:0,ā€œMaprangeā€:1000000000},{ā€œIsuidā€:false,ā€œIsgidā€:true,ā€œHostidā€:165536,ā€œNsidā€:0,ā€œMaprangeā€:1000000000}]ā€™
  • lxc start box

This should then remap your container (can take a while if large).

5

I just tried to add the parameter to the configuration but I noticed that it was at each start of the container overwrite to the default value:

volatile.idmap.current: ā€˜[{ā€œIsuidā€:true,ā€œIsgidā€:false,ā€œHostidā€:1000000,ā€œNsidā€:0,ā€œMaprangeā€:1000000000},{ā€œIsuidā€:false,ā€œIsgidā€:true,ā€œHostidā€:1000000,ā€œNsidā€:0,ā€œMaprangeā€:1000000000}]ā€™

No change of ID is visible when I run your command:

lxc config set box volatile.idmap.current ā€˜[{ā€œIsuidā€:true,ā€œIsgidā€:false,ā€œHostidā€:165536,ā€œNsidā€:0,ā€œMaprangeā€:1000000000},{ā€œIsuidā€:false,ā€œIsgidā€:true,ā€œHostidā€:165536,ā€œNsidā€:0,ā€œMaprangeā€:1000000000}]ā€™

To apply it at startup I tried to modify the following parameter:

volatile.idmap.next: ā€˜[{ā€œIsuidā€:true,ā€œIsgidā€:false,ā€œHostidā€:165536,ā€œNsidā€:0,ā€œMaprangeā€:1000000000},{ā€œIsuidā€:false,ā€œIsgidā€:true,ā€œHostidā€:165536,ā€œNsidā€:0,ā€œMaprangeā€:1000000000}]ā€™

My container has updated the user and group id but it has been replaced by ā€œ165536ā€ and not by ā€œ0ā€:

root@box:~# ls -lha /
total 113K
drwxr-xr-x 22 165536 165536 22 Apr 5 2018 .
drwxr-xr-x 22 165536 165536 22 Apr 5 2018 ā€¦
drwxr-xr-x 2 165536 165536 172 Nov 7 12:42 bin
drwxr-xr-x 3 165536 165536 3 Apr 5 2018 boot
drwxr-xr-x 8 root root 500 Nov 21 10:29 dev
drwxr-xr-x 105 165536 165536 199 Nov 21 10:29 etc
drwxr-xr-x 3 165536 165536 3 Apr 10 2018 home
drwxr-xr-x 22 165536 165536 25 Dec 16 2018 lib
drwxr-xr-x 2 165536 165536 3 Dec 16 2018 lib64
drwxr-xr-x 7 165536 165536 7 Oct 3 2018 media
drwxr-xr-x 2 165536 165536 2 Apr 5 2018 mnt
drwxr-xr-x 2 165536 165536 2 Apr 5 2018 opt
dr-xr-xr-x 1000 nobody nogroup 0 Nov 21 10:29 proc
drwx------ 8 165536 165536 17 Nov 14 23:35 root
drwxr-xr-x 16 root root 680 Nov 21 10:29 run
drwxr-xr-x 2 165536 165536 229 Oct 10 10:41 sbin
drwxr-xr-x 2 165536 165536 3 May 23 2018 snap
drwxr-xr-x 3 165536 165536 3 Apr 16 2018 srv
dr-xr-xr-x 13 nobody nogroup 0 Nov 19 11:56 sys
drwxrwxrwt 3 root root 3 Nov 21 10:39 tmp
drwxr-xr-x 10 165536 165536 10 Apr 5 2018 usr

Additional information:

$ ls -lh /var/snap/lxd/common/mntns/var/snap/lxd/common/lxd/storage-pools/default/containers/box/rootfs/
total 106K
drwxr-xr-x 2 331072 331072 172 Nov 7 12:42 bin
drwxr-xr-x 3 331072 331072 3 Apr 5 2018 boot
drwxr-xr-x 4 331072 331072 90 Apr 5 2018 dev
drwxr-xr-x 105 331072 331072 199 Nov 21 10:29 etc
drwxr-xr-x 3 331072 331072 3 Apr 10 2018 home
drwxr-xr-x 22 331072 331072 25 Dec 16 2018 lib
drwxr-xr-x 2 331072 331072 3 Dec 16 2018 lib64
drwxr-xr-x 7 331072 331072 7 Oct 3 2018 media
drwxr-xr-x 2 331072 331072 2 Apr 5 2018 mnt
drwxr-xr-x 2 331072 331072 2 Apr 5 2018 opt
drwxr-xr-x 2 331072 331072 2 Apr 12 2016 proc
drwx------ 8 331072 331072 17 Nov 14 23:35 root
drwxr-xr-x 4 331072 331072 4 Apr 5 2018 run
drwxr-xr-x 2 331072 331072 229 Oct 10 10:41 sbin
drwxr-xr-x 2 331072 331072 3 May 23 2018 snap
drwxr-xr-x 3 331072 331072 3 Apr 16 2018 srv
drwxr-xr-x 2 331072 331072 2 Feb 5 2016 sys
drwxrwxrwt 3 165536 165536 3 Nov 21 10:39 tmp
drwxr-xr-x 10 331072 331072 10 Apr 5 2018 usr
drwxr-xr-x 14 331072 331072 16 Apr 11 2018 var

6

Try:

  • lxc stop box
  • lxc config set volatile.last_state.idmap ā€˜[{ā€œIsuidā€:true,ā€œIsgidā€:false,ā€œHostidā€: 165536,ā€œNsidā€:0,ā€œMaprangeā€:1000000000},{ā€œIsuidā€:false,ā€œIsgidā€:true,ā€œHostidā€:165536,ā€œNsidā€:0,ā€œMaprangeā€:1000000000}]ā€™
  • lxc start box
1 Like
7

You probably mean:

lxc config set box volatile.last_state.idmap ā€˜[{ā€œIsuidā€:true,ā€œIsgidā€:false,ā€œHostidā€: 165536,ā€œNsidā€:0,ā€œMaprangeā€:1000000000},{ā€œIsuidā€:false,ā€œIsgidā€:true,ā€œHostidā€:165536,ā€œNsidā€:0,ā€œMaprangeā€:1000000000}]ā€™

I have regained all my rights to my files and folders. Everything works thanks to your help.