Mount problem LXC 3.0.3

I am trying to mount the java directory into an unprivileged LXC container on Ubuntu 18.04. This worked fine in Ubuntu 17.10 but I have no success in Ubuntu 18.04.
The container needs a bridge network device. That initially did not worked, but after I installed 3.0.3 from https://www.ubuntuupdates.org/package/core/bionic/universe/proposed/lxc it worked.
(dpkg -i liblxc1_3.0.3-0ubuntu1~18.04.1_amd64.deb liblxc-common_3.0.3-0ubuntu1~18.04.1_amd64.deb lxc_3.0.3-0ubuntu1~18.04.1_all.deb lxcfs_3.0.3-0ubuntu1~18.04.1_amd64.deb lxc-utils_3.0.3-0ubuntu1~18.04.1_amd64.deb)

When I add:
lxc.mount.entry = /usr/lib/jvm/java-8-openjdk-amd64 opt/java none defaults,bind,relative,create=dir 0 0
I get an error:
ubuntu:~$ lxc-start -n centos2 -F -o out.tmp -l=TRACE
lxc-start: centos2: utils.c: safe_mount: 1179 No such file or directory - Failed to mount “/usr/lib/x86_64-linux-gnu/lxc//usr/lib/jvm/java-8-openjdk-amd64” onto “/usr/lib/x86_64-linux-gnu/lxc/opt/java”
lxc-start: centos2: conf.c: mount_entry: 2019 No such file or directory - Failed to mount “/usr/lib/x86_64-linux-gnu/lxc//usr/lib/jvm/java-8-openjdk-amd64” on “/usr/lib/x86_64-linux-gnu/lxc/opt/java”
lxc-start: centos2: conf.c: lxc_setup: 3611 Failed to setup mount entries
lxc-start: centos2: start.c: do_start: 1263 Failed to setup container “centos2” lxc-start: centos2: sync.c: __sync_wait: 62 An error occurred in another process (expected sequence number 5)
lxc-start: centos2: start.c: __lxc_start: 1939 Failed to spawn container “centos2” lxc-start: centos2: tools/lxc_start.c: main: 330 The container failed to start

I can see that the /opt/java directory is successfully created inside the containers rootfs.
I got the same error even when I rename java-8-openjdk-amd64, so the path does not exist when I try to start the container.
lxc.mount.entry = /usr/lib/jvm/java-8-openjdk-amd64_none opt/java none defaults,bind,relative,create=dir 0 0

Do I have to change any file permissions, ACL:s or AppArmor preferences to make it work or are my lxc.mount.entry configuration wrong?
/Arvid

@brauner can you take a look at this?

Please show the config file.
I can’t reproduce this.
For some reason lxc seems to prepend the mount point for the rootfs to the path:

/usr/lib/x86_64-linux-gnu/lxc//usr/lib/jvm/java-8-openjdk-amd64

and that’s very odd.

Thank you!
Here are the config files. ( comments removed) All lxc.include files are not changed, but I include them to show the entire configuration.
The rootfs is created from : lxc-create -t download -n centos2

cat .local/share/lxc/centos2/config
lxc.include = /usr/share/lxc/config/common.conf
lxc.include = /usr/share/lxc/config/userns.conf
lxc.arch = x86_64
lxc.include = /etc/lxc/default.conf
lxc.idmap = u 0 231072 65536
lxc.idmap = g 0 231072 65536
lxc.rootfs.path = dir:/home/testing/.local/share/lxc/centos2/rootfs
lxc.uts.name = centos2
lxc.net.0.type = veth
lxc.net.0.flags = up
lxc.net.0.link = brtcp
lxc.mount.entry = /usr/lib/jvm/java-8-openjdk-amd64 opt/java none defaults,bind,relative,create=dir 0 0

cat /etc/lxc/default.conf
lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:3e:xx:xx:xx

cat /usr/share/lxc/config/common.conf
lxc.tty.dir = lxc
lxc.pty.max = 1024
lxc.tty.max = 4
lxc.cap.drop = mac_admin mac_override sys_time sys_module sys_rawio
lxc.hook.clone = /usr/share/lxc/hooks/clonehostname
lxc.cgroup.devices.deny = a
lxc.cgroup.devices.allow = c : m
lxc.cgroup.devices.allow = b : m
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
lxc.cgroup.devices.allow = c 1:7 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:2 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 10:229 rwm
lxc.mount.auto = cgroup:mixed proc:mixed sys:mixed
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none bind,optional 0 0
lxc.seccomp.profile = /usr/share/lxc/config/common.seccomp
lxc.include = /usr/share/lxc/config/common.conf.d/

cat /usr/share/lxc/config/common.conf.d/00-lxcfs.conf
lxc.hook.mount = /usr/share/lxcfs/lxc.mount.hook
lxc.hook.post-stop = /usr/share/lxcfs/lxc.reboot.hook

cat /usr/share/lxc/config/common.seccomp
2
blacklist
reject_force_umount # comment this to allow umount -f; not recommended
[all]
kexec_load errno 1
open_by_handle_at errno 1
init_module errno 1
finit_module errno 1
delete_module errno 1

cat /usr/share/lxc/config/userns.conf
lxc.cgroup.devices.deny =
lxc.cgroup.devices.allow =
lxc.cap.drop =
lxc.cap.keep =
lxc.tty.dir =
lxc.mount.auto = sys:rw
/Arvid

Please remove relative option in mount entry

1 Like

Thank you very much!

I can confirm that if a remove ’relative’ from the mount line everything works as expected.

Old line:

lxc.mount.entry = /usr/lib/jvm/java-8-openjdk-amd64 opt/java none defaults,bind,relative,create=dir 0 0

Updated line:

lxc.mount.entry = /usr/lib/jvm/java-8-openjdk-amd64 opt/java none defaults,bind,create=dir 0 0

Trying to mount a host folder in LXC unprivileged container (being run by non-root user) on fedora 29 host. When the container is started, it fails to start and gives error:

lxc-start: centos1: utils.c: safe_mount: 1179 Permission denied - Failed to mount "/host/path" onto "/usr/lib64/lxc/rootfs/ctfolder"

Config line being used:

lxc.mount.entry = /host/path ctfolder none defaults,bind,create=dir 0 0

It appears that wrong rootfs location is being prepended to mount point.
Correct rootfs location (which should be used) is /home//.local/share/lxc/centos1/rootfs

LXC: 3.0.3
Kernel: 4.20.8

Does anyone have any idea what might be going wrong?

The mount path is correct, that’s not the issue (it’s a temporary mount path in a separate mntns).

It may be that the root user in the container cannot see /host/path due to permissions or that you have sub-mounts on /host/path which triggers the kernel over-mounting protection.

Thank you for the prompt reply and resolution! It indeed was a permission issue. /usr/lib64/lxc/rootfs/ctfolder in error log was red herring.

Did a quick check by setting a temp host folder to “777” and mounting it inside container - it works!

It now seems obvious that “root inside container should have correct permissions to access/modify host folder being mounted.”