Unprivileged LXC on Yocto

LXC
#1

Hello,

I have a problem with running unprivileged LXC on Yocto. LXC version is 3.1.0 and privileged LXC works fine.

Log file : https://pastebin.com/zYJFZs2Y

/etc/subuid and /etc/subgid is set to :

root:1000000:65536

lxc config is :

lxc.idmap = u 0 1000000 65536
lxc.idmap = g 0 1000000 65536
lxc.log.file = /home/user/martini/fedora/logfile.log
lxc.console.logfile = /home/user/martini/fedora/console.log
lxc.rootfs.path = /home/user/fedora/rootfs

Output of cat /proc/self/mounts : https://pastebin.com/Umwd3U5r

lxc-checkconfig shows that everything is enabled.

Thanks in advance.

#2

If you can, would be nice to switch to LXC 4.0.x so it’s a bit easier to figure this kind of stuff out. Also bumping logging with lxc.log.level=trace would help too.

#3

@pericdario

Are you by chance running LVM volumes? Also, what do you get when you run this:

/sbin/sysctl -a | grep kernel.unprivileged

#4

Hi @williamblair,

Output of /sbin/sysctl -a | grep kernel.unprivileged is :

sysctl: error reading key 'net.ipv6.conf.all.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.default.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.eth0.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.ip6_vti0.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.ip6tnl0.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.ip_vti0.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.lo.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.lxcbr0.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.sit0.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.tunl0.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.usb0.stable_secret': Input/output error
sysctl: error reading key 'net.ipv6.conf.vethL8SA7O.stable_secret': Input/output error

I changed owner for /dev/mmcblk0p10 to 1000000:1000000 and mount error for /proc disappeared. Rootfs is located at /dev/mmcblk0p10 so maybe it was one of the causes for the problem.

The new error I’m getting is :

conf - conf.c:lxc_mount_rootfs:1342 - Failed to mount rootfs “/home/user/fedora/rootfs” onto “/usr/lib64/lxc/rootfs” with options “(null)”.

Full log : https://pastebin.com/4H84TSwS

#5

@pericdario I was expecting something like this:

root@lxc-host-machine:/home/lxc# /sbin/sysctl -a | grep kernel.unprivileged
kernel.unprivileged_bpf_disabled = 0
kernel.unprivileged_userns_apparmor_policy = 1
kernel.unprivileged_userns_clone = 0

I don’t know why you are getting input/output errors. su to root and type this directly:

/sbin/sysctl -n kernel.unprivileged_userns_clone

Do you get a 0 or a 1 returned?

Also, what does your container config look like? Or did you put in pastebin and I missed it. It should look something like this:

`lxc.group = onboot
lxc.start.autostart = 1

lxc.include = /usr/share/lxc/config/common.conf
lxc.include = /usr/share/lxc/config/userns.conf
lxc.arch = linux64

lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536
lxc.rootfs.path = btrfs:/home/lxc/.local/share/lxc/wt-relay-01/rootfs
lxc.uts.name = wt-relay-01

lxc.net.0.type = veth
lxc.net.0.link = br0

#6

Hi @stgraber,

I have tried switching to LXC 4, however I faced Namespace error:
Failed to clone a new set of namespaces,
that was issued on this post: https://github.com/lxc/lxd/issues/4720
(even though lxc-checkconifg showed that everything was enabled)

You suggested that it was probably a kernel issue but I was able to use LXC 3 with no problems on the same kernel, so I had to go with LXC 3.

#7

@williamblair i missed out to copy :

kernel.unprivileged_bpf_disabled = 0

So the output looks like :

kernel.unprivileged_bpf_disabled = 0
sysctl: error reading key ‘net.ipv6.conf.all.stable_secret’: Input/output error
sysctl: error reading key ‘net.ipv6.conf.default.stable_secret’: Input/output error
sysctl: error reading key ‘net.ipv6.conf.eth0.stable_secret’: Input/output error
sysctl: error reading key ‘net.ipv6.conf.ip6_vti0.stable_secret’: Input/output error
sysctl: error reading key ‘net.ipv6.conf.ip6tnl0.stable_secret’: Input/output error
sysctl: error reading key ‘net.ipv6.conf.ip_vti0.stable_secret’: Input/output error
sysctl: error reading key ‘net.ipv6.conf.lo.stable_secret’: Input/output error
sysctl: error reading key ‘net.ipv6.conf.lxcbr0.stable_secret’: Input/output error
sysctl: error reading key ‘net.ipv6.conf.sit0.stable_secret’: Input/output error
sysctl: error reading key ‘net.ipv6.conf.tunl0.stable_secret’: Input/output error
sysctl: error reading key ‘net.ipv6.conf.usb0.stable_secret’: Input/output error

I ran the /sbin/sysctl -a | grep kernel.unprivileged command as root and the output was the same as in my prevous post

I posted LXC config in my first post:

lxc.idmap = u 0 1000000 65536
lxc.idmap = g 0 1000000 65536
lxc.log.file = /home/user/martini/fedora/logfile.log
lxc.console.logfile = /home/user/martini/fedora/console.log
lxc.rootfs.path = /home/user/fedora/rootfs

The output of /sbin/sysctl -n kernel.unprivileged_userns_clone :
sysctl: error: 'kernel.unprivileged_userns_clone' is an unknown key

Maybe kernel.unprivileged_userns_clone is causing the problem?

#8

@pericdario type this in and then attempt to bring your container back up:

/sbin/sysctl -w kernel.unprivileged_userns_clone=1

If the container then works, yah!! Then, see if this will make it permanent through reboots:

echo 'kernel.unprivileged_userns_clone=1' > /etc/sysctl.d/userns.conf

I don’t know much about Yoco so it could be different.

#10

@williamblair i already tried that. Output is :

sysctl: error: 'kernel.unprivileged_userns_clone' is an unknown key

Maybe this is a kernel problem?

#11

Ok , what I was referring was regarding Debian use namespaces specifically (I just found this out) My apologies. So, in light of that, I would take a look and see if user namespaces are enabled on your distro build.

My apologies.

#12

Yes, user namespaces are enabled. This patch will probably solve a problem.

#13

Please keep us posted and hopefully we’ve helped save you some time at least.

#14

After applying patch i got back to problem with proc.

#15

What is happening now?

#16

@williamblair

This is a new log : https://pastebin.com/uL4anB2M
This is a old one without /proc problem : https://pastebin.com/4H84TSwS

I can’t figured out how “/proc error” disappeared before this patch for user namespace.
The good new is that this patch solved a problem :

 ERROR    conf - conf.c:lxc_mount_rootfs:1342 - Failed to mount rootfs "/home/user/fedora/rootfs" onto "/usr/lib64/lxc/rootfs" with options "(null)"

I tried again with changing owner of /dev/mmcblk0p10 but probably that didn’t solve a problem.

Is there possibility that lxc. log didn’t write problem, while the problem is still there?

For example i can mount proc like :

    mkdir /mnt/proc
    mount -t proc proc /mnt/pro
#17

@pericdario I don’t think I’m capable of assisting you. My apologies.

@stgraber any ideas?

1 Like
#18

@brauner tends to be the goto for weird environments

#19

@stgraber

This is a log from LXC 4 : https://pastebin.com/ssCHbEtB

Any idea?

#20

Ok, so still unable to mount proc.
Can you try mounting a clean copy of proc somewhere (like you did above) and keep it around, then see if LXC behaves better?

There is a kernel feature which prevents an unprivileged user from mounting a new copy of proc unless a non-overmounted copy of proc is available on the host somewhere accessible.

#21

Nothing changed :frowning: