Hi all,
I am trying to run incus inside incus (for some of our test-cases), both run incus version 6.0.3.
It works fine to create a container with debian, set security.nesting=true on the container, install incus inside of the container, start and run containers as root in the container.
My problem is that I created a user on the container that belongs to incus group and tried to manage incus as this user.
I can create and init container as non-root user but when I try to start it I just get an error:
lxc user-1000_test 20250306145827.367 ERROR cgroup2_devices - ../src/lxc/cgroups/cgroup2_devices.c:bpf_program_load_kernel:332 - Operation not permitted - Failed to load bpf program: (null)
lxc user-1000_test 20250306145827.370 ERROR idmap_utils - ../src/lxc/idmap_utils.c:lxc_map_ids:245 - newuidmap failed to write mapping "newuidmap: write to uid_map failed: Operation not permitted": newuidmap 1262 0 1000000 1000000000
lxc user-1000_test 20250306145827.370 ERROR start - ../src/lxc/start.c:lxc_spawn:1795 - Failed to set up id mapping.
lxc user-1000_test 20250306145827.370 ERROR lxccontainer - ../src/lxc/lxccontainer.c:wait_on_daemonized_start:837 - Received container state "ABORTING" instead of "RUNNING"
lxc user-1000_test 20250306145827.370 ERROR start - ../src/lxc/start.c:__lxc_start:2114 - Failed to spawn container "user-1000_test"
lxc user-1000_test 20250306145827.370 WARN start - ../src/lxc/start.c:lxc_abort:1037 - No such process - Failed to send SIGKILL via pidfd 17 for process 1262
Is it just a bad idea to run containers as a non-root user inside a container or am I missing something that will make it work?
Unless you have somehow provided your parent container with a LOT of uid/gid, you won’t be able to nest unprivileged containers inside of an unprivileged container.
But that’s not really a problem as “privileged” containers running inside of an unprivileged container are still effectively unprivileged containers.
So anyway, running incus profile set default security.privileged=true within the first level container should let you run nested containers just fine.
Note that incus admin init has logic to detect this case, so I’m surprised that it didn’t tell you about this situation and offered to set security.privileged=true for you.
Thanks @stgraber,
Yes, you are absolutely right, as the base container is unprivileged the nested container is still unprivileged.
I was just checking that I did not miss any obvious step and to me it is fine to run as incus-admin user.
When I run incus admin init inside the container the only different output I get compared to a “normal” init is:
We detected that you are running inside an unprivileged container.
This means that unless you manually configured your host otherwise,
you will not have enough uids and gids to allocate to your containers.
Your container's own allocation can be re-used to avoid the problem.
Doing so makes your nested containers slightly less safe as they could
in theory attack their parent container and gain more privileges than
they otherwise would.
So there is no info about security.privileged=true
Thanks for all the awesome work with incus, it is just great!!!
incus launch images:ubuntu/24.04/cloud foo -c security.nesting=true -c security.syscalls.intercept.mknod=true -c security.syscalls.intercept.setxattr=true
incus shell foo
### from here everything is inside "foo" ###
apt -y install curl
curl -fsSL https://pkgs.zabbly.com/key.asc -o /etc/apt/keyrings/zabbly.asc
sh -c 'cat <<EOF > /etc/apt/sources.list.d/zabbly-incus-lts-6.0.sources
Enabled: yes
Types: deb
URIs: https://pkgs.zabbly.com/incus/lts-6.0
Suites: $(. /etc/os-release && echo ${VERSION_CODENAME})
Components: main
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/zabbly.asc
EOF'
apt update
apt -y install incus-base # I got 1:6.0.6-ubuntu24.04-202603151019
incus admin init --minimal
incus profile set default security.privileged=true
incus launch images:ubuntu/24.04/cloud bar
incus shell bar
### from here everything is inside "bar" ###
resolvectl status
The physical host is ubuntu 22.04, kernel 5.15.0-173-generic, running incus 6.22 from Zabbly, and I see no errors in dmesg (which is where I’d expect to see apparmor problems)