Hi folks!
context! we’re building a bridge between Kubernetes’ CRI API and LXD.
We’re quite far right now, but we’re trying to smooth out some edges with regard to networking.
We’re using Kubernetes’ CNI to get an IP, but for that we need a PID, running in the container’s namespace.
Docker under Kubernetes solves this with the infamous “pause” container. We built our own pause container and are now trying to figure out how to start a second (and third, and etc…) container in the network namespace of the pause container.
lxc-start has a --share-net=, but LXD doesn’t seem to have an equivalent.
So we resorted to setting raw.lxc = lxc.namespace.share.net=pause, and then promptly hit another stump:
root@minion1:~# lxc info --show-log ubuntu | grep -v seccomp
Name: ubuntu
Remote: unix://
Architecture: x86_64
Created: 2018/06/14 13:53 UTC
Status: Stopped
Type: persistent
Profiles: default
Log:
lxc ubuntu 20180614144624.374 INFO lxc_container - lxccontainer.c:do_lxcapi_start:948 - Attempting to set proc title to [lxc monitor] /var/snap/lxd/common/lxd/containers ubuntu
lxc ubuntu 20180614144624.375 INFO lxc_start - start.c:lxc_check_inherited:285 - Closed inherited fd 3
lxc ubuntu 20180614144624.375 INFO lxc_start - start.c:lxc_check_inherited:285 - Closed inherited fd 5
lxc ubuntu 20180614144624.375 INFO lxc_start - start.c:lxc_check_inherited:285 - Closed inherited fd 17
lxc ubuntu 20180614144624.375 INFO lxc_lsm - lsm/lsm.c:lsm_init:46 - LSM security driver AppArmor
lxc ubuntu 20180614144624.375 INFO lxc_conf - conf.c:run_script_argv:369 - Executing script "/snap/lxd/current/bin/lxd callhook /var/snap/lxd/common/lxd 833 start" for container "ubuntu", config section "lxc"
lxc ubuntu 20180614144624.416 INFO lxc_start - start.c:lxc_init:858 - Container "ubuntu" is initialized
lxc ubuntu 20180614144624.419 INFO lxc_start - start.c:lxc_spawn:1641 - Cloned CLONE_NEWUSER
lxc ubuntu 20180614144624.419 INFO lxc_start - start.c:lxc_spawn:1641 - Cloned CLONE_NEWNS
lxc ubuntu 20180614144624.419 INFO lxc_start - start.c:lxc_spawn:1641 - Cloned CLONE_NEWPID
lxc ubuntu 20180614144624.419 INFO lxc_start - start.c:lxc_spawn:1641 - Cloned CLONE_NEWUTS
lxc ubuntu 20180614144624.419 INFO lxc_start - start.c:lxc_spawn:1641 - Cloned CLONE_NEWIPC
lxc ubuntu 20180614144624.420 WARN lxc_conf - conf.c:lxc_map_ids:2855 - newuidmap binary is missing
lxc ubuntu 20180614144624.420 WARN lxc_conf - conf.c:lxc_map_ids:2861 - newgidmap binary is missing
lxc ubuntu 20180614144624.420 INFO lxc_caps - caps.c:lxc_ambient_caps_up:175 - Last supported cap was 36
lxc ubuntu 20180614144624.440 WARN lxc_conf - conf.c:lxc_map_ids:2855 - newuidmap binary is missing
lxc ubuntu 20180614144624.440 WARN lxc_conf - conf.c:lxc_map_ids:2861 - newgidmap binary is missing
lxc ubuntu 20180614144624.441 NOTICE lxc_utils - utils.c:lxc_switch_uid_gid:2059 - Switched to gid 0.
lxc ubuntu 20180614144624.441 NOTICE lxc_utils - utils.c:lxc_switch_uid_gid:2065 - Switched to uid 0.
lxc ubuntu 20180614144624.441 NOTICE lxc_utils - utils.c:lxc_setgroups:2077 - Dropped additional groups.
lxc ubuntu 20180614144624.441 INFO lxc_start - start.c:do_start:1198 - Unshared CLONE_NEWCGROUP
lxc ubuntu 20180614144624.442 INFO lxc_conf - conf.c:setup_utsname:787 - Set hostname to "ubuntu"
lxc ubuntu 20180614144624.442 INFO lxc_conf - conf.c:mount_autodev:1194 - Preparing "/dev"
lxc ubuntu 20180614144624.442 INFO lxc_conf - conf.c:mount_autodev:1216 - Mounted tmpfs on "/var/snap/lxd/common/lxc//dev"
lxc ubuntu 20180614144624.442 INFO lxc_conf - conf.c:mount_autodev:1233 - Prepared "/dev"
lxc ubuntu 20180614144624.442 ERROR lxc_utils - utils.c:safe_mount:1671 - Operation not permitted - Failed to mount sysfs onto /var/snap/lxd/common/lxc//sys
lxc ubuntu 20180614144624.442 ERROR lxc_conf - conf.c:lxc_mount_auto_mounts:720 - Operation not permitted - Failed to mount "sysfs" on "/var/snap/lxd/common/lxc//sys" with flags 0
lxc ubuntu 20180614144624.442 ERROR lxc_conf - conf.c:lxc_setup:3412 - Failed to setup first automatic mounts
lxc ubuntu 20180614144624.442 ERROR lxc_start - start.c:do_start:1219 - Failed to setup container "ubuntu"
lxc ubuntu 20180614144624.442 ERROR lxc_sync - sync.c:__sync_wait:57 - An error occurred in another process (expected sequence number 5)
lxc ubuntu 20180614144624.442 ERROR lxc_start - start.c:__lxc_start:1887 - Failed to spawn container "ubuntu"
lxc ubuntu 20180614144624.443 WARN lxc_conf - conf.c:lxc_map_ids:2855 - newuidmap binary is missing
lxc ubuntu 20180614144624.443 WARN lxc_conf - conf.c:lxc_map_ids:2861 - newgidmap binary is missing
lxc ubuntu 20180614144624.444 INFO lxc_conf - conf.c:run_script_argv:369 - Executing script "/snap/lxd/current/lxcfs/lxc.reboot.hook" for container "ubuntu", config section "lxc"
lxc ubuntu 20180614144624.445 ERROR lxc_container - lxccontainer.c:wait_on_daemonized_start:834 - Received container state "ABORTING" instead of "RUNNING"
lxc 20180614144624.446 WARN lxc_commands - commands.c:lxc_cmd_rsp_recv:130 - Connection reset by peer - Failed to receive response for command "get_state"
lxc ubuntu 20180614144624.950 INFO lxc_conf - conf.c:run_script_argv:369 - Executing script "/snap/lxd/current/bin/lxd callhook /var/snap/lxd/common/lxd 833 stop" for container "ubuntu", config section "lxc"