LXC opensuse unprivileged container

seven · March 25, 2019, 12:37pm

@stgraber @Jeremy Hello, today i revisited it, i somehow got it working; an issue though, i was trying to limit cpu, but i got the following error;

lxc-start -n my-cont -d -F
lxc-start: cgroups/cgfsng.c: lxc_cgroup_set_data: 1931 Failed to setup limits for the "cpuset" controller. The controller seems to be unused by "cgfsng" cgroup driver or not enabled on the cgroup hierarchy
                                                                                                  lxc-start: cgroups/cgfsng.c: cgfsng_setup_limits: 1968 No such file or directory - Error setting cpuset.cpus to 0 for my-cont
             lxc-start: start.c: lxc_spawn: 1356 Failed to setup cgroup limits for container "my-cont".
                                                                                                      lxc-start: start.c: __lxc_start: 1530 Failed to spawn container "my-cont".
                                                                       lxc-start: tools/lxc_start.c: main: 368 The container failed to start.
lxc-start: tools/lxc_start.c: main: 372 Additional information can be obtained by setting the --logfile and --logpriority options.

for more details on this, i filed an issue here;

stgraber · March 25, 2019, 7:57pm

Sounds like your kernel is missing that cgroup controller or your host doesn’t have it mounted?

seven · March 25, 2019, 9:19pm

@stgraber in either cases, what would i do please?

stgraber · March 27, 2019, 4:54pm

Either make the cpuset controller work on your system or don’t use any cpuset.cpus limits as those require that cgroup.

seven · March 28, 2019, 9:39am

@stgraber i get it, but if i knew how to make it work, i would not be posting here. can you help fixing the issue please?

stgraber · March 28, 2019, 2:33pm

I don’t have any opensuse knowledge so can’t help much there, was just trying to give you some pointers so you can hopefully do some googling and find someone who explains how to do this on your distro.

The second option may be easier for you though, just don’t use the cpuset cgroup.
Edit your container’s config and remove any line that reference lxc.cgroup.cpuset (looks like you have lxc.cgroup.cpuset.cpus=0) and your container should start fine.

seven · March 28, 2019, 3:57pm

@stgraber i understand, i been googling since i got the issue, can not find any solution, rather its more ubuntu/debian related and the only source of info is this forum or the github issues section, disabling cpu limit/ memory/swap is not an option, if i was to use lxc instead of kvm i need to be doing it right.
imagine i had ubuntu instead of suse, what will you do to fix this issue, change permission of something? add code of line somewhere? just something that can really help solving it, if it works on ubuntu it has to work on other linux destro, its just the matter of knowing what to modify, i think.

seven · March 29, 2019, 7:14am

@stgraber , i kept experimenting and i got the cpu to be limited to core 0 i think, i had to add cpuset,cpu,cpuacct to my common-sesion-pc, as follows;
session optional pam_cgfs.so -c freezer,memory,cpuset,cpu,cpuacct,
i do not know what dose that mean, adding this lines, can you explain it please? plus am unable to have limit for swaplxc-start: cgroups/cgfsng.c: cgfsng_setup_limits: 1968 Permission denied - Error setting memory.memsw.limit_in_bytes to 1G for my-cont, last question, where do i find all this limits or options, are they listed in some web-page?

seven · March 29, 2019, 8:07am

even container start, i kept debugging using a file, here is some of its contents;

      lxc-start 20190329080105.142 WARN     lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
      lxc-start 20190329080105.142 INFO     lxc_utils - utils.c:get_rundir:284 - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 20190329080105.142 WARN     lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
      lxc-start 20190329080105.142 DEBUG    lxc_start - start.c:setup_signal_fd:288 - Set SIGCHLD handler with file descriptor: 7.
      lxc-start 20190329080105.142 DEBUG    console - console.c:lxc_console_peer_default:450 - process does not have a controlling terminal
      lxc-start 20190329080105.142 DEBUG    lxc_conf - conf.c:chown_mapped_root:2830 - trying to chown "/dev/pts/7" to 1001
lxc-start 20190329080105.155 DEBUG    lxc_conf - conf.c:idmaptool_on_path_and_privileged:2516 - The binary "/usr/bin/newuidmap" does have the setuid bit set.
      lxc-start 20190329080105.155 DEBUG    lxc_conf - conf.c:idmaptool_on_path_and_privileged:2516 - The binary "/usr/bin/newgidmap" does have the setuid bit set.
      lxc-start 20190329080105.155 DEBUG    lxc_conf - conf.c:lxc_map_ids:2604 - Functional newuidmap and newgidmap binary found.
lxc-start 20190329080105.165 DEBUG    lxc_cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits:1971 - cgroup 'cpuset.cpus' set to '0'
      lxc-start 20190329080105.165 INFO     lxc_cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits:1976 - cgroup has been setup
      lxc-start 20190329080105.166 DEBUG    lxc_conf - conf.c:idmaptool_on_path_and_privileged:2516 - The binary "/usr/bin/newuidmap" does have the setuid bit set.
      lxc-start 20190329080105.166 DEBUG    lxc_conf - conf.c:idmaptool_on_path_and_privileged:2516 - The binary "/usr/bin/newgidmap" does have the setuid bit set.
      lxc-start 20190329080105.166 DEBUG    lxc_conf - conf.c:lxc_map_ids:2604 - Functional newuidmap and newgidmap binary found.
      lxc-start 20190329080105.181 WARN     lxc_cgfsng - cgroups/cgfsng.c:chown_cgroup_wrapper:1465 - Error chmoding /sys/fs/cgroup/unified/user.slice/user-1000.slice/session-1.scope/user/lxc/0/lxc/kaltura: No such file or directory
      lxc-start 20190329080105.183 INFO     lxc_network - network.c:lxc_create_network_unpriv_exec:2081 - Execing lxc-user-nic create /home/lxc/.local/share/lxc kaltura 4044 veth br0 (null)
      lxc-start 20190329080105.289 NOTICE   lxc_utils - utils.c:lxc_switch_uid_gid:2070 - Switched to gid 0.
      lxc-start 20190329080105.289 NOTICE   lxc_utils - utils.c:lxc_switch_uid_gid:2076 - Switched to uid 0.
      lxc-start 20190329080105.289 NOTICE   lxc_utils - utils.c:lxc_setgroups:2088 - Dropped additional groups.
 lxc-start 20190329080105.350 INFO     lxc_conf - conf.c:lxc_fill_autodev:1165 - Populating "/dev"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_fill_autodev:1209 - Bind mounted "/dev/null" onto "/usr/lib64/lxc/rootfs/dev/null"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_fill_autodev:1209 - Bind mounted "/dev/zero" onto "/usr/lib64/lxc/rootfs/dev/zero"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_fill_autodev:1209 - Bind mounted "/dev/full" onto "/usr/lib64/lxc/rootfs/dev/full"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_fill_autodev:1209 - Bind mounted "/dev/urandom" onto "/usr/lib64/lxc/rootfs/dev/urandom"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_fill_autodev:1209 - Bind mounted "/dev/random" onto "/usr/lib64/lxc/rootfs/dev/random"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_fill_autodev:1209 - Bind mounted "/dev/tty" onto "/usr/lib64/lxc/rootfs/dev/tty"
      lxc-start 20190329080105.350 INFO     lxc_conf - conf.c:lxc_fill_autodev:1216 - Populated "/dev"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_setup_dev_console:1537 - mounted pts device "/dev/pts/7" onto "/usr/lib64/lxc/rootfs/dev/console"
      lxc-start 20190329080105.350 INFO     lxc_utils - utils.c:lxc_mount_proc_if_needed:1755 - I am 1, /proc/self points to "1"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_rootfs_pivot_root:1068 - pivot_root syscall to '/usr/lib64/lxc/rootfs' successful
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_pivot_root:1377 - finished pivot root
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_setup_devpts:1416 - mount new devpts instance with options "newinstance,ptmxmode=0666,mode=0620,gid=5"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_setup_devpts:1436 - created dummy "/dev/ptmx" file as bind mount target
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_setup_devpts:1441 - bind mounted "/dev/pts/ptmx" to "/dev/ptmx"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_allocate_ttys:905 - allocated pty "/dev/pts/0" with master fd 11 and slave fd 14
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_allocate_ttys:905 - allocated pty "/dev/pts/1" with master fd 15 and slave fd 16
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_allocate_ttys:905 - allocated pty "/dev/pts/2" with master fd 17 and slave fd 18
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_allocate_ttys:905 - allocated pty "/dev/pts/3" with master fd 19 and slave fd 20
      lxc-start 20190329080105.350 INFO     lxc_conf - conf.c:lxc_allocate_ttys:925 - finished allocating 4 pts devices
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_setup_ttys:862 - Bind mounted "/dev/pts/0" onto "/dev/tty1"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_setup_ttys:862 - Bind mounted "/dev/pts/1" onto "/dev/tty2"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_setup_ttys:862 - Bind mounted "/dev/pts/2" onto "/dev/tty3"
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:lxc_setup_ttys:862 - Bind mounted "/dev/pts/3" onto "/dev/tty4"
      lxc-start 20190329080105.350 INFO     lxc_conf - conf.c:lxc_setup_ttys:871 - Finished setting up 4 /dev/tty<N> device(s)
      lxc-start 20190329080105.350 INFO     lxc_conf - conf.c:setup_personality:1477 - set personality to '0x0'
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_caps:2329 - drop capability 'mac_admin' (33)
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_caps:2329 - drop capability 'mac_override' (32)
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_caps:2329 - drop capability 'sys_time' (25)
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_caps:2329 - drop capability 'sys_module' (16)
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_caps:2329 - drop capability 'sys_rawio' (17)
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_caps:2329 - drop capability 'sys_nice' (23)
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_caps:2329 - drop capability 'sys_pacct' (20)
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_caps:2329 - drop capability 'sys_rawio' (17)
      lxc-start 20190329080105.350 DEBUG    lxc_conf - conf.c:setup_caps:2338 - capabilities have been setup
      lxc-start 20190329080105.350 NOTICE   lxc_conf - conf.c:lxc_setup:3221 - Container "kaltura" is set up
      lxc-start 20190329080105.350 INFO     lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:238 - changed apparmor profile to lxc-container-default-cgns
      lxc-start 20190329080105.351 INFO     lxc_cgfsng - cgroups/cgfsng.c:cgfsng_setup_limits:1976 - cgroup has been setup
      lxc-start 20190329080105.351 NOTICE   lxc_start - start.c:start:1603 - Exec'ing "/sbin/init".
      lxc-start 20190329080105.353 NOTICE   lxc_start - start.c:post_start:1614 - Started "/sbin/init" with pid "4044".
      lxc-start 20190329080105.353 INFO     lxc_utils - utils.c:get_rundir:284 - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 20190329080105.353 WARN     lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
      lxc-start 20190329080105.353 INFO     lxc_utils - utils.c:get_rundir:284 - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 20190329080105.353 WARN     lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
      lxc-start 20190329080105.353 NOTICE   lxc_start - start.c:signal_handler:337 - Received SIGCHLD from pid 4039 instead of container init 4044.

please have a look, dose any seem wrong/ requires to be fixed?

stgraber · March 29, 2019, 2:12pm

Nope, looks fine

seven · March 29, 2019, 4:16pm

@stgraber what would you say about post 28? Btw i do not seem to be able to autostart a container.