I’m running LXC on a remote system connected via sshd (you can also take a local SCHED_RR ‘bash’ as a replacement - see bottom of post).
When the ssh server was started with regular scheduler (SCHED_OTHER) i can start/stop/attach containers without problems.
$ ps ax --format uname,pid,ppid,tty,cmd,cls,ni,pri,rtprio |grep sshd
USER PID PPID TT CMD CLS NI PRI RTPRIO
root 9379 1 ? /usr/sbin/sshd -D TS 0 19 -
root 9446 9379 ? sshd: root@pts/0 TS 0 19 -
But when sshd was spawned with a realtime scheduler (SCHED_RR) the lxc commands do not work anymore (already running or otherwise spawned containers still work properly though in the background).
$ chrt -r 90 /usr/sbin/sshd
$ ps ax --format uname,pid,ppid,tty,cmd,cls,ni,pri,rtprio |grep sshd
USER PID PPID TT CMD CLS NI PRI RTPRIO
root 10326 1 ? /usr/sbin/sshd -D RR - 130 90
root 10351 10326 ? sshd: root@pts/0 RR - 130 90
$ lxc-start -n alpine
lxc-start alpine 20200303215813.785 ERROR lxc_cgfs - cgroups/cgfs.c:lxc_cgroupfs_enter:1239 - Invalid argument - Could not add pid 10671 to cgroup /lxc/alpine: internal error
lxc-start alpine 20200303215813.941 ERROR lxc_container - lxccontainer.c:wait_on_daemonized_start:760 - Received container state "ABORTING" instead of "RUNNING"
lxc-start alpine 20200303215813.941 ERROR lxc_start_ui - tools/lxc_start.c:main:371 - The container failed to start.
lxc-start alpine 20200303215813.941 ERROR lxc_start_ui - tools/lxc_start.c:main:373 - To get more details, run the container in foreground mode.
lxc-start alpine 20200303215813.941 ERROR lxc_start_ui - tools/lxc_start.c:main:375 - Additional information can be obtained by setting the --logfile and --logpriority options.
lxc-start alpine 20200303215813.941 ERROR lxc_start - start.c:__lxc_start:1459 - Failed to spawn container "alpine".
lxc-start alpine 20200303222405.881 INFO lxc_start_ui - tools/lxc_start.c:main:280 - using rcfile /srv/lxc/alpine/config
lxc-start alpine 20200303222405.881 INFO lxc_confile - confile.c:set_config_idmaps:1556 - Read uid map: type u nsid 0 hostid 400000 range 65536
lxc-start alpine 20200303222405.881 INFO lxc_confile - confile.c:set_config_idmaps:1556 - Read uid map: type g nsid 0 hostid 400000 range 65536
lxc-start alpine 20200303222405.882 INFO lxc_container - lxccontainer.c:do_lxcapi_start:883 - Attempting to set proc title to [lxc monitor] /srv/lxc alpine
lxc-start alpine 20200303222405.882 INFO lxc_utils - utils.c:setproctitle:1472 - setting cmdline failed - Invalid argument
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:435 - processing: .reject_force_umount # comment this to allow umount -f; not recommended.
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:610 - Adding native rule for reject_force_umount action 0(kill).
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:do_resolve_add_rule:276 - Setting Seccomp rule to reject force umounts.
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:614 - Adding compat rule for reject_force_umount action 0(kill).
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:do_resolve_add_rule:276 - Setting Seccomp rule to reject force umounts.
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:do_resolve_add_rule:276 - Setting Seccomp rule to reject force umounts.
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:435 - processing: .[all].
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:435 - processing: .kexec_load errno 1.
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:610 - Adding native rule for kexec_load action 327681(errno).
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:614 - Adding compat rule for kexec_load action 327681(errno).
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:435 - processing: .open_by_handle_at errno 1.
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:610 - Adding native rule for open_by_handle_at action 327681(errno).
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:614 - Adding compat rule for open_by_handle_at action 327681(errno).
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:435 - processing: .init_module errno 1.
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:610 - Adding native rule for init_module action 327681(errno).
lxc-start alpine 20200303222405.882 INFO lxc_seccomp - seccomp.c:parse_config_v2:614 - Adding compat rule for init_module action 327681(errno).
lxc-start alpine 20200303222405.883 INFO lxc_seccomp - seccomp.c:parse_config_v2:435 - processing: .finit_module errno 1.
lxc-start alpine 20200303222405.883 INFO lxc_seccomp - seccomp.c:parse_config_v2:610 - Adding native rule for finit_module action 327681(errno).
lxc-start alpine 20200303222405.883 INFO lxc_seccomp - seccomp.c:parse_config_v2:614 - Adding compat rule for finit_module action 327681(errno).
lxc-start alpine 20200303222405.883 INFO lxc_seccomp - seccomp.c:parse_config_v2:435 - processing: .delete_module errno 1.
lxc-start alpine 20200303222405.883 INFO lxc_seccomp - seccomp.c:parse_config_v2:610 - Adding native rule for delete_module action 327681(errno).
lxc-start alpine 20200303222405.883 INFO lxc_seccomp - seccomp.c:parse_config_v2:614 - Adding compat rule for delete_module action 327681(errno).
lxc-start alpine 20200303222405.883 INFO lxc_seccomp - seccomp.c:parse_config_v2:624 - Merging in the compat Seccomp ctx into the main one.
lxc-start alpine 20200303222405.883 WARN lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
lxc-start alpine 20200303222405.883 WARN lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
lxc-start alpine 20200303222405.883 DEBUG lxc_start - start.c:setup_signal_fd:301 - Set SIGCHLD handler with file descriptor: 7.
lxc-start alpine 20200303222405.883 DEBUG console - console.c:lxc_console_peer_default:450 - process does not have a controlling terminal
lxc-start alpine 20200303222405.883 INFO lxc_start - start.c:lxc_init:680 - container "alpine" is initialized
lxc-start alpine 20200303222405.883 DEBUG storage - storage/storage.c:get_storage_by_name:231 - Detected rootfs type "dir"
lxc-start alpine 20200303222405.884 INFO lxc_network - network.c:instantiate_veth:171 - Retrieved mtu 1500 from br-alpine
lxc-start alpine 20200303222405.885 INFO lxc_network - network.c:instantiate_veth:197 - Attached "vethF1ORYV" to bridge "br-alpine"
lxc-start alpine 20200303222405.885 DEBUG lxc_network - network.c:instantiate_veth:214 - Instantiated veth "vethF1ORYV/vethUT8VKV", index is "36"
lxc-start alpine 20200303222405.885 INFO lxc_cgroup - cgroups/cgroup.c:cgroup_init:67 - cgroup driver cgroupfs initing for alpine
lxc-start alpine 20200303222405.923 INFO lxc_start - start.c:lxc_spawn:1259 - Cloned CLONE_NEWUSER.
lxc-start alpine 20200303222405.923 INFO lxc_start - start.c:lxc_spawn:1259 - Cloned CLONE_NEWNS.
lxc-start alpine 20200303222405.923 INFO lxc_start - start.c:lxc_spawn:1259 - Cloned CLONE_NEWPID.
lxc-start alpine 20200303222405.923 INFO lxc_start - start.c:lxc_spawn:1259 - Cloned CLONE_NEWUTS.
lxc-start alpine 20200303222405.923 INFO lxc_start - start.c:lxc_spawn:1259 - Cloned CLONE_NEWIPC.
lxc-start alpine 20200303222405.923 DEBUG lxc_conf - conf.c:idmaptool_on_path_and_privileged:2601 - The binary "/usr/bin/newuidmap" does have the setuid bit set.
lxc-start alpine 20200303222405.923 DEBUG lxc_conf - conf.c:idmaptool_on_path_and_privileged:2601 - The binary "/usr/bin/newgidmap" does have the setuid bit set.
lxc-start alpine 20200303222405.923 DEBUG lxc_conf - conf.c:lxc_map_ids:2689 - Functional newuidmap and newgidmap binary found.
lxc-start alpine 20200303222405.925 INFO lxc_start - start.c:do_start:848 - Unshared CLONE_NEWNET.
lxc-start alpine 20200303222405.925 ERROR lxc_cgfs - cgroups/cgfs.c:lxc_cgroupfs_enter:1239 - Invalid argument - Could not add pid 14075 to cgroup /lxc/alpine: internal error
lxc-start alpine 20200303222406.730 INFO lxc_network - network.c:lxc_delete_network_priv:2539 - Removed interface "(null)" with index 36
lxc-start alpine 20200303222406.810 WARN lxc_network - network.c:lxc_delete_network_priv:2557 - Failed to remove interface "vethF1ORYV" from "br-alpine": Invalid argument
lxc-start alpine 20200303222406.810 DEBUG lxc_network - network.c:lxc_delete_network:3124 - Deleted network devices
lxc-start alpine 20200303222406.811 WARN lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
lxc-start alpine 20200303222406.811 WARN lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
lxc-start alpine 20200303222406.811 ERROR lxc_container - lxccontainer.c:wait_on_daemonized_start:760 - Received container state "ABORTING" instead of "RUNNING"
lxc-start alpine 20200303222406.811 ERROR lxc_start_ui - tools/lxc_start.c:main:371 - The container failed to start.
lxc-start alpine 20200303222406.812 ERROR lxc_start_ui - tools/lxc_start.c:main:373 - To get more details, run the container in foreground mode.
lxc-start alpine 20200303222406.812 ERROR lxc_start_ui - tools/lxc_start.c:main:375 - Additional information can be obtained by setting the --logfile and --logpriority options.
lxc-start alpine 20200303222406.813 ERROR lxc_start - start.c:__lxc_start:1459 - Failed to spawn container "alpine".
lxc-start alpine 20200303222406.813 WARN lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
lxc-start alpine 20200303222406.813 WARN lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
lxc-start alpine 20200303222406.133 DEBUG lxc_conf - conf.c:idmaptool_on_path_and_privileged:2601 - The binary "/usr/bin/newuidmap" does have the setuid bit set.
lxc-start alpine 20200303222406.133 DEBUG lxc_conf - conf.c:idmaptool_on_path_and_privileged:2601 - The binary "/usr/bin/newgidmap" does have the setuid bit set.
lxc-start alpine 20200303222406.133 DEBUG lxc_conf - conf.c:lxc_map_ids:2689 - Functional newuidmap and newgidmap binary found.
lxc-start alpine 20200303222406.136 WARN lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
lxc-start alpine 20200303222406.136 WARN lxc_monitor - monitor.c:lxc_monitor_fifo_send:111 - Failed to open fifo to send message: No such file or directory.
$ cat /proc/self/cgroup
1:cpuset,cpu,cpuacct,blkio,memory,devices,freezer,net_cls,pids:/
Its critical for me to have a working sshd. It happened already that a process ran away with my system and the userspace was unusable (including sshd). That’s why i want to have the sshd in SCHED_RR (and not just a different niceness).
Is there any chance to get the lxc binaries working under SCHED_RR?
Linux 5.4.22 x86_64
lxc v2.1.1
Edit: Also tested on Arch Linux with lxc 3.2.1. Same problem.
$ chrt -r 90 bash
$ lxc-start -n alpine