Lxd and cgrulesengd

I try use cgrulesengd in container. When i run daemon it created need cgroup but when try put process to task got

Warning: cannot write tid 27136 to /sys/fs/cgroup/cpu,cpuacct/kola//tasks:Permission denied
Warning: cgroup_attach_task_pid failed: 50016
Warning: failed to apply the rule. Error was: 50016
cgroup change pid 27136 failed

How i can fix that? I just need cgroup per user for account cpu time and memory

@brauner any idea why this wouldn’t be allowed?

That’s really not a lot of information to go on. :slight_smile:
Have the cpu and cpuacct controllers been delegated to you, i.e. are they writable for container root? EPERM very much seems like they might not.
Please show:

ls -al /sys/fs/cgroup/cpu,cpuacct/

I check it before wrote topic

ls -al /sys/fs/cgroup/cpu,cpuacct/
total 0
drwxrwxr-x 4 nobody root 0 Dec 17 15:45 .
drwxr-xr-x 15 root root 380 Dec 7 10:50 …
-rw-r–r-- 1 nobody nogroup 0 Dec 18 03:59 cgroup.clone_children
-rw-rw-r-- 1 nobody root 0 Dec 18 03:58 cgroup.procs
-rw-r–r-- 1 nobody nogroup 0 Dec 7 10:50 cpu.cfs_period_us
-rw-r–r-- 1 nobody nogroup 0 Dec 18 03:59 cpu.cfs_quota_us
-rw-r–r-- 1 nobody nogroup 0 Dec 18 03:59 cpu.shares
-r–r--r-- 1 nobody nogroup 0 Dec 18 03:59 cpu.stat
-r–r--r-- 1 nobody nogroup 0 Dec 18 03:59 cpuacct.stat
-rw-r–r-- 1 nobody nogroup 0 Dec 18 03:59 cpuacct.usage
-r–r--r-- 1 nobody nogroup 0 Dec 18 03:59 cpuacct.usage_all
-r–r--r-- 1 nobody nogroup 0 Dec 18 03:59 cpuacct.usage_percpu
-r–r--r-- 1 nobody nogroup 0 Dec 18 03:59 cpuacct.usage_percpu_sys
-r–r--r-- 1 nobody nogroup 0 Dec 18 03:59 cpuacct.usage_percpu_user
-r–r--r-- 1 nobody nogroup 0 Dec 18 03:59 cpuacct.usage_sys
-r–r--r-- 1 nobody nogroup 0 Dec 18 03:59 cpuacct.usage_user
drwxr-xr-x 2 root root 0 Dec 17 15:45 kola

And in

ls -al /sys/fs/cgroup/cpu,cpuacct/kola/
total 0
drwxr-xr-x 2 root root 0 Dec 17 15:45 .
drwxrwxr-x 4 nobody root 0 Dec 17 15:45 …
-rw-r–r-- 1 root root 0 Dec 17 15:45 cgroup.clone_children
-rw-r–r-- 1 root root 0 Dec 17 15:45 cgroup.procs
-rw-r–r-- 1 root root 0 Dec 17 15:45 cpu.cfs_period_us
-rw-r–r-- 1 root root 0 Dec 17 15:45 cpu.cfs_quota_us
-rw-r–r-- 1 root root 0 Dec 17 15:45 cpu.shares
-r–r--r-- 1 root root 0 Dec 17 15:45 cpu.stat
-r–r--r-- 1 root root 0 Dec 17 15:45 cpuacct.stat
-rw-r–r-- 1 root root 0 Dec 17 15:45 cpuacct.usage
-r–r--r-- 1 root root 0 Dec 17 15:45 cpuacct.usage_all
-r–r--r-- 1 root root 0 Dec 17 15:45 cpuacct.usage_percpu
-r–r--r-- 1 root root 0 Dec 17 15:45 cpuacct.usage_percpu_sys
-r–r--r-- 1 root root 0 Dec 17 15:45 cpuacct.usage_percpu_user
-r–r--r-- 1 root root 0 Dec 17 15:45 cpuacct.usage_sys
-r–r--r-- 1 root root 0 Dec 17 15:45 cpuacct.usage_user
-rw-r–r-- 1 root root 0 Dec 17 15:45 notify_on_release
-rw-r–r-- 1 root root 0 Dec 17 15:51 tasks

And cgrulesengd run from root. When i try add it with manual, i got strange things:

try add user process

echo 3044 > /sys/fs/cgroup/cpuacct/kola/tasks
bash: echo: write error: Permission denied

try add own process

echo 2895 > /sys/fs/cgroup/cpuacct/kola/tasks
cat /sys/fs/cgroup/cpuacct/kola/tasks
2895

I try

lxc config set c1 raw.lxc “lxc.apparmor.profile=unconfined”

With this profile cgrulesengd not create group when it need

So that’s related to permission checking. In order to move someone else’s task into a cgroup in the legacy hierarchy you need to:

  • have global root, i.e. uid 0 in the initial user namespace as your effective uid
  • your effective uid and the task’s uid need to be identical
  • your saved uid and the task’s uid need to be identical

If none of these conditions hold you can’t move the task.

And now root user in this container not priviledged (uid in host node 1000000). And so i can’t do that right?

Depends, can you show:

cat /proc/self/status

and

cat /proc/<process-failed-to-move>/status
at /proc/self/status 
Name:   cat
Umask:  0022
State:  R (running)
Tgid:   29060
Ngid:   0
Pid:    29060
PPid:   29022
TracerPid:      0
Uid:    0       0       0       0
Gid:    0       0       0       0
FDSize: 256
Groups:  
NStgid: 29060
NSpid:  29060
NSpgid: 29060
NSsid:  29022
VmPeak:     9352 kB
VmSize:     9352 kB
VmLck:         0 kB
VmPin:         0 kB
VmHWM:       596 kB
VmRSS:       596 kB
RssAnon:              64 kB
RssFile:             532 kB
RssShmem:              0 kB
VmData:      312 kB
VmStk:       132 kB
VmExe:        32 kB
VmLib:      2108 kB
VmPTE:        56 kB
VmSwap:        0 kB
HugetlbPages:          0 kB
CoreDumping:    0
THP_enabled:    1
Threads:        1
SigQ:   0/127384
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000001000
SigCgt: 0000000000000000
CapInh: 0000000000000000
CapPrm: 0000003fffffffff
CapEff: 0000003fffffffff
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000
NoNewPrivs:     0
Seccomp:        2
Speculation_Store_Bypass:       thread force mitigated
Cpus_allowed:   000f
Cpus_allowed_list:      0-3
Mems_allowed:   00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001
Mems_allowed_list:      0
voluntary_ctxt_switches:        0
nonvoluntary_ctxt_switches:     0

And process

Name:   mysqld
Umask:  0026
State:  S (sleeping)
Tgid:   9550
Ngid:   0
Pid:    9550
PPid:   1
TracerPid:      0
Uid:    106     106     106     106
Gid:    109     109     109     109
FDSize: 128
Groups: 109 
NStgid: 9550
NSpid:  9550
NSpgid: 9550
NSsid:  9550
VmPeak:  5387844 kB
VmSize:  5387844 kB
VmLck:         0 kB
VmPin:         0 kB
VmHWM:    518708 kB
VmRSS:    518632 kB
RssAnon:          494864 kB
RssFile:           23768 kB
RssShmem:              0 kB
VmData:  1799860 kB
VmStk:       132 kB
VmExe:     54384 kB
VmLib:      9056 kB
VmPTE:      1588 kB
VmSwap:        0 kB
HugetlbPages:          0 kB
CoreDumping:    0
THP_enabled:    1
Threads:        74
SigQ:   0/127384
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000084807
SigIgn: 0000000000003000
SigCgt: 00000001800006e8
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000
NoNewPrivs:     0
Seccomp:        2
Speculation_Store_Bypass:       thread force mitigated
Cpus_allowed:   000f
Cpus_allowed_list:      0-3
Mems_allowed:   00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001
Mems_allowed_list:      0
voluntary_ctxt_switches:        9335
nonvoluntary_ctxt_switches:     47

So this specific codepath in the kernel:

	cred = current_cred();
	tcred = get_task_cred(task);
	if (!uid_eq(cred->euid, GLOBAL_ROOT_UID) &&
	    !uid_eq(cred->euid, tcred->uid) &&
	    !uid_eq(cred->euid, tcred->suid))
		ret = -EACCES;

is what I think causes the behavior you’re seeing. And tbh it looks like a bug to me. It looks like the first uid_eq should probably be more clever and look at the userns root id (probably a little more complicated than that but eh. The legacy cgroup hierarchy is not updated anymore so I’m not sure it’s worth fixing it.
What you could do is delegate a subcgroup to the task you’re interested in and then drop to the tasks effective uid and move it into the cgroup.

Or just simple answer this not work in unpriveledged containers right? :slight_smile: