LXD CLI client processes escaping from system.slice to user.slice (cgroups, systemd)

LXD
,
#1

Hey, can somebody please explain to me why processes from lxc CLI kinda “escape” to user.slice (or to be more precise the app.slice inside user.slice) even when they were started as a child process from a service belonging to the system.slice?

This causes our lxc processes getting killed by systemd (PID 1), when we’re using an SSH session for the root user. When the last SSH session has ended, systemd waits 10 seconds and then sends SIGTERM to any leftover processes of inside user-0.slice.

Is there any way to keep lxc processes under the system.slice where they started from? Why are they “escaping” at all? Is this some kind of snap feature?

Note: This does not affect the LXD daemon.
Environment: Ubuntu 22.04 with LXD 5.0.1

Thank you for any help in advance!

#2

Can you show an example with a set of reproducer steps to show what you mean please?

#3

Also you said this doesn’t affect LXD, but you’ve marked it as LXD category, so is this only with the lxc-* commands or do you mean the lxc client command for LXD?

#4

This affects the command line client for LXD which is named lxc. So i mean commands like lxc init or lxc list. I will provide an example for our system daemon running the client as soon as possible.

#5

With the following example, you should be able to see, that lxc list will escape to the user.slice.

Write the following bash script to /root/lister.sh and make it executable.

#!/bin/bash

while true; do
  /snap/bin/lxc list
  sleep 1
done

Write the following service file to /etc/systemd/system/lister.service

[Unit]
Description=lister
After=network.target

[Service]
Type=simple
ExecStart=/root/lister.sh
StandardOutput=null
StandardError=null

[Install]
WantedBy=multi-user.target

Run systemctl daemon-reload && systemctl start lister.service to start the service.
Repeatedly check the output from systemctl --user status and systemctl status lister.service to catch a running lxc list process.

You will notice that lxc list will never appear in the process tree of lister.service (like I would expect), but always in user@0.service.

Here is my output from systemctl --user status:

● system-1-dc
    State: running
     Jobs: 0 queued
   Failed: 0 units
    Since: Fri 2022-12-02 09:21:54 CET; 6min ago
   CGroup: /user.slice/user-0.slice/user@0.service
           ├─app.slice 
           │ ├─dbus.service 
           │ │ └─1319131 /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
           │ └─snap.lxd.lxc.75f6210c-a78a-47de-af5b-3235a2e67b51.scope 
           │   └─1325357 /snap/snapd/17883/usr/lib/snapd/snap-confine --base core20 snap.lxd.lxc /usr/lib/snapd/snap-exec lxd.lxc list
           └─init.scope 
             ├─1318832 /lib/systemd/systemd --user
             └─1318833 (sd-pam)

Here is my output from systemctl status lister.service:

● lister.service - lister
     Loaded: loaded (/etc/systemd/system/lister.service; disabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-12-02 09:28:05 CET; 7s ago
   Main PID: 1324702 (lister.sh)
      Tasks: 1 (limit: 9404)
     Memory: 404.0K
        CPU: 407ms
     CGroup: /system.slice/lister.service
             └─1324702 /bin/bash /root/lister.sh
1 Like
#6

Although I’d wish there would be some explanation from the maintainers, it seems that this behavior is not specific to LXD, but rather to all snap-packaged applications.

I’d like to make clear that auto-moving a process to user.slice can cause problems with systemd sessions cleanups. When the last session for a user ends, systemd (by default) kills all remaining processes from its user.slice. Which is an absolute bummer if you want to use lxc commands inside a service running at system.slice.

In case somebody else has this problem: You can disable systemd session cleanup for a specific user by running loginctl enable-linger $user.

#7

Any thoughts/idea on this @stgraber ?

Thanks