Shadowsocks-libev under Incus

I recently switched from LXD to Incus just because of the bad policy Canonical had against the good Linux Containers team
Everything is working fine so far, but for those wondering why the shadowsocks systemd service is not working under Incus, here’s there’s a quick fix ( even if i don’t know if it’s an appropriate one ).

To fix the problem, edit the systemd service using your favorite editor ( in my case, nano ):

sudo nano /lib/systemd/system/shadowsocks-libev.service

In the file, look for DynamicUser=true and switch the value from true to false.

The new file will look like this:

#  This file is part of shadowsocks-libev.
#
#  Shadowsocks-libev is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 3 of the License, or
#  (at your option) any later version.
#
#  This file is default for Debian packaging. See also
#  /etc/default/shadowsocks-libev for environment variables.

[Unit]
Description=Shadowsocks-libev Default Server Service
Documentation=man:shadowsocks-libev(8)
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
DynamicUser=false
EnvironmentFile=/etc/default/shadowsocks-libev
LimitNOFILE=32768
ExecStart=/usr/bin/ss-server -c $CONFFILE $DAEMON_ARGS

[Install]
WantedBy=multi-user.target

Save the file and reload the systemd daemon with the following command:

sudo systemctl daemon-reload

After this, just restart shadowsocks-libev service with:

sudo systemctl restart shadowsocks-libev

I hope this solution will help someone. Thank you.

1 Like

The DynamicUser documentation, systemd.exec
DynamicUser does a lot of things and it’s not clear what part could cause the problem.

What error do you get when you keep DynamicUser enabled? If there is a specific message, post it so that others that Google for that message, can get directly here.

My understanding of DynamicUser is that it makes use of high uid/gid which may not be available in the container.

It’s actually one of the leading reasons behind the recent work by @amikhalitsyn and myself to get a new concept of isolated user namespaces going in the kernel, this would then provide the entire uid/gid space for each container, allowing for such dynamic allocation of high uid/gid without problems.

I get this error. I run a Debian 12 container on a Debian 12 host ( node ) and the only user i have is root:

Feb 17 18:08:55 wg-debian12 (s-server)[1025]: shadowsocks-libev.service: Failed to update dynamic user credentials: Permission denied
Feb 17 18:08:55 wg-debian12 (s-server)[1025]: shadowsocks-libev.service: Failed at step USER spawning /usr/bin/ss-server: Permission denied

EDIT: i forgot to mention that i used lxd-to-incus for the migration from LXD to Incus. On LXD this problem, even if i had just root as user, was not present.

Can you show cat /proc/self/uid_map from inside the container?

It’s plausible that your switch from LXD to Incus has resulted in a much smaller uid/gid map being picked up, causing this issue.

Can you also show cat /etc/subuid and cat /etc/subgid from the host please?

Here it is :

root@wg-debian12:~# cat /proc/self/uid_map 
         0    1000000       1000
      1000       1000          1
      1001    1001001  999998999

Here they are:

tacco@buttalapasta:~$  cat /etc/subuid 
tacco:100000:65536
root:1000:1
root:1000000:1000000000
tacco@buttalapasta:~$ cat /etc/subgid 
tacco:100000:65536
root:1000:1
root:1000000:1000000000

Just for curiosity, i tried to create a new container based on ubuntu 22.04. The problem persists with the same error.

EDIT: But the output from cat /proc/self/uid_map inside the new container is different:

root@first:~# cat /proc/self/uid_map 
         0    1000000 1000000000

Can you check in dmesg for any related DENIED type log entries when the unit tries to start?

if with unit you mean the container, this is the message i receive:

root@wg-debian12:~# dmesg
dmesg: read kernel buffer failed: Operation not permitted

No, I meant on the host system.

From the host the command dmesg | grep 'DENIED' is giving me zero results.

EDIT: i tried to restart the VPS and this is the result:

[   19.755559] audit: type=1400 audit(1708284249.260:34): apparmor="DENIED" operation="file_lock" profile="incus-antmedia_</var/lib/incus>" pid=1721 comm="(crub_all)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   19.755568] audit: type=1400 audit(1708284249.260:35): apparmor="DENIED" operation="file_lock" profile="incus-antmedia_</var/lib/incus>" pid=1721 comm="(crub_all)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   20.031054] audit: type=1400 audit(1708284249.536:36): apparmor="DENIED" operation="file_lock" profile="incus-first_</var/lib/incus>" pid=1892 comm="(crub_all)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   20.031071] audit: type=1400 audit(1708284249.536:37): apparmor="DENIED" operation="file_lock" profile="incus-first_</var/lib/incus>" pid=1892 comm="(crub_all)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   20.124895] audit: type=1400 audit(1708284249.632:38): apparmor="DENIED" operation="file_lock" profile="incus-first_</var/lib/incus>" pid=1916 comm="(ostnamed)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   21.748946] audit: type=1400 audit(1708284251.256:44): apparmor="DENIED" operation="file_lock" profile="incus-nextcloud-ubuntu2204_</var/lib/incus>" pid=2344 comm="(crub_all)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   21.748955] audit: type=1400 audit(1708284251.256:45): apparmor="DENIED" operation="file_lock" profile="incus-nextcloud-ubuntu2204_</var/lib/incus>" pid=2344 comm="(crub_all)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   22.249915] audit: type=1400 audit(1708284251.756:46): apparmor="DENIED" operation="file_lock" profile="incus-nextcloud-ubuntu2204_</var/lib/incus>" pid=2629 comm="(ostnamed)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   22.249923] audit: type=1400 audit(1708284251.756:47): apparmor="DENIED" operation="file_lock" profile="incus-nextcloud-ubuntu2204_</var/lib/incus>" pid=2629 comm="(ostnamed)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   22.351037] audit: type=1400 audit(1708284251.856:49): apparmor="DENIED" operation="file_lock" profile="incus-pihole-debian12_</var/lib/incus>" pid=2639 comm="(crub_all)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   22.351045] audit: type=1400 audit(1708284251.856:50): apparmor="DENIED" operation="file_lock" profile="incus-pihole-debian12_</var/lib/incus>" pid=2639 comm="(crub_all)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   22.964153] audit: type=1400 audit(1708284252.468:51): apparmor="DENIED" operation="file_lock" profile="incus-pihole-debian12_</var/lib/incus>" pid=2773 comm="(ostnamed)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   22.964163] audit: type=1400 audit(1708284252.468:52): apparmor="DENIED" operation="file_lock" profile="incus-pihole-debian12_</var/lib/incus>" pid=2773 comm="(ostnamed)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   26.395432] audit: type=1400 audit(1708284255.900:63): apparmor="DENIED" operation="file_lock" profile="incus-wpstream-debian12_</var/lib/incus>" pid=3577 comm="(ostnamed)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"
[   26.395437] audit: type=1400 audit(1708284255.900:64): apparmor="DENIED" operation="file_lock" profile="incus-wpstream-debian12_</var/lib/incus>" pid=3577 comm="(ostnamed)" family="unix" sock_type="dgram" protocol=0 requested_mask="send"

So, something related to apparmor.