Binding container to /var/run/aesmd/aesm.socket on the host

[0x416e746f6e]

If I need to bind to unix:/var/run/aesmd/aesm.socket inside the container (mapping it exactly to /var/run/aesmd/aesm.socket on the host), how can I do that?

If I do:

lxc config device add my_container \
    lxdsocket proxy \
    connect="unix:/var/run/aesmd/aesm.socket" \
    listen="unix:/var/run/aesmd/aesm.socket" \
    bind=container \
    uid=998 \
    gid=998 \
    security.uid=998 \
    security.gid=998 \
    mode=0777

…then I get:

Error: Failed to start device "lxdsocket": Error occurred when starting proxy device: 
Error: Failed to listen on /var/run/aesmd/aesm.socket: listen unix /var/run/aesmd/aesm.socket: bind: no such file or directory

And if I do:

lxc config device add my_container \
    lxdsocket proxy \
    connect="unix:/var/run/aesmd/aesm.socket" \
    listen="unix:/var/run/aesm.socket" \
    bind=container \
    uid=998 \
    gid=998 \
    security.uid=998 \
    security.gid=998 \
    mode=0777

…then I get:

Error: Failed to start device "lxdsocket": Error occurred when starting proxy device: 
Error: Failed to listen on /var/run/aesm.socket: listen unix /var/run/aesm.socket: bind: permission denied

[qkiel]

Maybe try to bind aesm.socket inside the container to /mnt/ folder and then create a link from it to the proper location:

listen: /mnt/aesm.socket

then inside container:

ln -sf /mnt/aesm.socket /var/run/aesmd/aesm.socket

If link disappears on reboot, you can script this like here:

Incus / LXD profile for GUI apps: Wayland, X11 and Pulseaudio

I made a profile for running GUI apps in an Incus / LXD container. It supports wayland, X11 and pulseaudio. This profile is based on work of Justin Ludwig, thank you! https://blog.swwomm.com/2022/08/lxd-containers-for-wayland-gui-apps.html Profile has been tested using: both Incus and LXD Ubuntu 22.04 as a host Ubuntu 22.04 container images, both from community repository images:ubuntu/jammy/cloud and official Canonical repository ubuntu:22.04 This post is divided into four parts: Profile …

[0x416e746f6e]

thanks @qkiel, that helped with binding the socket inside the container. however, if I try to use it the connection fails.

trying to trace it inside container with strace socat - UNIX-CONNECT:/mnt/aesm.socket:

...
socket(AF_UNIX, SOCK_STREAM, 0)         = 5
recvfrom(3, 0x7ffe88cd5180, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
fcntl(5, F_SETFD, FD_CLOEXEC)           = 0
recvfrom(3, 0x7ffe88cd5180, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(3, 0x7ffe88cd5090, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
connect(5, {sa_family=AF_UNIX, sun_path="/mnt/aesm.socket"}, 18) = 0
recvfrom(3, 0x7ffe88cd5090, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
getsockname(5, {sa_family=AF_UNIX}, [112 => 2]) = 0
recvfrom(3, 0x7ffe88cd5540, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
pselect6(6, [0 5], [1 5], [], NULL, NULL) = 3 (in [5], out [1 5])
recvfrom(3, 0x7ffe88cd5540, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(3, 0x7ffe88cd4fd0, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
read(5, "", 8192)                       = 0
recvfrom(3, 0x7ffe88cd4fd0, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(3, 0x7ffe88cd5540, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
pselect6(1, [0], [], [], {tv_sec=0, tv_nsec=500000000}, NULL) = 0 (Timeout)
recvfrom(3, 0x7ffe88cd5540, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, SNDCTL_TMR_START or TCSETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
shutdown(5, SHUT_RDWR)                  = 0
recvfrom(3, 0x7ffe88cd5780, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
exit_group(0)                           = ?
+++ exited with 0 +++

and on the host with strace socat - UNIX-CONNECT:/var/run/aesmd/aesm.socket it works fine:

...
socket(AF_UNIX, SOCK_STREAM, 0)         = 5
recvfrom(3, 0x7ffca514fb10, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
fcntl(5, F_SETFD, FD_CLOEXEC)           = 0
recvfrom(3, 0x7ffca514fb10, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(3, 0x7ffca514fa20, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
connect(5, {sa_family=AF_UNIX, sun_path="/var/run/aesmd/aesm.socket"}, 28) = 0
recvfrom(3, 0x7ffca514fa20, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
getsockname(5, {sa_family=AF_UNIX}, [112 => 2]) = 0
recvfrom(3, 0x7ffca514fed0, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
pselect6(6, [0 5], [1 5], [], NULL, NULL) = 2 (out [1 5])
recvfrom(3, 0x7ffca514fed0, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
recvfrom(3, 0x7ffca514fed0, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
pselect6(6, [0 5], [], [], NULL, NULL^C)  = ? ERESTARTNOHAND (To be restarted if no handler)
... and here it waits for the input
strace: Process 6618 detached

it seems an issue lies with the proxy device. I tried different configurations of UID/GID, but no success.

is there a way I could debug the proxy device maybe?

[qkiel]

You can try type: disk device instead of type: proxy.

First, you need to add idmaps to your container by adding raw.idmap key. If the user on the host has UID 1000 and GID 1000 (check using id -u and id -g commands) and the user inside the container has UID 998 and GID 998, then the command would be:

lxc config set <container_name> raw.idmap "both 1000 998"

Next add the device and restart your container:

lxc config device add <container_name> <device_name> disk source=/var/run/aesmd/aesm.socket path=/mnt/aesm.socket

Unfortunately, I don’t know hot to debug the proxy device.

Edit: One more thing came to my mind. When using proxy device, are security.uid and security.gid set to your user’s uid/gid on the host as well as uid and gid set to your user’s uid/gid inside the container?

[0x416e746f6e]

@qkiel

lxc config device add <container_name> <device_name> disk source=/var/run/aesmd/aesm.socket path=/mnt/aesm.socket

Unfortunately, that does not work:

# lxc config device add gramine aesmd disk source=/var/run/aesmd/aesm.socket path=/mnt/aesm.socket
Error: Failed add validation for device "aesmd": Missing source path "/var/run/aesmd/aesm.socket" for disk "aesmd"

^-- this happens even if I am running a privileged container with raw.idmap at both 0 0

@qkiel

One more thing came to my mind. When using proxy device, are security.uid and security.gid set to your user’s uid/gid on the host as well as uid and gid set to your user’s uid/gid inside the container?

Yes. Also tried both, privileged (uid 0) and unprivileged user.

[0x416e746f6e]

hmm…

look what I found:

# cat ./var/snap/lxd/common/lxd/logs/gramine/proxy.aesm.log
Status: Started
Warning: Failed to connect to target: dial unix /var/lib/snapd/hostfs/var/run/aesmd/aesm.socket: connect: no such file or directory
Warning: Failed to prepare new listener instance: dial unix /var/lib/snapd/hostfs/var/run/aesmd/aesm.socket: connect: no such file or directory
Warning: Failed to connect to target: dial unix /var/lib/snapd/hostfs/var/run/aesmd/aesm.socket: connect: no such file or directory
Warning: Failed to prepare new listener instance: dial unix /var/lib/snapd/hostfs/var/run/aesmd/aesm.socket: connect: no such file or directory
Warning: Failed to connect to target: dial unix /var/lib/snapd/hostfs/var/run/aesmd/aesm.socket: connect: no such file or directory
Warning: Failed to prepare new listener instance: dial unix /var/lib/snapd/hostfs/var/run/aesmd/aesm.socket: connect: no such file or directory

[qkiel]

Maybe try sharing the whole /var/run/aesmd directory:

lxc config device add <container_name> <device_name> disk source=/var/run/aesmd path=/mnt/aesmd

[0x416e746f6e]

tried that as well already, but without any luck.

however, after reading this comment, I found the culprit:

• instead of using connect="unix:/var/run/aesmd/aesm.socket" I had to use just connect="unix:/run/aesmd/aesm.socket" (no /var).
• inside container, I needed to create a symlink with mkdir /run/aesmd && ln -s /mnt/aesm.socket /run/aesmd/aesm.socket

and it finally works as expected (in privileged container; now I will try to make it work for unprivileged one)

thanks a lot @qkiel for looking into this together with me and for giving me some inspiration!