Headless wayland container streaming via Sunshine; sway / libinput not finding input devices

I have just made the switch over to Incus and let me preface this with a big thank you to all contributors who have seen to making the process as painless as possible. Needless to say, the conversion went without a hitch :slight_smile:

That being said, there’s a little pet project of mine that I have been trying to get off the ground for quite a while now without final success - streaming a headless wayland container via Sunshine.

Background

There is a long history of running GUI apps in the container on the host (most recently this thread), which is not what I am trying to achieve since the host is purely accessed via SSH and is neither running a X nor wayland server. There is another topic where a user was having some success starting a container-only X-server and this is pretty much my direction using wlroots and sway in this case.

Setup info

I’m currently using Incus 5.0 from Debian Testing on Bookworm running on a Hetzner dedicated server. The container to be streamed is running Debian 12 as well with the following configuration -

container (device) configuration:

devices:
  desk.igpu:
    gid: "44"
    pci: "0000:00:02.0"
    type: gpu
  desk.input-joystick:
    gid: "1000"
    path: /dev/input/event7
    required: "false"
    source: /dev/input/event7
    type: unix-char
    uid: "1000"
  desk.input-keyboard:
    gid: "1000"
    path: /dev/input/event6
    required: "false"
    source: /dev/input/event6
    type: unix-char
    uid: "1000"
  desk.input-mouse:
    gid: "1000"
    path: /dev/input/event4
    required: "false"
    source: /dev/input/event4
    type: unix-char
    uid: "1000"
  desk.input-touchscreen:
    gid: "1000"
    path: /dev/input/event5
    required: "false"
    source: /dev/input/event5
    type: unix-char
    uid: "1000"
  desk.tty0:
    gid: "1000"
    path: /dev/tty0
    source: /dev/tty0
    type: unix-char
    uid: "1000"
  desk.tty1:
    gid: "1000"
    path: /dev/tty1
    source: /dev/tty1
    type: unix-char
    uid: "1000"
  desk.uinput:
    gid: "1000"
    path: /dev/uinput
    type: unix-char
    uid: "1000"
  desk.wireguard:
    connect: udp:10.200.200.2:60820
    listen: udp:external-ip:60820
    nat: "true"
    type: proxy

Since Sunshine creates virtual input devices via uinput, I have forwarded “/dev/uinput” accordingly and made sure that the resulting input devices are added as well (the configuration is rather static since no devices are connected / disconnected from the server). Side-node: unix-hotplug does not add any devices to the container, but unix-char seems to work.

You’ll also find the iGPU as well as a port for WireGuard access forwarded to the container (with wayvnc and Sunshine bound to the VPN).

sway is started with the following script after logging in to the container via ssh. “libinput” as a backend works when forwarding “/dev/tty0” and “/dev/tty1”, though some errors (*) can be seen during startup. Alternatively, I can start with libinput using the “incus console [container]” command and then just running the below script script. The “headless” backend accesses a virtual display created by sway.

sway-startup.sh script:

#!/bin/bash                                                                                                                                                                                                     
                                                                                                                                                                                                                
export DESKTOP_SESSION="sway"                                                                                                                                                                                   
export XDG_CURRENT_DESKTOP="sway"                                                                                                                                                                               
export XDG_DATA_DIRS="/usr/local/share:/usr/share:/var/lib/flatpak/exports/share:/home/desk/.local/share/flatpak/exports/share"                                                                                 
export XDG_SESSION_DESKTOP="sway"                                                                                                                                                                               
export XDG_SESSION_TYPE="wayland"                                                                                                                                                                               
export WLR_BACKENDS="headless,libinput"                                                                                                                                                                         
export WLR_LIBINPUT_NO_DEVICES="1"                                                                                                                                                                              
                                                                                                                                                                                                                
exec /usr/bin/sway -c /home/desk/.config/sway/desk

(*) errors when starting sway with libinput backend via ssh:

Jan 29 15:59:24 desk ssh[166884]: 00:00:00.001 [ERROR] [wlr] [libseat] [common/terminal.c:208] Could not set VT mode to enable process switching: Operation not permitted
Jan 29 15:59:24 desk ssh[166884]: 00:00:00.001 [ERROR] [wlr] [libseat] [common/terminal.c:248] Could not set KD keyboard mode to disabled: Operation not permitted
Jan 29 15:59:24 desk ssh[166884]: 00:00:00.001 [ERROR] [wlr] [libseat] [common/terminal.c:275] Could not set KD graphics mode to graphics: Operation not permitted

What works?

I tried connecting to the session using both, wayvnc and Sunshine with the former working pretty much flawlessly (*) and the latter only showing the stream without mouse and keyboard input being possible. This in itself is quite amazing imho since accelerated video (h.265 encoding for the Sunshine stream) and 3D seem to work without issues.

(*) Why not just use wayvnc then? The issue here is that while the stream uses h.264 encoding, the VNC screen capture method itself is way too slow to deliver full screen video or any high speed motion output such as scrolling a web page at higher resolutions. It also currently lacks sound support (the developer is looking into options), which, while not critical, would be nice to have.

wayvnc uses the wlr-virtual-{pointer,keyboard} protocol to add its virtual input devices, which are listed properly when querying with “swaymsg -t get_inputs”. Also, any game controller connected via Sunshine can be used since those are not managed by sway but accessed directly. I only tested with “Tuxkart”, so the controller may or may not be picked up by other titles, but that is not an issue for now.

What doesn’t?

This is not the case for Sunshine though, which creates its devices via uinput. The /dev/input/eventX entries get added correctly on the host, can be forwarded to the container and confirmed working by using e.g. evtest, but do not show up as libinput devices and consequently can’t be used to control the sway session.

I’d be grateful for any clues as to what might be missing here, if this can be made to work at all with the setup described above. This is not purely an incus issue, but there are probably quite a few people around who have tried something along those lines and might be able to provide input :slight_smile:

1 Like

Thanks for this!

If you want to offload the 2D/3D work to the host GPU (your local computer), you would need to use some software that performs just that. On the server there is a corresponding accelerated software GPU, otherwise you end up using llvmpipe which is not performant.

  1. virgl, for VMs
  2. VirtualGL

Also see this, Tutorial: How to run a full desktop environment on LXD The issue here is that all these tricks we do to run GUI apps in containers, are ways to glue together the container to the host. Containers created for isolation but we pierce holes into this isolation, and make devices of the hosts appear in the container. In some cases, the software needs some small changes, and @tarruda does exactly that.

1 Like

@simos Thank you for your answer and regarding the points you mentioned:

2D/3D acceleration: This already works well in the container by simply making the GPU accessible as outlined above. I made sure that Sunshine is actually using hardware encoding when connecting to the container and the apps running inside proper acceleration.

DE in LXD / Incus: I’d prefer to use wayland instead of X and the current state (video + sound working) feels like we’re 90% there with a relatively simple configuration (no building / patching required). The only thing missing is the virtual mouse and keyboard input, which I am not sure can be made to work with this setup, but maybe somebody else has an idea.

If you do not get input (pun not intended) from someone else, I suggest to post a cheat sheet on how to setup the container. A minimal setup, with software encoding, accessible from a browser.

1 Like

Will do, but some “progress” (more like “dirty hack”) in the meantime: After failing to get the Sunshine input devices to register in the container, I have given up on that route, also since Lennart Poettering mentioned that using udev in containers is not possible, see udevadm trigger fails when run in lxd container · Issue #28156 · systemd/systemd · GitHub -

“sysfs is not virtualized for containers on Linux kernel. Running udev inside containers hence is not supported. udevadm trigger is conditioned on /sys/ being writable, which is how container managers should communicate that udev is not supposed to run.”

I tried setting “sys:rw” with the “lxc.mount.auto” option, but “udevadm trigger” reported many errors nevertheless. Not sure if there is a way around this, but the configuration is getting way too complex (and potentially insecure) for my taste once we start passing “raw.lxc” commands …

So what I resorted to in the end was using wayvnc / VNC with its working virtual mouse and keyboard for input (*) and Sunshine / Moonlight for display & sound by starting a VNC connection with TigerVNC, making its window transparent and and “overlaying” it on the Moonlight window. I have not yet found a way not to send a picture via VNC, but TigerVNC can be configured not to use much data, so this works for now to have a fully accelerated cloud desktop.

This also leads to a drastically simplified configuration since only the iGPU needs to be passed to the container -

devices:
  desk.igpu:
    gid: "44"
    pci: "0000:00:02.0"
    type: gpu

(*) There are some programs to pass on only mouse and keyboard input (Barrier, Synergy, …), but those either don’t support wayland (yet) or need libinput / udev as well.

1 Like