Incus / LXD profile for GUI apps: Wayland, X11 and Pulseaudio

:warning: Update: A profile for running GUI apps in Incus Ubuntu containers with Ubuntu 24.04 as host.

Previous profiles are now here:

  • Preliminary profile for Ubuntu 24.04
  • Profile for Ubuntu 22.04 with kernel 6.5.0+
  • Profile for Ubuntu 22.04 with older kernels

The main goal this time around was to simplify the profile as much as possible. This means instead of relying on cloud-init and systemd service to remount sockets, I created a bare bone incus_ct_gui.sh script that you can push to the container and run once to get everything set up. You’ll find it in the Script section below.

Beside using this script, you’ll have to make a copy of the Xwaylandauth cookie on the host with a fixed name (it changes every time the host boots). This is necessary for X11 / xWayland sockets to work. See the What’s new section below.

This post is divided into four parts:

  1. Profile
  2. Script
  3. What’s new
  4. Troubleshooting

Profile

config:
  raw.idmap: |-
    uid 1000 1000
    gid 1000 1000
description: Requires 'incus_ct_gui.sh' script. GUI Wayland and xWayland profile with pipewire and pulseaudio, shifting enabled for all socket.
devices:
  gpu:
    type: gpu
    gid: 44
  xwayland_socket:
    type: disk
    shift: true
    source: /tmp/.X11-unix/X1
    path: /mnt/.container_sockets/X1
  xauthority_cookie:
    type: disk
    shift: true
    source: /run/user/1000/.mutter-Xwaylandauth.copy
    path: /mnt/.container_sockets/.mutter-Xwaylandauth.copy
  wayland_socket:
    type: disk
    shift: true
    source: /run/user/1000/wayland-0
    path: /mnt/.container_sockets/wayland-0
  pipewire_socket:
    type: disk
    shift: true
    source: /run/user/1000/pipewire-0
    path: /mnt/.container_sockets/pipewire-0
  pipewire_manager_socket:
    type: disk
    shift: true
    source: /run/user/1000/pipewire-0-manager
    path: /mnt/.container_sockets/pipewire-0-manager
  pulseaudio_socket:
    type: disk
    shift: true
    source: /run/user/1000/pulse/native
    path: /mnt/.container_sockets/native

Note 1: Profile requires creating a copy of Xwaylandauth cookie with the new name .mutter-Xwaylandauth.copy, more on that in What’s new section below.

Note 2: Profile assumes that your user on the host has a UID and GID 1000, as well as the user in the container has a UID and GID 1000. You can check this using the id -u and id -g commands.

raw.idmap: |-
  uid 1000 1000
  gid 1000 1000

means:

raw.idmap: |-
  uid <host_user_uid> <container_user_uid>
  gid <host_user_gid> <container_user_gid>

Note 3: The GID for GPU is 44 because this is the GID for video group in Ubuntu containers.

Note 4: Profile adds access to the Wayland, xWayland, xwaylandauth cookie (also know as xauthority cookie), pipewire and pulseaudio sockets, which will be available in /mnt/.container_sockets/ folder and have to be linked to their proper locations using a script.

Script

You can name this script anyway you want, for example incus_ct_gui.sh. It will:

  • Install pulseaudio-utils and dbus-user-session packages
  • Add default container user to the video and render groups (this is not strictly necessary, but it helps with hardware video decoding, etc.)
  • Set up environment variables in the $HOME/.profile file
  • Add commands to the $HOME/.profile file that link sockets from /mnt/.container_sockets/ folder to their proper locations (idea come from this post by @catfish)
#!/bin/bash
#
# This script should be run inside Incus containers. It links sockets added by the 'gui' profile, from ${mnt_dir} to their proper locations on every login, installs a couple of packages and sets up necessary environment variables.

readonly mnt_dir="/mnt/.container_sockets"
readonly run_dir="$XDG_RUNTIME_DIR"
readonly tmp_dir="/tmp/.X11-unix"

function modify_profile() {
  # Add socket linking commands to the "$HOME/.profile" file, copy Xwaylandauth cookie and set up necessary environment variables.
  # Now on every user login sockets from ${mnt_dir} will be linked to their proper locations.
  cat << EOF >> "$HOME/.profile"
[[ ! -d "${run_dir}/pulse" ]] && mkdir -m 700 "${run_dir}/pulse"
[[ -S "${mnt_dir}/native" && -d "${run_dir}/pulse" && ! -e "${run_dir}/pulse/native" ]] && ln -s "${mnt_dir}/native" "${run_dir}/pulse/native"
[[ -S "${mnt_dir}/pipewire-0" && ! -e "${run_dir}/pipewire-0" ]] && ln -s "${mnt_dir}/pipewire-0" "${run_dir}/pipewire-0"
[[ -S "${mnt_dir}/pipewire-0-manager" && ! -e "${run_dir}/pipewire-0-manager" ]] && ln -s "${mnt_dir}/pipewire-0-manager" "${run_dir}/pipewire-0-manager"
[[ -S "${mnt_dir}/wayland-0" && ! -e "${run_dir}/wayland-0" ]] && ln -s "${mnt_dir}/wayland-0" "${run_dir}/wayland-0"
[[ -S "${mnt_dir}/X1" && ! -e "${tmp_dir}/X1" ]] && ln -s "${mnt_dir}/X1" "${tmp_dir}/X1"
[[ -f "${mnt_dir}/.mutter-Xwaylandauth.copy" && ! -e "${run_dir}/.mutter-Xwaylandauth.copy" ]] && cp "${mnt_dir}/.mutter-Xwaylandauth.copy" "${run_dir}/.mutter-Xwaylandauth.copy"
export WAYLAND_DISPLAY=wayland-0
export XDG_SESSION_TYPE=wayland
export QT_QPA_PLATFORM=wayland
export DISPLAY=:1
export XAUTHORITY=${run_dir}/.mutter-Xwaylandauth.copy
EOF
}

function main() {
  # Add user to the video and render groups.
  sudo usermod -a -G video,render "$LOGNAME"
  sudo apt update && sudo apt upgrade -y && sudo apt install -y pulseaudio-utils dbus-user-session
  modify_profile
}

main

Push the script to the container, log in, execute it (for containers other than Ubuntu, change the default username) and reboot the container:

incus file push --gid=1000 --uid=1000 --mode=0744 /<path>/incus_ct_gui.sh <instance_name>/home/ubuntu/
incus exec <instance_name> -- sudo --user ubuntu --login
./incus_ct_gui.sh

Now you can start apps like this:

incus exec <instance_name> -- sudo --login --user ubuntu bash -ilc "<command>"

What’s new

If you previously used Ubuntu 22.04 profile, you’ll notice three things in 24.04 version:

  • X11 / xWayland sockets require Xwaylandauth cookie (also know as Xauthority cookie)
  • PipeWire is the new multimedia framework for Ubuntu and uses two new sockets
  • I use both shift: true and raw.idmap for shifting at the same time

Xauthority cookie is a bit tricky, because it has a random suffix that changes every time the host boots, for example .mutter-Xwaylandauth.77CDN2. But we can use Startup Applications on the host with the following entry to create a copy of that cookie with a fixed name on every boot:

bash -c "cp -f $XAUTHORITY $XDG_RUNTIME_DIR/.mutter-Xwaylandauth.copy"

Profile will make that copy accessible inside container.

Adding support for PipeWire requires two new sockets on top of previous PulseAudio: pipewire-0 and pipewire-0-manager.

Using both shift: true and raw.idmap at the same time eliminates a lot of weird errors when using snap packages, flatpaks and sometimes even regular debs. From my testing shift: true takes precedent over raw.idmap and using both at the same time should have no negative consequences for other devices that use shifting.

You can test that by creating a container with a purposefully wrong raw.idmap and sharing a folder with it. Look at permissions for that folder inside the container. Now set the shift: true for that folder and notice the change in permissions.

Troubleshooting

Sockets

Run those commands inside a container to check if all sockets are where they should be:

ll /tmp/.X11-unix/X?
ll /run/user/*/
ll /run/user/*/pulse/

xWayland

If you have problems with running xWayland / X11 apps just after the host boots, try
launching any X11 app on host, for example xclock.

If you still have problems with apps using xWayland, try X11 socket instead. You can add it to the container like this:

incus config device add <instance_name> x11_socket disk shift=true source=/tmp/.X11-unix/X0 path=/mnt/.container_sockets/X0

Then in .profile file inside container change DISPLAY environment variable to export DISPLAY=:0, then add this line and reboot it:

[[ -S "/mnt/.container_sockets/X0" && ! -e "/tmp/.X11-unix/X0" ]] && ln -s "/mnt/.container_sockets/X0" "/tmp/.X11-unix/X0"

You can completely disable xWayland socket by overriding it with this command:

incus config device add <instance_name> xwayland_socket none

Shifting

To change raw.idmap, set a new value with this command:

printf "uid $(id -u) 1000\ngid $(id -g) 1000" | incus config set <instance_name> raw.idmap -

To unset raw.idmap, use this command:

incus config unset <instance_name> raw.idmap

Audio

To test audio, run pactl info command couple of times, fast. If it shows Connection failure: Access denied at least once, then try copying pulseaudio cookie from host ~/.config/pulse/cookie to the container:

incus file push -p --mode=600 --gid=1000 --uid=1000 ~/.config/pulse/cookie <instance_name>/home/ubuntu/.config/pulse/

If you still have problems with audio, you may try disabling shared memory inside container in /etc/pulse/client.conf config file manually or with this command:

sed -i "s/; enable-shm = yes/enable-shm = no/g" /etc/pulse/client.conf
5 Likes
Preliminary profile for Ubuntu 24.04

This is a preliminary profile for running GUI apps in Incus / LXD containers with Ubuntu 24.04 as the host. I’m not yet switching to Noble Numbat, so the GUI profile modifications are the result of limited testing. Expect some issues in edge cases.

What changed since Jammy Jellyfish profile:

  • PipeWire is the new multimedia framework for Ubuntu
  • X11 / xWayland sockets require Xauthority cookie
  • AppArmor and the Ubuntu kernel restricts the use of unprivileged user namespaces (in my testing, this mainly affected web browsers)
  • The default user ubuntu in Noble Numbat containers no longer belongs to video and render groups
  • Cloud-init in Noble Numbat containers has some issues running my set_up_sockets.sh script placed in /var/lib/cloud/scripts/per-boot/

PipeWire

I added two new sockets pipewire-0 and pipewire-0-manager. Even though Ubuntu 24.04 uses PipeWire, we still need to copy the pulse cookie from the host to the containers (--uid=1000 --gid=1000 are the UID and GID of the user inside the container):

incus file push -p --mode=600 --uid=1000 --gid=1000 ~/.config/pulse/cookie <container_name>/home/<username>/.config/pulse/

I don’t know if you still need the pulseaudio-utils package in every container. If you have any audio problems, you can also try installing pipewire-utils.

Xauthority

This cookie is created in the /run/user/1000/ folder each time the host starts and has a random suffix, for example .mutter-Xwaylandauth.77CDN2. Because we can’t create a disk device for a file with .* mask at the end, we need a workaround. On the host we can use Startup Applications with the following entry to create a copy of the cookie with a fixed name on every boot:

bash -c "cp -f $XAUTHORITY $XDG_RUNTIME_DIR/.mutter-Xwaylandauth.incus"

Now we can pass that cookie to the container using a regular disk device:

  xauthority_cookie:
    type: disk
    shift: true
    source: /run/user/1000/.mutter-Xwaylandauth.incus
    path: /mnt/.container_sockets/.mutter-Xwaylandauth.incus

Finally, in the container we need to set an environment variable with the path to the cookie:

echo "export XAUTHORITY=/mnt/.container_sockets/.mutter-Xwaylandauth.incus" >> /home/ubuntu/.profile

Without that we would get an error:

Authorization required, but no authorization protocol specified
Error: Can't open display: :0

userns restriction

AppArmor and the Ubuntu kernel restricts the use of unprivileged user namespaces. Most applications are not affected, and for those that are, the problem can usually be resolved by installing the apparmor package, which puts several pre-built profiles in the /etc/apparmor.d/ folder.

The snap version of Firefox generates some additional errors that I haven’t found a solution for:

[512] Wayland Proxy [0x79f9b9151bd0] Error: CheckWaylandDisplay(): Failed to connect to Wayland display '/run/user/1000/snap.firefox/wayland-0' error: Permission denied
Error: we don't have any display, WAYLAND_DISPLAY='wayland-0' DISPLAY='(null)'

When I run tail -f -n 30 /var/log/syslog command on the host, I get apparmor="DENIED" operation="connect" error when Firefox starts:

2024-05-02T20:06:37.464526+00:00 test kernel: audit: type=1400 audit(1714680397.462:280): apparmor="DENIED" operation="connect" class="file" namespace="root//incus-t_<var-lib-incus>" profile="snap.firefox.firefox" name="/run/user/1000/wayland-0" pid=10024 comm="firefox" requested_mask="wr" denied_mask="wr" fsuid=1001000 ouid=1000

It seems to be snap specific, because Firefox installed from .deb works perfectly fine. Any help would be appreciated.

Cloud-init scripts

For some reason cloud-init in Noble Numbat containers has issues running my set_up_sockets.sh script placed in /var/lib/cloud/scripts/per-boot/ (it’s executed too soon?). As a workaround, I added a script to the profile that creates a service that runs set_up_sockets.sh every time the container starts.

Profile

Note: this profile assumes that your host user has UID and GID 1000. You can check those using id -u and id -g commands. If they are different, change the following lines in the devices: section only (not in the scripts, they in turn assume that the default container user has UID 1000):

  • every /run/user/1000/... to your host user UID /run/user/<host_user_uid>/...
  • every security.uid: "1000" to your host user UID security.uid: "<host_user_uid>"
  • every security.gid: "1000" to your host user GID security.gid: "<host_user_gid>"

Profile for Ubuntu 24.04 as the host:

config:
  security.nesting: "true"
  cloud-init.user-data: |
    #cloud-config
    package_update: true
    package_upgrade: true
    package_reboot_if_required: true
    packages:
      - pulseaudio-utils
      - dbus-user-session
    write_files:
    - path: /var/lib/cloud/scripts/per-boot/set_up_sockets.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_gid="$( getent passwd ${user_uid} | cut -d: -f4 )"
        if [[ -n ${user_gid} ]]; then
          mnt_dir="/mnt/.container_sockets"
          run_dir="/run/user/${user_uid}"
          [[ ! -d "${run_dir}" ]] && mkdir -m 700 -p "${run_dir}" && chown ${user_uid}:${user_gid} "${run_dir}"
          [[ ! -d "${run_dir}/pulse" && -d "${run_dir}" ]] && mkdir -m 700 "${run_dir}/pulse" && chown -R ${user_uid}:${user_gid} "${run_dir}/pulse"
          [[ -S "${mnt_dir}/wayland-0" && -d "${run_dir}" && ! -e "${run_dir}/wayland-0" ]] && touch "${run_dir}/wayland-0" && sudo mount --bind "${mnt_dir}/wayland-0" "${run_dir}/wayland-0"
          [[ -S "${mnt_dir}/pipewire-0" && -d "${run_dir}" && ! -e "${run_dir}/pipewire-0" ]] && touch "${run_dir}/pipewire-0" && sudo mount --bind "${mnt_dir}/pipewire-0" "${run_dir}/pipewire-0"
          [[ -S "${mnt_dir}/pipewire-0-manager" && -d "${run_dir}" && ! -e "${run_dir}/pipewire-0-manager" ]] && touch "${run_dir}/pipewire-0-manager" && sudo mount --bind "${mnt_dir}/pipewire-0-manager" "${run_dir}/pipewire-0-manager"
          [[ -S "${mnt_dir}/native" && -d "${run_dir}/pulse" && ! -e "${run_dir}/pulse/native" ]] && touch "${run_dir}/pulse/native" && sudo mount --bind "${mnt_dir}/native" "${run_dir}/pulse/native"
        fi
    - path: /var/lib/cloud/scripts/per-once/set_up_sockets_service.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_name="$( getent passwd ${user_uid} | cut -d: -f1 )"
        home_dir="$( getent passwd ${user_uid} | cut -d: -f6 )"
        if [[ -d "${home_dir}" && -n ${user_name} ]]; then
          run_as_user="runuser -u ${user_name} --"
          service_dir="${home_dir}/.config/systemd/user"
          target_dir="${service_dir}/default.target.wants"
          ${run_as_user} mkdir -p "${target_dir}"
          ${run_as_user} touch "${service_dir}/set_up_sockets.service"
          cat >> ${service_dir}/set_up_sockets.service << EOF
          [Unit]
          Description=Run set_up_sockets.sh on every boot
          After=local-fs.target
          [Service]
          Type=oneshot
          ExecStart=/var/lib/cloud/scripts/per-boot/set_up_sockets.sh
          [Install]
          WantedBy=default.target
        EOF
          ${run_as_user} ln -s "${service_dir}/set_up_sockets.service" "${target_dir}/"
        fi
    - path: /var/lib/cloud/scripts/per-once/set_up_env_vars.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_name="$( getent passwd ${user_uid} | cut -d: -f1 )"
        [[ -n ${user_name} ]] && usermod -a -G render,video ${user_name}
        mnt_dir="/mnt/.container_sockets"
        home_dir="$( getent passwd ${user_uid} | cut -d: -f6 )"
        profile="${home_dir}/.profile"
        bashrc="${home_dir}/.bashrc"
        if [[ -f "${profile}" ]]; then
          echo "export WAYLAND_DISPLAY=wayland-0" >> "${profile}"
          echo "export XDG_SESSION_TYPE=wayland" >> "${profile}"
          echo "export QT_QPA_PLATFORM=wayland" >> "${profile}"
          echo "export DISPLAY=:1" >> "${profile}"
          echo "export XAUTHORITY=${mnt_dir}/.mutter-Xwaylandauth.incus" >> "${profile}"
        fi
    - path: /var/lib/cloud/scripts/per-once/change_gid.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_name="$( getent passwd ${user_uid} | cut -d: -f1 )"
        user_gid="$( getent passwd ${user_name} | cut -d: -f4 )"
        home_dir="$( getent passwd ${user_name} | cut -d: -f6 )"
        if [[ -n ${user_name} && ! ${user_uid} == ${user_gid} ]]; then
          group_to_move="$( getent group ${user_uid} | cut -d: -f1 )"
          if [[ -n ${group_to_move} ]]; then
            for gid in {1000..6000}; do
              return_value="$( getent group ${gid} )"
              if [[ -z ${return_value} ]]; then
                groupmod -g ${gid} ${group_to_move}
                break
              fi
            done
          fi
          users_group="$( getent group ${user_gid} | cut -d: -f1 )"
          groupmod -g ${user_uid} ${users_group}
          chown -R ${user_uid}:${user_uid} "${home_dir}"
        fi
description: GUI Wayland and xWayland profile with pipewire and pulseaudio, shifting enabled
devices:
  gpu:
    type: gpu
    gid: 44
  xwayland_socket:
    type: proxy
    bind: container
    security.gid: "1000"
    security.uid: "1000"
    connect: unix:@/tmp/.X11-unix/X1
    listen: unix:@/tmp/.X11-unix/X1
  xauthority_cookie:
    type: disk
    shift: true
    source: /run/user/1000/.mutter-Xwaylandauth.incus
    path: /mnt/.container_sockets/.mutter-Xwaylandauth.incus
  wayland_socket:
    type: disk
    shift: true
    source: /run/user/1000/wayland-0
    path: /mnt/.container_sockets/wayland-0
  pipewire_socket:
    type: disk
    shift: true
    source: /run/user/1000/pipewire-0
    path: /mnt/.container_sockets/pipewire-0
  pipewire_manager_socket:
    type: disk
    shift: true
    source: /run/user/1000/pipewire-0-manager
    path: /mnt/.container_sockets/pipewire-0-manager
  pulseaudio_socket:
    type: disk
    shift: true
    source: /run/user/1000/pulse/native
    path: /mnt/.container_sockets/native
Profile for Ubuntu 22.04 with kernel 6.5.0+

Ubuntu 22.04 has a new kernel 6.5.0 and now VFS idmap shifting is working for some sockets (wayland and pulseaudio, but not X11), so I updated my profile for GUI apps to take advantage of this.

What’s new

Scripts do exactly what they did before, but are rewritten and are placed in /var/lib/cloud/scripts/ folder, which means they’ll be executed by cloud-init automatically. Subfolder /per-once means that the script will be run once on first boot of an instance, and it won’t be run again even if you clone an instance or create a new instance from a saved image. Subfolder /per-boot means that the script will be run on every boot.

Also, scripts don’t assume the default user in a container is named ubuntu anymore, instead they assume the user’s UID is 1000. This way they don’t have to be tweaked to works for other distros.

  • Script set_up_sockets.sh puts wayland and pulseaudio sockets in proper locations inside container.
  • Script set_up_env_vars.sh sets up required environment variables for the user inside container.
  • Script change_gid.sh makes sure the default user in container with UID 1000 has his GID also 1000. For example, in containers images:ubuntu/jammy/cloud default user ubuntu has GID 1002, which makes sockets and shared folders in container belong to ubuntu:lxd instead of ubuntu:ubuntu. In short, if there is a mismatch this script moves a group that has GID 1000 to the first free GID and assigns 1000 to the group of default user, then fixes ownership of user’s home folder.

I switched from X11 to xWayland socket, and it works perfectly well. You can always change it back by switching socket X1 to X0 and corresponding environment variable from DISPLAY=:1 to DISPLAY=:0.

When using VFS idmap shifting for pulseuadio socket, you have to copy pulse cookie from host into container (--uid=1000 --gid=1000 are the UID and GID of the user inside container):

incus file push -p --mode=600 --uid=1000 --gid=1000 ~/.config/pulse/cookie <container_name>/home/<username>/.config/pulse/

Profile

As before, this profile is for:

  • both Incus and LXD
  • Ubuntu 22.04 container images, both from community repository images:ubuntu/jammy/cloud and official Canonical repository ubuntu:22.04
  • but should work for other distros with very little modifications
config:
  security.nesting: "true"
  cloud-init.user-data: |
    #cloud-config
    package_update: true
    package_upgrade: true
    package_reboot_if_required: true
    packages:
      - pulseaudio-utils
    write_files:
    - path: /var/lib/cloud/scripts/per-boot/set_up_sockets.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_name=$( getent passwd ${user_uid} | cut -d: -f1 )
        user_gid=$( getent passwd ${user_name} | cut -d: -f4 )
        if [[ -n ${user_name} ]]; then
          mnt_dir=/mnt/.container_sockets
          run_dir=/run/user/${user_uid}
          [[ ! -d "${run_dir}" ]] && mkdir -p "${run_dir}" && chmod 700 "${run_dir}" && chown ${user_uid}:${user_gid} "${run_dir}"
          [[ ! -d "${run_dir}/pulse" ]] && mkdir -p "${run_dir}/pulse" && chmod 700 "${run_dir}/pulse" && chown ${user_uid}:${user_gid} "${run_dir}/pulse"
          [[ -S "${mnt_dir}/wayland-0" ]] && [[ -d "${run_dir}" ]] && [[ ! -e "${run_dir}/wayland-0" ]] && touch "${run_dir}/wayland-0" && sudo mount --bind "${mnt_dir}/wayland-0" "${run_dir}/wayland-0"
          [[ -S "${mnt_dir}/native" ]]  && [[ -d "${run_dir}/pulse" ]] && [[ ! -e "${run_dir}/pulse/native" ]] && touch "${run_dir}/pulse/native" && sudo mount --bind "${mnt_dir}/native" "${run_dir}/pulse/native"
        fi
    - path: /var/lib/cloud/scripts/per-once/set_up_env_vars.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        home_dir="$( getent passwd ${user_uid} | cut -d: -f6 )"
        profile="${home_dir}/.profile"
        if [[ -f "${profile}" ]]; then
          echo "export WAYLAND_DISPLAY=wayland-0" >> "${profile}"
          echo "export XDG_SESSION_TYPE=wayland" >> "${profile}"
          echo "export QT_QPA_PLATFORM=wayland" >> "${profile}"
          echo "export DISPLAY=:1" >> "${profile}"
        fi
    - path: /var/lib/cloud/scripts/per-once/change_gid.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_name=$( getent passwd ${user_uid} | cut -d: -f1 )
        user_gid=$( getent passwd ${user_name} | cut -d: -f4 )
        home_dir=$( getent passwd ${user_name} | cut -d: -f6 )
        if [[ -n ${user_name} && ! ${user_uid} == ${user_gid} ]]; then
          group_to_move=$( getent group ${user_uid} | cut -d: -f1 )
          if [[ -n ${group_to_move} ]]; then
            for gid in {1000..6000}; do
              return_value=$( getent group ${gid} )
              if [[ -z ${return_value} ]]; then
                groupmod -g ${gid} ${group_to_move}
                break
              fi
            done
          fi
          users_group=$( getent group ${user_gid} | cut -d: -f1 )
          groupmod -g ${user_uid} ${users_group}
          chown -R ${user_uid}:${user_uid} "${home_dir}"
        fi
description: GUI Wayland and xWayland profile with pulseaudio, shifting enabled
devices:
  gpu:
    type: gpu
    gid: 44
  pulseaudio_socket:
    type: disk
    shift: true
    source: /run/user/1000/pulse/native
    path: /mnt/.container_sockets/native
  wayland_socket:
    type: disk
    shift: true
    source: /run/user/1000/wayland-0
    path: /mnt/.container_sockets/wayland-0
  xwayland_socket:
    type: proxy
    bind: container
    security.gid: "1000"
    security.uid: "1000"
    connect: unix:@/tmp/.X11-unix/X1
    listen: unix:@/tmp/.X11-unix/X1

Note: profile assumes your user on host (not in container) has UID and GID 1000. You can check it using id -u and id -g commands. If that’s not true, change the following lines to match your UID and GID on host:

  pulseaudio_socket:
    source: /run/user/1000/pulse/native
  wayland_socket:
    source: /run/user/1000/wayland-0
  xwayland_socket:
    security.gid: "1000"
    security.uid: "1000"

Where:

  • /run/user/1000/... means /run/user/<host_user_uid>/...
  • security.gid: "1000" means security.gid: "<host_user_uid>"
  • security.uid: "1000" means security.uid: "<host_user_gid>"

Troubleshooting

See the first post.

If you notice problems with snap packages installed in container, you can try an intermediate profile:

Profile for snaps troubleshooting

This profile uses new scripts to set up sockets, but keeps the raw.dmap setting. Like the old profile, it assumes that your user on the host has a UID and GID 1000, as well as the user in the container has a UID and GID 1000. You can check this using the id -u and id -g commands.

raw.idmap: |-
  uid 1000 1000
  gid 1000 1000

means:

raw.idmap: |-
  uid <host_user_uid> <container_user_uid>
  gid <host_user_gid> <container_user_gid>

Profile for snaps troubleshooting:

config:
  raw.idmap: |-
    uid 1000 1000
    gid 1000 1000
  security.nesting: "true"
  cloud-init.user-data: |
    #cloud-config
    package_update: true
    package_upgrade: true
    package_reboot_if_required: true
    packages:
      - pulseaudio-utils
    write_files:
    - path: /var/lib/cloud/scripts/per-boot/set_up_sockets.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_name=$( getent passwd ${user_uid} | cut -d: -f1 )
        user_gid=$( getent passwd ${user_name} | cut -d: -f4 )
        if [[ -n ${user_name} ]]; then
          mnt_dir=/mnt/.container_sockets
          run_dir=/run/user/${user_uid}
          tmp_dir=/tmp/.X11-unix
          [[ ! -d "${tmp_dir}" ]] && mkdir -p "${tmp_dir}" && chmod 777 "${tmp_dir}"
          [[ ! -d "${run_dir}" ]] && mkdir -p "${run_dir}" && chmod 700 "${run_dir}" && chown ${user_uid}:${user_gid} "${run_dir}"
          [[ ! -d "${run_dir}/pulse" ]] && mkdir -p "${run_dir}/pulse" && chmod 700 "${run_dir}/pulse" && chown ${user_uid}:${user_gid} "${run_dir}/pulse"
          [[ -S "${mnt_dir}/X0" ]] && [[ -d "${tmp_dir}" ]] && [[ ! -e "${tmp_dir}/X0" ]] && touch "${tmp_dir}/X0" && sudo mount --bind "${mnt_dir}/X0" "${tmp_dir}/X0"
          [[ -S "${mnt_dir}/wayland-0" ]] && [[ -d "${run_dir}" ]] && [[ ! -e "${run_dir}/wayland-0" ]] && touch "${run_dir}/wayland-0" && sudo mount --bind "${mnt_dir}/wayland-0" "${run_dir}/wayland-0"
          [[ -S "${mnt_dir}/native" ]]  && [[ -d "${run_dir}/pulse" ]] && [[ ! -e "${run_dir}/pulse/native" ]] && touch "${run_dir}/pulse/native" && sudo mount --bind "${mnt_dir}/native" "${run_dir}/pulse/native"
        fi
    - path: /var/lib/cloud/scripts/per-once/set_up_env_vars.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        home_dir="$( getent passwd ${user_uid} | cut -d: -f6 )"
        profile="${home_dir}/.profile"
        if [[ -f "${profile}" ]]; then
          echo "export WAYLAND_DISPLAY=wayland-0" >> "${profile}"
          echo "export XDG_SESSION_TYPE=wayland" >> "${profile}"
          echo "export QT_QPA_PLATFORM=wayland" >> "${profile}"
          echo "export DISPLAY=:0" >> "${profile}"
        fi
    - path: /var/lib/cloud/scripts/per-once/change_gid.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_name=$( getent passwd ${user_uid} | cut -d: -f1 )
        user_gid=$( getent passwd ${user_name} | cut -d: -f4 )
        home_dir=$( getent passwd ${user_name} | cut -d: -f6 )
        if [[ -n ${user_name} && ! ${user_uid} == ${user_gid} ]]; then
          group_to_move=$( getent group ${user_uid} | cut -d: -f1 )
          if [[ -n ${group_to_move} ]]; then
            for gid in {1000..6000}; do
              return_value=$( getent group ${gid} )
              if [[ -z ${return_value} ]]; then
                groupmod -g ${gid} ${group_to_move}
                break
              fi
            done
          fi
          users_group=$( getent group ${user_gid} | cut -d: -f1 )
          groupmod -g ${user_uid} ${users_group}
          chown -R ${user_uid}:${user_uid} "${home_dir}"
        fi
description: GUI Wayland and X11 profile with pulseaudio, raw.idmap
devices:
  gpu:
    type: gpu
    gid: 44
  pulseaudio_socket:
    type: disk
    source: /run/user/1000/pulse/native
    path: /mnt/.container_sockets/native
  wayland_socket:
    type: disk
    source: /run/user/1000/wayland-0
    path: /mnt/.container_sockets/wayland-0
  x11_socket:
    source: /tmp/.X11-unix/X0
    path: /mnt/.container_sockets/X0
    type: disk

Bonus profile for Arch containers

The only difference is that in Arch video group has GID 985 and default user is not added to it. So GPU device has this GID and script set_up_env_vars.sh has a new line usermod -a -G video ${user_name}. It also sets missing environment variable XDG_RUNTIME_DIR.

Remember to copy pulse cookie from host into container as shown before, or audio won’t work.

config:
  security.nesting: "true"
  cloud-init.user-data: |
    #cloud-config
    package_update: true
    package_upgrade: true
    package_reboot_if_required: true
    packages:
      - libpulse
    write_files:
    - path: /var/lib/cloud/scripts/per-boot/set_up_sockets.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_name=$( getent passwd ${user_uid} | cut -d: -f1 )
        user_gid=$( getent passwd ${user_name} | cut -d: -f4 )
        if [[ -n ${user_name} ]]; then
          mnt_dir=/mnt/.container_sockets
          run_dir=/run/user/${user_uid}
          [[ ! -d "${run_dir}" ]] && mkdir -p "${run_dir}" && chmod 700 "${run_dir}" && chown ${user_uid}:${user_gid} "${run_dir}"
          [[ ! -d "${run_dir}/pulse" ]] && mkdir -p "${run_dir}/pulse" && chmod 700 "${run_dir}/pulse" && chown ${user_uid}:${user_gid} "${run_dir}/pulse"
          [[ -S "${mnt_dir}/wayland-0" ]] && [[ -d "${run_dir}" ]] && [[ ! -e "${run_dir}/wayland-0" ]] && touch "${run_dir}/wayland-0" && sudo mount --bind "${mnt_dir}/wayland-0" "${run_dir}/wayland-0"
          [[ -S "${mnt_dir}/native" ]]  && [[ -d "${run_dir}/pulse" ]] && [[ ! -e "${run_dir}/pulse/native" ]] && touch "${run_dir}/pulse/native" && sudo mount --bind "${mnt_dir}/native" "${run_dir}/pulse/native"
        fi
    - path: /var/lib/cloud/scripts/per-once/set_up_env_vars.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_name=$( getent passwd ${user_uid} | cut -d: -f1 )
        home_dir="$( getent passwd ${user_name} | cut -d: -f6 )"
        profile="${home_dir}/.bash_profile"
        usermod -a -G video ${user_name}
        if [[ -f "${profile}" ]]; then
          echo "export WAYLAND_DISPLAY=wayland-0" >> "${profile}"
          echo "export XDG_SESSION_TYPE=wayland" >> "${profile}"
          echo "export QT_QPA_PLATFORM=wayland" >> "${profile}"
          echo "export DISPLAY=:1" >> "${profile}"
          echo "export XDG_RUNTIME_DIR=/run/user/${user_uid}" >> "${profile}"
        fi
    - path: /var/lib/cloud/scripts/per-once/change_gid.sh
      permissions: 0755
      content: |
        #!/bin/bash
        user_uid=1000
        user_name=$( getent passwd ${user_uid} | cut -d: -f1 )
        user_gid=$( getent passwd ${user_name} | cut -d: -f4 )
        home_dir=$( getent passwd ${user_name} | cut -d: -f6 )
        if [[ -n ${user_name} && ! ${user_uid} == ${user_gid} ]]; then
          group_to_move=$( getent group ${user_uid} | cut -d: -f1 )
          if [[ -n ${group_to_move} ]]; then
            for gid in {1000..6000}; do
              return_value=$( getent group ${gid} )
              if [[ -z ${return_value} ]]; then
                groupmod -g ${gid} ${group_to_move}
                break
              fi
            done
          fi
          users_group=$( getent group ${user_gid} | cut -d: -f1 )
          groupmod -g ${user_uid} ${users_group}
          chown -R ${user_uid}:${user_uid} "${home_dir}"
        fi
description: GUI Wayland and xWayland profile with pulseaudio, shifting enabled
devices:
  gpu:
    type: gpu
    gid: 985
  pulseaudio_socket:
    type: disk
    shift: true
    source: /run/user/1000/pulse/native
    path: /mnt/.container_sockets/native
  wayland_socket:
    type: disk
    shift: true
    source: /run/user/1000/wayland-0
    path: /mnt/.container_sockets/wayland-0
  xwayland_socket:
    type: proxy
    bind: container
    security.gid: "1000"
    security.uid: "1000"
    connect: unix:@/tmp/.X11-unix/X1
    listen: unix:@/tmp/.X11-unix/X1
Profile for Ubuntu 22.04 with older kernels

I made a profile for running GUI apps in an Incus / LXD container. It supports wayland, X11 and pulseaudio. This profile is based on work of Justin Ludwig, thank you!
https://blog.swwomm.com/2022/08/lxd-containers-for-wayland-gui-apps.html

Profile has been tested using:

  • both Incus and LXD
  • Ubuntu 22.04 as a host
  • Ubuntu 22.04 container images, both from community repository images:ubuntu/jammy/cloud and official Canonical repository ubuntu:22.04

This post is divided into four parts:

  1. Profile
  2. Explanation
  3. Testing
  4. Troubleshooting

1. Profile

Note 1: profile assumes that your user on the host has a UID and GID 1000, as well as the user in the container has a UID and GID 1000. You can check this using the id -u and id -g commands.

raw.idmap: |-
  uid 1000 1000
  gid 1000 1000

means:

raw.idmap: |-
  uid <host_user_uid> <container_user_uid>
  gid <host_user_gid> <container_user_gid>

Note 2: those Ubuntu images have a default ubuntu user, which is hard-coded in the profile.
Note 3: profile adds some environment variables to .profile file inside container. Every time you change a profile using command incus profile edit <profile_name> it will be applied once again to all containers using it and therefore those environment variables will be duplicated in .profile files. This doesn’t break anything, just be aware of that.

Profile:

config:
  raw.idmap: |-
    uid 1000 1000
    gid 1000 1000
  security.nesting: "true"
  user.user-data: |
    #cloud-config
    package_update: true
    package_upgrade: true
    package_reboot_if_required: true
    packages:
      - pulseaudio-utils
    write_files:
    - path: /usr/local/bin/mystartup.sh
      permissions: 0755
      content: |
        #!/bin/sh
        uid=$(id -u)
        run_dir=/run/user/$uid
        mkdir -p $run_dir && chmod 700 $run_dir && chown $uid:$uid $run_dir
        ln -sf /mnt/.container_wayland_socket $run_dir/wayland-0
        mkdir -p $run_dir/pulse && chmod 700 $run_dir/pulse && chown $uid:$uid $run_dir/pulse
        ln -sf /mnt/.container_pulseaudio_socket $run_dir/pulse/native
        tmp_dir=/tmp/.X11-unix
        mkdir -p $tmp_dir
        ln -sf /mnt/.container_x11_socket $tmp_dir/X0
    - path: /usr/local/etc/mystartup.service
      content: |
        [Unit]
        After=local-fs.target
        [Service]
        Type=oneshot
        ExecStart=/usr/local/bin/mystartup.sh
        [Install]
        WantedBy=default.target
    runcmd:
    - mkdir -p /home/ubuntu/.config/systemd/user/default.target.wants
    - ln -s /usr/local/etc/mystartup.service /home/ubuntu/.config/systemd/user/default.target.wants/mystartup.service
    - ln -s /usr/local/etc/mystartup.service /home/ubuntu/.config/systemd/user/mystartup.service
    - chown -R ubuntu:ubuntu /home/ubuntu
    - echo 'export WAYLAND_DISPLAY=wayland-0' >> /home/ubuntu/.profile
    - echo 'export XDG_SESSION_TYPE=wayland' >> /home/ubuntu/.profile
    - echo 'export QT_QPA_PLATFORM=wayland' >> /home/ubuntu/.profile
    - echo 'export DISPLAY=:0' >> /home/ubuntu/.profile
description: GUI Wayland and X11 profile with pulseaudio
devices:
  gpu:
    type: gpu
    gid: 44
  wayland_socket:
    source: /run/user/1000/wayland-0
    path: /mnt/.container_wayland_socket
    type: disk
  x11_socket:
    source: /tmp/.X11-unix/X0
    path: /mnt/.container_x11_socket
    type: disk
  pulseaudio_socket:
    source: /run/user/1000/pulse/native
    path: /mnt/.container_pulseaudio_socket
    type: disk

The easiest way to use a profile like that is to copy it into a text file, then create an empty profile in Incus:

incus profile create <profile_name>

and update that profile with the file’s content:

incus profile edit <profile_name> < /<path>/<file_name>

2. Explanation

For an in depth explanation of how this profile creates a script and startup systemd service that links all sockets to their usual location inside the container, please read Justin’s post at his blog. Here I’ll explain only tweaks I made:

  • Installing pulseaudio-utils package will create pulseaudio cookie inside container.
  • I added X11 socket for apps that don’t use Wayland yet.
  • All sockets are shared as type: disk, not type: proxy device.
  • Adding GPU with gid: 44 enables GPU hardware video acceleration in containers. See Testing section below.
  • Key security.nesting: "true" is for Steam, Docker, etc.

3. Testing

Launch container using this profile:

incus launch images:ubuntu/jammy/cloud -p default -p <profile_name> <container_name>

Now login into your container:

incus exec <container_name> -- sudo --user ubuntu --login

Pulseaudio socket is called native and you can find it at /run/user/1000/pulse/. Wayland socket wayland-0 is in /run/user/1000/ folder, and X11 socket X0 is in /tmp/.X11-unix/ folder. On host in /tmp/.X11-unix/ folder you will find also Xwayland socket as X1.

All those sockets inside container should be visible at /mnt/ and linked into proper folders. Run those command to check if that’s true:

ll /mnt/.container*
ll /tmp/.X11-unix/X?
ll /run/user/*/wayland-?
ll /run/user/*/pulse/native

Check if most important environment variables are set, mainly WAYLAND_DISPLAY=wayland-0 and DISPLAY=:0:

printenv | grep -i display

Check if your user inside container is part of the video group using groups command. Then see if video render is owned by this group using ll /dev/dri/ command. Output should show root video (without gid: 44 it would be root root):

crw-rw---- 1 root video 226,   0 Nov 15 08:41 card0
crw-rw---- 1 root video 226, 128 Nov 15 08:41 renderD128

To test pulseaudio run pactl info command couple of times. If it shows Connection failure: Access denied at least once, then see Troubleshooting section.

For a final test, install Chrome, then run it in Wayland or X11 mode and watch any Youtube video with sound:

sudo apt update && sudo apt upgrade -y
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo apt install ~/google-chrome-stable_current_amd64.deb
sudo apt install libegl1

Without libegl1 package, Chrome will complain. Depending on which ubuntu image you used to create the container, Chrome will spill out some errors, but should work perfectly fine. To run Chrome in Wayland mode, use this command:

google-chrome --enable-features=UseOzonePlatform --ozone-platform=wayland

To run it in X11 mode, simply use this command:

google-chrome

Now on the host you can run xlsclients and see if Chrome shows up when run in X11 mode and if it’s absent when run in Wayland mode.

4. Troubleshooting

If pulseaudio test pactl info showed Connection failure: Access denied, then try copying pulseaudio cookie from host ~/.config/pulse/cookie to container:

incus file push -p --mode=600 --gid=1000 --uid=1000 ~/.config/pulse/cookie <container_name>/home/ubuntu/.config/pulse/

If you still have problems with pulseaudio you may try to disable shared memory inside container in /etc/pulse/client.conf config file manually or with this command:

sed -i "s/; enable-shm = yes/enable-shm = no/g" /etc/pulse/client.conf

If Chrome doesn’t start in X11 mode, you can try changing in profile socket X0 to X1 and corresponding environment variable from DISPLAY=:0 to DISPLAY=:1 .

2 Likes

Thanks for this. The instructions on how to run GUI apps in system containers have gone a long way.

It was thanks to your blog posts that I learned about lxd and containers. Isolating things is tremendously useful when playing with software like Webots, etc. So thank you :kissing_heart:

1 Like

How difficult might it be to get this profile working on a Debian 12 host and Debian 12 container?

Would it not be possible to create a container image that is specifically designed to run X11 applications?

This profile should work as is on a Debian 12 host and Debian 12 container.

What do you mean by “specifically designed to run X11 applications”? Without Wayland and Pulseaudio bits?

I mean a container image that includes an X Window server and PulseAudio or other sound libraries that would support a graphical desktop environment. In effect, it would already include the profile that you created or a similar one.

Locally, there are two ways to do that:

  • You can create an image from an instance or a snapshot that uses this profile, and use it as a base for new instances.
  • If you’re familiar with distrobuilder, you can build your own images from scratch with anything you want.
1 Like

This post has been moved here.

4 Likes

@eliranwong what exactly wasn’t working for you in 24.04 version? I’m still on 22.04, so my tests were rather preliminary.

@qkiel Thanks for work and reply. I tried both of your 22.04 and 24.04 profiles. Both have issues. If you don’t mind, I shared a bit here:

Regarding, I encountered three major issues with your 22.04 profile:

  1. I tried several gui apps, one required /dev/dri devices. This issue is easy to be resolved by adding the /dev/dri devices.
  2. Snap version of firefox cannot be loaded most of the time
  3. It is unacceptably slow to load GTK-based applications, e.g. gedit.

You may find my solutions for issue 2 & 3 from my github notes

Regarding your 24.04 profile, it was not working for display. I am afraid I did not record full details, but might be something about xauth. I didn’t go deep into the issue, as soon as I find my display solution work better and more universal.

For your information, I use the following for display instead:

  x11_server:
    bind: container
    connect: unix:@/tmp/.X11-unix/X1
    listen: unix:@/tmp/.X11-unix/X1
    type: proxy

Reasons:

  1. It saves me time from dealing with xauth issues, simply run “xhost +local:” in host to make the display works.
  2. It display snap applications, e.g. firefox, running on 22.04. The disk-type device, suggested in your 22.04 profile does not work with snap apps.
  3. It is like an universal solution that suits for both wayland and x11 users. As wayland offer backward x11 comtability via xwayland, this solution works on my 24.04 device that runs wayland. The proxy device works with connection either to X0 or X1.
  4. I use and develop some qt-based applications, some qt-based apps does not work with touch screen well on wayland. In addition, moving windows programatically does not work well with wayland. Therefore, I need to use QT_QPA_PLATFORM=xcb anyway.

I would like to note here that a construct like unix:@/tmp/.X11-unix/X1 is an abstract Unix socket, which means it’s not an actual file on the filesystem. This is important because snap packages cannot read files or enter directories that start with a dot. Such files and directories are inaccessible to snap packages.

A diff between your two profiles shows that @eliranwong has added more system variables that may or may not be used in the container. Those are targeting X11.

Thanks @simos May I add two further notes:

  1. The profile “x11_pipewire_shift” that you used for comparison above does not work with snap packages. The profile with an extra word “snap”, i.e. x11_pipewire_snap_shift, is the profile that works with snap on 22.04.

  2. For running on 24.04 LTS, as noted in file ubuntu_24.04_LTS_tested.md I need to use:

  pulseovernetwork:
    bind: container
    connect: tcp:127.0.0.1:4713
    listen: tcp:127.0.0.1:4713
    type: proxy

INSTEAD OF

  pipewire_socket:
    type: disk
    shift: true
    source: /run/user/1000/pipewire-0
    path: /mnt/.container_sockets/pipewire-0

in order for both sound and microphone to work.

Some snap packages do not have access to /mnt/, unless they have the removable disk or something permission. On top of that, snap packages cannot access the hidden folder .contrainer_sockets.

I would suggest to create a folder in the home directory of the non-root user (in the container), such as /home/ubuntu/INCUS/ and place there any sockets and mount points.

1 Like

All the sockets are mount --bind to their proper locations, e.g. /run/user/1000/, so I don’t think that snaps’ restrictions on /mnt or hidden folders is an issue here.

In Ubuntu 24.04 there were some changes to the Apparmor profiles (mentioned in this post), that I didn’t look into much.

@eliranwong what are the implications of using xhost +local:? I used xauthority magic cookie because I didn’t want to loosen X11 security too much.

On the security implications of running a GUI app in a container (any container offering) is that for the X11 protocol a malicious app in the container can intercept keystrokes from the host.

On the other hand Wayland is supposed to be able to avoid this though I do not know if we need to setup something extra.

Actually the Xauthority magic cookie is located in /mnt/.container_sockets folder and maybe that explains some of the issues after all. I need to experiment some more, thanks for the tip.

I revised my github notes about 24.04 setup as I tested more. I managed to test the following features with both my 22.04 and 24.04 setup:

Tested Hardware: GPU, Sound, Microphone, Camera

Tested GUI: X11 apps, Qt-based GUI apps, GTK-based apps, Snap apps, System-tray apps

Tested input method: ibus

Tested AI Tools: Llama.cpp

I really like incus, even system-tray apps are displayed on host top bar. Setting input method like ibus is so straightforward. It was a painful experience when I tried that on Crostini on ChromeOS. Incus is really awesome.

1 Like

I run a Compositor which doesn’t generate a Authority cookie at /run/user , nor anywhere on the filesystem. The compositor is Hyprland; and it has been disclosed multiple times by the maintainer that any Wayland/XOrg session interaction must be authorized by an authentication-agent, typically polkit-kde-agent , and that it is by design that an xauthority cookie will never be exported… How would you config having the host machine be prompted by it’s authentication agent whenever trying to launch an app on the container?