[Snap: LXD 4.11] [solved] Can't get GPU HW acceleration for non-root/sudo/su user within container

LXD
1

[SOLUTION]: add gid: “44” under mygpu section of x11 profile, thanks, Simos!

Dear community, please advise why I encounter following: I can’t get GPU HW acceleration within container for non-root user. If I use sudo - all works.

Host/guest: ubuntu 20.10 groovy arm64 on raspberry pi 4.
Lxd 4.11 from snap
Unprivileged container

card1 is listed in printouts as I’m using hdmi1 (2nd HDMI) on my RPi, 1st HDMI out is not in use (nothing attached to it)

Sudo: glxinfo -B

Vendor: Broadcom (0x14e4)
Device: V3D 4.2 (0xffffffff)
Accelerated: yes
OpenGL renderer string: V3D 4.2
OpenGL ES profile version string: OpenGL ES 3.1 Mesa 20.2.6
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

Regular:

libGL error: failed to create dri screen
libGL error: failed to load driver: vc4
libGL error: failed to open /dev/dri/card1: Permission denied
libGL error: failed to open /dev/dri/card1: Permission denied
libGL error: failed to load driver: vc4
Vendor: Mesa/X.org (0xffffffff)
Device: llvmpipe (LLVM 11.0.0, 128 bits) (0xffffffff)
Accelerated: no
OpenGL vendor string: Mesa/X.org
OpenGL renderer string: llvmpipe (LLVM 11.0.0, 128 bits)
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 20.2.6
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20'

Container’s “id”:

uid=1000(ubuntu) gid=1002(ubuntu) groups=1002(ubuntu),4(adm),20(dialout),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),1000(lxd),1001(netdev)

x11 profile:

config:
  environment.DISPLAY: :0
  user.user-data: |
    #cloud-config
    packages:
      - x11-apps
      - mesa-utils
description: GUI LXD profile
devices:
  X0:
    bind: container
    connect: unix:@/tmp/.X11-unix/X0
    listen: unix:@/tmp/.X11-unix/X0
    security.gid: "1000"
    security.uid: "1000"
    type: proxy
  mygpu:
    type: gpu
name: x11
used_by:

Container was launched via “lxc launch images:ubuntu/20.10/cloud --profile default --profile PulseAudio --profile x11 --profile network-ext game-container”
Container is executed via “lxc exec game-container – sudo --user ubuntu --login”

Container config

architecture: aarch64
config:
  environment.DISPLAY: :0
  image.architecture: arm64
  image.description: Ubuntu groovy arm64 (20210208_07:42)
  image.os: Ubuntu
  image.release: groovy
  image.serial: "20210208_07:42"
  image.type: squashfs
  image.variant: cloud
  user.network-config: |
    version: 1
    config:
      - type: physical
        name: eth0-int
        subnets:
          - type: dhcp
            ipv4: true
      - type: physical
        name: eth1-ext
        subnets:
          - type: dhcp
            ipv4: true
  user.user-data: |
    #cloud-config
    packages:
      - x11-apps
      - mesa-utils
  volatile.base_image: f1dd4b5d02fc7598399716c6c668dab62995a64ac070d51ffade142b40d9c29c
  volatile.eth0-ext.host_name: veth61205fxx
  volatile.eth0-ext.hwaddr: 00:16:3e:xx:xx:xx
  volatile.eth0.host_name: veth8105e5xx
  volatile.eth0.hwaddr: 00:16:3e:xx:xx:xx
  volatile.eth1-ext.host_name: mac9dc2d9xx
  volatile.eth1-ext.hwaddr: 00:16:3e:xx:xx:xx
  volatile.eth1-ext.last_state.created: "false"
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.power: RUNNING
  volatile.uuid: 5373be6d-b834-4336-b577-3e3889xxxxd3
devices:
  PASocket:
    bind: container
    connect: unix:/run/user/1000/pulse/native
    gid: "1002"
    listen: unix:/home/ubuntu/.pulse-native
    mode: "0777"
    security.gid: "1000"
    security.uid: "1000"
    type: proxy
    uid: "1000"
  X0:
    bind: container
    connect: unix:@/tmp/.X11-unix/X0
    listen: unix:@/tmp/.X11-unix/X0
    security.gid: "1000"
    security.uid: "1000"
    type: proxy
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
  eth0-ext:
    name: eth0-int
    nictype: bridged
    parent: lxdbr0
    type: nic
  eth1-ext:
    name: eth1-ext
    nictype: macvlan
    parent: eth0
    type: nic
  mygpu:
    type: gpu
  root:
    path: /
    pool: default
    type: disk
ephemeral: false
profiles:
- default
- PulseAudio
- x11
- network-ext
stateful: false
description: ""
2

I edited your post to use backticks for the code environment. I also changed the category from LXC to LXD.

1 Like
3

I can see /dev/dri/card1. Is there a card0 as well on the RPi?
Is the container a privileged container?

4

Thank you for your response and formatting!

The container is unprivileged (and hopefully it will stay so) - I’ve set it up based on your guides (big thanks for that!).

I guess card0 states for hdmi0 - 1st HDMI out on RPi. I’m currently using hdmi1 - 2nd HDMI out on Raspberry (due to dimensions of microHDMI adapter that collides with USB-C power plug if to use 1st HDMI out).

Just in case, I’ve added output of “lxc config show --expanded game-container” into original post.

I see there is some overlap with gid between host/guest (guest user’s gid is 1002 whereis host’s is 1000, and guest’s gid 1000 belongs to lxd) but that’s rather guessing.

5

Can you locate and check the file permissions for that vc4 driver?

1 Like
6

Thank you very much! Though I guess I should have thought of it myself due to kinda self-explaining log. Indeed video out belonged to root group instead of video (gid 44).
What was missing is gid: 44 under mygpu in x11 profile. Adding it solved the issue and now my glxinfo -B works just fine under non-root user.

As I was setting up my container also by following your articles/guides, I’ll recheck if I’ve missed any step there.

7

Thanks for reaching to the solution. Where I tested, I did not get an issue with permissions. On PC, the file in /dev/dri/ all belong to root.

Which exact device was at issue here? Can you ls -l that device, showing the change?

The documentation on the GPU LXD device is at https://linuxcontainers.org/lxd/docs/master/instances#type-gpu

8

it’s /dev/dri/card* (card0, card1) that belonged to group root within container, though on my host they belong to group video(44). both are crw-rw----

I’ve spotted it and haven’t checked further re vc4 drivers etc. I’ve just updated my x11 profile to include gid 44 and that solved the issue.

9

Thanks, I updated the post, https://blog.simos.info/running-x11-software-in-lxd-containers/

10

Thanks! Really helpful article. The only thing re last update: there is a typo in profile snippet: gid: 4 instead of gid:44.

11

Thanks! Typo has been fixed.