Use LXD-Container with primary (and only) GPU?

Hey there,

I read a lot about LXD in the last weeks, but I could not find the answer to one very specific question:

I would like to use several containers an a single-gpu system, one of them being an OS for browsing and mail. Obviously, this system needs to be GUI-based. for security reasons I don’t want to use the host system for this. Even a host-X-server and a container-X-client would not really reduce the security risk afaik. So my Idea is to access the host OS only via SSH and hand over the GPU to the browsing-container, such that it “feels” like being the major OS. IS that possible and how?

And no, I cannot add a secondary GPU, because the system is based on a ARM-SoC


You could potentially use a LXD VM and pass in the GPU from the host.

See Instances | LXD

Thank you for the quick reply.

following your link there are two option, physical passthrough (container or vm) or a virtual gpu (mdev, vm only). Do I assume correctly that vGPU is not very useful for browsers because of lacking hardware acceleration?
Do I understand your post correctly that you suggest vm with physical passthrough? if so, what is the advantage compared to container with physical passthrough? If both is supported I would expect container being the better choice because of performance reasons, isn’t it?

Thanks again.

@stgraber @monstermunchkin any thoughts on this?

X over SSH is no more secure than direct X passthrough. X clients on the remote SSH server can still access every feature of your X server and eavesdrop on all other clients, so that won’t help you.

You can’t really start an X server from within a container so that’s also not really an option.

With a VM, you can’t pass your sole GPU to a VM, this will make your host kernel panic as it will find itself with its VGA output ripped from underneath it. In theory if you can boot in full text mode over a serial link and never have the firmware or OS initialize the GPU, then you could pass it to a VM. But that may be very tricky to do.

One option could be to setup a Wayland display server on the host, then give access to containers so they can talk to it. X applications would then run over a dedicated Xwayland instance inside of the container.

This would then avoid the usual issues around eavesdropping or abuse of X11 protocol as that would effectively be run inside the container which runs the untrusted application.

But while this is certainly doable (that’s in fact what Google does on ChromeOS), it may need quite a bit of work on your part to find or build a suitable Wayland compositor.

1 Like

Thanks again for the quick reply. I did not intend to use X over SSH. My intention was (is) a non-GUI host and X-server and X-client in a container, because of the security issues with X-server on the host OS (which @stgraber mentions as well). Even if Wayland adresses some issues of X11, it is still a complex channel that I would prefer to avoid between container and host.
My original idea was an Ubuntu Server host and Alpine-Linux clients (as I read that they are quite minimal and preconfigured quite securely).

So if I understand you correctly, the options are:
a) configure host to use /dev/ttyS0 (serial port console) instead of “vga”

  • what are “typical” pitfalls that initialise the GPU anyway then? Can’t I hide the device from other processes early at boot time?
  • Can I run and use a ssh server as I am used to to when using ttyS0 (as it is text-based I assume so)?

b) use Wayland-Host

  • Bonus: separate containers can be used, for example one for browser and one for mail
  • need to configure X-Wayland in Every GUI-Container, security better than X11, but still some more risk than isolated container/vm.

c) Use VM and mdev-GPU:

  • rather easy and safe to configure
  • no hardware acceleration in Browser (and elsewhere, e.g. media codecs)

right now I think I’d try c because it probably the easiest to implement, as I know close-to-nothing about configuring wayland and about what may initialise the GPU.

An update on my efforts for interested readers:

a) seems to be impossible for Pi 4, as the GPU is part of the SoC and I was told that during the boot process the GPU is actually initialized first and then the gpu initializes the cpu.

c) mdev seems impossible as well. usinglxc info --resources, only CPUs, memory, NICs and Disks are shown (up-to-date Ubuntu Server 20.4.3)

Regarding option c I read [Howto] Use the Host's Wayland and XWayland Servers inside containers by @toby63

I wonder about

find method to prevent implications by XWayland on host/container-seperation

Shouldn’t this be solvable by running a Wayland server on the host and XWayland-servers on each container/vm/guest that run on the hosts Wayland server? Of course, Apps are not isolated within the same XWayland, but the XWaylands should be isolated towards each other and towards the host, aren’t they?

Update: Actually two differences.
This was also my idea, with two (little) differences:

  1. The XWayland Server is started with a compositor, so I think it only starts alongside another Wayland server (.e.g. weston that runs inside weston (or another Wayland server)).

  2. I don’t think that it’s possible to run Wayland servers inside a container.
    Instead the second Wayland server would need to be run on the host as well.
    But I didn’t test this scenario yet and I am also not sure, whether this really works as intended.
    Doubt comes for example from my experience with the regular scenario, in which my desktop environment on the host automatically tries to use the XWayland server as well.
    Maybe you could reach out to the folks at the XServer community (they also develop XWayland and Wayland).

If I understand you correctly you say that each XWayland server starts its own Wayland-server (compositor), is that what you intended to say?

Do you have a more specific reference? Googeling “XServer community” does not seem to return useful results :smiley:


Actually no.
The XWayland Server is usually started by a compositor.
But honestly said I don’t know whether it can also be started on its own.

My original idea was to start a second wayland server on the host, alongside a (second) XWayland server that is somehow inaccessible by host applications (at least they should not use it by default). And then make this second XWayland server accessible in the container.

See: X.Org