Hello, I would like to have a way to access a desktop environment from within the container, probably using VNC or X11. I thought I could use the Ubuntu headless container and install an environment like XFCE on top of it. Is this something you would recommend, or would it be better to use a VM directly?
I only need to be able to send mouse and keyboard inputs programmatically, and take screenshots, and i felt using the container instead of the VM would be more efficient
Welcome!
Thanks for describing your use-case. If you are interested in controlling a browser window with the mouse and keyboard, you may be interested in using Puppeteer, Playwright or something similar.
See How to install and use Puppeteer in an Incus container – Mi blog lah! on how to set this up in an Incus container. The default example in such software is to load some Web page and then take a screenshot. You can also manipulate the mouse and keyboard. While developing your script, you can view visually the browser window and the automation actions. When you have developed the script, then you can run it in headless mode.
If you want to run any graphical window, then you would need to use some UI testing framework for Linux. Those allow to programmatically send inputs to window widgets. There are a few options, they depend on what toolkit you are interested in (Qt, GTK, etc).
I have never had an issue with installing “tasksel” and then choosing a desktop environment to run inside of an Ubuntu container. I have three such containers running in my Home Lab at the moment.