Incus failing to re-add devices from profile

I created a new profile to add in X11 mounts. It works when i initially create the container, applying it with the launch -p profile. however, once i stop the container and start it again, the devices are no longer present. i can’t seem to get the devices back either. it shows my profile as being applied, the profile shows the devices should be there, but the container doesn’t have them.

here’s my incus config (after stopping and starting the container)

architecture: x86_64
config:
  image.architecture: amd64
  image.description: Ubuntu noble amd64 (20250201_07:42)
  image.os: Ubuntu
  image.release: noble
  image.requirements.cgroup: v2
  image.serial: "20250201_07:42"
  image.type: squashfs
  image.variant: default
  volatile.base_image: 10443562fa9d988df158a575aec1aa080c0884a052a63b64d55782f7859a263a
  volatile.cloud-init.instance-id: 57b28140-7ea1-42b0-aca2-868993cd6680
  volatile.eth0.host_name: vethfe4d6525
  volatile.eth0.hwaddr: 00:16:3e:43:90:36
  volatile.eth0.name: eth0
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
  volatile.last_state.ready: "false"
  volatile.uuid: 9f461dc0-226f-4922-981f-2d6fed427462
  volatile.uuid.generation: 615161e9-c33d-4650-960b-2574642fd4c6
devices: {}
ephemeral: false
profiles:
- default
- x11-forward
stateful: false
description: ""

Note the x11 profile:

incus profile show x11-forward 
config:
  environment.DISPLAY: :0
description: Profile to forward host’s X11 socket
devices:
  X0:
    bind: container
    connect: unix:/tmp/.X11-unix/X0
    listen: unix:/tmp/.X11-unix/X0
    security.gid: "1000"
    security.uid: "1000"
    type: proxy
  gpu:
    gid: "44"
    type: gpu
name: x11-forward
used_by:
- /1.0/instances/brave
project: default

So we can see, both the container and profile think the profile is applied to the container. However, the container doesn’t show any of the devices from the profile.

incus profile assign brave default,x11-forward – this doesn’t have any affect. the profile is still “applied” but doesn’t add the devices.

To be clear, it DOES work fine when i use incus launch with the profile, but completely fails to re-assign the devices after a stop/start.

incus version 
Client version: 6.9
Server version: 6.9

Use --expanded for incus config show otherwise it won’t show you the stuff coming from profiles.

Fair enough. here’s the expanded config:

incus config show --expanded brave
architecture: x86_64
config:
  environment.DISPLAY: :0
  image.architecture: amd64
  image.description: Ubuntu noble amd64 (20250201_07:42)
  image.os: Ubuntu
  image.release: noble
  image.requirements.cgroup: v2
  image.serial: "20250201_07:42"
  image.type: squashfs
  image.variant: default
  limits.cpu: "8"
  limits.memory: 4GiB
  security.guestapi: "false"
  volatile.base_image: 10443562fa9d988df158a575aec1aa080c0884a052a63b64d55782f7859a263a
  volatile.cloud-init.instance-id: 57b28140-7ea1-42b0-aca2-868993cd6680
  volatile.eth0.hwaddr: 00:16:3e:43:90:36
  volatile.eth0.name: eth0
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: STOPPED
  volatile.uuid: 9f461dc0-226f-4922-981f-2d6fed427462
  volatile.uuid.generation: 615161e9-c33d-4650-960b-2574642fd4c6
devices:
  X0:
    bind: container
    connect: unix:/tmp/.X11-unix/X0
    listen: unix:/tmp/.X11-unix/X0
    security.gid: "1000"
    security.uid: "1000"
    type: proxy 
  eth0:
    network: incusbr0
    type: nic   
  gpu:
    gid: "44"   
    type: gpu   
  root:
    path: /
    pool: default
    type: disk  
ephemeral: false
profiles:
- default
- x11-forward   
stateful: false 
description: "" 

Here’s the next command where i try to list the mount that should be in there:

incus exec brave -- bash -c "ls -la /tmp/.X11-unix/"
total 34
drwxrwxrwt 2 root root 2 Feb  2 17:47 .
drwxrwxrwt 8 root root 8 Feb  2 17:48 ..

So i guess we can at least confirm that the config for brave believes it should have that mount, but it still does not actually have that mount.

I don’t know why this socket is disappearing, but you can try adding it as an abstract unix socket, with a @ character (socket will not show up in /tmp/.X11-unix/ using ll):

  X0:
    bind: instance
    connect: unix:@/tmp/.X11-unix/X0
    listen: unix:@/tmp/.X11-unix/X0
    security.gid: "1000"
    security.uid: "1000"
    type: proxy

If you don’t want to use profile, you can use this command:

incus config device add <instance_name> <device_name> proxy bind=instance listen=unix:@/tmp/.X11-unix/X0 connect=unix:@/tmp/.X11-unix/X0 security.uid=$(id -u) security.gid=$(id -g)

Another solution:
Add the socket to the /mnt/ folder and make a link to the /tmp/.X11-unix/ every time the container starts, using .profile file of the default user inside container:

devices:
  X0:
    bind: container
    connect: unix:/tmp/.X11-unix/X0
    listen: unix:/mnt/.X0
    mode: "0775"
    uid: "1000"
    gid: "1000"
    security.gid: "1000"
    security.uid: "1000"
    type: proxy

Add this at the end of the .profile file of the default user inside container:

[[ -S "/mnt/.X0" && ! -e "/tmp/.X11-unix/X0" ]] && ln -s "/mnt/.X0" "/tmp/.X11-unix/X0"

ok interesting, if i put the socket in /mnt then it seems to come back on start up. So apparently how incus handles adding in devices from the select incus profile differs based on if it’s the first start of the container (incus launch) or subsequent starts of the contains (stop/start). my assumption is that it would handle those cases the same, but that doesn’t appear to be the case.

This feels like a bug to me. I can use the /mnt as a workaround for now, but honestly if i specify a device mount in the /tmp/.X11-unix directory in my profile, it really should come back when i stop/start it.

EDIT: I can also confirm your first solution works as well. I don’t fully understand the abstract unix socket and why i can’t see it, but it seems to allow me to do X11 as well, without needing to make symbolic links and stuff.

It’s not Incus’ fault. Linux distributions wipe/reset /tmp on boot.
Some literally wipe the content, others mount a tmpfs on /tmp and /run.
In all cases, you really can’t rely on anything you put on top of /tmp or /run to survive.

So if you add something after boot, it works fine because /tmp was already reset.
If you add something before boot, then the init system in the container will wipe it.

Could also just have a validation step after start command that ensures the mounts were successful? Either way, sounds like you don’t consider it a bug. I’d then advocate for a warning at least if a mount is in those folders that the user shouldn’t expect it to be there. Like on start, validate the mount points and print a warning to the user if they use it.

The mount was successful, Incus mounts all of those before init starts, it’s init that then covers them up or unmounts them.

If the actual mount would have failed, the container wouldn’t have started (unless you set optional=true).

We also have no idea what the init system will do in the container.

For example, an Android or busybox container will not touch /run or /tmp, so the mounts are perfectly fine with those.