Docker inside Incus: mkdir 'operation not permitted' on idmapped directory

Incus
1

I’m running into a permission issue when trying to use Docker inside an Incus container(NixOS) with idmapped directory.Host path /mnt/Kiwi/VM/SCALE4stor/DockerConfig/nixos-rl is bind-mounted into the container at the same path.

UID/GID settings:

I’ve mapped UID/GID 5000 from the host to the container using TrueNAS GUI. And the directory is owned by UID/GID 5000, which I’ve also set via the TrueNAS GUI.

Incus host (TrueNAS SCALE) permission setting:

root@KiwiServer:~# ls /mnt/Kiwi/VM/SCALE4stor/DockerConfig/nixos-rl -al
total 37
drwxrwx---  5 rl-man   rl  5 May 12 13:45 .
drwxrwx---  6 rootless rl  6 May 13 15:30 ..
drwxrwx--- 22 rl-man   rl 22 May 13 13:54 DOCKER_DATA
drwxrwx---  4 rl-man   rl  6 May 13 14:06 komodo
drwxrwx---  2 rl-man   rl  2 May 12 13:33 root-data
root@KiwiServer:~# id rl-man 
uid=5000(rl-man) gid=5000(rl) groups=5000(rl)

Inside the NixOS container, I created matching user/group:

users.users.rl-man = {
  isNormalUser = true;
  uid = 5000;
  group = "rl";  
};

users.groups.rl = {
  gid = 5000;
  members = [ "root" ];
};

Container config includes for docker :

root@KiwiServer:~# incus config show nixos-rl 
architecture: x86_64
config:
  boot.autostart: "false"
  image.architecture: amd64
  image.description: Nixos 24.11 amd64 (20250511_03:47)
  image.os: Nixos
  image.release: "24.11"
  image.requirements.secureboot: "false"
  image.serial: "20250511_03:47"
  image.type: squashfs
  image.variant: default
  linux.kernel_modules: br_netfilter
  raw.idmap: |-
    uid 568 568
    uid 529288 529288
    uid 5000 5000
    gid 568 568
    gid 3004 3004
    gid 5000 5000
  raw.lxc: lxc.apparmor.profile=unconfined
  security.nesting: "true"
  security.syscalls.intercept.mknod: "true"
  security.syscalls.intercept.setxattr: "true"
  user.autostart: "true"
#.......
devices:
  dir1:
    path: /mnt/Kiwi/VM/SCALE4stor/DockerConfig
    recursive: "true"
    source: /mnt/Kiwi/VM/SCALE4stor/DockerConfig
    type: disk

Issue:

From the container’s root shell, the directory appears accessible:

[root@nixos-rl:~]# ls /mnt/Kiwi/VM/SCALE4stor/DockerConfig/nixos-rl -al
total 37
drwxrwx---  5 rl-man rl  5 May 12 13:45 .
drwxrwx---  6 nobody rl  6 May 13 15:30 ..
drwxrwx--- 22 rl-man rl 22 May 13 13:54 DOCKER_DATA
drwxrwx---  4 rl-man rl  6 May 13 14:06 komodo
drwxrwx---  2 rl-man rl  2 May 12 13:33 root-data

But, Docker fails to mount or write into the directory:

[root@nixos-rl:~]# docker run --rm -v /mnt/Kiwi/VM/SCALE4stor/DockerConfig/nixos-rl/DOCKER_DATA:/data alpine touch /data/test
docker: Error response from daemon: error while creating mount source path '/mnt/Kiwi/VM/SCALE4stor/DockerConfig/nixos-rl/DOCKER_DATA': mkdir /mnt/Kiwi/VM/SCALE4stor/DockerConfig/nixos-rl: operation not permitted.

Strangely, the directory does exist and is writable from the shell. Trying to mkdir also shows “File exists”:

[root@nixos-rl:~]# mkdir /mnt/Kiwi/VM/SCALE4stor/DockerConfig/nixos-rl
mkdir: cannot create directory ‘/mnt/Kiwi/VM/SCALE4stor/DockerConfig/nixos-rl’: File exists

I also tried different Docker storage drivers like vfs and overlay2, but the issue persists with this kernel message.

[2460416.119237] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
[2460421.678622] overlayfs: failed to set xattr on upper
[2460421.680552] overlayfs: ...falling back to redirect_dir=nofollow.
[2460421.682715] overlayfs: ...falling back to uuid=null.
[2460421.684597] overlayfs: try mounting with 'userxattr' option