I’m trying to mount parts of my home directory on the underlying host into an Incus container. The container is restricted in a confined project. Therefore, I want to use an idmap to map my uid/gid inside the container to my uid/gid on the underlying host.
Unfortunately, I’m running into errors that I don’t fully understand. Not sure, if I’m doing something wrong, or if it’s a glitch in Incus itself? Here’s the gist (see tl;rd further down for more info):
- Project restriction config, where
40001is the uid/gid of my usermichaelon the host:restricted: "true" restricted.devices.disk: allow restricted.devices.disk.paths: /home/michael restricted.idmap.gid: "40001" restricted.idmap.uid: "40001" - Idmap config of the container in my restricted project:
raw.idmap: both 40001 40001 - Disk config of the container:
devices: host-bind-tofu-aws: path: /mnt/meeque/tofu/aws shift: "false" source: /home/michael/*****/opentofu/aws type: disk - Error message in container logs, when I try to start the container:
The container state remains stopped.lxc user-40001_container-aws-controller 20241008090532.702 ERROR conf - ../src/lxc/conf.c:lxc_map_ids:3704 - newuidmap failed to write mapping "newuidmap: uid range [40001-40002) -> [40001-40002) not allowed": newuidmap 643068 0 1000000 40001 40001 40001 1 40002 1040002 999959998
Any ideas what might be wrong here?
Workarounds that I’ve tried:
-
Omit the
idmap:
The container comes up fine. I use OpenTofu and Ansible to provision the same user (michael, 40001) inside the container. When Ilsthe mounted files, it shows uid/gid asnobody. Obviously this is expected, but it will not allow me to work with the mounted files as desired. -
Use
shiftproperty on the disk device:
When starting the container, I get the following error message:Failed to start device "host-bind-tofu-aws": The `shift` property cannot be used with a restricted source pathSeems to be a feature. Not sure, why it’s necessary?
-
Use
shiftproperty without source path restrictions in the project config:
restricted.devices.disk.paths=''
Theshiftproperty works now. I get the desired behavior inside the container.
However, on the underlying host, this would give my usermichaelthe same privileges asroot(without going throughsudoor similar). E.g. my user could create a new Incus container, bind mount/into it, and manipulate arbitrary host config files (e.g./etc/shadow) from inside the container.
Disclaimer: I’m fairly new to Incus, and I have never used lxc directly.
tl;dr:
Here’s the details, some of it redacted with *****:
Environment:
$ cat /etc/issue
Ubuntu 24.04.1
$ uname -a
Linux d*****t 6.8.0-45-generic #45-Ubuntu SMP PREEMPT_DYNAMIC Fri Aug 30 12:02:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
$ incus --version
6.0.0
Relevant configuration:
$ cat /etc/subuid
michael:165536:65536
common:231072:65536
root:1000000:1000000000
$ cat /etc/subgid
michael:165536:65536
common:231072:65536
root:1000000:1000000000
$ sudo incus admin init --dump
config:
images.auto_update_interval: "0"
networks:
- config:
ipv4.address: 192.168.66.1/24
ipv4.firewall: "false"
ipv4.nat: "true"
ipv6.address: none
description: ""
name: incusbr0
type: bridge
project: default
storage_pools:
- config:
source: /var/lib/incus/storage-pools/meeque-desktop
description: General storage pool for Incus instances on desktop hosts.
name: meeque-desktop
driver: dir
profiles:
- config: {}
description: Default profile
devices:
eth0:
name: eth0
network: incusbr0
type: nic
root:
path: /
pool: meeque-desktop
type: disk
name: default
projects:
- config:
features.images: "true"
features.networks: "true"
features.networks.zones: "true"
features.profiles: "true"
features.storage.buckets: "true"
features.storage.volumes: "true"
description: Default Incus project
name: default
- config:
features.images: "true"
features.networks: "false"
features.networks.zones: "true"
features.profiles: "true"
features.storage.buckets: "true"
features.storage.volumes: "false"
restricted: "true"
restricted.containers.nesting: block
restricted.devices.disk: allow
restricted.devices.disk.paths: /home/michael
restricted.devices.gpu: block
restricted.devices.nic: allow
restricted.idmap.gid: "40001"
restricted.idmap.uid: "40001"
restricted.networks.access: incusbr0
description: User restricted project for "michael" (40001)
name: user-40001
$ incus config show container-aws-controller
architecture: x86_64
config:
boot.autostart: "true"
cloud-init.user-data: |
#!/bin/sh
*****
image.architecture: amd64
image.description: Ubuntu noble amd64 (20241007_07:42)
image.os: Ubuntu
image.release: noble
image.requirements.cgroup: v2
image.serial: "20241007_07:42"
image.type: squashfs
image.variant: cloud
limits.cpu: "1"
limits.memory: 1GiB
raw.idmap: both 40001 40001
volatile.base_image: 40bc***121d
volatile.cloud-init.instance-id: 8048*****e6aa
volatile.eth0.host_name: veth7*****6
volatile.eth0.hwaddr: 00:*****:e8
volatile.eth0.name: eth0
volatile.idmap.base: "0"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":40001},{"Isuid":true,"Isgid":true,"Hostid":40001,"Nsid":40001,"Maprange":1},{"Isuid":true,"Isgid":false,"Hostid":1040002,"Nsid":40002,"Maprange":999959998},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":40001},{"Isuid":false,"Isgid":true,"Hostid":1040002,"Nsid":40002,"Maprange":999959998}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":40001},{"Isuid":true,"Isgid":true,"Hostid":40001,"Nsid":40001,"Maprange":1},{"Isuid":true,"Isgid":false,"Hostid":1040002,"Nsid":40002,"Maprange":999959998},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":40001},{"Isuid":false,"Isgid":true,"Hostid":1040002,"Nsid":40002,"Maprange":999959998}]'
volatile.last_state.idmap: '[]'
volatile.last_state.power: STOPPED
volatile.last_state.ready: "false"
volatile.uuid: a904*****8f143
volatile.uuid.generation: a904*****8f143
devices:
eth0:
ipv4.address: 192.168.66.32
network: incusbr0
type: nic
host-bind-tofu-aws:
path: /mnt/meeque/tofu/aws
shift: "false"
source: /home/michael/*****/opentofu/aws
type: disk
root:
path: /
pool: meeque-desktop
type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""
Commands that I’ve run:
$ incus start container-aws-controller
Error: Failed to run: /usr/libexec/incus/incusd forkstart user-40001_container-aws-controller /var/lib/incus/containers /run/incus/user-40001_container-aws-controller/lxc.conf: exit status 1
Try `incus info --show-log container-aws-controller` for more info
Name: container-aws-controller
Status: STOPPED
Type: container
Architecture: x86_64
Created: 2024/10/08 10:59 CEST
Last Used: 2024/10/08 11:05 CEST
Log:
lxc user-40001_container-aws-controller 20241008090532.702 ERROR conf - ../src/lxc/conf.c:lxc_map_ids:3704 - newuidmap failed to write mapping "newuidmap: uid range [40001-40002) -> [40001-40002) not allowed": newuidmap 643068 0 1000000 40001 40001 40001 1 40002 1040002 999959998
lxc user-40001_container-aws-controller 20241008090532.702 ERROR start - ../src/lxc/start.c:lxc_spawn:1788 - Failed to set up id mapping.
lxc user-40001_container-aws-controller 20241008090532.702 ERROR lxccontainer - ../src/lxc/lxccontainer.c:wait_on_daemonized_start:878 - Received container state "ABORTING" instead of "RUNNING"
lxc user-40001_container-aws-controller 20241008090532.703 ERROR start - ../src/lxc/start.c:__lxc_start:2107 - Failed to spawn container "user-40001_container-aws-controller"
lxc user-40001_container-aws-controller 20241008090532.703 WARN start - ../src/lxc/start.c:lxc_abort:1036 - No such process - Failed to send SIGKILL via pidfd 17 for process 643068
lxc 20241008090532.748 ERROR af_unix - ../src/lxc/af_unix.c:lxc_abstract_unix_recv_fds_iov:218 - Connection reset by peer - Failed to receive response
lxc 20241008090532.748 ERROR commands - ../src/lxc/commands.c:lxc_cmd_rsp_recv_fds:128 - Failed to receive file descriptors for command "get_init_pid"