From LXC to Incus mounting with rbind

IronBox · June 12, 2026, 2:46am

Hi, using Incus 6.0.6

On a host, I have the equivalent of /mnt/StoreA, /mnt/StoreB and /mnt/StoreC, each mounting a file system from different RAID sets. StoreA, the biggest in size is on ZFS the others on MD/RAID5+Ext4. In /etc/fstab I have a few bind mounts like this :

/mnt/StoreA/Overview/Share1 → /mnt/StoreA/DirX
/mnt/StoreA/Overview/Share2 → /mnt/StoreB/DirY
/mnt/StoreA/Overview/Share3 → /mnt/StoreC/DirZ

To be clear, something like this :
/mnt/StoreA/DirX /mnt/StoreA/Overview/Share1 none bind,ro,noatime,nofail 0 0

I have a classic (ie: privileged using lxc-start) LXC container with Samba which shares /mnt/StoreA/Overview across the network and it only takes lxc.mount.entry = /mnt/StoreA/Overview mnt/Store/Overview defaults,rbind,create=dir 0 0 to work properly.

Despite everything I’ve tried, I can not seem to find the right way to convert this to a Incus instance!

devices:
  Overview:
    path: /mnt/StoreA/Overview
    readonly: "true"
    source: /opt/Overview
    type: disk

Allows the instance to start but obviously, Share1, Share2 and Share3 are empty with the lack of rbind.

I’ve tried every possibility with propagation, especially the r* options and even tried raw.mount.options: defaults,rbind,ro but nothing worked.

The instance either starts but all Share* dirs are empty or the instance fails to start with strange errors in lxc.log like for example

lxc samba 20260612021122.346 ERROR    utils - ../src/lxc/utils.c:safe_mount:1220 - Invalid argument - Failed to mount "/var/lib/incus/devices/samba/disk.Overview.opt-Overview" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs/opt/Overview"
lxc samba 20260612021122.346 ERROR    conf - ../src/lxc/conf.c:mount_entry:2437 - Invalid argument - Failed to mount "/var/lib/incus/devices/samba/disk.Overview.opt-Overview" on "/usr/lib/x86_64-linux-gnu/lxc/rootfs/opt/Overview"
lxc samba 20260612021122.346 ERROR    conf - ../src/lxc/conf.c:lxc_setup:4410 - Failed to setup mount entries
lxc samba 20260612021122.346 ERROR    start - ../src/lxc/start.c:do_start:1272 - Failed to setup container "samba"
lxc samba 20260612021122.346 ERROR    sync - ../src/lxc/sync.c:sync_wait:34 - An error occurred in another process (expected sequence number 3)
lxc samba 20260612021122.358 WARN     network - ../src/lxc/network.c:lxc_delete_network_priv:3631 - Failed to rename interface with index 0 from "eth0" to its initial name "veth79d2cc69"
lxc samba 20260612021122.358 ERROR    start - ../src/lxc/start.c:__lxc_start:2107 - Failed to spawn container "samba"
lxc samba 20260612021122.358 WARN     start - ../src/lxc/start.c:lxc_abort:1036 - No such process - Failed to send SIGKILL via pidfd 17 for process 75081
lxc samba 20260612021122.358 ERROR    lxccontainer - ../src/lxc/lxccontainer.c:wait_on_daemonized_start:878 - Received container state "ABORTING" instead of "RUNNING"
lxc 20260612021122.461 ERROR    af_unix - ../src/lxc/af_unix.c:lxc_abstract_unix_recv_fds_iov:218 - Connection reset by peer - Failed to receive response
lxc 20260612021122.461 ERROR    commands - ../src/lxc/commands.c:lxc_cmd_rsp_recv_fds:128 - Failed to receive file descriptors for command "get_init_pid"

what is it trying to do here with /usr/lib/x86_64-linux-gnu/lxc/rootfs/opt/Overview?

Any hint pointing in the right direction would be appreciated

stgraber · June 12, 2026, 1:10pm

Did you try setting recursion=true on your disk device?

IronBox · June 12, 2026, 11:35pm

Sorry for the late reply. Yes I’ve tried but it badly messes with the host with recursion=true. First, I must remove readonly=true as per the error message

Config parsing error: Invalid devices: Device validation failed for "Overview": Recursive read-only bind-mounts aren't currently supported by the kernel
Press enter to open the editor again or ctrl+c to abort change

but no big deal since the bind mounts they refer to are already read-only in fstab.

Second, and that’s where the mess begins, upon start, the “samba” instance creates the following mounts on the host

StoreA/Main on /var/lib/incus/devices/samba/disk.Overview.opt-Overview type zfs (rw,noexec,noatime,xattr,posixacl,casesensitive)
StoreA/Main on /var/lib/incus/devices/samba/disk.Overview.opt-Overview/Share1 type zfs (ro,noatime,xattr,posixacl,casesensitive)
/dev/md0 on /var/lib/incus/devices/samba/disk.Overview.opt-Overview/Share2 type ext4 (ro,noatime,errors=remount-ro)
/dev/md1 on /var/lib/incus/devices/samba/disk.Overview.opt-Overview/Share3 type ext4 (ro,noatime,errors=remount-ro)

which is not expected nor desired but apparently everything seems to work… until I shutdown the instance, either with poweroff within it or with incus stop samba. It correctly stops the instance but I can’t restart it :

$ incus start samba
Error: Failed to start device "Overview": remove /var/lib/incus/devices/samba/disk.Overview.opt-Overview: device or resource busy
Try `incus info --show-log samba` for more info

And indeed, the host shows the mounts I pasted above. I have to manually umount them but by a cascading effect, the original bind mount it refers to also gets unmounted I have to issue a mount -a to recreate them from fstab

So, definitely not a valid solution.

I’m a bit in dismay here, I never thought something as straight forward as these bind mounts on the host would lead to a big problem where I can’t migrate the last 2 bare bones LXC instances to Incus without creating a big mess… I hope there are some unexplored solutions I can try to sort this out.

jochumdev · June 12, 2026, 11:53pm

You could try raw.lxc far from beeing an expert in that matters.

Something like:

$ incus config edit instance

config:
  raw.lxc: |-
    lxc.mount.entry = /mnt/StoreA/Overview mnt/Store/Overview defaults,rbind,create=dir 0 0

IronBox · June 13, 2026, 12:57am

Yeeeeessss! Very creative, I like it! I tried and it works precisely like in plain LXC

Even though it is not documented for lxc.mount.entry options, there’s a lxc.rootfs.options called idmap=X where X can be container “which will instruct LXC to use the container’s user namespace to idmap the rootfs” (sic!) I tried it and bingo! It perfectly worked, the mount point is “uid shifted” so users can write on some shares as they used to do on plain LXC.

So, this is the (Mandalorian) way to have shift=true when using raw.lxc: lxc.mount.entry = ...

Thank you very much, you saved my day!

Incus rocks, LXC rocks, and the help in this forum is very high level !

jochumdev · June 13, 2026, 12:59am

It’s fun to see your fun