Incus 0.x and Ceph RBD map is sometimes busy

Incus
1

Had an issue and tried to respawn the topic : HowTo: Delete Container with CEPH RBD volume giving Device or resource busy

I had this issue : sometime while deleting a container, the container RBD device is busy and so it failed to delete :

Deleting container ec-5aca19ff
Error: Stopping the instance failed: Failed unmounting instance: Failed to run: rbd --id admin --cluster ceph --pool customers-clouds.xxxx unmap container_ec-5aca19ff: exit status 16 (rbd: sysfs write failed
rbd: unmap failed: (16) Device or resource busy)
Error while doing : Deleting container ec-5aca19ff

It happens sometimes often… But I cannot find out the root cause until now :slight_smile: and I used the nsenter fix to unmap the device and do the delete job.

root@ceph06:~# rbd showmapped 
id  pool                              namespace  image                    snap  device   
...
4   customers-clouds.xxxx.xxx             container_ec-5aca19ff    -     /dev/rbd4
...
root@ceph06:~# grep rbd4 /proc/*/mountinfo
/proc/3377307/mountinfo:3897 3721 253:64 / /rootfs/var/lib/incus/storage-pools/default/containers/ec-5aca19ff rw,relatime - ext4 /dev/rbd4 rw,discard,stripe=16
/proc/3377309/mountinfo:3897 3721 253:64 / /rootfs/var/lib/incus/storage-pools/default/containers/ec-5aca19ff rw,relatime - ext4 /dev/rbd4 rw,discard,stripe=16
root@ceph06:~# ps -fauxw | grep 3377307
root     3377307  0.0  0.0   1072     4 ?        Ss   Apr02   0:00  \_ /run/podman-init -- /usr/bin/ceph-osd -n osd.24 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false
root@ceph06:~# ps -fauxw | grep 3377309
167      3377309  1.0  0.4 1921140 1280924 ?     Sl   Apr02  27:57      \_ /usr/bin/ceph-osd -n osd.24 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false
root@ceph06:~# nsenter -t 3377309 -m -- umount /rootfs/var/lib/incus/storage-pools/default/containers/ec-5aca19ff
root@ceph06:~# nsenter -t 3377307 -m -- umount /rootfs/var/lib/incus/storage-pools/default/containers/ec-5aca19ff
umount: /rootfs/var/lib/incus/storage-pools/default/containers/ec-5aca19ff: umount failed: No such file or directory.
root@ceph06:~# rbd unmap /dev/rbd4

What I had noticied is that everytime the PID of the holder is an podman OSD, and … everytime one of the latest added to the cluster (and not concerned by the crushmap, no data of my container are in this OSD)

So I tried to find a systematic reproduction scheme to find out the bug :

I had started a loop who create / start / add interfaces / stop / delete a container and it works for thousands iterations …

If I add an OSD while in loop it failed and the namespace who hold the rbd device is related to a PID of the brand new OSD podman !

This morning I will start another loop who create image / map / format / write something / unmount / umap / delete and see if the OSD add broke the chain.

So maybe more related to Ceph/RBD than Incus.

Let’s start bug hunt.

1 Like
2

Can you show the full cat /proc/self/mountinfo? I’m confused by that whole /rootfs thing.

3

Sure :slight_smile:

22 27 0:20 / /sys rw,nosuid,nodev,noexec,relatime shared:7 - sysfs sysfs rw
23 27 0:21 / /proc rw,nosuid,nodev,noexec,relatime shared:13 - proc proc rw
24 27 0:5 / /dev rw,nosuid,relatime shared:2 - devtmpfs udev rw,size=131976148k,nr_inodes=32994037,mode=755,inode64
25 24 0:22 / /dev/pts rw,nosuid,noexec,relatime shared:3 - devpts devpts rw,gid=5,mode=620,ptmxmode=000
26 27 0:23 / /run rw,nosuid,nodev,noexec,relatime shared:5 - tmpfs tmpfs rw,size=26401016k,mode=755,inode64
27 1 0:24 / / rw,nodev,noatime shared:1 - zfs rpool/ROOT/debian rw,xattr,noacl
28 22 0:6 / /sys/kernel/security rw,nosuid,nodev,noexec,relatime shared:8 - securityfs securityfs rw
29 24 0:25 / /dev/shm rw,nosuid,nodev shared:4 - tmpfs tmpfs rw,inode64
30 26 0:26 / /run/lock rw,nosuid,nodev,noexec,relatime shared:6 - tmpfs tmpfs rw,size=5120k,inode64
31 22 0:27 / /sys/fs/cgroup rw,nosuid,nodev,noexec,relatime shared:9 - cgroup2 cgroup2 rw
32 22 0:28 / /sys/fs/pstore rw,nosuid,nodev,noexec,relatime shared:10 - pstore pstore rw
33 22 0:29 / /sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime shared:11 - efivarfs efivarfs rw
34 22 0:30 / /sys/fs/bpf rw,nosuid,nodev,noexec,relatime shared:12 - bpf bpf rw,mode=700
35 23 0:31 / /proc/sys/fs/binfmt_misc rw,relatime shared:14 - autofs systemd-1 rw,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=30826
36 24 0:19 / /dev/mqueue rw,nosuid,nodev,noexec,relatime shared:15 - mqueue mqueue rw
37 24 0:32 / /dev/hugepages rw,relatime shared:16 - hugetlbfs hugetlbfs rw,pagesize=2M
38 22 0:7 / /sys/kernel/debug rw,nosuid,nodev,noexec,relatime shared:17 - debugfs debugfs rw
39 22 0:12 / /sys/kernel/tracing rw,nosuid,nodev,noexec,relatime shared:18 - tracefs tracefs rw
40 22 0:33 / /sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime shared:19 - fusectl fusectl rw
41 22 0:34 / /sys/kernel/config rw,nosuid,nodev,noexec,relatime shared:20 - configfs configfs rw
64 26 0:35 / /run/credentials/systemd-sysctl.service ro,nosuid,nodev,noexec,relatime shared:21 - ramfs ramfs rw,mode=700
66 26 0:36 / /run/credentials/systemd-sysusers.service ro,nosuid,nodev,noexec,relatime shared:22 - ramfs ramfs rw,mode=700
68 26 0:37 / /run/credentials/systemd-tmpfiles-setup-dev.service ro,nosuid,nodev,noexec,relatime shared:23 - ramfs ramfs rw,mode=700
94 27 8:8 / /boot/efi rw,relatime shared:33 - vfat /dev/sda8 rw,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro
97 27 0:40 / /home rw,nosuid,nodev,noatime shared:49 - zfs rpool/home rw,xattr,noacl
100 27 0:39 / /root rw,nosuid,nodev,noatime shared:51 - zfs rpool/home/root rw,xattr,noacl
103 27 0:44 / /var/tmp rw,nosuid,nodev,noatime shared:53 - zfs rpool/var/tmp rw,xattr,noacl
106 27 0:41 / /var/cache rw,nosuid,nodev,noexec,noatime shared:55 - zfs rpool/var/cache rw,xattr,noacl
109 27 0:42 / /var/spool rw,nosuid,nodev,noexec,noatime shared:57 - zfs rpool/var/spool rw,xattr,noacl
112 27 0:43 / /var/log rw,nosuid,nodev,noexec,noatime shared:59 - zfs rpool/var/log rw,xattr,noacl
147 26 0:45 / /run/credentials/systemd-tmpfiles-setup.service ro,nosuid,nodev,noexec,relatime shared:61 - ramfs ramfs rw,mode=700
181 35 0:46 / /proc/sys/fs/binfmt_misc rw,nosuid,nodev,noexec,relatime shared:63 - binfmt_misc binfmt_misc rw
261 27 0:50 / /var/lib/incus-lxcfs rw,nosuid,nodev,relatime shared:103 - fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
411 26 0:53 / /run/user/0 rw,nosuid,nodev,relatime shared:182 - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,inode64
433 26 0:55 / /run/user/1000 rw,nosuid,nodev,relatime shared:220 - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1000,gid=1000,inode64
444 100 0:56 / /root/zfs/graph/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,relatime shared:226 - zfs rpool/home/root/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,xattr,noacl
452 100 0:63 / /root/zfs/graph/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,relatime shared:229 - zfs rpool/home/root/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,xattr,noacl
461 100 0:70 / /root/zfs/graph/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,relatime shared:232 - zfs rpool/home/root/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,xattr,noacl
470 100 0:73 / /root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26 rw,relatime shared:235 - zfs rpool/home/root/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26 rw,xattr,noacl
479 100 0:76 / /root/zfs/graph/fd06bc512623d9dbd60c207df2a9f3d17729887649de1bdaebd7bfe22f5c6cf5 rw,relatime shared:238 - zfs rpool/home/root/fd06bc512623d9dbd60c207df2a9f3d17729887649de1bdaebd7bfe22f5c6cf5 rw,xattr,noacl
488 27 0:79 / /var/lib/incus/shmounts rw,relatime shared:241 - tmpfs tmpfs rw,size=100k,mode=711,inode64
529 27 0:80 / /var/lib/incus/guestapi rw,relatime shared:247 - tmpfs tmpfs rw,size=100k,mode=755,inode64
691 100 0:81 / /root/zfs/graph/1ae6bd179860c916c9484f526ab0c40889e2e93d38f3415d4894dae97e611477 rw,relatime shared:253 - zfs rpool/home/root/1ae6bd179860c916c9484f526ab0c40889e2e93d38f3415d4894dae97e611477 rw,xattr,noacl
544 100 0:88 / /root/zfs/graph/dd866148764c5ce43a68205c2ca851359c09aaa52291e8f00485c75e1f437550 rw,relatime shared:256 - zfs rpool/home/root/dd866148764c5ce43a68205c2ca851359c09aaa52291e8f00485c75e1f437550 rw,xattr,noacl
882 27 253:0 / /var/lib/incus/storage-pools/default/containers/ec-58ca16d9 rw,relatime shared:265 - ext4 /dev/rbd0 rw,discard,stripe=16
422 27 253:16 / /var/lib/incus/storage-pools/default/containers/ec-5cca1d25 rw,relatime shared:214 - ext4 /dev/rbd1 rw,discard,stripe=16
795 27 253:32 / /var/lib/incus/storage-pools/default/containers/ec-5eca204b rw,relatime shared:259 - ext4 /dev/rbd2 rw,discard,stripe=16
2661 27 253:48 / /var/lib/incus/storage-pools/default/containers/ec-08c998e9 rw,relatime shared:1112 - ext4 /dev/rbd3 rw,discard,stripe=16
3607 26 0:136 / /run/user/1002 rw,nosuid,nodev,relatime shared:1708 - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1002,gid=1003,inode64
3660 27 253:80 / /var/lib/incus/storage-pools/default/containers/iceph-r2b-fl1 rw,relatime shared:1734 - ext4 /dev/rbd5 rw,discard,stripe=16
3597 100 0:135 / /root/zfs/graph/f7d03fcf5c672119324f4b88154ebd3a913120f8efc2d03d88d10562b81f101d rw,relatime shared:789 - zfs rpool/home/root/f7d03fcf5c672119324f4b88154ebd3a913120f8efc2d03d88d10562b81f101d rw,xattr,noacl
3850 100 0:126 / /root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7 rw,relatime shared:2076 - zfs rpool/home/root/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7 rw,xattr,noacl
4

Seems related to rbd/ceph :frowning:

root@ceph06:~# exit=0 ; it=0 ; while [ $exit -eq 0 ] ; do bash -x ./test_rbd_flow.sh ; exit=$? ; it=$(($it+1)) ; echo $it ; done
+ rbd create image1 --size 1024 --pool customers-clouds.ix-mrs2.fr.eho
++ rbd map customers-clouds.ix-mrs2.fr.eho/image1
+ RBD_DEVICE=/dev/rbd0
+ mkfs.ext4 /dev/rbd0
mke2fs 1.47.0 (5-Feb-2023)
Discarding device blocks: done                            
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: 07d17285-717a-4bbb-9d2e-0a094cf2f1b9
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

+ mount /dev/rbd0 /media/test
+ dd if=/dev/zero of=/media/test/test.out
dd: writing to '/media/test/test.out': No space left on device
1960737+0 records in
1960736+0 records out
1003896832 bytes (1.0 GB, 957 MiB) copied, 5.23228 s, 192 MB/s
+ sleep 10
+ rm /media/test/test.out
+ umount /dev/rbd0
+ rbd unmap /dev/rbd0
+ rbd rm customers-clouds.ix-mrs2.fr.eho/image1
Removing image: 100% complete...done.
+ sleep 1
1
+ rbd create image1 --size 1024 --pool customers-clouds.ix-mrs2.fr.eho
++ rbd map customers-clouds.ix-mrs2.fr.eho/image1
+ RBD_DEVICE=/dev/rbd0
+ mkfs.ext4 /dev/rbd0
mke2fs 1.47.0 (5-Feb-2023)
Discarding device blocks: done                            
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: 3e7cce60-7b81-4d7e-a33d-4e540f01a152
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376
<===== INSERT OF A NEW OSD IN BACKGROUND ========>
Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

+ mount /dev/rbd0 /media/test
+ dd if=/dev/zero of=/media/test/test.out
dd: writing to '/media/test/test.out': No space left on device
1960737+0 records in
1960736+0 records out
1003896832 bytes (1.0 GB, 957 MiB) copied, 5.51992 s, 182 MB/s
+ sleep 10
+ rm /media/test/test.out
+ umount /dev/rbd0
+ rbd unmap /dev/rbd0
rbd: sysfs write failed
rbd: unmap failed: (16) Device or resource busy
+ exit 16
2

AND OFF COURSE :slight_smile:

root@ceph01:~# grep rbd0 /proc/*/mountinfo
/proc/1415299/mountinfo:1959 1837 252:0 / /rootfs/media/test rw,relatime - ext4 /dev/rbd0 rw,stripe=16
/proc/1415301/mountinfo:1959 1837 252:0 / /rootfs/media/test rw,relatime - ext4 /dev/rbd0 rw,stripe=16
root@ceph01:~# cat /proc/1415299/cmdline 
/run/podman-init--/usr/bin/ceph-osd-nosd.26-f--setuserceph--setgroupceph--default-log-to-file=false--default-log-to-journald=true--default-log-to-stderr=falseroot@ceph01-b15r56-2r016.ix-mrs2.fr.eho.admin:~# 
root@ceph01:~# cat /proc/1415301/cmdline 
/usr/bin/ceph-osd-nosd.26-f--setuserceph--setgroupceph--default-log-to-file=false--default-log-to-journald=true--default-log-to-stderr=falseroot@ceph01-b15r56-2r016.ix-mrs2.fr.eho.admin:~# 
root@ceph01-b15r56-2r016.ix-mrs2.fr.eho.admin:~#

why OSD26 holds the rbd map ? That’s a good question.

5

Can you look for grep /rootfs/ /proc/*/mountinfo to see what processes those are?

It looks like you have processes on your system which get a copy of the mount table, likely by creating a mount namespace and so find themselves keeping a mounted reference to those rbd volumes. This in turn prevents us from fully unmounting them, to Incus it looks like it’s unmounted but it can’t be unmapped as there’s another process holding a private copy of the mount which we can’t do anything about.

I suspect it’s related to those podman containers on your system, but that’s just a guess.

6 
/proc/1031681/mountinfo:2534 2484 253:32 /rootfs/boot /boot ro,relatime,idmapped shared:1115 master:788 - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/1031681/mountinfo:2641 2484 253:32 /rootfs/tmp/systemd-private-e251b72bf59e42f7bfee0d66046fdf9e-openvpn@openvpn_server.service-wkgn3h/tmp /tmp rw,relatime,idmapped shared:1171 master:788 - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/1031681/mountinfo:2642 2484 253:32 /rootfs/usr /usr ro,relatime,idmapped shared:1184 master:788 - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/1031681/mountinfo:2660 2484 253:32 /rootfs/var/tmp/systemd-private-e251b72bf59e42f7bfee0d66046fdf9e-openvpn@openvpn_server.service-aTcsP0/tmp /var/tmp rw,relatime,idmapped shared:2072 master:788 - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/2222944/mountinfo:1331 1323 253:0 /rootfs/tmp/systemd-private-f058936797d845b3b0a0a77648c4ac36-systemd-resolved.service-8FUBfk/tmp /tmp rw,relatime,idmapped shared:469 master:271 - ext4 /dev/rbd0 rw,discard,stripe=16
/proc/2222944/mountinfo:1332 1323 253:0 /rootfs/var/tmp/systemd-private-f058936797d845b3b0a0a77648c4ac36-systemd-resolved.service-9ttXj3/tmp /var/tmp rw,relatime,idmapped shared:470 master:271 - ext4 /dev/rbd0 rw,discard,stripe=16
/proc/2222954/mountinfo:1578 1528 253:0 /rootfs/etc /etc rw,relatime,idmapped shared:589 master:271 - ext4 /dev/rbd0 rw,discard,stripe=16
/proc/2222954/mountinfo:1592 1528 253:0 /rootfs/tmp/systemd-private-f058936797d845b3b0a0a77648c4ac36-systemd-logind.service-ceLoax/tmp /tmp rw,relatime,idmapped shared:592 master:271 - ext4 /dev/rbd0 rw,discard,stripe=16
/proc/2222954/mountinfo:1593 1528 253:0 /rootfs/var/lib/systemd/linger /var/lib/systemd/linger rw,relatime,idmapped shared:593 master:271 - ext4 /dev/rbd0 rw,discard,stripe=16
/proc/2222954/mountinfo:1594 1528 253:0 /rootfs/var/tmp/systemd-private-f058936797d845b3b0a0a77648c4ac36-systemd-logind.service-qJcKY7/tmp /var/tmp rw,relatime,idmapped shared:594 master:271 - ext4 /dev/rbd0 rw,discard,stripe=16
/proc/2235500/mountinfo:2055 2005 253:16 /rootfs/etc /etc rw,relatime,idmapped shared:854 master:453 - ext4 /dev/rbd1 rw,discard,stripe=16
/proc/2235500/mountinfo:2069 2005 253:16 /rootfs/tmp/systemd-private-bbf598abf9d440edb22117a2802410ba-systemd-logind.service-GSqRu1/tmp /tmp rw,relatime,idmapped shared:857 master:453 - ext4 /dev/rbd1 rw,discard,stripe=16
/proc/2235500/mountinfo:2070 2005 253:16 /rootfs/var/lib/systemd/linger /var/lib/systemd/linger rw,relatime,idmapped shared:858 master:453 - ext4 /dev/rbd1 rw,discard,stripe=16
/proc/2235500/mountinfo:2071 2005 253:16 /rootfs/var/tmp/systemd-private-bbf598abf9d440edb22117a2802410ba-systemd-logind.service-kx5Hx6/tmp /var/tmp rw,relatime,idmapped shared:859 master:453 - ext4 /dev/rbd1 rw,discard,stripe=16
/proc/2241160/mountinfo:1808 1800 253:16 /rootfs/tmp/systemd-private-bbf598abf9d440edb22117a2802410ba-systemd-resolved.service-vJFXbG/tmp /tmp rw,relatime,idmapped shared:734 master:453 - ext4 /dev/rbd1 rw,discard,stripe=16
/proc/2241160/mountinfo:1809 1800 253:16 /rootfs/var/tmp/systemd-private-bbf598abf9d440edb22117a2802410ba-systemd-resolved.service-PpQZIz/tmp /var/tmp rw,relatime,idmapped shared:735 master:453 - ext4 /dev/rbd1 rw,discard,stripe=16
/proc/28245/mountinfo:532 531 0:5 / /rootfs/dev rw,nosuid,relatime - devtmpfs udev rw,size=131976148k,nr_inodes=32994037,mode=755,inode64
/proc/28245/mountinfo:533 532 0:22 / /rootfs/dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
/proc/28245/mountinfo:534 532 0:25 / /rootfs/dev/shm rw,nosuid,nodev - tmpfs tmpfs rw,inode64
/proc/28245/mountinfo:535 532 0:19 / /rootfs/dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
/proc/28245/mountinfo:536 532 0:32 / /rootfs/dev/hugepages rw,relatime - hugetlbfs hugetlbfs rw,pagesize=2M
/proc/28245/mountinfo:537 531 0:23 / /rootfs/run rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=26401016k,mode=755,inode64
/proc/28245/mountinfo:538 537 0:26 / /rootfs/run/lock rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=5120k,inode64
/proc/28245/mountinfo:539 537 0:35 / /rootfs/run/credentials/systemd-sysctl.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/28245/mountinfo:540 537 0:36 / /rootfs/run/credentials/systemd-sysusers.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/28245/mountinfo:541 537 0:37 / /rootfs/run/credentials/systemd-tmpfiles-setup-dev.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/28245/mountinfo:542 537 0:45 / /rootfs/run/credentials/systemd-tmpfiles-setup.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/28245/mountinfo:543 537 0:53 / /rootfs/run/user/0 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,inode64
/proc/28245/mountinfo:545 537 0:55 / /rootfs/run/user/1000 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1000,gid=1000,inode64
/proc/28245/mountinfo:546 531 0:20 / /rootfs/sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
/proc/28245/mountinfo:547 546 0:6 / /rootfs/sys/kernel/security rw,nosuid,nodev,noexec,relatime - securityfs securityfs rw
/proc/28245/mountinfo:548 546 0:27 / /rootfs/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw
/proc/28245/mountinfo:549 546 0:28 / /rootfs/sys/fs/pstore rw,nosuid,nodev,noexec,relatime - pstore pstore rw
/proc/28245/mountinfo:550 546 0:29 / /rootfs/sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime - efivarfs efivarfs rw
/proc/28245/mountinfo:551 546 0:30 / /rootfs/sys/fs/bpf rw,nosuid,nodev,noexec,relatime - bpf bpf rw,mode=700
/proc/28245/mountinfo:552 546 0:7 / /rootfs/sys/kernel/debug rw,nosuid,nodev,noexec,relatime - debugfs debugfs rw
/proc/28245/mountinfo:553 546 0:12 / /rootfs/sys/kernel/tracing rw,nosuid,nodev,noexec,relatime - tracefs tracefs rw
/proc/28245/mountinfo:554 546 0:33 / /rootfs/sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime - fusectl fusectl rw
/proc/28245/mountinfo:555 546 0:34 / /rootfs/sys/kernel/config rw,nosuid,nodev,noexec,relatime - configfs configfs rw
/proc/28245/mountinfo:556 531 0:21 / /rootfs/proc rw,nosuid,nodev,noexec,relatime - proc proc rw
/proc/28245/mountinfo:557 556 0:31 / /rootfs/proc/sys/fs/binfmt_misc rw,relatime - autofs systemd-1 rw,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=30826
/proc/28245/mountinfo:558 557 0:46 / /rootfs/proc/sys/fs/binfmt_misc rw,nosuid,nodev,noexec,relatime - binfmt_misc binfmt_misc rw
/proc/28245/mountinfo:559 531 8:8 / /rootfs/boot/efi rw,relatime - vfat /dev/sda8 rw,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro
/proc/28245/mountinfo:560 531 0:40 / /rootfs/home rw,nosuid,nodev,noatime - zfs rpool/home rw,xattr,noacl
/proc/28245/mountinfo:561 531 0:39 / /rootfs/root rw,nosuid,nodev,noatime - zfs rpool/home/root rw,xattr,noacl
/proc/28245/mountinfo:562 561 0:56 / /rootfs/root/zfs/graph/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,relatime - zfs rpool/home/root/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,xattr,noacl
/proc/28245/mountinfo:563 561 0:63 / /rootfs/root/zfs/graph/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,relatime - zfs rpool/home/root/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,xattr,noacl
/proc/28245/mountinfo:564 563 0:63 / /rootfs/root/zfs/graph/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,relatime - zfs rpool/home/root/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,xattr,noacl
/proc/28245/mountinfo:565 531 0:44 / /rootfs/var/tmp rw,nosuid,nodev,noatime - zfs rpool/var/tmp rw,xattr,noacl
/proc/28245/mountinfo:566 531 0:41 / /rootfs/var/cache rw,nosuid,nodev,noexec,noatime - zfs rpool/var/cache rw,xattr,noacl
/proc/28245/mountinfo:567 531 0:42 / /rootfs/var/spool rw,nosuid,nodev,noexec,noatime - zfs rpool/var/spool rw,xattr,noacl
/proc/28245/mountinfo:568 531 0:43 / /rootfs/var/log rw,nosuid,nodev,noexec,noatime - zfs rpool/var/log rw,xattr,noacl
/proc/28245/mountinfo:569 531 0:50 / /rootfs/var/lib/incus-lxcfs rw,nosuid,nodev,relatime - fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
/proc/28247/mountinfo:532 531 0:5 / /rootfs/dev rw,nosuid,relatime - devtmpfs udev rw,size=131976148k,nr_inodes=32994037,mode=755,inode64
/proc/28247/mountinfo:533 532 0:22 / /rootfs/dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
/proc/28247/mountinfo:534 532 0:25 / /rootfs/dev/shm rw,nosuid,nodev - tmpfs tmpfs rw,inode64
/proc/28247/mountinfo:535 532 0:19 / /rootfs/dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
/proc/28247/mountinfo:536 532 0:32 / /rootfs/dev/hugepages rw,relatime - hugetlbfs hugetlbfs rw,pagesize=2M
/proc/28247/mountinfo:537 531 0:23 / /rootfs/run rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=26401016k,mode=755,inode64
/proc/28247/mountinfo:538 537 0:26 / /rootfs/run/lock rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=5120k,inode64
/proc/28247/mountinfo:539 537 0:35 / /rootfs/run/credentials/systemd-sysctl.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/28247/mountinfo:540 537 0:36 / /rootfs/run/credentials/systemd-sysusers.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/28247/mountinfo:541 537 0:37 / /rootfs/run/credentials/systemd-tmpfiles-setup-dev.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/28247/mountinfo:542 537 0:45 / /rootfs/run/credentials/systemd-tmpfiles-setup.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/28247/mountinfo:543 537 0:53 / /rootfs/run/user/0 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,inode64
/proc/28247/mountinfo:545 537 0:55 / /rootfs/run/user/1000 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1000,gid=1000,inode64
/proc/28247/mountinfo:546 531 0:20 / /rootfs/sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
/proc/28247/mountinfo:547 546 0:6 / /rootfs/sys/kernel/security rw,nosuid,nodev,noexec,relatime - securityfs securityfs rw
/proc/28247/mountinfo:548 546 0:27 / /rootfs/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw
/proc/28247/mountinfo:549 546 0:28 / /rootfs/sys/fs/pstore rw,nosuid,nodev,noexec,relatime - pstore pstore rw
/proc/28247/mountinfo:550 546 0:29 / /rootfs/sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime - efivarfs efivarfs rw
/proc/28247/mountinfo:551 546 0:30 / /rootfs/sys/fs/bpf rw,nosuid,nodev,noexec,relatime - bpf bpf rw,mode=700
/proc/28247/mountinfo:552 546 0:7 / /rootfs/sys/kernel/debug rw,nosuid,nodev,noexec,relatime - debugfs debugfs rw
/proc/28247/mountinfo:553 546 0:12 / /rootfs/sys/kernel/tracing rw,nosuid,nodev,noexec,relatime - tracefs tracefs rw
/proc/28247/mountinfo:554 546 0:33 / /rootfs/sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime - fusectl fusectl rw
/proc/28247/mountinfo:555 546 0:34 / /rootfs/sys/kernel/config rw,nosuid,nodev,noexec,relatime - configfs configfs rw
/proc/28247/mountinfo:556 531 0:21 / /rootfs/proc rw,nosuid,nodev,noexec,relatime - proc proc rw
/proc/28247/mountinfo:557 556 0:31 / /rootfs/proc/sys/fs/binfmt_misc rw,relatime - autofs systemd-1 rw,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=30826
/proc/28247/mountinfo:558 557 0:46 / /rootfs/proc/sys/fs/binfmt_misc rw,nosuid,nodev,noexec,relatime - binfmt_misc binfmt_misc rw
/proc/28247/mountinfo:559 531 8:8 / /rootfs/boot/efi rw,relatime - vfat /dev/sda8 rw,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro
/proc/28247/mountinfo:560 531 0:40 / /rootfs/home rw,nosuid,nodev,noatime - zfs rpool/home rw,xattr,noacl
/proc/28247/mountinfo:561 531 0:39 / /rootfs/root rw,nosuid,nodev,noatime - zfs rpool/home/root rw,xattr,noacl
/proc/28247/mountinfo:562 561 0:56 / /rootfs/root/zfs/graph/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,relatime - zfs rpool/home/root/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,xattr,noacl
/proc/28247/mountinfo:563 561 0:63 / /rootfs/root/zfs/graph/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,relatime - zfs rpool/home/root/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,xattr,noacl
/proc/28247/mountinfo:564 563 0:63 / /rootfs/root/zfs/graph/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,relatime - zfs rpool/home/root/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,xattr,noacl
/proc/28247/mountinfo:565 531 0:44 / /rootfs/var/tmp rw,nosuid,nodev,noatime - zfs rpool/var/tmp rw,xattr,noacl
/proc/28247/mountinfo:566 531 0:41 / /rootfs/var/cache rw,nosuid,nodev,noexec,noatime - zfs rpool/var/cache rw,xattr,noacl
/proc/28247/mountinfo:567 531 0:42 / /rootfs/var/spool rw,nosuid,nodev,noexec,noatime - zfs rpool/var/spool rw,xattr,noacl
/proc/28247/mountinfo:568 531 0:43 / /rootfs/var/log rw,nosuid,nodev,noexec,noatime - zfs rpool/var/log rw,xattr,noacl
/proc/28247/mountinfo:569 531 0:50 / /rootfs/var/lib/incus-lxcfs rw,nosuid,nodev,relatime - fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
/proc/298662/mountinfo:3239 3188 253:48 /rootfs/etc /etc rw,relatime,idmapped shared:1508 master:1190 - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/298662/mountinfo:3256 3188 253:48 /rootfs/tmp/systemd-private-7bd8a0dbde034e05b7d70938d784bebc-systemd-logind.service-jrzY6o/tmp /tmp rw,relatime,idmapped shared:1511 master:1190 - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/298662/mountinfo:3257 3188 253:48 /rootfs/var/lib/systemd/linger /var/lib/systemd/linger rw,relatime,idmapped shared:1512 master:1190 - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/298662/mountinfo:3258 3188 253:48 /rootfs/var/tmp/systemd-private-7bd8a0dbde034e05b7d70938d784bebc-systemd-logind.service-W9QuUU/tmp /var/tmp rw,relatime,idmapped shared:1513 master:1190 - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/304903/mountinfo:2995 2987 253:48 /rootfs/tmp/systemd-private-7bd8a0dbde034e05b7d70938d784bebc-systemd-resolved.service-PeABWK/tmp /tmp rw,relatime,idmapped shared:1388 master:1190 - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/304903/mountinfo:2996 2987 253:48 /rootfs/var/tmp/systemd-private-7bd8a0dbde034e05b7d70938d784bebc-systemd-resolved.service-B5hkGD/tmp /var/tmp rw,relatime,idmapped shared:1389 master:1190 - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/311828/mountinfo:3262 3094 253:48 /rootfs/boot /boot ro,relatime,idmapped shared:1515 master:1190 - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/311828/mountinfo:3268 3094 253:48 /rootfs/tmp/systemd-private-7bd8a0dbde034e05b7d70938d784bebc-openvpn@openvpn_server.service-8NNDGO/tmp /tmp rw,relatime,idmapped shared:1518 master:1190 - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/311828/mountinfo:3269 3094 253:48 /rootfs/usr /usr ro,relatime,idmapped shared:1519 master:1190 - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/311828/mountinfo:3270 3094 253:48 /rootfs/var/tmp/systemd-private-7bd8a0dbde034e05b7d70938d784bebc-openvpn@openvpn_server.service-CTTtyQ/tmp /var/tmp rw,relatime,idmapped shared:1520 master:1190 - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/3526824/mountinfo:2643 2589 253:32 /rootfs/etc /etc rw,relatime,idmapped shared:1178 master:788 - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/3526824/mountinfo:2657 2589 253:32 /rootfs/tmp/systemd-private-e251b72bf59e42f7bfee0d66046fdf9e-systemd-logind.service-M8tlaB/tmp /tmp rw,relatime,idmapped shared:1181 master:788 - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/3526824/mountinfo:2658 2589 253:32 /rootfs/var/lib/systemd/linger /var/lib/systemd/linger rw,relatime,idmapped shared:1182 master:788 - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/3526824/mountinfo:2659 2589 253:32 /rootfs/var/tmp/systemd-private-e251b72bf59e42f7bfee0d66046fdf9e-systemd-logind.service-6mp8e1/tmp /var/tmp rw,relatime,idmapped shared:1183 master:788 - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/3532771/mountinfo:2396 2388 253:32 /rootfs/tmp/systemd-private-e251b72bf59e42f7bfee0d66046fdf9e-systemd-resolved.service-w8BUy5/tmp /tmp rw,relatime,idmapped shared:1058 master:788 - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/3532771/mountinfo:2397 2388 253:32 /rootfs/var/tmp/systemd-private-e251b72bf59e42f7bfee0d66046fdf9e-systemd-resolved.service-39y8jt/tmp /var/tmp rw,relatime,idmapped shared:1059 master:788 - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/434248/mountinfo:4206 4198 253:80 /rootfs/tmp/systemd-private-a13d619a83da4c6cbbd579812b816777-systemd-resolved.service-uqPQ32/tmp /tmp rw,relatime,idmapped shared:2004 master:1740 - ext4 /dev/rbd5 rw,discard,stripe=16
/proc/434248/mountinfo:4207 4198 253:80 /rootfs/var/tmp/systemd-private-a13d619a83da4c6cbbd579812b816777-systemd-resolved.service-xhhNGp/tmp /var/tmp rw,relatime,idmapped shared:2005 master:1740 - ext4 /dev/rbd5 rw,discard,stripe=16
/proc/434255/mountinfo:4343 4293 253:80 /rootfs/etc /etc rw,relatime,idmapped shared:2066 master:1740 - ext4 /dev/rbd5 rw,discard,stripe=16
/proc/434255/mountinfo:4357 4293 253:80 /rootfs/tmp/systemd-private-a13d619a83da4c6cbbd579812b816777-systemd-logind.service-VwK2bM/tmp /tmp rw,relatime,idmapped shared:2069 master:1740 - ext4 /dev/rbd5 rw,discard,stripe=16
/proc/434255/mountinfo:4358 4293 253:80 /rootfs/var/lib/systemd/linger /var/lib/systemd/linger rw,relatime,idmapped shared:2070 master:1740 - ext4 /dev/rbd5 rw,discard,stripe=16
/proc/434255/mountinfo:4359 4293 253:80 /rootfs/var/tmp/systemd-private-a13d619a83da4c6cbbd579812b816777-systemd-logind.service-1VnnnG/tmp /var/tmp rw,relatime,idmapped shared:2071 master:1740 - ext4 /dev/rbd5 rw,discard,stripe=16
/proc/492604/mountinfo:3807 3806 0:5 / /rootfs/dev rw,nosuid,relatime - devtmpfs udev rw,size=131976148k,nr_inodes=32994037,mode=755,inode64
/proc/492604/mountinfo:3808 3807 0:22 / /rootfs/dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
/proc/492604/mountinfo:3809 3807 0:25 / /rootfs/dev/shm rw,nosuid,nodev - tmpfs tmpfs rw,inode64
/proc/492604/mountinfo:3810 3807 0:19 / /rootfs/dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
/proc/492604/mountinfo:3811 3807 0:32 / /rootfs/dev/hugepages rw,relatime - hugetlbfs hugetlbfs rw,pagesize=2M
/proc/492604/mountinfo:3812 3806 0:23 / /rootfs/run rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=26401016k,mode=755,inode64
/proc/492604/mountinfo:3813 3812 0:26 / /rootfs/run/lock rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=5120k,inode64
/proc/492604/mountinfo:3814 3812 0:35 / /rootfs/run/credentials/systemd-sysctl.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/492604/mountinfo:3815 3812 0:36 / /rootfs/run/credentials/systemd-sysusers.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/492604/mountinfo:3816 3812 0:37 / /rootfs/run/credentials/systemd-tmpfiles-setup-dev.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/492604/mountinfo:3817 3812 0:45 / /rootfs/run/credentials/systemd-tmpfiles-setup.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/492604/mountinfo:3818 3812 0:53 / /rootfs/run/user/0 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,inode64
/proc/492604/mountinfo:3819 3812 0:55 / /rootfs/run/user/1000 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1000,gid=1000,inode64
/proc/492604/mountinfo:3820 3812 0:136 / /rootfs/run/user/1002 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1002,gid=1003,inode64
/proc/492604/mountinfo:3821 3806 0:20 / /rootfs/sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
/proc/492604/mountinfo:3822 3821 0:6 / /rootfs/sys/kernel/security rw,nosuid,nodev,noexec,relatime - securityfs securityfs rw
/proc/492604/mountinfo:3823 3821 0:27 / /rootfs/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw
/proc/492604/mountinfo:3824 3821 0:28 / /rootfs/sys/fs/pstore rw,nosuid,nodev,noexec,relatime - pstore pstore rw
/proc/492604/mountinfo:3825 3821 0:29 / /rootfs/sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime - efivarfs efivarfs rw
/proc/492604/mountinfo:3826 3821 0:30 / /rootfs/sys/fs/bpf rw,nosuid,nodev,noexec,relatime - bpf bpf rw,mode=700
/proc/492604/mountinfo:3827 3821 0:7 / /rootfs/sys/kernel/debug rw,nosuid,nodev,noexec,relatime - debugfs debugfs rw
/proc/492604/mountinfo:3828 3821 0:12 / /rootfs/sys/kernel/tracing rw,nosuid,nodev,noexec,relatime - tracefs tracefs rw
/proc/492604/mountinfo:3829 3821 0:33 / /rootfs/sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime - fusectl fusectl rw
/proc/492604/mountinfo:3830 3821 0:34 / /rootfs/sys/kernel/config rw,nosuid,nodev,noexec,relatime - configfs configfs rw
/proc/492604/mountinfo:3831 3806 0:21 / /rootfs/proc rw,nosuid,nodev,noexec,relatime - proc proc rw
/proc/492604/mountinfo:3832 3831 0:31 / /rootfs/proc/sys/fs/binfmt_misc rw,relatime - autofs systemd-1 rw,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=30826
/proc/492604/mountinfo:3833 3832 0:46 / /rootfs/proc/sys/fs/binfmt_misc rw,nosuid,nodev,noexec,relatime - binfmt_misc binfmt_misc rw
/proc/492604/mountinfo:3834 3806 8:8 / /rootfs/boot/efi rw,relatime - vfat /dev/sda8 rw,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro
/proc/492604/mountinfo:3835 3806 0:40 / /rootfs/home rw,nosuid,nodev,noatime - zfs rpool/home rw,xattr,noacl
/proc/492604/mountinfo:3836 3806 0:39 / /rootfs/root rw,nosuid,nodev,noatime - zfs rpool/home/root rw,xattr,noacl
/proc/492604/mountinfo:3837 3836 0:56 / /rootfs/root/zfs/graph/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,relatime - zfs rpool/home/root/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,xattr,noacl
/proc/492604/mountinfo:3838 3836 0:63 / /rootfs/root/zfs/graph/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,relatime - zfs rpool/home/root/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,xattr,noacl
/proc/492604/mountinfo:3839 3836 0:70 / /rootfs/root/zfs/graph/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,relatime - zfs rpool/home/root/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,xattr,noacl
/proc/492604/mountinfo:3840 3836 0:73 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26 rw,relatime - zfs rpool/home/root/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26 rw,xattr,noacl
/proc/492604/mountinfo:3841 3836 0:76 / /rootfs/root/zfs/graph/fd06bc512623d9dbd60c207df2a9f3d17729887649de1bdaebd7bfe22f5c6cf5 rw,relatime - zfs rpool/home/root/fd06bc512623d9dbd60c207df2a9f3d17729887649de1bdaebd7bfe22f5c6cf5 rw,xattr,noacl
/proc/492604/mountinfo:3842 3836 0:81 / /rootfs/root/zfs/graph/1ae6bd179860c916c9484f526ab0c40889e2e93d38f3415d4894dae97e611477 rw,relatime - zfs rpool/home/root/1ae6bd179860c916c9484f526ab0c40889e2e93d38f3415d4894dae97e611477 rw,xattr,noacl
/proc/492604/mountinfo:3843 3836 0:88 / /rootfs/root/zfs/graph/dd866148764c5ce43a68205c2ca851359c09aaa52291e8f00485c75e1f437550 rw,relatime - zfs rpool/home/root/dd866148764c5ce43a68205c2ca851359c09aaa52291e8f00485c75e1f437550 rw,xattr,noacl
/proc/492604/mountinfo:3844 3836 0:135 / /rootfs/root/zfs/graph/f7d03fcf5c672119324f4b88154ebd3a913120f8efc2d03d88d10562b81f101d rw,relatime - zfs rpool/home/root/f7d03fcf5c672119324f4b88154ebd3a913120f8efc2d03d88d10562b81f101d rw,xattr,noacl
/proc/492604/mountinfo:3845 3836 0:126 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7 rw,relatime - zfs rpool/home/root/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7 rw,xattr,noacl
/proc/492604/mountinfo:3846 3845 0:126 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7 rw,relatime - zfs rpool/home/root/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7 rw,xattr,noacl
/proc/492604/mountinfo:3847 3846 0:20 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
/proc/492604/mountinfo:3848 3847 0:6 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/kernel/security rw,nosuid,nodev,noexec,relatime - securityfs securityfs rw
/proc/492604/mountinfo:3849 3847 0:27 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw
/proc/492604/mountinfo:3858 3847 0:28 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/fs/pstore rw,nosuid,nodev,noexec,relatime - pstore pstore rw
/proc/492604/mountinfo:3859 3847 0:29 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime - efivarfs efivarfs rw
/proc/492604/mountinfo:3860 3847 0:30 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/fs/bpf rw,nosuid,nodev,noexec,relatime - bpf bpf rw,mode=700
/proc/492604/mountinfo:3861 3847 0:7 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/kernel/debug rw,nosuid,nodev,noexec,relatime - debugfs debugfs rw
/proc/492604/mountinfo:3862 3847 0:12 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/kernel/tracing rw,nosuid,nodev,noexec,relatime - tracefs tracefs rw
/proc/492604/mountinfo:3863 3847 0:33 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime - fusectl fusectl rw
/proc/492604/mountinfo:3864 3847 0:34 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/kernel/config rw,nosuid,nodev,noexec,relatime - configfs configfs rw
/proc/492604/mountinfo:3865 3806 0:44 / /rootfs/var/tmp rw,nosuid,nodev,noatime - zfs rpool/var/tmp rw,xattr,noacl
/proc/492604/mountinfo:3866 3806 0:41 / /rootfs/var/cache rw,nosuid,nodev,noexec,noatime - zfs rpool/var/cache rw,xattr,noacl
/proc/492604/mountinfo:3867 3806 0:42 / /rootfs/var/spool rw,nosuid,nodev,noexec,noatime - zfs rpool/var/spool rw,xattr,noacl
/proc/492604/mountinfo:3868 3806 0:43 / /rootfs/var/log rw,nosuid,nodev,noexec,noatime - zfs rpool/var/log rw,xattr,noacl
/proc/492604/mountinfo:3869 3806 0:50 / /rootfs/var/lib/incus-lxcfs rw,nosuid,nodev,relatime - fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
/proc/492604/mountinfo:3870 3806 0:79 / /rootfs/var/lib/incus/shmounts rw,relatime - tmpfs tmpfs rw,size=100k,mode=711,inode64
/proc/492604/mountinfo:3871 3806 0:80 / /rootfs/var/lib/incus/guestapi rw,relatime - tmpfs tmpfs rw,size=100k,mode=755,inode64
/proc/492604/mountinfo:3872 3806 253:0 / /rootfs/var/lib/incus/storage-pools/default/containers/ec-58ca16d9 rw,relatime - ext4 /dev/rbd0 rw,discard,stripe=16
/proc/492604/mountinfo:3873 3806 253:16 / /rootfs/var/lib/incus/storage-pools/default/containers/ec-5cca1d25 rw,relatime - ext4 /dev/rbd1 rw,discard,stripe=16
/proc/492604/mountinfo:3874 3806 253:32 / /rootfs/var/lib/incus/storage-pools/default/containers/ec-5eca204b rw,relatime - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/492604/mountinfo:3875 3806 253:48 / /rootfs/var/lib/incus/storage-pools/default/containers/ec-08c998e9 rw,relatime - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/492604/mountinfo:3876 3806 253:80 / /rootfs/var/lib/incus/storage-pools/default/containers/iceph-r2b-fl1 rw,relatime - ext4 /dev/rbd5 rw,discard,stripe=16
/proc/492604/mountinfo:3877 3806 253:64 / /rootfs/var/lib/incus/storage-pools/default/containers/test-startdeletestop rw,relatime - ext4 /dev/rbd4 rw,discard,stripe=16
/proc/492606/mountinfo:3807 3806 0:5 / /rootfs/dev rw,nosuid,relatime - devtmpfs udev rw,size=131976148k,nr_inodes=32994037,mode=755,inode64
/proc/492606/mountinfo:3808 3807 0:22 / /rootfs/dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
/proc/492606/mountinfo:3809 3807 0:25 / /rootfs/dev/shm rw,nosuid,nodev - tmpfs tmpfs rw,inode64
/proc/492606/mountinfo:3810 3807 0:19 / /rootfs/dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
/proc/492606/mountinfo:3811 3807 0:32 / /rootfs/dev/hugepages rw,relatime - hugetlbfs hugetlbfs rw,pagesize=2M
/proc/492606/mountinfo:3812 3806 0:23 / /rootfs/run rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=26401016k,mode=755,inode64
/proc/492606/mountinfo:3813 3812 0:26 / /rootfs/run/lock rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=5120k,inode64
/proc/492606/mountinfo:3814 3812 0:35 / /rootfs/run/credentials/systemd-sysctl.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/492606/mountinfo:3815 3812 0:36 / /rootfs/run/credentials/systemd-sysusers.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/492606/mountinfo:3816 3812 0:37 / /rootfs/run/credentials/systemd-tmpfiles-setup-dev.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/492606/mountinfo:3817 3812 0:45 / /rootfs/run/credentials/systemd-tmpfiles-setup.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/492606/mountinfo:3818 3812 0:53 / /rootfs/run/user/0 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,inode64
/proc/492606/mountinfo:3819 3812 0:55 / /rootfs/run/user/1000 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1000,gid=1000,inode64
/proc/492606/mountinfo:3820 3812 0:136 / /rootfs/run/user/1002 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1002,gid=1003,inode64
/proc/492606/mountinfo:3821 3806 0:20 / /rootfs/sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
/proc/492606/mountinfo:3822 3821 0:6 / /rootfs/sys/kernel/security rw,nosuid,nodev,noexec,relatime - securityfs securityfs rw
/proc/492606/mountinfo:3823 3821 0:27 / /rootfs/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw
/proc/492606/mountinfo:3824 3821 0:28 / /rootfs/sys/fs/pstore rw,nosuid,nodev,noexec,relatime - pstore pstore rw
/proc/492606/mountinfo:3825 3821 0:29 / /rootfs/sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime - efivarfs efivarfs rw
/proc/492606/mountinfo:3826 3821 0:30 / /rootfs/sys/fs/bpf rw,nosuid,nodev,noexec,relatime - bpf bpf rw,mode=700
/proc/492606/mountinfo:3827 3821 0:7 / /rootfs/sys/kernel/debug rw,nosuid,nodev,noexec,relatime - debugfs debugfs rw
/proc/492606/mountinfo:3828 3821 0:12 / /rootfs/sys/kernel/tracing rw,nosuid,nodev,noexec,relatime - tracefs tracefs rw
/proc/492606/mountinfo:3829 3821 0:33 / /rootfs/sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime - fusectl fusectl rw
/proc/492606/mountinfo:3830 3821 0:34 / /rootfs/sys/kernel/config rw,nosuid,nodev,noexec,relatime - configfs configfs rw
/proc/492606/mountinfo:3831 3806 0:21 / /rootfs/proc rw,nosuid,nodev,noexec,relatime - proc proc rw
/proc/492606/mountinfo:3832 3831 0:31 / /rootfs/proc/sys/fs/binfmt_misc rw,relatime - autofs systemd-1 rw,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=30826
/proc/492606/mountinfo:3833 3832 0:46 / /rootfs/proc/sys/fs/binfmt_misc rw,nosuid,nodev,noexec,relatime - binfmt_misc binfmt_misc rw
/proc/492606/mountinfo:3834 3806 8:8 / /rootfs/boot/efi rw,relatime - vfat /dev/sda8 rw,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro
/proc/492606/mountinfo:3835 3806 0:40 / /rootfs/home rw,nosuid,nodev,noatime - zfs rpool/home rw,xattr,noacl
/proc/492606/mountinfo:3836 3806 0:39 / /rootfs/root rw,nosuid,nodev,noatime - zfs rpool/home/root rw,xattr,noacl
/proc/492606/mountinfo:3837 3836 0:56 / /rootfs/root/zfs/graph/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,relatime - zfs rpool/home/root/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,xattr,noacl
/proc/492606/mountinfo:3838 3836 0:63 / /rootfs/root/zfs/graph/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,relatime - zfs rpool/home/root/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,xattr,noacl
/proc/492606/mountinfo:3839 3836 0:70 / /rootfs/root/zfs/graph/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,relatime - zfs rpool/home/root/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,xattr,noacl
/proc/492606/mountinfo:3840 3836 0:73 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26 rw,relatime - zfs rpool/home/root/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26 rw,xattr,noacl
/proc/492606/mountinfo:3841 3836 0:76 / /rootfs/root/zfs/graph/fd06bc512623d9dbd60c207df2a9f3d17729887649de1bdaebd7bfe22f5c6cf5 rw,relatime - zfs rpool/home/root/fd06bc512623d9dbd60c207df2a9f3d17729887649de1bdaebd7bfe22f5c6cf5 rw,xattr,noacl
/proc/492606/mountinfo:3842 3836 0:81 / /rootfs/root/zfs/graph/1ae6bd179860c916c9484f526ab0c40889e2e93d38f3415d4894dae97e611477 rw,relatime - zfs rpool/home/root/1ae6bd179860c916c9484f526ab0c40889e2e93d38f3415d4894dae97e611477 rw,xattr,noacl
/proc/492606/mountinfo:3843 3836 0:88 / /rootfs/root/zfs/graph/dd866148764c5ce43a68205c2ca851359c09aaa52291e8f00485c75e1f437550 rw,relatime - zfs rpool/home/root/dd866148764c5ce43a68205c2ca851359c09aaa52291e8f00485c75e1f437550 rw,xattr,noacl
/proc/492606/mountinfo:3844 3836 0:135 / /rootfs/root/zfs/graph/f7d03fcf5c672119324f4b88154ebd3a913120f8efc2d03d88d10562b81f101d rw,relatime - zfs rpool/home/root/f7d03fcf5c672119324f4b88154ebd3a913120f8efc2d03d88d10562b81f101d rw,xattr,noacl
/proc/492606/mountinfo:3845 3836 0:126 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7 rw,relatime - zfs rpool/home/root/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7 rw,xattr,noacl
/proc/492606/mountinfo:3846 3845 0:126 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7 rw,relatime - zfs rpool/home/root/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7 rw,xattr,noacl
/proc/492606/mountinfo:3847 3846 0:20 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
/proc/492606/mountinfo:3848 3847 0:6 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/kernel/security rw,nosuid,nodev,noexec,relatime - securityfs securityfs rw
/proc/492606/mountinfo:3849 3847 0:27 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw
/proc/492606/mountinfo:3858 3847 0:28 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/fs/pstore rw,nosuid,nodev,noexec,relatime - pstore pstore rw
/proc/492606/mountinfo:3859 3847 0:29 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime - efivarfs efivarfs rw
/proc/492606/mountinfo:3860 3847 0:30 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/fs/bpf rw,nosuid,nodev,noexec,relatime - bpf bpf rw,mode=700
/proc/492606/mountinfo:3861 3847 0:7 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/kernel/debug rw,nosuid,nodev,noexec,relatime - debugfs debugfs rw
/proc/492606/mountinfo:3862 3847 0:12 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/kernel/tracing rw,nosuid,nodev,noexec,relatime - tracefs tracefs rw
/proc/492606/mountinfo:3863 3847 0:33 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime - fusectl fusectl rw
/proc/492606/mountinfo:3864 3847 0:34 / /rootfs/root/zfs/graph/4fda23b47befabff67ab0d22e226aa0e404f69d17570866446c9c773133c96c7/sys/kernel/config rw,nosuid,nodev,noexec,relatime - configfs configfs rw
/proc/492606/mountinfo:3865 3806 0:44 / /rootfs/var/tmp rw,nosuid,nodev,noatime - zfs rpool/var/tmp rw,xattr,noacl
/proc/492606/mountinfo:3866 3806 0:41 / /rootfs/var/cache rw,nosuid,nodev,noexec,noatime - zfs rpool/var/cache rw,xattr,noacl
/proc/492606/mountinfo:3867 3806 0:42 / /rootfs/var/spool rw,nosuid,nodev,noexec,noatime - zfs rpool/var/spool rw,xattr,noacl
/proc/492606/mountinfo:3868 3806 0:43 / /rootfs/var/log rw,nosuid,nodev,noexec,noatime - zfs rpool/var/log rw,xattr,noacl
/proc/492606/mountinfo:3869 3806 0:50 / /rootfs/var/lib/incus-lxcfs rw,nosuid,nodev,relatime - fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
/proc/492606/mountinfo:3870 3806 0:79 / /rootfs/var/lib/incus/shmounts rw,relatime - tmpfs tmpfs rw,size=100k,mode=711,inode64
/proc/492606/mountinfo:3871 3806 0:80 / /rootfs/var/lib/incus/guestapi rw,relatime - tmpfs tmpfs rw,size=100k,mode=755,inode64
/proc/492606/mountinfo:3872 3806 253:0 / /rootfs/var/lib/incus/storage-pools/default/containers/ec-58ca16d9 rw,relatime - ext4 /dev/rbd0 rw,discard,stripe=16
/proc/492606/mountinfo:3873 3806 253:16 / /rootfs/var/lib/incus/storage-pools/default/containers/ec-5cca1d25 rw,relatime - ext4 /dev/rbd1 rw,discard,stripe=16
/proc/492606/mountinfo:3874 3806 253:32 / /rootfs/var/lib/incus/storage-pools/default/containers/ec-5eca204b rw,relatime - ext4 /dev/rbd2 rw,discard,stripe=16
/proc/492606/mountinfo:3875 3806 253:48 / /rootfs/var/lib/incus/storage-pools/default/containers/ec-08c998e9 rw,relatime - ext4 /dev/rbd3 rw,discard,stripe=16
/proc/492606/mountinfo:3876 3806 253:80 / /rootfs/var/lib/incus/storage-pools/default/containers/iceph-r2b-fl1 rw,relatime - ext4 /dev/rbd5 rw,discard,stripe=16
/proc/492606/mountinfo:3877 3806 253:64 / /rootfs/var/lib/incus/storage-pools/default/containers/test-startdeletestop rw,relatime - ext4 /dev/rbd4 rw,discard,stripe=16
/proc/51436/mountinfo:619 618 0:5 / /rootfs/dev rw,nosuid,relatime - devtmpfs udev rw,size=131976148k,nr_inodes=32994037,mode=755,inode64
/proc/51436/mountinfo:620 619 0:22 / /rootfs/dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
/proc/51436/mountinfo:621 619 0:25 / /rootfs/dev/shm rw,nosuid,nodev - tmpfs tmpfs rw,inode64
/proc/51436/mountinfo:622 619 0:19 / /rootfs/dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
/proc/51436/mountinfo:623 619 0:32 / /rootfs/dev/hugepages rw,relatime - hugetlbfs hugetlbfs rw,pagesize=2M
/proc/51436/mountinfo:624 618 0:23 / /rootfs/run rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=26401016k,mode=755,inode64
/proc/51436/mountinfo:625 624 0:26 / /rootfs/run/lock rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=5120k,inode64
/proc/51436/mountinfo:626 624 0:35 / /rootfs/run/credentials/systemd-sysctl.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/51436/mountinfo:627 624 0:36 / /rootfs/run/credentials/systemd-sysusers.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/51436/mountinfo:628 624 0:37 / /rootfs/run/credentials/systemd-tmpfiles-setup-dev.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/51436/mountinfo:629 624 0:45 / /rootfs/run/credentials/systemd-tmpfiles-setup.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/51436/mountinfo:630 624 0:53 / /rootfs/run/user/0 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,inode64
/proc/51436/mountinfo:632 624 0:55 / /rootfs/run/user/1000 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1000,gid=1000,inode64
/proc/51436/mountinfo:633 618 0:20 / /rootfs/sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
/proc/51436/mountinfo:634 633 0:6 / /rootfs/sys/kernel/security rw,nosuid,nodev,noexec,relatime - securityfs securityfs rw
/proc/51436/mountinfo:635 633 0:27 / /rootfs/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw
/proc/51436/mountinfo:636 633 0:28 / /rootfs/sys/fs/pstore rw,nosuid,nodev,noexec,relatime - pstore pstore rw
/proc/51436/mountinfo:637 633 0:29 / /rootfs/sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime - efivarfs efivarfs rw
/proc/51436/mountinfo:638 633 0:30 / /rootfs/sys/fs/bpf rw,nosuid,nodev,noexec,relatime - bpf bpf rw,mode=700
/proc/51436/mountinfo:639 633 0:7 / /rootfs/sys/kernel/debug rw,nosuid,nodev,noexec,relatime - debugfs debugfs rw
/proc/51436/mountinfo:640 633 0:12 / /rootfs/sys/kernel/tracing rw,nosuid,nodev,noexec,relatime - tracefs tracefs rw
/proc/51436/mountinfo:641 633 0:33 / /rootfs/sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime - fusectl fusectl rw
/proc/51436/mountinfo:642 633 0:34 / /rootfs/sys/kernel/config rw,nosuid,nodev,noexec,relatime - configfs configfs rw
/proc/51436/mountinfo:643 618 0:21 / /rootfs/proc rw,nosuid,nodev,noexec,relatime - proc proc rw
/proc/51436/mountinfo:644 643 0:31 / /rootfs/proc/sys/fs/binfmt_misc rw,relatime - autofs systemd-1 rw,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=30826
/proc/51436/mountinfo:645 644 0:46 / /rootfs/proc/sys/fs/binfmt_misc rw,nosuid,nodev,noexec,relatime - binfmt_misc binfmt_misc rw
/proc/51436/mountinfo:646 618 8:8 / /rootfs/boot/efi rw,relatime - vfat /dev/sda8 rw,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro
/proc/51436/mountinfo:647 618 0:40 / /rootfs/home rw,nosuid,nodev,noatime - zfs rpool/home rw,xattr,noacl
/proc/51436/mountinfo:648 618 0:39 / /rootfs/root rw,nosuid,nodev,noatime - zfs rpool/home/root rw,xattr,noacl
/proc/51436/mountinfo:649 648 0:56 / /rootfs/root/zfs/graph/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,relatime - zfs rpool/home/root/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,xattr,noacl
/proc/51436/mountinfo:650 648 0:63 / /rootfs/root/zfs/graph/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,relatime - zfs rpool/home/root/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,xattr,noacl
/proc/51436/mountinfo:651 648 0:70 / /rootfs/root/zfs/graph/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,relatime - zfs rpool/home/root/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,xattr,noacl
/proc/51436/mountinfo:652 651 0:70 / /rootfs/root/zfs/graph/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,relatime - zfs rpool/home/root/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,xattr,noacl
/proc/51436/mountinfo:653 618 0:44 / /rootfs/var/tmp rw,nosuid,nodev,noatime - zfs rpool/var/tmp rw,xattr,noacl
/proc/51436/mountinfo:654 618 0:41 / /rootfs/var/cache rw,nosuid,nodev,noexec,noatime - zfs rpool/var/cache rw,xattr,noacl
/proc/51436/mountinfo:655 618 0:42 / /rootfs/var/spool rw,nosuid,nodev,noexec,noatime - zfs rpool/var/spool rw,xattr,noacl
/proc/51436/mountinfo:656 618 0:43 / /rootfs/var/log rw,nosuid,nodev,noexec,noatime - zfs rpool/var/log rw,xattr,noacl
/proc/51436/mountinfo:657 618 0:50 / /rootfs/var/lib/incus-lxcfs rw,nosuid,nodev,relatime - fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
/proc/51438/mountinfo:619 618 0:5 / /rootfs/dev rw,nosuid,relatime - devtmpfs udev rw,size=131976148k,nr_inodes=32994037,mode=755,inode64
/proc/51438/mountinfo:620 619 0:22 / /rootfs/dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
/proc/51438/mountinfo:621 619 0:25 / /rootfs/dev/shm rw,nosuid,nodev - tmpfs tmpfs rw,inode64
/proc/51438/mountinfo:622 619 0:19 / /rootfs/dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
/proc/51438/mountinfo:623 619 0:32 / /rootfs/dev/hugepages rw,relatime - hugetlbfs hugetlbfs rw,pagesize=2M
/proc/51438/mountinfo:624 618 0:23 / /rootfs/run rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=26401016k,mode=755,inode64
/proc/51438/mountinfo:625 624 0:26 / /rootfs/run/lock rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=5120k,inode64
/proc/51438/mountinfo:626 624 0:35 / /rootfs/run/credentials/systemd-sysctl.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/51438/mountinfo:627 624 0:36 / /rootfs/run/credentials/systemd-sysusers.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/51438/mountinfo:628 624 0:37 / /rootfs/run/credentials/systemd-tmpfiles-setup-dev.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/51438/mountinfo:629 624 0:45 / /rootfs/run/credentials/systemd-tmpfiles-setup.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/51438/mountinfo:630 624 0:53 / /rootfs/run/user/0 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,inode64
/proc/51438/mountinfo:632 624 0:55 / /rootfs/run/user/1000 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1000,gid=1000,inode64
/proc/51438/mountinfo:633 618 0:20 / /rootfs/sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
/proc/51438/mountinfo:634 633 0:6 / /rootfs/sys/kernel/security rw,nosuid,nodev,noexec,relatime - securityfs securityfs rw
/proc/51438/mountinfo:635 633 0:27 / /rootfs/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw
/proc/51438/mountinfo:636 633 0:28 / /rootfs/sys/fs/pstore rw,nosuid,nodev,noexec,relatime - pstore pstore rw
/proc/51438/mountinfo:637 633 0:29 / /rootfs/sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime - efivarfs efivarfs rw
/proc/51438/mountinfo:638 633 0:30 / /rootfs/sys/fs/bpf rw,nosuid,nodev,noexec,relatime - bpf bpf rw,mode=700
/proc/51438/mountinfo:639 633 0:7 / /rootfs/sys/kernel/debug rw,nosuid,nodev,noexec,relatime - debugfs debugfs rw
/proc/51438/mountinfo:640 633 0:12 / /rootfs/sys/kernel/tracing rw,nosuid,nodev,noexec,relatime - tracefs tracefs rw
/proc/51438/mountinfo:641 633 0:33 / /rootfs/sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime - fusectl fusectl rw
/proc/51438/mountinfo:642 633 0:34 / /rootfs/sys/kernel/config rw,nosuid,nodev,noexec,relatime - configfs configfs rw
/proc/51438/mountinfo:643 618 0:21 / /rootfs/proc rw,nosuid,nodev,noexec,relatime - proc proc rw
/proc/51438/mountinfo:644 643 0:31 / /rootfs/proc/sys/fs/binfmt_misc rw,relatime - autofs systemd-1 rw,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=30826
/proc/51438/mountinfo:645 644 0:46 / /rootfs/proc/sys/fs/binfmt_misc rw,nosuid,nodev,noexec,relatime - binfmt_misc binfmt_misc rw
/proc/51438/mountinfo:646 618 8:8 / /rootfs/boot/efi rw,relatime - vfat /dev/sda8 rw,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro
/proc/51438/mountinfo:647 618 0:40 / /rootfs/home rw,nosuid,nodev,noatime - zfs rpool/home rw,xattr,noacl
/proc/51438/mountinfo:648 618 0:39 / /rootfs/root rw,nosuid,nodev,noatime - zfs rpool/home/root rw,xattr,noacl
/proc/51438/mountinfo:649 648 0:56 / /rootfs/root/zfs/graph/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,relatime - zfs rpool/home/root/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,xattr,noacl
/proc/51438/mountinfo:650 648 0:63 / /rootfs/root/zfs/graph/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,relatime - zfs rpool/home/root/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,xattr,noacl
/proc/51438/mountinfo:651 648 0:70 / /rootfs/root/zfs/graph/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,relatime - zfs rpool/home/root/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,xattr,noacl
/proc/51438/mountinfo:652 651 0:70 / /rootfs/root/zfs/graph/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,relatime - zfs rpool/home/root/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,xattr,noacl
/proc/51438/mountinfo:653 618 0:44 / /rootfs/var/tmp rw,nosuid,nodev,noatime - zfs rpool/var/tmp rw,xattr,noacl
/proc/51438/mountinfo:654 618 0:41 / /rootfs/var/cache rw,nosuid,nodev,noexec,noatime - zfs rpool/var/cache rw,xattr,noacl
/proc/51438/mountinfo:655 618 0:42 / /rootfs/var/spool rw,nosuid,nodev,noexec,noatime - zfs rpool/var/spool rw,xattr,noacl
/proc/51438/mountinfo:656 618 0:43 / /rootfs/var/log rw,nosuid,nodev,noexec,noatime - zfs rpool/var/log rw,xattr,noacl
/proc/51438/mountinfo:657 618 0:50 / /rootfs/var/lib/incus-lxcfs rw,nosuid,nodev,relatime - fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
/proc/53390/mountinfo:718 717 0:5 / /rootfs/dev rw,nosuid,relatime - devtmpfs udev rw,size=131976148k,nr_inodes=32994037,mode=755,inode64
/proc/53390/mountinfo:719 718 0:22 / /rootfs/dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
/proc/53390/mountinfo:720 718 0:25 / /rootfs/dev/shm rw,nosuid,nodev - tmpfs tmpfs rw,inode64
/proc/53390/mountinfo:721 718 0:19 / /rootfs/dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
/proc/53390/mountinfo:722 718 0:32 / /rootfs/dev/hugepages rw,relatime - hugetlbfs hugetlbfs rw,pagesize=2M
/proc/53390/mountinfo:723 717 0:23 / /rootfs/run rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=26401016k,mode=755,inode64
/proc/53390/mountinfo:724 723 0:26 / /rootfs/run/lock rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=5120k,inode64
/proc/53390/mountinfo:725 723 0:35 / /rootfs/run/credentials/systemd-sysctl.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/53390/mountinfo:726 723 0:36 / /rootfs/run/credentials/systemd-sysusers.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/53390/mountinfo:727 723 0:37 / /rootfs/run/credentials/systemd-tmpfiles-setup-dev.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/53390/mountinfo:728 723 0:45 / /rootfs/run/credentials/systemd-tmpfiles-setup.service ro,nosuid,nodev,noexec,relatime - ramfs ramfs rw,mode=700
/proc/53390/mountinfo:729 723 0:53 / /rootfs/run/user/0 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,inode64
/proc/53390/mountinfo:731 723 0:55 / /rootfs/run/user/1000 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=26401012k,nr_inodes=6600253,mode=700,uid=1000,gid=1000,inode64
/proc/53390/mountinfo:732 717 0:20 / /rootfs/sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
/proc/53390/mountinfo:733 732 0:6 / /rootfs/sys/kernel/security rw,nosuid,nodev,noexec,relatime - securityfs securityfs rw
/proc/53390/mountinfo:734 732 0:27 / /rootfs/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw
/proc/53390/mountinfo:735 732 0:28 / /rootfs/sys/fs/pstore rw,nosuid,nodev,noexec,relatime - pstore pstore rw
/proc/53390/mountinfo:736 732 0:29 / /rootfs/sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime - efivarfs efivarfs rw
/proc/53390/mountinfo:737 732 0:30 / /rootfs/sys/fs/bpf rw,nosuid,nodev,noexec,relatime - bpf bpf rw,mode=700
/proc/53390/mountinfo:738 732 0:7 / /rootfs/sys/kernel/debug rw,nosuid,nodev,noexec,relatime - debugfs debugfs rw
/proc/53390/mountinfo:739 732 0:12 / /rootfs/sys/kernel/tracing rw,nosuid,nodev,noexec,relatime - tracefs tracefs rw
/proc/53390/mountinfo:740 732 0:33 / /rootfs/sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime - fusectl fusectl rw
/proc/53390/mountinfo:741 732 0:34 / /rootfs/sys/kernel/config rw,nosuid,nodev,noexec,relatime - configfs configfs rw
/proc/53390/mountinfo:742 717 0:21 / /rootfs/proc rw,nosuid,nodev,noexec,relatime - proc proc rw
/proc/53390/mountinfo:743 742 0:31 / /rootfs/proc/sys/fs/binfmt_misc rw,relatime - autofs systemd-1 rw,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=30826
/proc/53390/mountinfo:744 743 0:46 / /rootfs/proc/sys/fs/binfmt_misc rw,nosuid,nodev,noexec,relatime - binfmt_misc binfmt_misc rw
/proc/53390/mountinfo:745 717 8:8 / /rootfs/boot/efi rw,relatime - vfat /dev/sda8 rw,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro
/proc/53390/mountinfo:746 717 0:40 / /rootfs/home rw,nosuid,nodev,noatime - zfs rpool/home rw,xattr,noacl
/proc/53390/mountinfo:747 717 0:39 / /rootfs/root rw,nosuid,nodev,noatime - zfs rpool/home/root rw,xattr,noacl
/proc/53390/mountinfo:748 747 0:56 / /rootfs/root/zfs/graph/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,relatime - zfs rpool/home/root/4c5e10b4ee1509790c2fe4fbcc9a5ecd1507e09fbdf8b6c6aeec304eef489f81 rw,xattr,noacl
/proc/53390/mountinfo:749 747 0:63 / /rootfs/root/zfs/graph/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,relatime - zfs rpool/home/root/f10d2789ab3ba1c4479673c8438d78330267a9ab023f6b0a202173329592f8b4 rw,xattr,noacl
/proc/53390/mountinfo:750 747 0:70 / /rootfs/root/zfs/graph/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,relatime - zfs rpool/home/root/da1c7d356be59e818b9e4662fd916f503b0608b0231780ecd107b087cb9ea953 rw,xattr,noacl
/proc/53390/mountinfo:751 747 0:73 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26 rw,relatime - zfs rpool/home/root/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26 rw,xattr,noacl
/proc/53390/mountinfo:752 751 0:73 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26 rw,relatime - zfs rpool/home/root/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26 rw,xattr,noacl
/proc/53390/mountinfo:753 752 0:20 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
/proc/53390/mountinfo:754 753 0:6 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/sys/kernel/security rw,nosuid,nodev,noexec,relatime - securityfs securityfs rw
/proc/53390/mountinfo:755 753 0:27 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw
/proc/53390/mountinfo:756 753 0:28 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/sys/fs/pstore rw,nosuid,nodev,noexec,relatime - pstore pstore rw
/proc/53390/mountinfo:757 753 0:29 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/sys/firmware/efi/efivars rw,nosuid,nodev,noexec,relatime - efivarfs efivarfs rw
/proc/53390/mountinfo:758 753 0:30 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/sys/fs/bpf rw,nosuid,nodev,noexec,relatime - bpf bpf rw,mode=700
/proc/53390/mountinfo:759 753 0:7 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/sys/kernel/debug rw,nosuid,nodev,noexec,relatime - debugfs debugfs rw
/proc/53390/mountinfo:760 753 0:12 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/sys/kernel/tracing rw,nosuid,nodev,noexec,relatime - tracefs tracefs rw
/proc/53390/mountinfo:761 753 0:33 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime - fusectl fusectl rw
/proc/53390/mountinfo:762 753 0:34 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/sys/kernel/config rw,nosuid,nodev,noexec,relatime - configfs configfs rw
/proc/53390/mountinfo:763 752 0:5 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/dev rw,nosuid,relatime - devtmpfs udev rw,size=131976148k,nr_inodes=32994037,mode=755,inode64
/proc/53390/mountinfo:764 763 0:22 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
/proc/53390/mountinfo:765 763 0:25 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/dev/shm rw,nosuid,nodev - tmpfs tmpfs rw,inode64
/proc/53390/mountinfo:766 763 0:19 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
/proc/53390/mountinfo:767 763 0:32 / /rootfs/root/zfs/graph/1d7cf7b50b0ed09c2288c9b2d9f0aa61b9351aff606410ffafa8fed85154dd26/dev/hugepages rw,relatime - hugetlbfs hugetlbfs rw,pagesize=2M
/proc/53390/mountinfo:768 717 0:44 / /rootfs/var/tmp rw,nosuid,nodev,noatime - zfs rpool/var/tmp rw,xattr,noacl
... And more lines ... do yo need them ?
7

My last test seems to demontrate this : I get the busy on unmap on an podman OSD without incus .

8

The problem is that podman images are the way that ceph is distributed now :frowning:

9

Yeah, that’s why I’ve been working on GitHub - stgraber/incus-deploy: Deployment playbooks, configurations and scripts for Incus which doesn’t use cephadm/podman :slight_smile:

It may be worth talking to someone who better understands podman. Chances are that there’s a way to not have it take a full copy of the root mount table or if it does briefly, at least get rid of those mounts shortly afterwards so it doesn’t hold an indefinite reference like it does now.

1 Like
10

To have a nice removal/move of your containers in this case :

You should use this little script :

#/bin/sh
for osd in $(podman ps | grep osd | cut -f 1 -d " ")
do
	podman exec $osd mount | grep "incus/storage-pool" 
	if [ $? -eq 0 ]
	then
	echo "$osd is bad"
	for mountpoint in $(podman exec $osd mount | grep "incus/storage-pool" | awk '{print $3}')
	do 
	   echo unmount $mountpoint in Podman $osd
	   podman exec $osd umount ${mountpoint}
	done
	fi
done

I cleanup the RBD in hold in the podman OSD. To open a ticket on Ceph it seems it needs to be a “member” for … 10k/year !

1 Like