Best way to forward block devices to containers

1

I tried to create profiles with devices such as,
incus profile device add partition_sda1 mypartition unix-block source=/dev/sda1 gid=6
and
incus profile device add partition_41eea4a1-b2d0-48ad-8b40-0240c9e08dda mypartition unix-block source=/dev/disk/by-uuid/41eea4a1-b2d0-48ad-8b40-0240c9e08dda gid=6

I have my user in the container added to the disk group (gid=6); probably doesn’t matter…
Yet, I get with both:

> sudo mount /dev/sda1 /mnt/disk1
mount: /mnt/disk1: permission denied.
       dmesg(1) may have more information after failed mount system call.

There’s nothing on dmesg. Fedora 40 host and client.

2

Mounting block devices inside of a container is a very very unsafe operation and is therefore blocked by the kernel for all unprivileged containers and usually blocked by AppArmor or similar in privileged containers.

Incus has some ways around that, in order of preference:

  • security.syscalls.intercept.mount=true and security.syscalls.intercept.mount.fuse=ext4=fuse2fs. This option is safe and relies on fuse2fs being present in the container, effectively redirecting unsafe mount calls to the userspace FUSE implementation.
  • security.syscalls.intercept.mount=true and security.syscalls.intercept.mount.allowed=ext4. This is unsafe but allows an unprivileged container to mount any ext4 filesystem with full root privileges without having to resort to a privileged container.
  • security.privileged=true. This is the least safe option as now the entire container is running with full root privileges.
1 Like
3

What filesystems can participate in the CSV to this option ( security.syscalls.intercept.mount.allowed)? The reference does not mention it.

And could you please tell if it’s possible to forward raw block devices (no filesystem and maybe not even paritions)? Only with privileged containers I imagine, but my use case would allow that.

4

I wanted to see if it was possible to run Scrutiny in Incus, but I think it’s a bridge too far - GitHub - AnalogJ/scrutiny: Hard Drive S.M.A.R.T Monitoring, Historical Trends & Real World Failure Thresholds - it needs the /dev/disks as a device, /run/dev, and --cap-add SYS_RAWIO - does that have an Incus OCI equivalent?

5

You’d need a privileged container for that. There’s no way for an unprivileged container to have RAWIO on a block device.

6

Yeah that’s how the Docker container works too, I was expecting that.

7

I would love if someone shed light on this. As I tested there’s no validation when setting this value.

8

Another relevant dilemma of mine is :smiley::

  1. Can I mount one device (an ext4 partition) RW on a container and expect RO mounts in other containers to work as well in conjuction?
    1.1. What changes if I used fuse2fs?
    1.2. Multiple RW mounts?
9

moved here