How to add a disk raw partition to a LXD container?

Hi, I’d like to add a disk raw partition to a LXD container.
I’d like to access it like /dev/sda2 inside the container.
I’m using zfs for LXD storage backend.
Could you tell me how to do that?
Thanks!

Hi,

i would try it with a unix-block device:

https://lxd.readthedocs.io/en/latest/containers/#type-unix-block

Something like:

lxc config device add <container> <name_in_container> unix-block source=/dev/zvol/$yourzfs

Just do some experiments. Your favourite search engine will also give you some hints.

1 Like

I connected a USB stick on my computer. I used gnome-disks to unmount it from the host, but still not disconnect it (not power down). The device is /dev/sdb1.

$ lxc launch ubuntu:18.04 rawdisk
Creating rawdisk
Starting rawdisk
$ lxc config device add rawdisk myrawdisk unix-block source=/dev/sdb1
Device myrawdisk added to rawdisk

In the container, I run

$ sudo fdisk /dev/sdb1

Welcome to fdisk (util-linux 2.31.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk /dev/sdb1: 3.9 GiB, 4126146560 bytes, 8058880 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Command (m for help): q

You have full access to the raw block device in the container.
If you want to mount it, see Unix-block device mounting
If you still want to mount it, you can look for some FUSE package that supports your filesystem.

2 Likes

Thanks for your comments.

simos’ answer works as expected. However I’d like to use disk space from zfs pool.

I did some experiment and found I cannot use zfs dataset as source for lxc config device add unix-block.

As a workaround, I create a file for raw partition in a zfs dataset /tank1/storage1

$ sudo dd if=/dev/zero of=/tank1/storage1/disk1.img bs=8M count=100

Then I mount it as a loopback device.

$ sudo losetup $(sudo losetup -f) /tank1/storage1/disk1.img
$ losetup -a
...(snip)...
/dev/loop3: []: (/tank1/storage1/disk1.img)

Finally I can add this device to a container as a unix-block

$ lxc config device add bionic storage1 unix-block source=/dev/loop3 path=/dev/sdb

In the container, I can see the device.

root@bionic:~# df -h /dev/sdb
Filesystem      Size  Used Avail Use% Mounted on
tank1/snap/var  382G  1.4G  380G   1% /dev/sdb

The size in the output is not correct, but I can see the correct size with fdisk.

root@bionic:~# fdisk /dev/sdb

Welcome to fdisk (util-linux 2.31.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x24f32ec9.

Command (m for help): p
Disk /dev/sdb: 800 MiB, 838860800 bytes, 1638400 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x24f32ec9

Command (m for help): q

i suggest you to create a real raw image file with qemu-img and use this. That will most probably solve your issue.

If ever possible you should just create a zfs vol/dataset and use this directly.

Going from blockdevice -> mount filesystem -> create raw file -> mount raw file as blockdevice
can be a way, but you loose flexibility and just add complexity without real gain ( imho ) if you leave your zfs ecosystem like this.

Hi, sorry to bring this up again, when we add a unix-block and refer to /dev/sdb1, does LXD maps it to UUID or just de sdX?
Just that I have had situations where you have to pull the drive or a faulty cable, then the OS map it to a different sdX, in this case might break the device added to the container.

Here are the docs for the unix-block device:

It uses the source and path settings. So if you want to refer to a device by UUID you should use the /dev/disk/by-uuid/ paths.

1 Like

I think it might be working but I keep getting and issue of permission denied on mount.
path: /dev/sdb
source: /dev/disk/by-uuid/*******************
type: unix-block

already added
raw.lxc: lxc.cgroup.devices.allow = b 8:* rwm
But it did not work, any suggestions?
Thanks.

You won’t be able to mount the device inside the container without using syscall interception.

See