Hello,
i have two storage pools on my system
I would like to mount two devices in my lxd vm.
root device on the ssd "/dev/sda
my storage from the second pool hdd "/dev/sdb
unfortunately i can’t find out how to do this with the help of the documentation.
mfg
Please can you show the output of lxc storage ls and lxc storage volume ls <pool> for the two storage pools you are referring to. Also which volume are you trying to attach to a VM?
lxc storage ls
+--------+-------------+--------+--------------+---------+
| NAME | DESCRIPTION | DRIVER | SOURCE | USED BY |
+--------+-------------+--------+--------------+---------+
| LXDhdd | | btrfs | /dev/sda | 30 |
+--------+-------------+--------+--------------+---------+
| LXDpro | | btrfs | /dev/nvme1n1 | 5 |
+--------+-------------+--------+--------------+---------+
lxc storage volume ls LXDpro
+----------------------+----------------------------------+-------------+--------------+---------+
| TYPE | NAME | DESCRIPTION | CONTENT TYPE | USED BY |
+----------------------+----------------------------------+-------------+--------------+---------+
| container | c1 | | filesystem | 1 |
+----------------------+----------------------------------+-------------+--------------+---------+
| virtual-machine | v1 | | block | 1 |
+----------------------+----------------------------------+-------------+--------------+---------+
lxc storage volume ls LXDhdd
+-----------------+------------------------------------------------------------------+-------------+--------------+---------+
| TYPE | NAME | DESCRIPTION | CONTENT TYPE | USED BY |
+-----------------+------------------------------------------------------------------+-------------+--------------+---------+
| image | 2f456602cd86a6dc4d50102f028729421fece7e68f7c8aec6d8af5d9bbc681bf | | block | 1 |
+-----------------+------------------------------------------------------------------+-------------+--------------+---------+
| virtual-machine | ceph | | block | 1 |
+-----------------+------------------------------------------------------------------+-------------+--------------+---------+
You can use the disk device type to attach an existing block custom volume to a VM:
lxc storage volume create <pool> myvol --type=block
lxc init images:ubuntu/focal v1 --vm
lxc config device add v1 mydisk disk source=myvol pool=<pool>
lxc start v1
lxc exec v1 -- fdisk /dev/sdb
Welcome to fdisk (util-linux 2.34).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x348ab859.
Command (m for help): p
Disk /dev/sdb: 9.32 GiB, 10003415040 bytes, 19537920 sectors
See https://linuxcontainers.org/lxd/docs/master/instances#type-disk for more details.
Thank you, it worked out wonderfully, unfortunately it is a bit more careful than expected
Glad it worked 
What do you mean by “careful”?
I just mean that I imagined it to be easier
If you have an existing block device that isn’t managed by LXD (by that I mean doesn’t show in lxc storage volume ls <pool> (perhaps just an existing disk or partition device in the system) then you can directly add it as a disk using:
lxc config device add v1 mydisk disk source=/dev/blah
Hi thomas,
is it possible to add the same device to multiple containers? without errors if multiple containers are writing to the disks.
i want to resetup everything. I was thinking of the following.
2 x m.2 sata 500gb raid 1 (os)
2 x 1tb sata raid 1 for all the containers (only the containers)
4 x 1 tb sata raid 10 for all the data inside the containers. I want to add this device to all the containers that are using a lot of storage(data)
All filesystems will be btrfs. What do you think?
Is this possible? and will this be a good setup?
kind regards,
Chris Kruijntjens.
You cannot add same block disk device to multiple VMs at the same time as they would conflict.
Instead you could mount the block disk device on the host and then share that directory as a disk share into the VMs, e.g.
lxc config device add v1 mydisk disk source=/dir/where/disk/is/mounted
Hi Thomas,
If i use your command tot add device to a lxd container. I can do the same for other containers. It works. However from your reply i onderstand i would get issues in the future?
For the earlier command:
lxc config device add v1 mydisk disk source=/dev/blah
Then yes it probably does allow you to add the same block device to multiple VMs concurrently, however it probably won’t work well if you actually use the block device concurrently from multiple VMs.
However it is allowed from LXD’s perspective because you may want to coordinate access to the underlying block device from inside the VMs (perhaps a network aware cluster filesystem).
But if its a standard block device with a filesystem in it then it will not work.
Hi Thomas,
thank you for the info. Can you advise me how would you install the following.
i have. 2 x 500gb m.2 sata. -Here i will install the os.
i have 2 x 1 tb ssd btrfs raid 1. -Here i want to install all my containers
i have 4 x 1 TB ssd btrfs raid10 -Here i want to store all data from the containers. (user data, mail etc etc)
How would you share the 4tb storage to other containers? or is your advise to do a different setup?
You can create two BTRFS storage pools on top of your relevant existing filesystems (using the source property pointing to the existing mount path when doing lxc storage create), and then you can create instances using the smaller storage pool (using either the -s flag to lxc launch or specified inside the profile) and custom volumes using the larger storage pool (which can be of type filesystem or block, with only filesystem volumes being suitable for use with multiple instances concurrently as discussed).