Moving default storage pool to another (larger) partition

LXD
,
1

Hi, I have the following problem:

  • I have a (relatively) small SSD where’s my system (Linux Mint 20.2, if it matters) and a (much larger) HD array completely handled by ZFS and mounted on my /home.
  • I installed LXD from snap as advised and I created a few containers.
  • I actually need just one of them, but as I use it to launch a rather large yocto build it needs a huge amount of disk.
  • Compilation died due to lack of space.
  • Now I have NO free space on main disk:
mcon@cinderella:/tmp$ df
Filesystem      1K-blocks       Used  Available Use% Mounted on
udev             16369964          0   16369964   0% /dev
tmpfs             3283676       2516    3281160   1% /run
/dev/sda5       146595780  139084284          0 100% /
tmpfs            16418360     250420   16167940   2% /dev/shm
tmpfs                5120          4       5116   1% /run/lock
tmpfs            16418360          0   16418360   0% /sys/fs/cgroup
/dev/sda1          129039       5034     124005   4% /boot/efi
homez/mcon     5077915776 1713844864 3364070912  34% /home/mcon
homez/mauro    3364088960      18048 3364070912   1% /home/mauro
homez          3780690560  416619648 3364070912  12% /homez
homez/etherpad 3364324608     253696 3364070912   1% /homez/etherpad
/dev/loop1          63360      63360          0 100% /snap/core20/1242
/dev/loop2          74880      74880          0 100% /snap/lxd/22114
/dev/loop0          63488      63488          0 100% /snap/core20/1270
/dev/loop3          43264      43264          0 100% /snap/snapd/14066
/dev/loop4          74752      74752          0 100% /snap/lxd/22147
/dev/loop5          44416      44416          0 100% /snap/snapd/14295
tmpfs             3283672         68    3283604   1% /run/user/1000
tmpfs                1024          0       1024   0% /var/snap/lxd/common/ns
  • I killed the container (lxc stop bionic && lxc delete bionic) and now I got my disk space back (although it seems there has been some “leakage”, but I’m not positive on that).

Question is: how can I setup things so that container data is stored in my zfs array?
I have a relatively large space there (4x3T in RAIDZ2 configuration giving me ~6T usable space; about one half is actually in use) but I’m unsure about how to proceed ans if btrfs can actually live alongside with zfz (I would like to avoid reformatting/repartitioning, if at all possible).

Note: my current setup is:

mcon@cinderella:/tmp$ lxc storage list
+---------+--------+------------------------------------------------+-------------+---------+
|  NAME   | DRIVER |                     SOURCE                     | DESCRIPTION | USED BY |
+---------+--------+------------------------------------------------+-------------+---------+
| default | dir    | /var/snap/lxd/common/lxd/storage-pools/default |             | 1       |
+---------+--------+------------------------------------------------+-------------+---------+

I see there DRIVER -> dir so I guess all btrf vs zfs debate shouldn’t really matter, right?

What are my options?
Please consider I’m a relative newbie to LXD.

Any advice would be very welcome.

2

Hi Mauro,
If I understand your post correctly, you have already a zfs pool then you can create a separate zfs filesystem and stop the mentioned container then move the container to the new zfs storage, like that.

lxc storage create ztest zfs source=zpool/test --> create a new storage, I named it ztest
lxc stop <container_name>
lxc move <container_name> -s ztest

Regards.

3

Thanks @cemzafer.
I need to understand a little better how lxc storage create works (pointers to docs/tutorials welcome!).
My zfs pool is called homez and I assume ztest in your example is the name of newly created storage (to be used in the lxc move command).
I have only a foggy idea about real meaning of the source=zpool/test part.
Can you elaborate, please?

I also have another, related, question:
I will have to create another container; how can I specify to create it directly in the newly created ztest (or whatever, of course)?

Thanks in Advance.

4

Hi,
You can think zpool/test as a directory. In my example I just want to separate the lxc filesystem from the existing filesystems. But you can instead use just like that. lxc storage create ztest zfs source=<poolname> which in your case is homez. I use lxc storage name as ztest you can give anyname you want also the rest will be done by lxd automatically (the creation of the filesystems under pool).

lxc launch images:ubuntu/20.04/cloud newubuntu -s ztest → You can specify the storage with “s” parameter and the storage name ztest is used cos we created the lxd storage name as ztest in this example.
Regards.