Hi cli0,
I was having the same challenge and my need was exactly the same as yours. I understand from your description, that you ran lxd init and you went with the option for local directory storage, right?
1- Anyways, even if this is not the case, let’s see how to create a new storage pool
$ lxc storage list
The above command will show you the storage pools available for your containers and VMs. In my setup, the output is
+----------+--------+------------------------------------------------+-------------+---------+---------+
| NAME | DRIVER | SOURCE | DESCRIPTION | USED BY | STATE |
+----------+--------+------------------------------------------------+-------------+---------+---------+
| default | dir | /var/snap/lxd/common/lxd/storage-pools/default | | 1 | CREATED |
+----------+--------+------------------------------------------------+-------------+---------+---------+
| zfs-pool | zfs | zfs-pool | | 9 | CREATED |
+----------+--------+------------------------------------------------+-------------+---------+---------+
So I have 3 storage pools that I can use with my profiles or I can use directly in the command line when launching a new instance (container or VM).
Ref: https://linuxcontainers.org/lxd/docs/master/howto/storage_pools/
You can create a new storage pool in command line as mentioned in the doc link above, choose the driver you prefer (zfs, dir, btrfs, etc…), let’s quickly create a new pool with dir driver
$ lxc storage create dir-pool dir source=/virtual-hdd/test-dir/
Storage pool dir-pool created
$ lxc storage list
+----------+--------+------------------------------------------------+-------------+---------+---------+
| NAME | DRIVER | SOURCE | DESCRIPTION | USED BY | STATE |
+----------+--------+------------------------------------------------+-------------+---------+---------+
| default | dir | /var/snap/lxd/common/lxd/storage-pools/default | | 1 | CREATED |
+----------+--------+------------------------------------------------+-------------+---------+---------+
| dir-pool | dir | /virtual-hdd/test-dir/ | | 0 | CREATED |
+----------+--------+------------------------------------------------+-------------+---------+---------+
| zfs-pool | zfs | zfs-pool | | 9 | CREATED |
+----------+--------+------------------------------------------------+-------------+---------+---------+
Now I have my new storage pool created and ready to be used. Let’s use it directly to launch a new container
$ lxc launch images:ubuntu/20.04 -s dir-pool c1
Creating c2
Starting c2
$ lxc list
+-----------------+---------+-----------------------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------------+---------+-----------------------+------+-----------+-----------+
| c2 | RUNNING | 10.252.104.51 (eth0) | | CONTAINER | 0 |
To make sure that my new container is really in the new dir-pool, run the following
$ lxc storage info dir-pool
info:
description: ""
driver: dir
name: dir-pool
space used: 20.82GiB
total space: 491.08GiB
used by:
instances:
- c2
2- Now to the second part of your question, how to move existing containers and VMs to the new storage pool, to free the space on your OS partition. You can just run lxc move as follows
lxc move <container|VM name> -s <destination_storage_pool>
I created the container c1 in dir-pool and I want to move it to zfs-pool
$ lxc move c1 -s zfs-pool
You might need to stop the instance before moving, or you will get the error I got
VM
$ lxc move v1 -s dir-pool
Error: Migration operation failure: Stateful stop requires migration.stateful to be set to true
Container
$ lxc move c1 -s dir-pool
Error: Migration operation failure: Unable to perform live container migration. CRIU isn't installed. To migrate the container, stop the container before migration or install CRIU
3- Finally, how to make it the default behaviour to create new instances in the new storage pool? Answer: You can: a- just modify the default profile to point to the newly created storage pool
$ lxc profile edit default
This will open an editor to the profile yaml config
..
root:
path: /
pool: zfs-pool # change this to be -> dir-pool (or whatever you called your new storage pool. And do not edit the previous and next lines. Just this one.
type: disk
..
or, b- create a new profile (if for some reason you want to keep the default profile as is), by copying the default one to a new one and modifying the new profile, and then you will need to specify it with every instance launch
$ lxc profile copy default new-profile
$ lxc profile edit new-profile #then edit the new profile the same way you would have done it with the default profile
$ lxc launch images:ubuntu/20.04 -p new-profile c3 #you will need to specify the profile every time you launch a new instance
References: