Ghost ZFS images dataset clean up

Hi Guys,

I new at LXD and ZFS ecosystem, I notice and have several images dataset on my host, but I don’t have much images or containers running. It seems to be taken a lot of space.

# lxc list -f csv | wc -l
7
# lxc image list
+-------+--------------+--------+--------------------------------------+--------------+-----------+----------+-------------------------------+
| ALIAS | FINGERPRINT  | PUBLIC |             DESCRIPTION              | ARCHITECTURE |   TYPE    |   SIZE   |          UPLOAD DATE          |
+-------+--------------+--------+--------------------------------------+--------------+-----------+----------+-------------------------------+
|       | 9b2301ec680f | no     | Debian buster amd64 (20220224_06:50) | x86_64       | CONTAINER | 75.24MB  | Feb 24, 2022 at 10:42am (UTC) |
+-------+--------------+--------+--------------------------------------+--------------+-----------+----------+-------------------------------+
|       | 5025f5034c0a | no     | Centos 6 x86_64 (20201202_07:08)     | x86_64       | CONTAINER | 119.05MB | Apr 15, 2021 at 11:31am (UTC) |
+-------+--------------+--------+--------------------------------------+--------------+-----------+----------+-------------------------------+
|       | b031ef0d28a8 | no     | Centos 7 amd64 (20220224_07:08)      | x86_64       | CONTAINER | 84.49MB  | Feb 24, 2022 at 10:42am (UTC) |
+-------+--------------+--------+--------------------------------------+--------------+-----------+----------+-------------------------------+
# zfs list
zpool1/lxdpool1/images/010c79ad44b26b04456f2dbe716a18aa6030388b7a1bc19ed6012dd1ba649bd0                 415M  21.8G      415M  /var/snap/lxd/common/lxd/storage-pools/lxdpool1/images/010c79ad44b26b04456f2dbe716a18aa6030388b7a1bc19ed6012dd1ba649bd0
zpool1/lxdpool1/images/01687841140c517eceef734a390a7f04c2972ad8078b3f21188f19a95322efa8                 415M  21.8G      415M  /var/snap/lxd/common/lxd/storage-pools/lxdpool1/images/01687841140c517eceef734a390a7f04c2972ad8078b3f21188f19a95322efa8
zpool1/lxdpool1/images/01c29383240a18c93edbeeb9748e374572afcd3e0afe9c4fcdcac37fc025ac73                 415M  21.8G      415M  /var/snap/lxd/common/lxd/storage-pools/lxdpool1/images/01c29383240a18c93edbeeb9748e374572afcd3e0afe9c4fcdcac37fc025ac73
zpool1/lxdpool1/images/02cd90bcc4670d6150d7565784f85816b1afc46e5c8d6301f77b87a01dca0722                 378M  21.8G      378M  /var/snap/lxd/common/lxd/storage-pools/lxdpool1/images/02cd90bcc4670d6150d7565784f85816b1afc46e5c8d6301f77b87a01dca0722
zpool1/lxdpool1/images/031234bd21df2c33c307664703c075ed2e4c7aee2509feb157ed54b505a98c87                 268M  21.8G      268M  /var/snap/lxd/common/lxd/storage-pools/lxdpool1/images/031234bd21df2c33c307664703c075ed2e4c7aee2509feb157ed54b505a98c87
zpool1/lxdpool1/images/0d04983fd90915e43037278ce4c3c79cbdfda7ef390bad8c794c879a4f6feaa0                  27K  95.3M       27K  /var/snap/lxd/common/lxd/storage-pools/lxdpool1/images/0d04983fd90915e43037278ce4c3c79cbdfda7ef390bad8c794c879a4f6feaa0
zpool1/lxdpool1/images/0d04983fd90915e43037278ce4c3c79cbdfda7ef390bad8c794c879a4f6feaa0.block          4.01G  21.8G     4.01G  -
zpool1/lxdpool1/images/0dcbf4eae5f50dc6d7b844771141438108d3a78003fbfa4e256a13b23ca19598                 377M  21.8G      377M  /var/snap/lxd/common/lxd/storage-pools/lxdpool1/images/0dcbf4eae5f50dc6d7b844771141438108d3a78003fbfa4e256a13b23ca19598

# zfs list | grep images  | wc -l
242

I tried to add images.remote_cache_expiry: "1" and restart LXD but it doesn’t changed anything.

I would like know if someone can help me figure out what is causing this and how to do a clean up.

Some info:

OS: Ubuntu 20.04.1
LXD: 4.23

# lxc config show
config:
  core.https_address: 0.0.0.0
  core.trust_password: true
  images.remote_cache_expiry: "1"

When you create an instance on a storage pool from an image, the image is downloaded locally, and then unpacked into a volume on the storage pool (a so-called “image” volume). On storage pools like ZFS that support efficient snapshots, instances are then created by taking a snapshot of the image volume for the instance volume. This allows quick instance creation and efficient storage space, as only the differences between the instance and the image volume actually take up space.

So you maybe seeing image volumes because they are the basis of your instances.