Disk usage increasing by 4-5 GB per day

1

LXD version 3.18, using snap. I have one storage pool set up, the default zfs pool. I’m running Ubuntu 18.04 and only using the computer for running the containers.

The disk space used by the ZFS pool is increasing by 4-5 GB per day. Nothing in my containers should be generating anywhere near that amount of data, and when I check the amount of space used by all containers it is very small.

> df -h .
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       458G   94G  341G  22% /

> sudo du -sh /var/snap/lxd/common/mntns/var/snap/lxd/common/lxd/disks
83G	/var/snap/lxd/common/mntns/var/snap/lxd/common/lxd/disks

> zpool list
NAME      SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
default   186G  5.41G   181G         -    11%     2%  1.00x  ONLINE  -

> zfs list
NAME                                                                                      USED  AVAIL  REFER  MOUNTPOINT
default                                                                                  5.41G   175G    24K  none

I went through and added up the usage of every container using
sudo du -hs /var/snap/lxd/common/mntns/var/snap/lxd/common/lxd/storage-pools/default/containers/<container>/rootfs
and it was about 11GB.

As a test I stopped all but one container for about 9 hours. The size of the container grew by about 10MB, but disk usage grew by 1.8GB.

Any ideas about what could be going on?

2

That’s not particularly surprising. File back zpool will not reduce the size of the .img file once allocated, the file will effectively grow until it reaches the limit you’ve set during lxd init at which point ZFS will do more active block reuse within the file.

3

Thanks, I didn’t realize ZFS would continue growing without reuse like that. I originally set up a 200 GB pool because I had no idea how much I’d need. Seems like I can reduce it significantly so I don’t have such huge daily backups to do.

I still don’t understand why the ZFS file is growing at about 180 times the rate of what the containers are growing though. Maybe something to do with block sizes in the containers vs zfs pool?