I use ZFS as the storage backend and have 4 Linux containers running on it. The OS is Ubuntu20.04 and the LXD version is 4.06.
My biggest concern is that the 4 running containers are growing in size everyday and the largest one is growing by 4GB everyday. I will soon running out of disk space. Here is some outputs from
zfs list for the last there days.
The output format is
NAME USED AVAIL REFER MOUNTPOINT
default/containers/repo 49.0G 165G 27.1G /var/snap/lxd/common/lxd/storage-pools/default/containers/repo
default/containers/repo 53.0G 161G 26.7G /var/snap/lxd/common/lxd/storage-pools/default/containers/repo
default/containers/repo 57.1G 157G 26.5G /var/snap/lxd/common/lxd/storage-pools/default/containers/repo
I have no clue on what’s going on here. Please give some advices and instructions how I can resolve this issue.
BTW, two more things I noticed.
- The stopped container doesn’t grow in size, only the running ones.
- I noticed that size differences between the column
USEDand the column
REFERin the above
zfs listoutputs. When I copied the snapshot of the running container to the backup server. The size of the backup container is the same as the size shown in the column
REFERnot the column
USED. I am not clear what it means.
Some background if helps:
I started to use LXD several years ago and the version was 2.21 on Ubuntu16.04. It have been running with no big problems and therefore I never tempted to upgrade to newer version.
Starting about 1 to 2 week ago, all the containers suddenly became no responsive. Then I noticed that the
AVAIL disk space was running out. It became
0 literally. I then deleted some snapshots to make some free disk space, several hundreds MB. However, all the free space were gone the 2nd day. As a matter of fact, I noticed the free space was shrinking by several MB every a few minutes.
So I built a new server with much larger SSD and the latest Ubuntu and LXD. I copied the containers from the old server to the new one and started them again. Then I noticed that issue mentioned in the beginning of this post. Please advise and thank you!