High memory consumption after deleting some containers

Hello, I am currently using xubuntu 20.04 with lxc/lxd version Client version: 4.4,Server version: 4.4, my test machine has 6GB of ram, I just tried to delete few stopped containers, and then noticed the lxd process is taking 1.8GB of ram
1818.0m 128.2m 27.4m S 0.0 2.2 0:12.65 lxd

At the moment I don’t have any containers running on the system
lxc list
±-----±------±-----±-----±-----±----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
±-----±------±-----±-----±-----±----------+

take it easy, 1818.0m is vm size , not real memory usage,
the second column 128.2m is the memory lxd consumed.