Lox
August 5, 2019, 12:16am
1
Hello,
I am trying to grow an lxd root storage. I added the root device with the proper quota, but it behaves weirdly: the root disk quota seems to have been changed on the zfs side, but looks pretty full. From the container, nothing changed:
# zfs list | grep nextcloud
nvme/lxd/containers/nextcloud 36,4G 854M 3,93G /var/lib/lxd/storage-pools/nvme/containers/nextcloud
# du -h /var/lib/lxd/storage-pools/nvme/containers/nextcloud
4,0G /var/lib/lxd/storage-pools/nvme/containers/nextcloud
# lxc exec nextcloud -- df -h
Filesystem Size Used Avail Use% Mounted on
nvme/lxd/containers/nextcloud 4.8G 4.0G 854M 83% /
#lxc config show nextcloud --expanded
devices:
root:
path: /
pool: nvme
size: 40GB
type: disk
stgraber
August 5, 2019, 1:31am
2
Could it be that your zpool itself is running out of diskspace and so cannot provide the amount of space you have configured for that particular dataset?
Lox
August 7, 2019, 7:53am
3
It can’t be:
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
data 5,44T 1,25T 4,18T - 2% 23% 1.00x ONLINE -
nvme 390G 100G 290G - 52% 25% 1.00x ONLINE -
stgraber
August 8, 2019, 10:51pm
4
Can you show zfs get all nvme/lxd/containers/nextcloud?
Might also be useful to see:
zfs get quota -r
zfs get refquota -r
Lox
August 9, 2019, 6:20am
5
Sure:
zfs get all nvme/lxd/containers/nextcloud
NAME PROPERTY VALUE SOURCE
nvme/lxd/containers/nextcloud type filesystem -
nvme/lxd/containers/nextcloud creation mer. avril 17 21:53 2019 -
nvme/lxd/containers/nextcloud used 28,9G -
nvme/lxd/containers/nextcloud available 9,31G -
nvme/lxd/containers/nextcloud referenced 3,99G -
nvme/lxd/containers/nextcloud compressratio 1.67x -
nvme/lxd/containers/nextcloud mounted yes -
nvme/lxd/containers/nextcloud origin nvme/lxd/images/663f6663aed66a22dd708c4b07514748221522b810008c55002fcc1dd81af377@readonly -
nvme/lxd/containers/nextcloud quota 38,2G local
nvme/lxd/containers/nextcloud reservation none default
nvme/lxd/containers/nextcloud recordsize 128K default
nvme/lxd/containers/nextcloud mountpoint /var/lib/lxd/storage-pools/nvme/containers/nextcloud local
nvme/lxd/containers/nextcloud sharenfs off default
nvme/lxd/containers/nextcloud checksum on default
nvme/lxd/containers/nextcloud compression lz4 inherited from nvme
nvme/lxd/containers/nextcloud atime on default
nvme/lxd/containers/nextcloud devices on inherited from nvme/lxd
nvme/lxd/containers/nextcloud exec on inherited from nvme/lxd
nvme/lxd/containers/nextcloud setuid on inherited from nvme/lxd
nvme/lxd/containers/nextcloud readonly off default
nvme/lxd/containers/nextcloud zoned off default
nvme/lxd/containers/nextcloud snapdir hidden default
nvme/lxd/containers/nextcloud aclinherit restricted default
nvme/lxd/containers/nextcloud createtxg 33928 -
nvme/lxd/containers/nextcloud canmount noauto local
nvme/lxd/containers/nextcloud xattr sa inherited from nvme/lxd
nvme/lxd/containers/nextcloud copies 1 default
nvme/lxd/containers/nextcloud version 5 -
nvme/lxd/containers/nextcloud utf8only off -
nvme/lxd/containers/nextcloud normalization none -
nvme/lxd/containers/nextcloud casesensitivity sensitive -
nvme/lxd/containers/nextcloud vscan off default
nvme/lxd/containers/nextcloud nbmand off default
nvme/lxd/containers/nextcloud sharesmb off default
nvme/lxd/containers/nextcloud refquota none default
nvme/lxd/containers/nextcloud refreservation none default
nvme/lxd/containers/nextcloud guid 3430061508082321110 -
nvme/lxd/containers/nextcloud primarycache all default
nvme/lxd/containers/nextcloud secondarycache all default
nvme/lxd/containers/nextcloud usedbysnapshots 25,1G -
nvme/lxd/containers/nextcloud usedbydataset 3,76G -
nvme/lxd/containers/nextcloud usedbychildren 0B -
nvme/lxd/containers/nextcloud usedbyrefreservation 0B -
nvme/lxd/containers/nextcloud logbias latency default
nvme/lxd/containers/nextcloud dedup off default
nvme/lxd/containers/nextcloud mlslabel none default
nvme/lxd/containers/nextcloud sync standard default
nvme/lxd/containers/nextcloud dnodesize legacy default
nvme/lxd/containers/nextcloud refcompressratio 1.70x -
nvme/lxd/containers/nextcloud written 1,62M -
nvme/lxd/containers/nextcloud logicalused 48,1G -
nvme/lxd/containers/nextcloud logicalreferenced 6,78G -
nvme/lxd/containers/nextcloud volmode default default
nvme/lxd/containers/nextcloud filesystem_limit none default
nvme/lxd/containers/nextcloud snapshot_limit none default
nvme/lxd/containers/nextcloud filesystem_count none default
nvme/lxd/containers/nextcloud snapshot_count none default
nvme/lxd/containers/nextcloud snapdev hidden default
nvme/lxd/containers/nextcloud acltype posixacl inherited from nvme/lxd
nvme/lxd/containers/nextcloud context none default
nvme/lxd/containers/nextcloud fscontext none default
nvme/lxd/containers/nextcloud defcontext none default
nvme/lxd/containers/nextcloud rootcontext none default
nvme/lxd/containers/nextcloud relatime on inherited from nvme
nvme/lxd/containers/nextcloud redundant_metadata all default
nvme/lxd/containers/nextcloud overlay off default
zfs get quota -r nvme/lxd/containers/nextcloud
NAME PROPERTY VALUE SOURCE
nvme/lxd/containers/nextcloud quota 38,2G local
nvme/lxd/containers/nextcloud@zfs-auto-snap_monthly-2019-06-30-1952 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_weekly-2019-07-13-1947 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_weekly-2019-07-20-1947 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-25-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-26-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-27-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_weekly-2019-07-27-1947 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-28-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-29-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-30-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-31-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_monthly-2019-07-31-1952 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-01-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-02-1926 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-03-1926 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_weekly-2019-08-03-1947 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-04-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-05-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-06-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-07-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-0717 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-0817 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-0917 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1017 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1117 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1217 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1317 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1417 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1517 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1617 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1717 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1817 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1917 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-08-1925 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-2017 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-2117 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-2217 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-2317 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0017 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0117 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0217 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0317 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0417 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0517 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_frequent-2019-08-09-0530 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_frequent-2019-08-09-0545 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_frequent-2019-08-09-0600 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_frequent-2019-08-09-0615 quota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0617 quota - -
# zfs get refquota -r nvme/lxd/containers/nextcloud
NAME PROPERTY VALUE SOURCE
nvme/lxd/containers/nextcloud refquota none default
nvme/lxd/containers/nextcloud@zfs-auto-snap_monthly-2019-06-30-1952 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_weekly-2019-07-13-1947 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_weekly-2019-07-20-1947 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-25-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-26-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-27-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_weekly-2019-07-27-1947 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-28-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-29-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-30-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-07-31-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_monthly-2019-07-31-1952 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-01-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-02-1926 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-03-1926 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_weekly-2019-08-03-1947 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-04-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-05-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-06-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-07-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-0717 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-0817 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-0917 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1017 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1117 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1217 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1317 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1417 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1517 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1617 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1717 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1817 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-1917 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_daily-2019-08-08-1925 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-2017 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-2117 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-2217 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-08-2317 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0017 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0117 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0217 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0317 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0417 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0517 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_frequent-2019-08-09-0530 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_frequent-2019-08-09-0545 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_frequent-2019-08-09-0600 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_frequent-2019-08-09-0615 refquota - -
nvme/lxd/containers/nextcloud@zfs-auto-snap_hourly-2019-08-09-0617 refquota - -
Lox
August 9, 2019, 6:22am
6
Lox:
nvme/lxd/containers/nextcloud used 28,9G
Are snapshot counted in quota ?!?
Xaoc
August 9, 2019, 6:30am
7
Yes they are! Instead of quota you can use “requota”. refquota is controling actual data limitations and isn’t include snapshots
Lox
August 9, 2019, 6:30am
8
It seems so. Found that https://github.com/lxc/lxd/issues/2354
So by default LXD uses “quota”, it seems little sense to me but I see that I can make LXD use refquota.
If I do so will it be changed for all containers ? If not what would be the way way for doing it ?
Lox
August 9, 2019, 7:06am
9
That was it.
I changed the global configuration, when I set a root size I really expect to have the corresponding root “disk” size from inside the container:
lxc storage set nvme volume.zfs.use_refquota true
I then removed zfs quotas manually for each containers
zfs set quota=none nvme/lxd/containers/[container_name]
I then edited accordingly profiles and configs that had quotas set using lxc profile edit [profile_name] and lxc config edit [conainer_name]
It seems to do the job. Did I miss something in that process ?