Hm, not really atm.
How exactly are the zfs tools calculating disk usage. Because du seems to be working just fine, so that doesn’t look like a bug in shiftfs.
As far as I know, it’s kernel side tracking as the output is near instant and matches that of
@brauner Can you try to reproduce this issue with the instructions above and see if anything jumps out as far as behavior?
If things are counting twice somehow, a simple dd of 100MB should show 200MB used on zfs, hopefully making things easier to track down.
I actually tried that on my virtual setup and the results looked correct. Then I ran it again overwriting the same file to see if freeing up space was the issue but that also worked correctly.
The command I used was:
lxc exec disco -- dd if=/dev/urandom of=/swapfile bs=1M count=1K
NAME USED AVAIL REFER pool/lxd/containers/disco 1.01G 1.37G 1.47G pool/lxd/containers/disco-shiftfs 1006M 1.37G 1.45G
NAME USED AVAIL REFER pool/lxd/containers/disco 1.01G 1.39G 1.47G pool/lxd/containers/disco-shiftfs 994M 1.39G 1.44G
Here’s some more oddness. I just ran
poweroff inside of each container in preparation for rebooting my physical host.
NAME USED AVAIL REFER pool/lxd/containers/disco 195M 23.2G 651M pool/lxd/containers/disco-shiftfs 3.15G 23.2G 3.60G
NAME USED AVAIL REFER pool/lxd/containers/disco 195M 26.1G 648M pool/lxd/containers/disco-shiftfs 194M 26.1G 649M
Well, that’s special, so on stop it suddenly decides to sync back to reality.
@brauner any luck playing with this in a test VM to see what may tickle zfs into counting writes multiple time?
Not yet, but will soon!
I finally figured out a way to more easily demonstrate the issue. This is on a newly installed virtual test system.
NAME USED AVAIL REFER pool/lxd/containers/disco 8.08M 9.14G 484M pool/lxd/containers/disco-shiftfs 5.82M 9.14G 487M
Create 1GB file with dd:
NAME USED AVAIL REFER pool/lxd/containers/disco 1.01G 7.14G 1.47G pool/lxd/containers/disco-shiftfs 1.01G 7.14G 1.48G
Remove the file:
NAME USED AVAIL REFER pool/lxd/containers/disco 8.09M 8.14G 484M pool/lxd/containers/disco-shiftfs 1.01G 8.14G 1.48G
Stop the containers:
NAME USED AVAIL REFER pool/lxd/containers/disco 8.14M 9.14G 484M pool/lxd/containers/disco-shiftfs 3.99M 9.14G 484M