I was looking for a way to have loopback mounted storage start empty as a sparse file and grow as needed, but also shrink when you remove files from the upper layer.
I tested this with a sparse ext4 formatted file mounted with losetup (mount loopback). It starts very small (About 4M for the basically empty filesystem) and grows as you add files to it on the mountpoint. Then I deleted files, nothing happened, but when I ran fstrim on that loopback mountpoint, it did shrink the underlay file again to about 4M. I tried mounting with discard directly and got an error, not sure why, but fstrim works, which is good enough I think. Seems like it works well, the loopback device can pass discard operations to the underlying file which, if sparse, will shrink again. This is perfect.
Can Incus take advantage of this when using file based storage?
The way discard is handled varies a fair bit between drivers.
ZFS should basically just work as it doesn’t use a regular loop device, but you may need to kick its trimming or auto-trim logic.
LVM normally forwards discard all the way through, so I’d expect that one to work fine unless something at the loop level is preventing the discards from being passed through to the filesystem.
btrfs would indeed need to be mounted with discard, which I thought we were doing, but similarly to LVM, something at the loop device level may interfere with it.
I only tested this manually with ext4 in a loopback to prove the concept. My reasoning was:
I create and destroy a lot of containers constantly, so I like the filesystems that can help accelerating those tasks
I like to use ext4 as my main OS filesystem, and I don’t like thin providioned LVM (had huge problems with it in the past) so that means using either btrfs or zfs on loopback over ext4
I would like to create a giant loopback file and have it expand and shrink dinamically without worrying that it will take a lot of space unnecessarily after some time
I currently have btrfs as my main filesystem because I wanted to avoid that problem from the start, but I want to move away from it and back to ext4, so I wondered if this shrinking feature was already working with some of the storage backends and I did not realize it.
Looks like incus issues discard with VM. but not containers.
Anyway you can use systemd fstrim timer or cron jobs to do this on idle.
LVM thin pool does not support forward discard to underlying loop device.