I’ve a home grown orchestration system written in python which controls the lifecycle of instances for a distributed mahousive in memory database.
I current used rsync with lz4 to copy the project/role directory structures… Which is pretty fast and efficient.
Am thinking of bundling those as squashes but just wanted to check if they are directly mountable as incus configurable devices…
Do you mean squashfs? squashfs is read-only.
You could directly mount a host filesystem read-only, bypassing squashfs entirely. Maybe this is what you do today?
Or you could mount the squashfs on the host, and then share it into the container(s).
Hmm, I don’t remember if we have the logic to set up loop devices if the source of a disk device is a file rather than a block device.
Unpacking it and then sharing that works… But on some systems I’ve seen the use of overlayfs to allow changes to go to a separate directory on write.
So you mount your squashfs r/o on once place and then run a overlayfs that reads from squashes but writes to a second.
I just wondered if that was implemented in LXD/Incus shares… When I get the chance I’ll try it out.
Though from my experience with sharing just based sshfs mounts into instances it doesn’t work as a share…
No, we don’t support mounting overlayfs within Incus. That’s something that’s very common with application containers as their layered approach often requires overlayfs.
You could obviously do it yourself, basically get a .squashfs, mount it somewhere, say /mnt/foo/read-only, then create /mnt/foo/delta and /mnt/foo/read-write and set up overlayfs with /mnt/foo/read-only as the underlay, /mnt/foo/delta as the overlay and /mnt/foo/read-write as the mount point, then you can pass /mnt/foo/read-write as a disk to the container.
We may in time add some support for directly having Incus do something like that, but overlayfs has had some annoying limitations around combined use with zfs and with vfs idmap which would make it work pretty inconsistently as a feature of Incus.