I’m looking for a way to automatically create a volume when deploying a container, similar to how the root disk is created. This would help store important data in a separate volume, similar to Docker.
Currently, volume creation exists, but it’s manual, requiring the creation of the volume and its assignment to the container. I explored the lxops tool, which achieves this, but it uses its own CLI and doesn’t integrate with the Incus API.
I tried configuring the profile with the following:
devices:
data:
path: /data
pool: default
type: disk
However, I expected to see:
zfs list
default/containers/u1
default/containers/u1/data
Since this doesn’t work, is there any plan to add automatic volume creation in future updates? This feature would greatly simplify container management, especially with OCI image support.
It’s something we’ve been thinking about on and off but haven’t come up with a satisfying design.
Having a custom storage volume get created as part of instance creation then brings up a bunch of potential issues around, what do you do when an instance gets snapshotted or backed up, what do you do when the instance gets deleted, what if another instance now starts using that volume too, what about instance copy/migration, …
I suspect it’s something we will eventually address, possibly with a different volume type which is effectively tracked as an extension of the instance, can’t be attached to anything else, … but that will come with quite a lot of complexity for all the remaining cases above.
Yeah, I’ve been doing that stuff through Tofu in my case, works pretty well as it then keeps track of everything and appropriately destroys what’s needed.
But we have some work around importing .ova type images and the like which can have multiple disks. We can certainly do it with the current logic but some of the awkwardness around volume ownership will creep up there.
I’m curious how exactly to set up automatic volume creation and management via Tofu. If possible, could you share a sample configuration or a description of the process? I’m especially interested in how situations are handled when a container is deleted or associated data needs to be cleaned up.
About New volume type
The idea of a new volume type that is bound to a container and inaccessible to others sounds really promising. Perhaps such functionality could include automatic deletion of volumes when the container is deleted, or an option to exclude them from snapshots and backups. Do you think this could simplify the implementation and remove some of the complexity you mentioned?
In addition to the topic of volume creation, I’d like to propose another feature for Incus: incus rebuild --all. This command could update all containers, significantly simplifying management in cases of large-scale changes.
Regarding automatic volume creation, it might be worth considering an approach similar to root disk creation. For instance, a second root disk with a unique name could be created, like this:
zfs list
default/containers/u1
default/containers/u1/data-u1
default/containers/u2
default/containers/u2/data-u2
Each volume would be associated only with its respective container, solving the issue of volumes being used by other containers. While this solution currently works only with ZFS (and possibly BTRFS?), it could be extended to other storage pools.
I understand this is just a concept, and its implementation might be challenging. However, adding features like rebuild --all and automatic volume creation could be a major step forward. Similar functionality already exists in Docker and tools like lxops.
I believe it’s worth exploring the creation of such an advanced and in-demand technology, which would greatly simplify Incus container management.
Just following up on this. Any thoughts or suggestions on the proposed features?
Would it be possible to include such functionality in the Incus roadmap? If so, do you have any estimation on when it might be considered or implemented?
Thank you very much for your time and consideration!
All proposed solutions so far come with major complexity when considering all the different storage features we have to support. The reality is that this isn’t currently a priority for us, so I wouldn’t really expect to see movement on this for another year or so.
As mentioned above, we do have plans to add OVA image importing for VMs which will have us internally create custom volumes for a new VM, that may evolve into having a more formal relationship between instance and those volumes, which could one day evolve into having multi-disk instances but it’s going to take a while and there will be many bumps along that road.