Is there any support planned for using firecracker with incus?
Specific use case: I’d like to be able to set up nfs shares between containers, but it appears that due to current kernel limitations of nfs and namespaces, that isn’t possible without interacting with the host or using fuse. The current alternative is to boot multiple full vms using the qemu backend and use nfs that way.
I have not looked too much into what this would require in incus source yet, but current thought is use a single default kernel image (possibly configurable per microvm) and the same btrfs subvolume as used for containers to boot a firecracker vm of a container image. That would give a lighter-weight option than qemu for tasks that need full separation from the host, but don’t really need a full bios boot, and this also gives the combination of btrfs subvolume with the separation of a vm.
I might start poking around to see how much work this would entail, adding a third type of MicroVM which uses the Container image. Has any work already been done on this front, or does it seem like just using qemu would be a better solution?
This is not currently planned. In theory I’m not opposed to having a separate dedicated instance type (alongside container and VM) for something like that but adding one of those can be quite a lot of work.
The separate instance type is basically required in this case as a lot of what we offer for containers just won’t work when a VM layer like firecracker is present.
That would affect device passthrough the most but also require a different way of pulling live information from the guest (likely needing running something like incus-agent as we do our VMs), causing changes to how we can attach/spawn processes inside of the instance, what kind of networking options are available, … basically creating a hybrid between our current VMs and containers.
Then there’s also the extra infrastructure that would be needed to provide the kernel images for this on all architectures we care about and some work on shim logic so our normal container images behave in such an environment as we wouldn’t want to have to publish a new microvm version of every one of our images, but instead rely on the normal container images with some kind of shim logic that makes the VM environment look more like a normal container so the container images work unmodified.