I wanted to suspend a vm, reboot the host and release it. I tried incus pause and then rebooted the host but the VM just started with a cold boot after the host came back.
Is there a way to suspend an incus instance so the host can be rebooted and then bring the instance back as it was before?
Having boot.autostart: false on the VM should prevent the resume on boot.
Thanks @stgraber. I am not concerned about the automated restart, I was more concerned about the fact that when the VM resumes, it doesn’t have its prior state but appears to cold boot.
I’ve looked into it a bit more and now realise that “pause” is a suspend-to-ram action and that state does not survive a reboot. I am looking for a “suspend to disk” (aka “hibernate”).
I think virsh managedsave is an example but I haven’t tried that (don’t have libvirt, only incus 
) but that’s what I am after…
I want to be able to “suspend” an incus VM to disk, power down the host, come back, boot the host, start the VM (resume from disk suspend) and carry on where I left off…
incus stop --stateful
Just tried that, it requires migration.stateful=true and it would seem that a VM with PCI passthrough won’t start when that is set. Does this mean that it isn’t possible to suspend a vm to disk if that vm has pci passthru?
Woud that be an Incus restriction or something to do with kvm/qemu ?
It’s a pretty core limitation of QEMU and most VM managers when passthrough is in use.
Basically for live migration or suspend to disk to work (same logic), you need to be able to capture all the runtime state. You can do that when the manager (QEMU) is in control of all devices, but you can’t do that when some state is stored inside of an external device, in this case your PCI device.
There are some tricks that exist for some of those, most notably some enterprise grade GPUs for cloud providers where they expose an special API for the VM managers to extract their own runtime state (memory, registers, …) to allow for live migration or suspend/resume using them, but that’s rather uncommon and not very standardized as far as I can tell.