Unattended upgrades and container migrations

Hi all,

I’m running Incus 7.0 LTS. I have a couple of questions:

  1. Unattended Upgrades: Is it safe and reliable to enable automatic updates on LTS versions, or is there a risk of structural changes breaking containers? Since it’s an LTS, I assume only bug and security fixes are pushed (avoiding breaking changes), but security patches are high priority.

  2. Live Migration: How does live migration work in Incus?

    • Does it support system containers, or VMs only?

    • Is it possible between two independent nodes with no shared storage (network only)?

Thanks!

Incus 7.0 LTS gets five years of support. Keep backups before installing updates.

VMs support live migration with migration.stateful=true. Container migration is more limited, but Incus 7.0.2 offers a faster option using incus move --refresh, with a short stop during the final transfer.

You can move instances between separate servers without clustering, provided their storage and network settings are compatible.

References:

That’s not true in incus: new features are quite aggressively back-ported to the LTS branch. I have not been bitten by it in the past, but I’d say it is a risk.

Another thing to beware of is that the zabbly packages between point releases (e.g. 1:7.0.2-ubuntu26.04-202610301448) have no corresponding git tags in the main source tree. AFAIK it’s handled via cherry-picking patches in the zabbly/incus repo, but that’s not tagged either.

However, as long as there are no changes to the database schema, it’s safe to roll back. I presume this is why the OCI container functionality was not back-ported to 6.0 LTS.

It’s really just VMs. In theory, live migration of containers is possible with CRIU, but it has so many limitations and caveats (e.g. your container cannot have a network interface!) that in real scenarios it never works.

However, incus 7.0.2 has recently back-ported the feature for “near-live” migration, where it will repeatedly send incremental storage updates until just before the actual cutover, for a very short off/on cycle. And that should work between independent nodes with no shared storage. I’ve used this approach manually in the past, using incus copy --refresh multiple times followed by a stop on the source, a final refresh, then a start on the destination

As for VMs: I have never tried live migration without shared storage, nor between independent nodes, but I don’t see any documented limitation which says this shouldn’t work.

Actually OCI didn’t need any database change and could have been backported to 6.0 LTS.
Same goes with storage drivers like Linstor and TrueNAS, they didn’t have any schema changes either and could have been backported.

But we don’t quite backport everything to LTS. We tend to backport all security (obviously) and bugfixes, then on the feature side, we usually backport most easy/isolated improvements, so things like new config keys, new options on existing configuration keys, new CLI tools and easy standalone or debug APIs.

We do however refrain from anything too significant or that would pull in new large dependencies which is why new storage drivers or new runtime backends (like for OCI) don’t get backported. And then as you mentioned, anything that needs database schema changes gets excluded from LTS backports, so things like the network address sets for example weren’t pushed to 6.0 LTS.

Currently we’ve not had any huge new feature or anything requiring a database change (though we have one coming soon on the DB front), so the LTS updates for 7.0 have been mostly identical to monthly releases, just pushed out a few weeks/months later and so generally less bumpy.