I have migrate the VM Talos 20GB size, from HDD to SSD. the both used LVM Cluster
incus storage list
+------------------------+------------+-------------+---------+---------+
| NAME | DRIVER | DESCRIPTION | USED BY | STATE |
+------------------------+------------+-------------+---------+---------+
| pool-iso-and-templates | dir | | 4 | CREATED |
+------------------------+------------+-------------+---------+---------+
| pool-qnap-hdd-01 | lvmcluster | | 24 | CREATED |
+------------------------+------------+-------------+---------+---------+
| pool-qnap-ssd-01 | lvmcluster | | 0 | CREATED |
+------------------------+------------+-------------+---------+---------+
During migrate we have an issue messages:
Instance move to destination failed on source:
Failed migration on source: Failed transferring snapshot disk:
Failed adding NBD device: Failed adding block device:
Monitor command timed out: "blockdev-add" after 5s
Failed waiting for migration control connection on source: context deadline exceeded
And process canceled. My Plan, I will migration vm with storages 3 TB. before I am doing that, we test migrate small size and have an issue like messages mentioned.
Those timeouts are a very recent addition to Incus and there to handle a number of QMP deadlocks and similar issues.
It’s certainly odd that 5s isn’t enough time to setup the new blockdev overlay though.
The actual operation here is asynchronous, so we’re not expecting any data to be transferred during those 5 seconds, just QEMU to have setup the built-in NBD server to access the data.
a. VM state: the VM was running during the move (live storage migration).
b. Reproducibility: tried the migration 3 times with the same 20GB running VM — fails identically every time with Monitor command timed out: "blockdev-add" after 5s.
c. Version: Incus 7.2 (Zabbly stable), package version:
d. Important data point: launching a fresh VM directly on the SSD pool works fine — no delays, boots normally. So the new pool itself looks healthy; the issue seems specific to the NBD/blockdev path used by live storage migration.
e. Storage context: both pools are lvmcluster with sanlock (lvmlockd) on iSCSI LUNs from the same QNAP, same iSCSI target — the SSD pool is a newly added LUN on that existing target. Single path per LUN via multipathd. Since sanlock lease acquisition on LV activation involves I/O to the lockspace, I wonder if that’s what occasionally pushes the blockdev-add setup past the 5s window.
Happy to test a patched build or gather QMP/debug logs if useful.
$ incus storage volume move pool-qnap-hdd-01/trim-vm-stg-t1-ms-sql-s21-dev-disk2 pool-qnap-ssd-01/trim-vm-stg-t1-ms-sql-s21-dev-disk2
Error: Create custom volume from copy failed: [Volume is not attached to running instance Error copying from migration connection to "/dev/dm-18": context canceled]
$ for i in `seq 2 4`; do incus storage volume copy pool-qnap-hdd-01/trim-vm-stg-t1-ms-sql-s21-dev-disk$i pool-qnap-ssd-01/trim-vm-stg-t1-ms-sql-s21-dev-disk$i; done
Error: Create custom volume from copy failed: [Error copying "/dev/nbd0" to migration connection: context canceled Volume is not attached to running instance]