Migrate failed, vm in pool but not instance list, how to remove from pool?

Incus
1

Apparently you can define an agent disk with source: agent:config as part of a profile and incus will accept it, but when you import a VM onto that profile it will fail.

As a result of the failure, the VM is not in the instance list, but it is listed as a vm in the storage pool.

However:

  • its not a custom volume, so I can’t remove it with incus storage volume delete
  • its not in the instance list, so I can’t remove it with incus delete
  • I can’t import the VM to a fixed profile because a VM with the same name exists in the storage pool.

Halp? The pool is ZFS.

2

What was the error you got during the initial import?

3

I’d have to recreate the conditions to get the exact message, but something about failing to add the device agent.

The failure happened after the raw img has been imported, hence the stuck state I’m currently in.

Right now it fails with:

Error: Failed sending filesystem volume: Failed to rsync: exit status 12 rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(232) [sender=3.4.1]

Which is not very descriptive but I eventually figured out is because the VM already exists in the pool but not the instance list, so it passes the check for a unique instance name but then fails when importing the disk.

4

❯ incus profile show test

config: {}
description: Test Profile
devices:
  agent:
    pool: incus-zfs
    source: agent:config
    type: disk
name: test
used_by: []
project: default

> incus-migrate

Instance to be created:
  Name: OpenWRT-Test
  Project: default
  Type: virtual-machine
  Source: openwrt-24.10.0-x86-64-generic-ext4-combined-efi.img
  Source format: raw
  Profiles:
  - test
  Config:
    security.secureboot: "false"

Please pick one of the options above [default=1]:
Error: Failed creating instance record: Failed initializing instance: Failed add validation for device "agent": Failed loading custom volume: Storage volume not found

Result: Its not in either incus list or incus storage volume list incus-zfs.

So its not the agent disk that caused the broken import, although its weird the I can incus profile device add test agent disk pool=incus-zfs source=agent:config like I would to an instance and it will accept it but then fail on import.

I removed the agent disk and added my OpenWRT type=block persistent storage custom volume with:

> incus profile device remove test agent
> incus storage volume attach-profile incus-zfs openwrt_opt test
> incus profile show test

config: {}
description: Test Profile
devices:
  openwrt_opt:
    pool: incus-zfs
    source: openwrt_opt
    type: disk
name: test
used_by: []
project: default

Import:

Instance to be created:
  Name: OpenWRT-Test
  Project: default
  Type: virtual-machine
  Source: openwrt-24.10.0-x86-64-generic-ext4-combined-efi.img
  Source format: raw
  Profiles:
  - test
  Storage pool: incus-zfs
  Config:
    security.secureboot: "false"

Please pick one of the options above [default=1]:
Transferring instance: OpenWRT-Test: 113.91MB (28.47MB/s)Error: Failed to prepare device "openwrt_opt" for migration: Not supported

Once again, incus list doesn’t show the instance, but:
> incus storage volume list incus-zfs shows:

+-----------------+------------------------------------------------------------------+----------------------------+--------------+---------+
|      TYPE       |                               NAME                               |        DESCRIPTION         | CONTENT-TYPE | USED BY |
+-----------------+------------------------------------------------------------------+----------------------------+--------------+---------+
<snip>
+-----------------+------------------------------------------------------------------+----------------------------+--------------+---------+
| virtual-machine | OpenWRT-24-10-0-25-02-12                                         |                            | block        | 1       |
+-----------------+------------------------------------------------------------------+----------------------------+--------------+---------+
| virtual-machine | OpenWRT-Test                                                     |                            | block        | 1       |
+-----------------+------------------------------------------------------------------+----------------------------+--------------+---------+

So now I have 2 instances that exist in pool but not in instance list and cannot be removed.

5

You may need some DB surgery in this case, something like incus admin sql global "DELETE FROM storage_volumes WHERE name='OpenWRT-Test'" or similar.

The import failure though is very odd and definitely caused a failure in an unexpected spot.
Could you file a bug at GitHub · Where software is built? The content of your last comment should do very well as a bug report for us to reproduce and fix this.

6

Done: