Let’s do a repro (a test case that reproduces the problem)!
I launch an Arch Linux VM in Incus, then in that VM I installed incus and created an Alpine VM (VM nesting). While doing this, I got an error.
In the following I managed to reproduce the following error.
Error: Failed setting up device via monitor: Failed adding block device for disk device “root”: Failed adding block device: Failed to find file descriptor with matching flags=0x4002
$ incus launch images:archlinux/cloud archlinux --vm
Launching archlinux
Error: Failed instance creation: The image used by this instance is incompatible with secureboot. Please set security.secureboot=false on the instance
$ incus config set archlinux security.secureboot=false
$ incus start archlinux
$ incus shell archlinux
Error: VM agent isn't currently running
$ incus shell archlinux
Error: VM agent isn't currently running
$ incus shell archlinux
[root@archlinux ~]# pacman -Syu
:: Synchronizing package databases...
core is up to date
...
Starting full system upgrade...
there is nothing to do
[root@archlinux ~]# pacman -S incus
...
(43/80) installing qemu-base
Optional dependencies for qemu-base !!!
qemu-user-static: for static user mode emulation of QEMU targets
samba: for SMB/CIFS server support
qemu-block-gluster: for Gluster block driver
qemu-block-iscsi: for iSCSI block driver
qemu-chardev-baum: for Baum chardev driver
qemu-docs: for documentation
qemu-emulators-full: for all system emulators
qemu-full: for a full QEMU installation
...
[root@archlinux ~]# incus admin init
Error: Failed to connect to local daemon: Get "http://unix.socket/1.0": dial unix /var/lib/incus/unix.socket: connect: no such file or directory
[root@archlinux ~]# systemctl start incus
[root@archlinux ~]# incus list
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+
[root@archlinux ~]# incus admin init
Would you like to use clustering? (yes/no) [default=no]:
Do you want to configure a new storage pool? (yes/no) [default=yes]:
Name of the new storage pool [default=default]:
Where should this storage pool store its data? [default=/var/lib/incus/storage-pools/default]:
Would you like to create a new local network bridge? (yes/no) [default=yes]:
What should the new bridge be called? [default=incusbr0]:
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
Would you like the server to be available over the network? (yes/no) [default=no]:
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]:
Would you like a YAML "init" preseed to be printed? (yes/no) [default=no]: yes
config: {}
networks:
- config:
ipv4.address: auto
ipv6.address: auto
description: ""
name: incusbr0
type: ""
project: default
storage_pools:
- config: {}
description: ""
name: default
driver: dir
profiles:
- config: {}
description: ""
devices:
eth0:
name: eth0
network: incusbr0
type: nic
root:
path: /
pool: default
type: disk
name: default
projects: []
cluster: null
[root@archlinux ~]# incus launch --vm images:alpine/edge/cloud alpine
Launching alpine
Error: Failed instance creation: Failed creating instance record: Failed initializing instance: Invalid config: No uid/gid allocation configured. In this mode, only privileged containers are supported
[root@archlinux ~]# cat /etc/sub?id
arch:100000:65536
arch:100000:65536
[root@archlinux ~]# vi /etc/subuid
[root@archlinux ~]# vi /etc/subgid
[root@archlinux ~]# cat /etc/sub?id
root:100000:65536
arch:100000:65536
root:100000:65536
arch:100000:65536
[root@archlinux ~]# incus launch --vm images:alpine/edge/cloud alpine
Launching alpine
Error: Failed instance creation: Failed creating instance record: Failed initializing instance: Invalid config: No uid/gid allocation configured. In this mode, only privileged containers are supported
[root@archlinux ~]# systemctl restart incus
[root@archlinux ~]# incus launch --vm images:alpine/edge/cloud alpine
Launching alpine
Error: Failed instance creation: The image used by this instance is incompatible with secureboot. Please set security.secureboot=false on the instance
[root@archlinux ~]# incus config set alpine security.secureboot=false
[root@archlinux ~]# incus start alpine
Error: Failed setting up device via monitor: Failed adding block device for disk device "root": Failed adding block device: Failed to find file descriptor with matching flags=0x4002
Try `incus info --show-log alpine` for more info
[root@archlinux ~]# incus info --show-log alpine
Name: alpine
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2024/09/18 12:13 UTC
Last Used: 1970/01/01 00:00 UTC
Log:
[root@archlinux ~]#
The Arch packaging for Incus does not add the necessary subuid/subgid, therefore they need to be added manually.
If you notice in my earlier terminal commands, I diligently kept all error messages as an educational instrument on how to deal with issues that may arise. Presumably, if someone looks into the commands and their output, they can get an idea on how something is done, and how to rectify errors without having to write explicitly about it.
I just looked into the Arch Wiki, a high quality wiki, and it does have a page for Incus. Also, high quality content. It mentions the issue with subuid, subgid, along with nice troubleshooting. Incus - ArchWiki
Only the first one is needed to get most QEMU 9.1 users to behave, the rest is to clean that up a bit and avoid other deprecation warnings.
Note that as I mentioned last night, there is an issue with QEMU 9.1 and live migration which leaves any live-migrated VM with a hung CPU0. Single CPU VMs therefore just die on migration while multi-CPU VMs will usually respond but with a very upset kernel and hung tasks…
I’ve not had time to try to figure out what’s going on there, but that’s why we’re still using QEMU 9.0 in our tests and in my packages.
I checked our reproducible example. I can’t reproduce the issue anymore, so it seems that the patches that made it into 6.5-2 fixed the issue with qemu 9.1.
