gber
February 18, 2025, 7:07am
1
I am unable to mount a host directory into a VM:
$ incus create images:debian/bookworm/cloud testvm --vm
$ incus config device add testvm shared disk source=/home/debian/shared path=/mnt/shared
$ incus start testvm
Inside the VM /mnt/shared has not been mounted and I can see the following in dmesg:
[ 2.811251] virtio-fs: tag <incus_shared> not found
[ 2.812627] 9pnet_virtio: no channels available for device incus_shared
Manually mounting it leads to the same result:
# mount -t virtiofs incus_shared /mnt/shared/
mount: /mnt/shared: wrong fs type, bad option, bad superblock on incus_shared, missing codepage or helper program, or other error.
dmesg(1) may have more information after failed mount system call.
[ 224.892681] virtio-fs: tag <incus_shared> not found
On the host I am using incus 6.0.3-2~bpo12+1 from backports on Debian bookworm and I’ve tried both Debian bookworm and Ubuntu noble VMs with the same result.
stgraber
February 18, 2025, 4:06pm
2
Do you see virtiofsd running on the host system?
gber
February 18, 2025, 4:53pm
3
No, it is not running although virtiofsd/qemu-system-common is installed.
stgraber
February 18, 2025, 5:18pm
4
Okay, so that’s the problem.incus monitor --pretty while you issue the incus start, that will give you a bunch of debug output which will hopefully include why virtiofsd isn’t starting up.
gber
February 19, 2025, 7:24am
5
I can’t spot anything here:
Summary
DEBUG [2025-02-19T06:11:52+01:00] Handling API request ip=@ method=GET protocol=tls url="/1.0?project=user-1000" username=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13
DEBUG [2025-02-19T06:11:52+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-19T06:11:52+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-19T06:11:52+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-19T06:11:52+01:00] Handling API request ip=@ method=GET protocol=tls url="/1.0/instances/testvm?project=user-1000" username=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13
DEBUG [2025-02-19T06:11:52+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-19T06:11:52+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-19T06:11:52+01:00] Handling API request ip=@ method=GET protocol=tls url="/1.0/events?project=user-1000" username=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13
DEBUG [2025-02-19T06:11:52+01:00] Event listener server handler started id=b39f5543-846a-4ab0-a6fd-6a828b182a09 local=/var/lib/incus/unix.socket remote=@
DEBUG [2025-02-19T06:11:52+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-19T06:11:52+01:00] Handling API request ip=@ method=PUT protocol=tls url="/1.0/instances/testvm/state?project=user-1000" username=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13
DEBUG [2025-02-19T06:11:52+01:00] Start started instance=testvm instanceType=virtual-machine project=user-1000 stateful=false
DEBUG [2025-02-19T06:11:52+01:00] Started operation class=task description="Starting instance" operation=89a5a2c2-6d82-4ccb-8170-ac13b3fe9e7b project=user-1000
DEBUG [2025-02-19T06:11:52+01:00] New operation class=task description="Starting instance" operation=89a5a2c2-6d82-4ccb-8170-ac13b3fe9e7b project=user-1000
DEBUG [2025-02-19T06:11:52+01:00] Instance operation lock created action=start instance=testvm project=user-1000 reusable=false
DEBUG [2025-02-19T06:11:52+01:00] MountInstance started driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-19T06:11:52+01:00] Handling API request ip=@ method=GET protocol=tls url="/1.0/operations/89a5a2c2-6d82-4ccb-8170-ac13b3fe9e7b?project=user-1000" username=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13
DEBUG [2025-02-19T06:11:52+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-19T06:11:52+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-19T06:11:52+01:00] Activated ZFS volume dev=data/incus/virtual-machines/user-1000_testvm.block driver=zfs pool=default volName=user-1000_testvm
DEBUG [2025-02-19T06:11:52+01:00] MountInstance finished driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-19T06:11:52+01:00] Mounted ZFS dataset dev=data/incus/virtual-machines/user-1000_testvm driver=zfs path=/var/lib/incus/storage-pools/default/virtual-machines/user-1000_testvm pool=default volName=user-1000_testvm
DEBUG [2025-02-19T06:11:52+01:00] Installing incus-agent installPath=/var/lib/incus/virtual-machines/user-1000_testvm/config/incus-agent instance=testvm instanceType=virtual-machine project=user-1000 srcPath=/usr/libexec/incus/incus-agent
DEBUG [2025-02-19T06:11:52+01:00] Generating NVRAM instance=testvm instanceType=virtual-machine project=user-1000
DEBUG [2025-02-19T06:11:52+01:00] Starting device device=eth0 instance=testvm instanceType=virtual-machine project=user-1000 type=nic
DEBUG [2025-02-19T06:11:52+01:00] Starting device device=shared instance=testvm instanceType=virtual-machine project=user-1000 type=disk
DEBUG [2025-02-19T06:11:52+01:00] Starting device device=root instance=testvm instanceType=virtual-machine project=user-1000 type=disk
DEBUG [2025-02-19T06:11:52+01:00] Starting QEMU command="[forklimits limit=memlock:unlimited:unlimited fd=3 fd=4 -- /usr/bin/qemu-system-x86_64 -S -name testvm -uuid 9fb6025f-e4a7-49c0-a12d-5eef893e2144 -daemonize -cpu host,hv_passthrough -nographic -serial chardev:console -nodefaults -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=allow,resourcecontrol=deny -readconfig /run/incus/user-1000_testvm/qemu.conf -spice unix=on,disable-ticketing=on,addr=/run/incus/user-1000_testvm/qemu.spice -pidfile /run/incus/user-1000_testvm/qemu.pid -D /var/log/incus/user-1000_testvm/qemu.log -smbios type=2,manufacturer=LinuxContainers,product=Incus -runas nobody]" instance=testvm instanceType=virtual-machine project=user-1000
DEBUG [2025-02-19T06:11:52+01:00] UpdateInstanceBackupFile started driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-19T06:11:52+01:00] UpdateInstanceBackupFile finished driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-19T06:11:52+01:00] Skipping unmount as in use driver=zfs pool=default refCount=1 volName=user-1000_testvm
DEBUG [2025-02-19T06:11:52+01:00] QMP monitor started path=/run/incus/user-1000_testvm/qemu.monitor
DEBUG [2025-02-19T06:11:52+01:00] Start finished instance=testvm instanceType=virtual-machine project=user-1000 stateful=false
DEBUG [2025-02-19T06:11:52+01:00] Success for operation class=task description="Starting instance" operation=89a5a2c2-6d82-4ccb-8170-ac13b3fe9e7b project=user-1000
DEBUG [2025-02-19T06:11:52+01:00] Instance operation lock finished action=start err="<nil>" instance=testvm project=user-1000 reusable=false
DEBUG [2025-02-19T06:11:52+01:00] Event listener server handler stopped listener=b39f5543-846a-4ab0-a6fd-6a828b182a09 local=/var/lib/incus/unix.socket remote=@
DEBUG [2025-02-19T06:12:03+01:00] Instance operation lock created action=restart instance=testvm project=user-1000 reusable=false
DEBUG [2025-02-19T06:12:03+01:00] onStop hook started instance=testvm instanceType=virtual-machine project=user-1000 target=reboot
DEBUG [2025-02-19T06:12:03+01:00] Instance initiated stop action=reboot instance=testvm instanceType=virtual-machine project=user-1000
DEBUG [2025-02-19T06:12:03+01:00] Instance stopped instance=testvm instanceType=virtual-machine project=user-1000 reason=guest-reset target=reboot
DEBUG [2025-02-19T06:12:03+01:00] Waiting for VM process to finish instance=testvm instanceType=virtual-machine project=user-1000
DEBUG [2025-02-19T06:12:03+01:00] QMP monitor stopped path=/run/incus/user-1000_testvm/qemu.monitor
DEBUG [2025-02-19T06:12:03+01:00] VM process finished instance=testvm instanceType=virtual-machine project=user-1000
DEBUG [2025-02-19T06:12:03+01:00] Stopping device device=shared instance=testvm instanceType=virtual-machine project=user-1000 type=disk
DEBUG [2025-02-19T06:12:03+01:00] Stopping device device=root instance=testvm instanceType=virtual-machine project=user-1000 type=disk
DEBUG [2025-02-19T06:12:03+01:00] Stopping device device=eth0 instance=testvm instanceType=virtual-machine project=user-1000 type=nic
DEBUG [2025-02-19T06:12:03+01:00] UnmountInstance started driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-19T06:12:03+01:00] Unmounted ZFS dataset dev=data/incus/virtual-machines/user-1000_testvm driver=zfs path=/var/lib/incus/storage-pools/default/virtual-machines/user-1000_testvm pool=default volName=user-1000_testvm
DEBUG [2025-02-19T06:12:03+01:00] UnmountInstance finished driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-19T06:12:03+01:00] Deactivated ZFS volume dev=data/incus/virtual-machines/user-1000_testvm.block driver=zfs pool=default volName=user-1000_testvm
DEBUG [2025-02-19T06:12:03+01:00] Instance operation lock inherited action=restart inheritedByAction=start instance=testvm project=user-1000 reusable=false
DEBUG [2025-02-19T06:12:03+01:00] Start started instance=testvm instanceType=virtual-machine project=user-1000 stateful=false
DEBUG [2025-02-19T06:12:03+01:00] MountInstance started driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-19T06:12:04+01:00] Activated ZFS volume dev=data/incus/virtual-machines/user-1000_testvm.block driver=zfs pool=default volName=user-1000_testvm
DEBUG [2025-02-19T06:12:04+01:00] MountInstance finished driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-19T06:12:04+01:00] Mounted ZFS dataset dev=data/incus/virtual-machines/user-1000_testvm driver=zfs path=/var/lib/incus/storage-pools/default/virtual-machines/user-1000_testvm pool=default volName=user-1000_testvm
DEBUG [2025-02-19T06:12:04+01:00] Skipping incus-agent install as unchanged installPath=/var/lib/incus/virtual-machines/user-1000_testvm/config/incus-agent instance=testvm instanceType=virtual-machine project=user-1000 srcPath=/usr/libexec/incus/incus-agent
DEBUG [2025-02-19T06:12:04+01:00] Starting device device=eth0 instance=testvm instanceType=virtual-machine project=user-1000 type=nic
DEBUG [2025-02-19T06:12:04+01:00] Starting device device=shared instance=testvm instanceType=virtual-machine project=user-1000 type=disk
DEBUG [2025-02-19T06:12:04+01:00] Starting device device=root instance=testvm instanceType=virtual-machine project=user-1000 type=disk
DEBUG [2025-02-19T06:12:04+01:00] Starting QEMU command="[forklimits limit=memlock:unlimited:unlimited fd=3 fd=4 -- /usr/bin/qemu-system-x86_64 -S -name testvm -uuid 9fb6025f-e4a7-49c0-a12d-5eef893e2144 -daemonize -cpu host,hv_passthrough -nographic -serial chardev:console -nodefaults -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=allow,resourcecontrol=deny -readconfig /run/incus/user-1000_testvm/qemu.conf -spice unix=on,disable-ticketing=on,addr=/run/incus/user-1000_testvm/qemu.spice -pidfile /run/incus/user-1000_testvm/qemu.pid -D /var/log/incus/user-1000_testvm/qemu.log -smbios type=2,manufacturer=LinuxContainers,product=Incus -runas nobody]" instance=testvm instanceType=virtual-machine project=user-1000
DEBUG [2025-02-19T06:12:04+01:00] UpdateInstanceBackupFile started driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-19T06:12:04+01:00] Skipping unmount as in use driver=zfs pool=default refCount=1 volName=user-1000_testvm
DEBUG [2025-02-19T06:12:04+01:00] UpdateInstanceBackupFile finished driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-19T06:12:04+01:00] QMP monitor started path=/run/incus/user-1000_testvm/qemu.monitor
DEBUG [2025-02-19T06:12:04+01:00] Start finished instance=testvm instanceType=virtual-machine project=user-1000 stateful=false
DEBUG [2025-02-19T06:12:04+01:00] onStop hook finished instance=testvm instanceType=virtual-machine project=user-1000 target=reboot
DEBUG [2025-02-19T06:12:04+01:00] Instance operation lock finished action=restart err="<nil>" instance=testvm project=user-1000 reusable=false
DEBUG [2025-02-19T06:12:14+01:00] Instance agent started instance=testvm instanceType=virtual-machine project=user-1000
DEBUG [2025-02-19T06:12:14+01:00] Sending request to Incus etag= method=PUT url="https://custom.socket/1.0"
DEBUG [2025-02-19T06:12:14+01:00] Connecting to a VM agent over a VM socket
DEBUG [2025-02-19T06:12:14+01:00]
{
"cid": 2,
"port": 5149,
"certificate": "-----BEGIN CERTIFICATE-----\nMIIB+DCCAX2gAwIBAgIQb9ablfkU4sWgLeJn+8+n0DAKBggqhkjOPQQDAzAvMRkw\nFwYDVQQKExBMaW51eCBDb250YWluZXJzMRIwEAYDVQQDDAlyb290QHdvcHIwHhcN\nMjQwNTI2MDg1NTQ1WhcNMzQwNTI0MDg1NTQ1WjAvMRkwFwYDVQQKExBMaW51eCBD\nb250YWluZXJzMRIwEAYDVQQDDAlyb290QHdvcHIwdjAQBgcqhkjOPQIBBgUrgQQA\nIgNiAARz6ZVsA37vCHaxV74puRx7wPOv3R6Sd4brrLO92t3GZ0h9V1r09/3lh/48\np2T6n3XwFGxM27PGBWiqefOJxrsfoJCWL2OU7Tu1U8Jo+tMrlcMBsGK7H7zZhnu4\npy/Sx8GjXjBcMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATAM\nBgNVHRMBAf8EAjAAMCcGA1UdEQQgMB6CBHdvcHKHBH8AAAGHEAAAAAAAAAAAAAAA\nAAAAAAEwCgYIKoZIzj0EAwMDaQAwZgIxAKooMM/r8SpCv3lUATFLmUIGJz9xsxrK\nhCMpRACXjXUhzDrzjPpVPq1ImTZdtYJZQgIxAIBWo32cLspcOoUTvff3YTsokf3n\nCdq8mp5YAGqPFX/FylK1AUHk6bGlvfmw0DC+9w==\n-----END CERTIFICATE-----\n",
"dev_incus": true
}
DEBUG [2025-02-19T06:12:14+01:00] Matched trusted cert fingerprint=0 subject="CN=root@wopr,O=Linux Containers"
But disk.shared.log contains this error:
fuse: unknown option(s): `--cache=never'
stgraber
February 19, 2025, 3:32pm
6
Ah, so you’re on a pretty old virtiofsd version, maybe one predating the rust rewrite?virtiofsd package which is currently only in testing and unstable as far as I can tell.
@gibmat is that something that could be backported too? The old virtiofsd is a completely different codebase, so supporting both may get tricky.
gibmat
February 19, 2025, 4:38pm
7
I think it would be pretty hard to get backported, for three reasons:
Like Go, Rust packages can have a ton of dependencies that would also require backporting. From a discussion on d-backports last August, one of the DDs who does rust packaging thinks that rust is essentially unbackportable .
The backported version of rust-virtiofsd would conflict with qemu-system-common’s /usr/lib/qemu/virtiofsd in bookworm. I don’t know how this would be handled; a simple conflict against qemu-system-common would prevent qemu’s use, which kind of defeats the purpose of the backport.
We’re ~six months from the trixie release, which means there’d be a pretty limited window of usefulness before backporting work would lose its value. bookworm backports will also be receiving fewer updates as the trixie freezes start in the next couple of months.
stgraber
February 19, 2025, 8:06pm
8
Right, so the best bet there may be to use cargo and build virtiofsd directly, putting it in /usr/local/bin/ and Incus should then favor that over the older one.
gber
February 20, 2025, 8:37am
9
Still not working for me, I see that virtiofsd is now running:
/usr/bin/virtiofsd --fd=3 --cache=never -o source=/var/lib/incus/devices/user-1000_testvm/disk.shared.home-debian-shared
disk.shared.log:
virtio_session_mount: Waiting for vhost-user socket connection...
incus monitor --pretty doesn’t show any errors:
DEBUG [2025-02-20T09:34:37+01:00] Event listener server handler started id=69f4e815-9465-4771-a824-b1e97dfb7b0e local=/var/lib/incus/unix.socket remote=@
DEBUG [2025-02-20T09:34:39+01:00] Handling API request ip=@ method=GET protocol=tls url="/1.0?project=user-1000" username=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13
DEBUG [2025-02-20T09:34:39+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-20T09:34:39+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-20T09:34:39+01:00] Handling API request ip=@ method=GET protocol=tls url="/1.0/instances/testvm?project=user-1000" username=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13
DEBUG [2025-02-20T09:34:39+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-20T09:34:39+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-20T09:34:39+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-20T09:34:39+01:00] Handling API request ip=@ method=GET protocol=tls url="/1.0/events?project=user-1000" username=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13
DEBUG [2025-02-20T09:34:39+01:00] Event listener server handler started id=7c9df969-e92a-4c65-92ec-d97f67fc944b local=/var/lib/incus/unix.socket remote=@
DEBUG [2025-02-20T09:34:39+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-20T09:34:39+01:00] Handling API request ip=@ method=PUT protocol=tls url="/1.0/instances/testvm/state?project=user-1000" username=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13
DEBUG [2025-02-20T09:34:39+01:00] Start started instance=testvm instanceType=virtual-machine project=user-1000 stateful=false
DEBUG [2025-02-20T09:34:39+01:00] New operation class=task description="Starting instance" operation=ad38fc71-672f-4ad7-b53e-82a58a36c21e project=user-1000
DEBUG [2025-02-20T09:34:39+01:00] Started operation class=task description="Starting instance" operation=ad38fc71-672f-4ad7-b53e-82a58a36c21e project=user-1000
DEBUG [2025-02-20T09:34:39+01:00] Instance operation lock created action=start instance=testvm project=user-1000 reusable=false
DEBUG [2025-02-20T09:34:39+01:00] MountInstance started driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-20T09:34:39+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-20T09:34:39+01:00] Handling API request ip=@ method=GET protocol=tls url="/1.0/operations/ad38fc71-672f-4ad7-b53e-82a58a36c21e?project=user-1000" username=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13
DEBUG [2025-02-20T09:34:39+01:00] Matched trusted cert fingerprint=d2f1d8a45e8b192a26a32d4a2178b258b20e126fa5b1264b0681acb1b8805d13 subject="CN=root@wopr,O=Linux Containers"
DEBUG [2025-02-20T09:34:40+01:00] Activated ZFS volume dev=data/incus/virtual-machines/user-1000_testvm.block driver=zfs pool=default volName=user-1000_testvm
DEBUG [2025-02-20T09:34:40+01:00] MountInstance finished driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-20T09:34:40+01:00] Mounted ZFS dataset dev=data/incus/virtual-machines/user-1000_testvm driver=zfs path=/var/lib/incus/storage-pools/default/virtual-machines/user-1000_testvm pool=default volName=user-1000_testvm
DEBUG [2025-02-20T09:34:40+01:00] Skipping incus-agent install as unchanged installPath=/var/lib/incus/virtual-machines/user-1000_testvm/config/incus-agent instance=testvm instanceType=virtual-machine project=user-1000 srcPath=/usr/libexec/incus/incus-agent
DEBUG [2025-02-20T09:34:40+01:00] Starting device device=eth0 instance=testvm instanceType=virtual-machine project=user-1000 type=nic
DEBUG [2025-02-20T09:34:40+01:00] Starting device device=root instance=testvm instanceType=virtual-machine project=user-1000 type=disk
DEBUG [2025-02-20T09:34:40+01:00] Starting device device=shared instance=testvm instanceType=virtual-machine project=user-1000 type=disk
DEBUG [2025-02-20T09:34:40+01:00] Starting QEMU command="[forklimits limit=memlock:unlimited:unlimited fd=3 fd=4 -- /usr/bin/qemu-system-x86_64 -S -name testvm -uuid 9fb6025f-e4a7-49c0-a12d-5eef893e2144 -daemonize -cpu host,hv_passthrough -nographic -serial chardev:console -nodefaults -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=allow,resourcecontrol=deny -readconfig /run/incus/user-1000_testvm/qemu.conf -spice unix=on,disable-ticketing=on,addr=/run/incus/user-1000_testvm/qemu.spice -pidfile /run/incus/user-1000_testvm/qemu.pid -D /var/log/incus/user-1000_testvm/qemu.log -smbios type=2,manufacturer=LinuxContainers,product=Incus -runas nobody]" instance=testvm instanceType=virtual-machine project=user-1000
DEBUG [2025-02-20T09:34:40+01:00] UpdateInstanceBackupFile started driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-20T09:34:40+01:00] Skipping unmount as in use driver=zfs pool=default refCount=1 volName=user-1000_testvm
DEBUG [2025-02-20T09:34:40+01:00] UpdateInstanceBackupFile finished driver=zfs instance=testvm pool=default project=user-1000
DEBUG [2025-02-20T09:34:40+01:00] QMP monitor started path=/run/incus/user-1000_testvm/qemu.monitor
DEBUG [2025-02-20T09:34:40+01:00] Start finished instance=testvm instanceType=virtual-machine project=user-1000 stateful=false
DEBUG [2025-02-20T09:34:40+01:00] Instance operation lock finished action=start err="<nil>" instance=testvm project=user-1000 reusable=false
DEBUG [2025-02-20T09:34:40+01:00] Success for operation class=task description="Starting instance" operation=ad38fc71-672f-4ad7-b53e-82a58a36c21e project=user-1000
DEBUG [2025-02-20T09:34:40+01:00] Event listener server handler stopped listener=7c9df969-e92a-4c65-92ec-d97f67fc944b local=/var/lib/incus/unix.socket remote=@
DEBUG [2025-02-20T09:34:51+01:00] Instance agent started instance=testvm instanceType=virtual-machine project=user-1000
DEBUG [2025-02-20T09:34:51+01:00]
{
"cid": 2,
"port": 5149,
"certificate": "-----BEGIN CERTIFICATE-----\nMIIB+DCCAX2gAwIBAgIQb9ablfkU4sWgLeJn+8+n0DAKBggqhkjOPQQDAzAvMRkw\nFwYDVQQKExBMaW51eCBDb250YWluZXJzMRIwEAYDVQQDDAlyb290QHdvcHIwHhcN\nMjQwNTI2MDg1NTQ1WhcNMzQwNTI0MDg1NTQ1WjAvMRkwFwYDVQQKExBMaW51eCBD\nb250YWluZXJzMRIwEAYDVQQDDAlyb290QHdvcHIwdjAQBgcqhkjOPQIBBgUrgQQA\nIgNiAARz6ZVsA37vCHaxV74puRx7wPOv3R6Sd4brrLO92t3GZ0h9V1r09/3lh/48\np2T6n3XwFGxM27PGBWiqefOJxrsfoJCWL2OU7Tu1U8Jo+tMrlcMBsGK7H7zZhnu4\npy/Sx8GjXjBcMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATAM\nBgNVHRMBAf8EAjAAMCcGA1UdEQQgMB6CBHdvcHKHBH8AAAGHEAAAAAAAAAAAAAAA\nAAAAAAEwCgYIKoZIzj0EAwMDaQAwZgIxAKooMM/r8SpCv3lUATFLmUIGJz9xsxrK\nhCMpRACXjXUhzDrzjPpVPq1ImTZdtYJZQgIxAIBWo32cLspcOoUTvff3YTsokf3n\nCdq8mp5YAGqPFX/FylK1AUHk6bGlvfmw0DC+9w==\n-----END CERTIFICATE-----\n",
"dev_incus": true
}
DEBUG [2025-02-20T09:34:51+01:00] Connecting to a VM agent over a VM socket
DEBUG [2025-02-20T09:34:51+01:00] Sending request to Incus etag= method=PUT url="https://custom.socket/1.0"
DEBUG [2025-02-20T09:34:52+01:00] Matched trusted cert fingerprint=0 subject="CN=root@wopr,O=Linux Containers"
But dmesg in testvm:
[ 3.298193] virtio-fs: tag <incus_shared> not found
[ 3.302882] 9pnet_virtio: no channels available for device incus_shared
stgraber
February 20, 2025, 4:14pm
10
That’s very weird…
stgraber@dakara:~$ incus create images:debian/bookworm/cloud testvm --vm
Creating testvm
stgraber@dakara:~$ incus config device add testvm shared disk source=/home/stgraber/Desktop/ path=/mnt/shared
Device shared added to testvm
stgraber@dakara:~$ incus start testvm
stgraber@dakara:~$ incus exec testvm bash
root@testvm:~# grep shared /proc/mounts
incus_shared /mnt/shared virtiofs rw,relatime 0 0
gber
February 20, 2025, 4:29pm
11
Just did exactly the same.
In the guest:
[ 2.944453] 9p: Installing v9fs 9p2000 file system support
[ 3.252654] virtio-fs: tag <incus_shared> not found
[ 3.259809] 9pnet_virtio: no channels available for device incus_shared
root@testvm:~# grep shared /proc/mounts
On the host:
# ps -ef | grep '[v]irtiofsd'
nobody 509871 207232 0 17:21 ? 00:00:00 /usr/bin/virtiofsd --fd=3 --cache=never -o source=/var/lib/incus/devices/user-1000_testvm/disk.shared.mnt-shared
nobody 509873 509871 0 17:21 ? 00:00:00 /usr/bin/virtiofsd --fd=3 --cache=never -o source=/var/lib/incus/devices/user-1000_testvm/disk.shared.mnt-shared
stgraber
February 20, 2025, 5:23pm
12
Ah, you’re in one of the per-user restricted projects, I wonder if there’s some bad logic in that situation…
Can you file an issue at GitHub · Where software is built ?
gber
February 21, 2025, 12:26pm
13
Indeed it works if I create the VM as root. And it even works with the virtiofsd shipping with qemu in Debian with minor adjustments:
diff --git a/internal/server/device/device_utils_disk.go b/internal/server/device/device_utils_disk.go
index c8da81b77..0f4c285a5 100644
--- a/internal/server/device/device_utils_disk.go
+++ b/internal/server/device/device_utils_disk.go
@@ -380,15 +380,15 @@ func DiskVMVirtiofsdStart(execPath string, inst instance.Instance, socketPath st
switch cacheOption {
case "metadata":
- cacheOption = "metadata"
+ cacheOption = "auto"
case "unsafe":
cacheOption = "always"
default:
- cacheOption = "never"
+ cacheOption = "none"
}
// Start the virtiofsd process in non-daemon mode.
- args := []string{"--fd=3", fmt.Sprintf("--cache=%s", cacheOption), "-o", fmt.Sprintf("source=%s", sharePath)}
+ args := []string{"--fd=3", "-o", fmt.Sprintf("cache=%s", cacheOption), "-o", fmt.Sprintf("source=%s", sharePath)}
proc, err := subprocess.NewProcess(cmd, args, logPath, logPath)
if err != nil {
return nil, nil, err
@gibmat Could you add this to the backport?
OK, will do
gber
February 22, 2025, 7:16am
14
gibmat
February 24, 2025, 2:43am
15
It looks straightforward enough; I’ll review the changes a bit closer and test on my end, and if things look good I’ll add it to the backport.
T_Sclly
March 13, 2025, 2:08am
16