How can I passthrough my SATA controller to a VM?

Hello,

This is my first post to this forum. I’ve read the incus guide and I’ve been trying since yesterday migrate my home server from Proxmox to Incus.

I’d like to passthrough a SATA controller to a VM in order to use it as a NAS server. The thing is I don’t know what more can I do to make it work.

Here follow the information I believe is relevant:

cat /proc/cmdline

BOOT_IMAGE=/boot/vmlinuz-linux-lts root=UUID=3954b3b2-67f2-45af-8124-a440163b1c03 rw zswap.enabled=0 rootfstype=ext4 loglevel=3 quiet intel_iommu=on iommu=pt pcie_port_pm=off

cat /etc/modprobe.d/vfio.conf

options vfio-pci ids=1b21:1166
softdep ahci pre: vfio-pci
cat /etc/modprobe.d/blacklist.conf

blacklist ahci
03:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1166 Serial ATA Controller [1b21:1166] (rev 02)
        Subsystem: ASMedia Technology Inc. Device [1b21:2116]
        Kernel driver in use: vfio-pci

That was the command I used to add the sata controller to the vm:

incus config device add nas asm1166 pci address=0000:03:00.0
cat /run/incus/nas/qemu.conf

# PCI card ("asm1166" device)
[device "dev-incus_asm1166"]
addr = "00.0"
multifunction = "on"
host = "0000:03:00.0"
driver = "vfio-pci"
bus = "qemu_pcie5"

Thing is as soon as I start the vm all incus setup freezes. I can interact with the host normally but no incus commands work so that I can’t even access the vm in order to see what is happening.

By the way, I have 3 zfs disks connected to this sata port that I’ll import to use in this vm.

Could anyone tell me what I’m doing wrong and point me in the right direction?

Can you run dmesg to see if the kernel is mad about something in particular when things get stuck?

Hi,

Thanks for your interest and prompt reply.

In fact I noticed msgs related to the disks. But it’s very odd because it worked both with Proxmox the day before yesterday and also worked when I configured the zpool on the host (I first thought of using the host itself as a nfs server, then I changed my mind and decided to run it in a vm and passthrough the controller).

The following messages happen on the host when I stop and then start again the vm.

After a long while the vm starts and there’s no more msgs on the host.

[ 9891.945740] device offline error, dev sda, sector 8 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0                                                                                         
[ 9891.945757] Buffer I/O error on dev sda, logical block 1, async page read                                                                                                                   
[ 9891.945793] sd 0:0:0:0: [sda] Attached SCSI disk                                                                                                                                            
[ 9892.016242] sd 0:0:0:0: [sda] Synchronizing SCSI cache                                                                                                                                      
[ 9892.017025] ata1.00: Entering standby power mode                                                                                                                                            
[ 9892.223221] ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)                                                                                                                          
[ 9892.224510] ata3.00: ATA-10: WDC WD60EFPX-68C5ZN0, 81.00A81, max UDMA/133                                                                                                                   
[ 9892.225971] ata3.00: 11721045168 sectors, multi 16: LBA48 NCQ (depth 32), AA                                                                                                                
[ 9892.225982] ata3.00: Features: NCQ-prio                                                                                                                                                     
[ 9892.227841] ata3.00: configured for UDMA/133
[ 9892.228176] scsi 2:0:0:0: Direct-Access     ATA      WDC WD60EFPX-68C 0A81 PQ: 0 ANSI: 5
[ 9892.229130] sd 2:0:0:0: [sda] 11721045168 512-byte logical blocks: (6.00 TB/5.46 TiB)
[ 9892.229142] sd 2:0:0:0: [sda] 4096-byte physical blocks
[ 9892.229274] sd 2:0:0:0: [sda] Write Protect is off
[ 9892.229328] sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 9892.230014] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 9892.230091] sd 2:0:0:0: [sda] Preferred minimum I/O size 4096 bytes
[ 9892.271157] device offline error, dev sdb, sector 8 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 9892.271175] Buffer I/O error on dev sdb, logical block 1, async page read
[ 9892.271265] sd 1:0:0:0: [sdb] Attached SCSI disk
[ 9892.319530] sd 1:0:0:0: [sdb] Synchronizing SCSI cache
[ 9892.320421] ata2.00: Entering standby power mode
[ 9892.540991] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 9892.542067] ata4.00: ATA-10: WDC WD80EFZZ-68BTXN0, 81.00A81, max UDMA/133
[ 9892.544676] ata4.00: 15628053168 sectors, multi 16: LBA48 NCQ (depth 32), AA
[ 9892.549100] ata4.00: configured for UDMA/133
[ 9892.549479] scsi 3:0:0:0: Direct-Access     ATA      WDC WD80EFZZ-68B 0A81 PQ: 0 ANSI: 5
[ 9892.550481] sd 3:0:0:0: [sdb] 15628053168 512-byte logical blocks: (8.00 TB/7.28 TiB)
[ 9892.550507] sd 3:0:0:0: [sdb] 4096-byte physical blocks
[ 9892.550554] sd 3:0:0:0: [sdb] Write Protect is off
[ 9892.550562] sd 3:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[ 9892.550603] sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 9892.550695] sd 3:0:0:0: [sdb] Preferred minimum I/O size 4096 bytes
[ 9892.578671] device offline error, dev sda, sector 8 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 9892.578678] Buffer I/O error on dev sda, logical block 1, async page read
[ 9892.578718] sd 2:0:0:0: [sda] Attached SCSI disk
[ 9892.646183] sd 2:0:0:0: [sda] Synchronizing SCSI cache
[ 9892.647131] ata3.00: Entering standby power mode
[ 9892.859650] ata5: SATA link down (SStatus 0 SControl 300)
[ 9892.920279] device offline error, dev sdb, sector 8 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 9892.920302] Buffer I/O error on dev sdb, logical block 1, async page read
[ 9892.920429] sd 3:0:0:0: [sdb] Attached SCSI disk
[ 9892.966175] sd 3:0:0:0: [sdb] Synchronizing SCSI cache
[ 9892.967151] ata4.00: Entering standby power mode
[ 9893.166158] ata6: SATA link down (SStatus 0 SControl 300)
[ 9895.141910] br0: port 2(tap23ed64b3) entered blocking state
[ 9895.141916] br0: port 2(tap23ed64b3) entered forwarding state

I’d really appreciate any suggestions.

Are those two WD drives attached to the controller you’re trying to move or to something else?

I’m trying to pass all drives attached to the sata controller. In fact there are 3 6Tb disks, not 2 (a raidz1) and a 8 Tb. I believe I didn’t pasted the whole dmesg.

It’s so weird, because as soon as the vm effectively starts (and it’s taking more than 10 min) everything works ok.

On the host side (the sde is the flash drive I used to install and nvme0n1 is the Proxmox installation that I haven’t removed yet):

NAME        FSTYPE     FSVER            LABEL       UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sde         iso9660    Joliet Extension ARCH_202507 2025-07-01-17-34-06-00                              
├─sde1      iso9660    Joliet Extension ARCH_202507 2025-07-01-17-34-06-00                              
└─sde2      vfat       FAT32            ARCHISO_EFI 6864-1C0E                                           
zd0         zfs_member 5000             zroot       11021497591369907639                                
zd16                                                                                                    
zram0       swap       1                zram0       4d77fc18-073f-49ad-9f0f-baa05405746c                [SWAP]
nvme1n1                                                                                                 
├─nvme1n1p1 vfat       FAT32            EFI         49A6-5B05                            1007,3M     1% /boot/efi
├─nvme1n1p2 ext4       1.0              root        2845b3b2-67f2-45af-8124-a440163b1c03   73,8G    10% /
└─nvme1n1p3 zfs_member 5000             incus-zfs   2545950505813272211                                 
nvme0n1                                                                                                 
├─nvme0n1p1                                                                                             
├─nvme0n1p2 vfat       FAT32                        5146-A3B7                                           
└─nvme0n1p3 zfs_member 5000             rpool       4007981877909047391

On the vm side:

 lsblk -f
NAME   FSTYPE     FSVER LABEL   UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sda                                                                                 
├─sda1 zfs_member 5000  storage 14411323417404523349                                
└─sda9                                                                              
sdb                                                                                 
├─sdb1 zfs_member 5000  storage 14411323417404523349                                
└─sdb9                                                                              
sdc                                                                                 
├─sdc1 zfs_member 5000  storage 14411323417404523349                                
└─sdc9                                                                              
sdd                                                                                 
├─sdd1 zfs_member 5000  bkps    13972754217946913792                                
└─sdd9                                                                              
sde                                                                                 
├─sde1 vfat       FAT32 UEFI    9FB5-59BE                              98,1M     0% /boot/efi
└─sde2 ext4       1.0   rootfs  8e961594-7c30-4079-b294-aee2458b6ffb    6,5G    32% /
 zpool status
  pool: bkps
 state: ONLINE
  scan: scrub repaired 0B in 16:51:59 with 0 errors on Sun Aug 10 17:16:00 2025
config:

        NAME                                    STATE     READ WRITE CKSUM
        bkps                                    ONLINE       0     0     0
          ata-WDC_WD80EFZZ-68BTXN0_WD-CA102ZNK  ONLINE       0     0     0

errors: No known data errors

  pool: storage
 state: ONLINE
  scan: resilvered 384K in 00:00:00 with 0 errors on Mon Aug 18 13:33:56 2025
config:

        NAME                                          STATE     READ WRITE CKSUM
        storage                                       ONLINE       0     0     0
          raidz1-0                                    ONLINE       0     0     0
            ata-WDC_WD60EFPX-68C5ZN0_WD-WX12DA3J07F6  ONLINE       0     0     0
            ata-WDC_WD60EFPX-68C5ZN0_WD-WX22DC322KP6  ONLINE       0     0     0
            ata-WDC_WD60EFPX-68C5ZN0_WD-WX22DC322SZ6  ONLINE       0     0     0

errors: No known data errors

Well, this was my first Incus installation. I’m not that familiarized with zfs. I don’t doubt that I may have done something stupid.

The host should really never show any log messages related to your drives if the driver was correctly tied to vfio-pci from the start.

So that suggests that your modprobe.d configuration isn’t working as intended.

Can you show the output of incus query /1.0/resources on the host following a fresh boot and prior to starting up the VM? That should show us all devices and what drives they’re bound to as well as the IOMMU groups.

Here you are (all the following info was collected with the vm turned off):

lspci -nnk

03:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1166 Serial ATA Controller [1b21:1166] (rev 02)
        Subsystem: ASMedia Technology Inc. Device [1b21:2116]
        Kernel driver in use: ahci

incus list

+-----------+---------+------------------------------+--------------------------+-----------------+-----------+
|   NAME    |  STATE  |             IPV4             |           IPV6           |      TYPE       | SNAPSHOTS |
+-----------+---------+------------------------------+--------------------------+-----------------+-----------+
| omv       | STOPPED |                              |                          | VIRTUAL-MACHINE | 0         |
+-----------+---------+------------------------------+--------------------------+-----------------+-----------+

incus query /1.0/resources

{
        "cpu": {
                "architecture": "x86_64",
                "sockets": [
                        {
                                "address_sizes": {
                                        "physical_bits": 39,
                                        "virtual_bits": 48
                                },
                                "cache": [
                                        {
                                                "level": 1,
                                                "size": 32768,
                                                "type": "Data"
                                        },
                                        {
                                                "level": 1,
                                                "size": 65536,
                                                "type": "Instruction"
                                        },
                                        {
                                                "level": 2,
                                                "size": 2097152,
                                                "type": "Unified"
                                        },
                                        {
                                                "level": 3,
                                                "size": 6291456,
                                                "type": "Unified"
                                        }
                                ],
                                "cores": [
                                        {
                                                "core": 0,
                                                "die": 0,
                                                "flags": [
                                                        "fpu",
                                                        "vme",
                                                        "de",
                                                        "pse",
                                                        "tsc",
                                                        "msr",
                                                        "pae",
                                                        "mce",
                                                        "cx8",
                                                        "apic",
                                                        "sep",
                                                        "mtrr",
                                                        "pge",
                                                        "mca",
                                                        "cmov",
                                                        "pat",
                                                        "pse36",
                                                        "clflush",
                                                        "dts",
                                                        "acpi",
                                                        "mmx",
                                                        "fxsr",
                                                        "sse",
                                                        "sse2",
                                                        "ss",
                                                        "ht",
                                                        "tm",
                                                        "pbe",
                                                        "syscall",
                                                        "nx",
                                                        "pdpe1gb",
                                                        "rdtscp",
                                                        "lm",
                                                        "constant_tsc",
                                                        "art",
                                                        "arch_perfmon",
                                                        "pebs",
                                                        "bts",
                                                        "rep_good",
                                                        "nopl",
                                                        "xtopology",
                                                        "nonstop_tsc",
                                                        "cpuid",
                                                        "aperfmperf",
                                                        "tsc_known_freq",
                                                        "pni",
                                                        "pclmulqdq",
                                                        "dtes64",
                                                        "monitor",
                                                        "ds_cpl",
                                                        "vmx",
                                                        "est",
                                                        "tm2",
                                                        "ssse3",
                                                        "sdbg",
                                                        "fma",
                                                        "cx16",
                                                        "xtpr",
                                                        "pdcm",
                                                        "pcid",
                                                        "sse4_1",
                                                        "sse4_2",
                                                        "x2apic",
                                                        "movbe",
                                                        "popcnt",
                                                        "tsc_deadline_timer",
                                                        "aes",
                                                        "xsave",
                                                        "avx",
                                                        "f16c",
                                                        "rdrand",
                                                        "lahf_lm",
                                                        "abm",
                                                        "3dnowprefetch",
                                                        "cpuid_fault",
                                                        "epb",
                                                        "cat_l2",
                                                        "cdp_l2",
                                                        "ssbd",
                                                        "ibrs",
                                                        "ibpb",
                                                        "stibp",
                                                        "ibrs_enhanced",
                                                        "tpr_shadow",
                                                        "flexpriority",
                                                        "ept",
                                                        "vpid",
                                                        "ept_ad",
                                                        "fsgsbase",
                                                        "tsc_adjust",
                                                        "bmi1",
                                                        "avx2",
                                                        "smep",
                                                        "bmi2",
                                                        "erms",
                                                        "invpcid",
                                                        "rdt_a",
                                                        "rdseed",
                                                        "adx",
                                                        "smap",
                                                        "clflushopt",
                                                        "clwb",
                                                        "intel_pt",
                                                        "sha_ni",
                                                        "xsaveopt",
                                                        "xsavec",
                                                        "xgetbv1",
                                                        "xsaves",
                                                        "split_lock_detect",
                                                        "user_shstk",
                                                        "avx_vnni",
                                                        "dtherm",
                                                        "ida",
                                                        "arat",
                                                        "pln",
                                                        "pts",
                                                        "hwp",
                                                        "hwp_notify",
                                                        "hwp_act_window",
                                                        "hwp_epp",
                                                        "hwp_pkg_req",
                                                        "vnmi",
                                                        "umip",
                                                        "pku",
                                                        "ospke",
                                                        "waitpkg",
                                                        "gfni",
                                                        "vaes",
                                                        "vpclmulqdq",
                                                        "rdpid",
                                                        "movdiri",
                                                        "movdir64b",
                                                        "fsrm",
                                                        "md_clear",
                                                        "serialize",
                                                        "arch_lbr",
                                                        "ibt",
                                                        "flush_l1d",
                                                        "arch_capabilities"
                                                ],
                                                "frequency": 2900,
                                                "threads": [
                                                        {
                                                                "id": 0,
                                                                "isolated": false,
                                                                "numa_node": 0,
                                                                "online": true,
                                                                "thread": 0
                                                        }
                                                ]
                                        },
                                        {
                                                "core": 1,
                                                "die": 0,
                                                "flags": [
                                                        "fpu",
                                                        "vme",
                                                        "de",
                                                        "pse",
                                                        "tsc",
                                                        "msr",
                                                        "pae",
                                                        "mce",
                                                        "cx8",
                                                        "apic",
                                                        "sep",
                                                        "mtrr",
                                                        "pge",
                                                        "mca",
                                                        "cmov",
                                                        "pat",
                                                        "pse36",
                                                        "clflush",
                                                        "dts",
                                                        "acpi",
                                                        "mmx",
                                                        "fxsr",
                                                        "sse",
                                                        "sse2",
                                                        "ss",
                                                        "ht",
                                                        "tm",
                                                        "pbe",
                                                        "syscall",
                                                        "nx",
                                                        "pdpe1gb",
                                                        "rdtscp",
                                                        "lm",
                                                        "constant_tsc",
                                                        "art",
                                                        "arch_perfmon",
                                                        "pebs",
                                                        "bts",
                                                        "rep_good",
                                                        "nopl",
                                                        "xtopology",
                                                        "nonstop_tsc",
                                                        "cpuid",
                                                        "aperfmperf",
                                                        "tsc_known_freq",
                                                        "pni",
                                                        "pclmulqdq",
                                                        "dtes64",
                                                        "monitor",
                                                        "ds_cpl",
                                                        "vmx",
                                                        "est",
                                                        "tm2",
                                                        "ssse3",
                                                        "sdbg",
                                                        "fma",
                                                        "cx16",
                                                        "xtpr",
                                                        "pdcm",
                                                        "pcid",
                                                        "sse4_1",
                                                        "sse4_2",
                                                        "x2apic",
                                                        "movbe",
                                                        "popcnt",
                                                        "tsc_deadline_timer",
                                                        "aes",
                                                        "xsave",
                                                        "avx",
                                                        "f16c",
                                                        "rdrand",
                                                        "lahf_lm",
                                                        "abm",
                                                        "3dnowprefetch",
                                                        "cpuid_fault",
                                                        "epb",
                                                        "cat_l2",
                                                        "cdp_l2",
                                                        "ssbd",
                                                        "ibrs",
                                                        "ibpb",
                                                        "stibp",
                                                        "ibrs_enhanced",
                                                        "tpr_shadow",
                                                        "flexpriority",
                                                        "ept",
                                                        "vpid",
                                                        "ept_ad",
                                                        "fsgsbase",
                                                        "tsc_adjust",
                                                        "bmi1",
                                                        "avx2",
                                                        "smep",
                                                        "bmi2",
                                                        "erms",
                                                        "invpcid",
                                                        "rdt_a",
                                                        "rdseed",
                                                        "adx",
                                                        "smap",
                                                        "clflushopt",
                                                        "clwb",
                                                        "intel_pt",
                                                        "sha_ni",
                                                        "xsaveopt",
                                                        "xsavec",
                                                        "xgetbv1",
                                                        "xsaves",
                                                        "split_lock_detect",
                                                        "user_shstk",
                                                        "avx_vnni",
                                                        "dtherm",
                                                        "ida",
                                                        "arat",
                                                        "pln",
                                                        "pts",
                                                        "hwp",
                                                        "hwp_notify",
                                                        "hwp_act_window",
                                                        "hwp_epp",
                                                        "hwp_pkg_req",
                                                        "vnmi",
                                                        "umip",
                                                        "pku",
                                                        "ospke",
                                                        "waitpkg",
                                                        "gfni",
                                                        "vaes",
                                                        "vpclmulqdq",
                                                        "rdpid",
                                                        "movdiri",
                                                        "movdir64b",
                                                        "fsrm",
                                                        "md_clear",
                                                        "serialize",
                                                        "arch_lbr",
                                                        "ibt",
                                                        "flush_l1d",
                                                        "arch_capabilities"
                                                ],
                                                "frequency": 2900,
                                                "threads": [
                                                        {
                                                                "id": 1,
                                                                "isolated": false,
                                                                "numa_node": 0,
                                                                "online": true,
                                                                "thread": 0
                                                        }
                                                ]
                                        },
                                        {
                                                "core": 2,
                                                "die": 0,
                                                "flags": [
                                                        "fpu",
                                                        "vme",
                                                        "de",
                                                        "pse",
                                                        "tsc",
                                                        "msr",
                                                        "pae",
                                                        "mce",
                                                        "cx8",
                                                        "apic",
                                                        "sep",
                                                        "mtrr",
                                                        "pge",
                                                        "mca",
                                                        "cmov",
                                                        "pat",
                                                        "pse36",
                                                        "clflush",
                                                        "dts",
                                                        "acpi",
                                                        "mmx",
                                                        "fxsr",
                                                        "sse",
                                                        "sse2",
                                                        "ss",
                                                        "ht",
                                                        "tm",
                                                        "pbe",
                                                        "syscall",
                                                        "nx",
                                                        "pdpe1gb",
                                                        "rdtscp",
                                                        "lm",
                                                        "constant_tsc",
                                                        "art",
                                                        "arch_perfmon",
                                                        "pebs",
                                                        "bts",
                                                        "rep_good",
                                                        "nopl",
                                                        "xtopology",
                                                        "nonstop_tsc",
                                                        "cpuid",
                                                        "aperfmperf",
                                                        "tsc_known_freq",
                                                        "pni",
                                                        "pclmulqdq",
                                                        "dtes64",
                                                        "monitor",
                                                        "ds_cpl",
                                                        "vmx",
                                                        "est",
                                                        "tm2",
                                                        "ssse3",
                                                        "sdbg",
                                                        "fma",
                                                        "cx16",
                                                        "xtpr",
                                                        "pdcm",
                                                        "pcid",
                                                        "sse4_1",
                                                        "sse4_2",
                                                        "x2apic",
                                                        "movbe",
                                                        "popcnt",
                                                        "tsc_deadline_timer",
                                                        "aes",
                                                        "xsave",
                                                        "avx",
                                                        "f16c",
                                                        "rdrand",
                                                        "lahf_lm",
                                                        "abm",
                                                        "3dnowprefetch",
                                                        "cpuid_fault",
                                                        "epb",
                                                        "cat_l2",
                                                        "cdp_l2",
                                                        "ssbd",
                                                        "ibrs",
                                                        "ibpb",
                                                        "stibp",
                                                        "ibrs_enhanced",
                                                        "tpr_shadow",
                                                        "flexpriority",
                                                        "ept",
                                                        "vpid",
                                                        "ept_ad",
                                                        "fsgsbase",
                                                        "tsc_adjust",
                                                        "bmi1",
                                                        "avx2",
                                                        "smep",
                                                        "bmi2",
                                                        "erms",
                                                        "invpcid",
                                                        "rdt_a",
                                                        "rdseed",
                                                        "adx",
                                                        "smap",
                                                        "clflushopt",
                                                        "clwb",
                                                        "intel_pt",
                                                        "sha_ni",
                                                        "xsaveopt",
                                                        "xsavec",
                                                        "xgetbv1",
                                                        "xsaves",
                                                        "split_lock_detect",
                                                        "user_shstk",
                                                        "avx_vnni",
                                                        "dtherm",
                                                        "ida",
                                                        "arat",
                                                        "pln",
                                                        "pts",
                                                        "hwp",
                                                        "hwp_notify",
                                                        "hwp_act_window",
                                                        "hwp_epp",
                                                        "hwp_pkg_req",
                                                        "vnmi",
                                                        "umip",
                                                        "pku",
                                                        "ospke",
                                                        "waitpkg",
                                                        "gfni",
                                                        "vaes",
                                                        "vpclmulqdq",
                                                        "rdpid",
                                                        "movdiri",
                                                        "movdir64b",
                                                        "fsrm",
                                                        "md_clear",
                                                        "serialize",
                                                        "arch_lbr",
                                                        "ibt",
                                                        "flush_l1d",
                                                        "arch_capabilities"
                                                ],
                                                "frequency": 2900,
                                                "threads": [
                                                        {
                                                                "id": 2,
                                                                "isolated": false,
                                                                "numa_node": 0,
                                                                "online": true,
                                                                "thread": 0
                                                        }
                                                ]
                                        },
                                        {
                                                "core": 3,
                                                "die": 0,
                                                "flags": [
                                                        "fpu",
                                                        "vme",
                                                        "de",
                                                        "pse",
                                                        "tsc",
                                                        "msr",
                                                        "pae",
                                                        "mce",
                                                        "cx8",
                                                        "apic",
                                                        "sep",
                                                        "mtrr",
                                                        "pge",
                                                        "mca",
                                                        "cmov",
                                                        "pat",
                                                        "pse36",
                                                        "clflush",
                                                        "dts",
                                                        "acpi",
                                                        "mmx",
                                                        "fxsr",
                                                        "sse",
                                                        "sse2",
                                                        "ss",
                                                        "ht",
                                                        "tm",
                                                        "pbe",
                                                        "syscall",
                                                        "nx",
                                                        "pdpe1gb",
                                                        "rdtscp",
                                                        "lm",
                                                        "constant_tsc",
                                                        "art",
                                                        "arch_perfmon",
                                                        "pebs",
                                                        "bts",
                                                        "rep_good",
                                                        "nopl",
                                                        "xtopology",
                                                        "nonstop_tsc",
                                                        "cpuid",
                                                        "aperfmperf",
                                                        "tsc_known_freq",
                                                        "pni",
                                                        "pclmulqdq",
                                                        "dtes64",
                                                        "monitor",
                                                        "ds_cpl",
                                                        "vmx",
                                                        "est",
                                                        "tm2",
                                                        "ssse3",
                                                        "sdbg",
                                                        "fma",
                                                        "cx16",
                                                        "xtpr",
                                                        "pdcm",
                                                        "pcid",
                                                        "sse4_1",
                                                        "sse4_2",
                                                        "x2apic",
                                                        "movbe",
                                                        "popcnt",
                                                        "tsc_deadline_timer",
                                                        "aes",
                                                        "xsave",
                                                        "avx",
                                                        "f16c",
                                                        "rdrand",
                                                        "lahf_lm",
                                                        "abm",
                                                        "3dnowprefetch",
                                                        "cpuid_fault",
                                                        "epb",
                                                        "cat_l2",
                                                        "cdp_l2",
                                                        "ssbd",
                                                        "ibrs",
                                                        "ibpb",
                                                        "stibp",
                                                        "ibrs_enhanced",
                                                        "tpr_shadow",
                                                        "flexpriority",
                                                        "ept",
                                                        "vpid",
                                                        "ept_ad",
                                                        "fsgsbase",
                                                        "tsc_adjust",
                                                        "bmi1",
                                                        "avx2",
                                                        "smep",
                                                        "bmi2",
                                                        "erms",
                                                        "invpcid",
                                                        "rdt_a",
                                                        "rdseed",
                                                        "adx",
                                                        "smap",
                                                        "clflushopt",
                                                        "clwb",
                                                        "intel_pt",
                                                        "sha_ni",
                                                        "xsaveopt",
                                                        "xsavec",
                                                        "xgetbv1",
                                                        "xsaves",
                                                        "split_lock_detect",
                                                        "user_shstk",
                                                        "avx_vnni",
                                                        "dtherm",
                                                        "ida",
                                                        "arat",
                                                        "pln",
                                                        "pts",
                                                        "hwp",
                                                        "hwp_notify",
                                                        "hwp_act_window",
                                                        "hwp_epp",
                                                        "hwp_pkg_req",
                                                        "vnmi",
                                                        "umip",
                                                        "pku",
                                                        "ospke",
                                                        "waitpkg",
                                                        "gfni",
                                                        "vaes",
                                                        "vpclmulqdq",
                                                        "rdpid",
                                                        "movdiri",
                                                        "movdir64b",
                                                        "fsrm",
                                                        "md_clear",
                                                        "serialize",
                                                        "arch_lbr",
                                                        "ibt",
                                                        "flush_l1d",
                                                        "arch_capabilities"
                                                ],
                                                "frequency": 2899,
                                                "threads": [
                                                        {
                                                                "id": 3,
                                                                "isolated": false,
                                                                "numa_node": 0,
                                                                "online": true,
                                                                "thread": 0
                                                        }
                                                ]
                                        }
                                ],
                                "frequency": 2899,
                                "frequency_minimum": 700,
                                "frequency_turbo": 3400,
                                "name": "Intel(R) N100",
                                "socket": 0,
                                "vendor": "GenuineIntel"
                        }
                ],
                "total": 4
        },
        "gpu": {
                "cards": [
                        {
                                "driver": "i915",
                                "driver_version": "6.12.42-1-lts",
                                "drm": {
                                        "card_device": "226:1",
                                        "card_name": "card1",
                                        "control_device": "226:1",
                                        "control_name": "controlD65",
                                        "id": 1,
                                        "render_device": "226:128",
                                        "render_name": "renderD128"
                                },
                                "numa_node": 0,
                                "pci_address": "0000:00:02.0",
                                "product": "Alder Lake-N [UHD Graphics]",
                                "product_id": "46d1",
                                "sriov": {
                                        "current_vfs": 0,
                                        "maximum_vfs": 7,
                                        "vfs": null
                                },
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086"
                        }
                ],
                "total": 1
        },
        "load": {
                "Average10Min": 0.12,
                "Average1Min": 1.37,
                "Average5Min": 0.36,
                "Processes": 334
        },
        "memory": {
                "hugepages_size": 2097152,
                "hugepages_total": 0,
                "hugepages_used": 0,
                "nodes": [
                        {
                                "hugepages_total": 0,
                                "hugepages_used": 0,
                                "numa_node": 0,
                                "total": 34225520640,
                                "used": 5397417984
                        }
                ],
                "total": 34225520640,
                "used": 2437464064
        },
        "network": {
                "cards": [
                        {
                                "driver": "igc",
                                "driver_version": "6.12.42-1-lts",
                                "firmware_version": "2017:888d",
                                "numa_node": 0,
                                "pci_address": "0000:04:00.0",
                                "ports": [
                                        {
                                                "address": "a8:b8:e0:06:aa:f9",
                                                "auto_negotiation": true,
                                                "id": "enp4s0",
                                                "link_detected": true,
                                                "link_duplex": "full",
                                                "link_speed": 1000,
                                                "port": 0,
                                                "port_type": "twisted pair",
                                                "protocol": "ethernet",
                                                "supported_modes": [
                                                        "10baseT/Half",
                                                        "10baseT/Full",
                                                        "100baseT/Half",
                                                        "100baseT/Full",
                                                        "1000baseT/Full",
                                                        "2500baseT/Full"
                                                ],
                                                "supported_ports": [
                                                        "twisted pair"
                                                ],
                                                "transceiver_type": "internal"
                                        }
                                ],
                                "product": "Ethernet Controller I226-V",
                                "product_id": "125c",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086"
                        },
                        {
                                "driver": "vfio-pci",
                                "driver_version": "6.12.42-1-lts",
                                "numa_node": 0,
                                "pci_address": "0000:05:00.0",
                                "product": "Ethernet Controller I226-V",
                                "product_id": "125c",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086"
                        }
                ],
                "total": 2
        },
        "pci": {
                "devices": [
                        {
                                "driver": "igen6_edac",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 1,
                                "numa_node": 0,
                                "pci_address": "0000:00:00.0",
                                "product": "Alder Lake-N Processor Host Bridge/DRAM Registers",
                                "product_id": "461c",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "i915",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 0,
                                "numa_node": 0,
                                "pci_address": "0000:00:02.0",
                                "product": "Alder Lake-N [UHD Graphics]",
                                "product_id": "46d1",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "xhci_hcd",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 2,
                                "numa_node": 0,
                                "pci_address": "0000:00:0d.0",
                                "product": "Alder Lake-N Thunderbolt 4 USB Controller",
                                "product_id": "464e",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "xhci_hcd",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 3,
                                "numa_node": 0,
                                "pci_address": "0000:00:14.0",
                                "product": "Alder Lake-N PCH USB 3.2 Gen 2x1 (10 Gb/s) xHCI Host Controller",
                                "product_id": "54ed",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "",
                                "driver_version": "",
                                "iommu_group": 3,
                                "numa_node": 0,
                                "pci_address": "0000:00:14.2",
                                "product": "Alder Lake-N PCH Shared SRAM",
                                "product_id": "54ef",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "mei_me",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 4,
                                "numa_node": 0,
                                "pci_address": "0000:00:16.0",
                                "product": "Alder Lake-N PCH HECI Controller",
                                "product_id": "54e0",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "intel-lpss",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 5,
                                "numa_node": 0,
                                "pci_address": "0000:00:19.0",
                                "product": "Alder Lake-N I2C Controller #4",
                                "product_id": "54c5",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "intel-lpss",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 5,
                                "numa_node": 0,
                                "pci_address": "0000:00:19.1",
                                "product": "Alder Lake-N I2C Controller #5",
                                "product_id": "54c6",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "sdhci-pci",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 6,
                                "numa_node": 0,
                                "pci_address": "0000:00:1a.0",
                                "product": "Alder Lake-N eMMC Controller",
                                "product_id": "54c4",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "pcieport",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 7,
                                "numa_node": 0,
                                "pci_address": "0000:00:1c.0",
                                "product": "Alder Lake-N PCI Express Root Port #1",
                                "product_id": "54b8",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "pcieport",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 8,
                                "numa_node": 0,
                                "pci_address": "0000:00:1c.1",
                                "product": "Alder Lake-N PCI Express Root Port #2",
                                "product_id": "54b9",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "pcieport",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 9,
                                "numa_node": 0,
                                "pci_address": "0000:00:1c.2",
                                "product": "Alder Lake-N PCI Express Root Port #3",
                                "product_id": "54ba",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "pcieport",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 10,
                                "numa_node": 0,
                                "pci_address": "0000:00:1c.3",
                                "product": "Alder Lake-N PCI Express Root Port #4",
                                "product_id": "54bb",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "pcieport",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 11,
                                "numa_node": 0,
                                "pci_address": "0000:00:1c.6",
                                "product": "Alder Lake-N PCI Express Root Port #7",
                                "product_id": "54be",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "",
                                "driver_version": "",
                                "iommu_group": 12,
                                "numa_node": 0,
                                "pci_address": "0000:00:1f.0",
                                "product": "Alder Lake-N PCH eSPI Controller",
                                "product_id": "5481",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "snd_hda_intel",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 12,
                                "numa_node": 0,
                                "pci_address": "0000:00:1f.3",
                                "product": "Alder Lake-N PCH High Definition Audio Controller",
                                "product_id": "54c8",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "i801_smbus",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 12,
                                "numa_node": 0,
                                "pci_address": "0000:00:1f.4",
                                "product": "Alder Lake-N SMBus",
                                "product_id": "54a3",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "intel-spi",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 12,
                                "numa_node": 0,
                                "pci_address": "0000:00:1f.5",
                                "product": "Alder Lake-N SPI (flash) Controller",
                                "product_id": "54a4",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "nvme",
                                "driver_version": "1.0",
                                "iommu_group": 13,
                                "numa_node": 0,
                                "pci_address": "0000:01:00.0",
                                "product": "NV2 NVMe SSD [SM2267XT] (DRAM-less)",
                                "product_id": "5017",
                                "vendor": "Kingston Technology Company, Inc.",
                                "vendor_id": "2646",
                                "vpd": {}
                        },
                        {
                                "driver": "nvme",
                                "driver_version": "1.0",
                                "iommu_group": 14,
                                "numa_node": 0,
                                "pci_address": "0000:02:00.0",
                                "product": "NV3 NVMe SSD [SM2268XT2] (DRAM-less)",
                                "product_id": "5028",
                                "vendor": "Kingston Technology Company, Inc.",
                                "vendor_id": "2646",
                                "vpd": {}
                        },
                        {
                                "driver": "ahci",
                                "driver_version": "3.0",
                                "iommu_group": 15,
                                "numa_node": 0,
                                "pci_address": "0000:03:00.0",
                                "product": "ASM1166 Serial ATA Controller",
                                "product_id": "1166",
                                "vendor": "ASMedia Technology Inc.",
                                "vendor_id": "1b21",
                                "vpd": {}
                        },
                        {
                                "driver": "igc",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 16,
                                "numa_node": 0,
                                "pci_address": "0000:04:00.0",
                                "product": "Ethernet Controller I226-V",
                                "product_id": "125c",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        },
                        {
                                "driver": "vfio-pci",
                                "driver_version": "6.12.42-1-lts",
                                "iommu_group": 17,
                                "numa_node": 0,
                                "pci_address": "0000:05:00.0",
                                "product": "Ethernet Controller I226-V",
                                "product_id": "125c",
                                "vendor": "Intel Corporation",
                                "vendor_id": "8086",
                                "vpd": {}
                        }
                ],
                "total": 23
        },
        "storage": {
                "disks": [
                        {
                                "block_size": 512,
                                "device": "259:4",
                                "device_id": "nvme-eui.00000000000000000026b7686528d845",
                                "device_path": "pci-0000:01:00.0-nvme-1",
                                "firmware_version": "SBM02103",
                                "id": "nvme0n1",
                                "model": "KINGSTON SNV2S1000G",
                                "numa_node": 0,
                                "partitions": [
                                        {
                                                "device": "259:5",
                                                "id": "nvme0n1p1",
                                                "partition": 1,
                                                "read_only": false,
                                                "size": 1031168
                                        },
                                        {
                                                "device": "259:6",
                                                "id": "nvme0n1p2",
                                                "partition": 2,
                                                "read_only": false,
                                                "size": 1073741824
                                        },
                                        {
                                                "device": "259:7",
                                                "id": "nvme0n1p3",
                                                "partition": 3,
                                                "read_only": false,
                                                "size": 999130078720
                                        }
                                ],
                                "pci_address": "0000:01:00.0",
                                "read_only": false,
                                "removable": false,
                                "rpm": 0,
                                "serial": "50026B7686528D84",
                                "size": 1000204886016,
                                "type": "nvme",
                                "wwn": "eui.00000000000000000026b7686528d845"
                        },
                        {
                                "block_size": 512,
                                "device": "259:0",
                                "device_id": "nvme-eui.00000000000000000026b768724c69b5",
                                "device_path": "pci-0000:02:00.0-nvme-1",
                                "firmware_version": "SDQ00103",
                                "id": "nvme1n1",
                                "model": "KINGSTON SNV3S1000G",
                                "numa_node": 0,
                                "partitions": [
                                        {
                                                "device": "259:1",
                                                "id": "nvme1n1p1",
                                                "partition": 1,
                                                "read_only": false,
                                                "size": 1073741824
                                        },
                                        {
                                                "device": "259:2",
                                                "id": "nvme1n1p2",
                                                "partition": 2,
                                                "read_only": false,
                                                "size": 95561973760
                                        },
                                        {
                                                "device": "259:3",
                                                "id": "nvme1n1p3",
                                                "partition": 3,
                                                "read_only": false,
                                                "size": 903567376384
                                        }
                                ],
                                "pci_address": "0000:02:00.0",
                                "read_only": false,
                                "removable": false,
                                "rpm": 0,
                                "serial": "50026B768724C69B",
                                "size": 1000204886016,
                                "type": "nvme",
                                "wwn": "eui.00000000000000000026b768724c69b5"
                        },
                        {
                                "block_size": 4096,
                                "device": "8:0",
                                "device_id": "wwn-0x50014ee21629a6b8",
                                "device_path": "pci-0000:03:00.0-ata-1.0",
                                "firmware_version": "81.00A81",
                                "id": "sda",
                                "model": "WDC WD60EFPX-68C5ZN0",
                                "numa_node": 0,
                                "partitions": [
                                        {
                                                "device": "8:1",
                                                "id": "sda1",
                                                "partition": 1,
                                                "read_only": false,
                                                "size": 6001165074432
                                        },
                                        {
                                                "device": "8:9",
                                                "id": "sda9",
                                                "partition": 9,
                                                "read_only": false,
                                                "size": 8388608
                                        }
                                ],
                                "read_only": false,
                                "removable": false,
                                "rpm": 5400,
                                "serial": "WD-WX22DC322KP6",
                                "size": 6001175126016,
                                "type": "sata"
                        },
                        {
                                "block_size": 4096,
                                "device": "8:16",
                                "device_id": "wwn-0x50014ee2161be494",
                                "device_path": "pci-0000:03:00.0-ata-2.0",
                                "firmware_version": "81.00A81",
                                "id": "sdb",
                                "model": "WDC WD60EFPX-68C5ZN0",
                                "numa_node": 0,
                                "partitions": [
                                        {
                                                "device": "8:17",
                                                "id": "sdb1",
                                                "partition": 1,
                                                "read_only": false,
                                                "size": 6001165074432
                                        },
                                        {
                                                "device": "8:25",
                                                "id": "sdb9",
                                                "partition": 9,
                                                "read_only": false,
                                                "size": 8388608
                                        }
                                ],
                                "read_only": false,
                                "removable": false,
                                "rpm": 5400,
                                "serial": "WD-WX12DA3J07F6",
                                "size": 6001175126016,
                                "type": "sata"
                        },
                        {
                                "block_size": 4096,
                                "device": "8:32",
                                "device_id": "wwn-0x50014ee2c0d5172b",
                                "device_path": "pci-0000:03:00.0-ata-3.0",
                                "firmware_version": "81.00A81",
                                "id": "sdc",
                                "model": "WDC WD60EFPX-68C5ZN0",
                                "numa_node": 0,
                                "partitions": [
                                        {
                                                "device": "8:33",
                                                "id": "sdc1",
                                                "partition": 1,
                                                "read_only": false,
                                                "size": 6001165074432
                                        },
                                        {
                                                "device": "8:41",
                                                "id": "sdc9",
                                                "partition": 9,
                                                "read_only": false,
                                                "size": 8388608
                                        }
                                ],
                                "read_only": false,
                                "removable": false,
                                "rpm": 5400,
                                "serial": "WD-WX22DC322SZ6",
                                "size": 6001175126016,
                                "type": "sata"
                        },
                        {
                                "device": "8:48",
                                "device_id": "wwn-0x50014ee214e465bb",
                                "device_path": "pci-0000:03:00.0-ata-4.0",
                                "firmware_version": "81.00A81",
                                "id": "sdd",
                                "model": "WDC WD80EFZZ-68BTXN0",
                                "numa_node": 0,
                                "partitions": [
                                        {
                                                "device": "8:49",
                                                "id": "sdd1",
                                                "partition": 1,
                                                "read_only": false,
                                                "size": 8001553432576
                                        },
                                        {
                                                "device": "8:57",
                                                "id": "sdd9",
                                                "partition": 9,
                                                "read_only": false,
                                                "size": 8388608
                                        }
                                ],
                                "read_only": false,
                                "removable": false,
                                "rpm": 5640,
                                "serial": "WD-CA102ZNK",
                                "size": 8001563222016,
                                "type": "sata"
                        },
                        {
                                "block_size": 512,
                                "device": "8:64",
                                "device_id": "usb-Kingston_DT_101_G2_000AEB91BD29F070359F12C8-0:0",
                                "device_path": "pci-0000:00:14.0-usbv2-0:4:1.0-scsi-0:0:0:0",
                                "firmware_version": "1.00",
                                "id": "sde",
                                "model": "DT 101 G2",
                                "numa_node": 0,
                                "partitions": [
                                        {
                                                "device": "8:65",
                                                "id": "sde1",
                                                "partition": 1,
                                                "read_only": false,
                                                "size": 1169784832
                                        },
                                        {
                                                "device": "8:66",
                                                "id": "sde2",
                                                "partition": 2,
                                                "read_only": false,
                                                "size": 187695104
                                        }
                                ],
                                "read_only": false,
                                "removable": true,
                                "rpm": 1,
                                "serial": "000AEB91BD29F070359F12C8",
                                "size": 8005787648,
                                "type": "usb",
                                "usb_address": "3:2"
                        }
                ],
                "total": 23
        },
        "system": {
                "chassis": {
                        "serial": "Default string",
                        "type": "Desktop",
                        "vendor": "Default string",
                        "version": "Default string"
                },
                "family": "Default string",
                "firmware": {
                        "date": "06/12/2024",
                        "vendor": "American Megatrends International, LLC.",
                        "version": "5.27"
                },
                "motherboard": {
                        "product": "",
                        "serial": "",
                        "vendor": "CWWK",
                        "version": ""
                },
                "product": "CW-NAS-ADLN-K",
                "serial": "Default string",
                "sku": "Default string",
                "type": "physical",
                "uuid": "03000200-0400-0500-0006-000700080009",
                "vendor": "CWWK",
                "version": "Default string"
        },
        "usb": {
                "devices": [
                        {
                                "bus_address": 3,
                                "device_address": 2,
                                "interfaces": [
                                        {
                                                "class": "",
                                                "class_id": 8,
                                                "driver": "usb-storage",
                                                "driver_version": "6.12.42-1-lts",
                                                "number": 0,
                                                "subclass": "",
                                                "subclass_id": 6
                                        }
                                ],
                                "product": "DT 101 G2",
                                "product_id": "1642",
                                "serial": "",
                                "speed": 480,
                                "vendor": "",
                                "vendor_id": "0951"
                        }
                ],
                "total": 1
        }
}

And that’s what happens on the host when I start the vm:

dmesg

[   77.026494] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[   77.026905] sd 0:0:0:0: [sda] Stopping disk
[   77.252661] sd 1:0:0:0: [sdb] Synchronizing SCSI cache
[   77.253258] sd 1:0:0:0: [sdb] Stopping disk
[   77.485972] sd 2:0:0:0: [sdc] Synchronizing SCSI cache
[   77.486573] sd 2:0:0:0: [sdc] Stopping disk
[   77.702376] sd 3:0:0:0: [sdd] Synchronizing SCSI cache
[   77.702990] sd 3:0:0:0: [sdd] Stopping disk

Right, so that shows 4 drives on the SATA controller:

         {
                                "driver": "ahci",
                                "driver_version": "3.0",
                                "iommu_group": 15,
                                "numa_node": 0,
                                "pci_address": "0000:03:00.0",
                                "product": "ASM1166 Serial ATA Controller",
                                "product_id": "1166",
                                "vendor": "ASMedia Technology Inc.",
                                "vendor_id": "1b21",
                                "vpd": {}
                        },

We can see it’s bound to the ahci driver, so the host system is going to have initialized the drives before Incus/QEMU yank it out, likely causing a bunch of confusion.

If you don’t need the ahci driver at all on the system, you may be able to force the driver to be vfio-pci through the kernel command line with:

vfio-pci.ids=1b21:1166

That will hopefully kick in earlier, before the system gets to do anything with ahci.

cat /proc/cmdline

BOOT_IMAGE=/boot/vmlinuz-linux-lts root=UUID=2845b3b2-67f2-45af-8124-a440163b1c03 rw zswap.enabled=0 rootfstype=ext4 loglevel=3 quiet intel_iommu=on iommu=pt pcie_port_pm=off vfio-pci.ids=1b21:1166

It made no difference… As I said, this is my first time installing using it, I made a lot of tests, probably messed something… maybe it’s time to just reinstall everything and see as things go.

Anyway, I do appreciate your unvaluable help! :slight_smile:

Hmm, that’s odd, you’d think that telling the vfio-pci driver to take over the device would be enough.

I guess ahci may be a kernel built-in driver and vfio-pci a regular kernel module, explaining why vfio-pci doesn’t trigger early enough to take over the device.

Can you check lsmod | grep ahci?
I’m guessing it won’t be listed in there.

Sorry for the delay… different time zones…

It looks like you got it right!

I had never noticed that because I never needed before but it seems that ahci is really a kernel built-in driver and not a module in Arch Linux’s linux-lts kernel (maybe the other versions too?)…

I stopped the vm and set the boot.autostart config to false.

Then I rebooted and that’s the result:

uname -r

6.12.42-1-lts

lspci -nkk

03:00.0 0106: 1b21:1166 (rev 02)
        Subsystem: 1b21:2116
        Kernel driver in use: ahci

lsmod | grep ahci returns nothing, as you supposed. The driver is being used but not as a kernel module.

Now I’ll go search if there’s a way around it. If not, I’ll probably have to use another distro.

Not sure if it was mentioned but instead of passing the sata device into the VM you could just mount a dataset into you VM and treat it like a real ZFS pool. Have a look at Is it possible to mount nested ZFS datasets from within a container? which explains what I mean by that.

Basically leave the core disk managment to your Incus host and just use storage pools in your container, applications and VMs. Puts a pretty clear line on responsibilities. Work pretty well in my installations with less complexity.

2 Likes

I do appreciate your input. As I said, I’m quite new to Incus and still don’t know all of its possibilities.

Do you know if doing as you’re suggesting I can use thes mount points as a share to my own PC (and other in my LAN) which aren’t Incus vms or containers?

That’s one option and it should work fine for filesystems but probably won’t be the best fit if the intent is to run something like TrueNAS or a similar NAS appliance in the VM.

Instead a better approach may be to pass through the individual disks rather than the controller card, so something like this:

incus config device add VM sda disk source=/dev/sda io.bus=nvme
incus config device add VM sdb disk source=/dev/sdb io.bus=nvme
incus config device add VM sdc disk source=/dev/sdc io.bus=nvme
incus config device add VM sdd disk source=/dev/sdd io.bus=nvme

This will then expose your 4 physical SATA drives as 4 virtual NVME drives in the VM.
NVME here is simply used for its slightly lower overhead which may be relevant for a NAS application.

1 Like

Hey guys,

Sorry for the delay. I endedp up deciding to run a nfs server on the host itself. I agree with @osch that running services on the host isn’t the most elegant solution (and I was trying to avoid that) but it’s easier and, well… creating and passing a pre-existing zfs dataset to the vm running the nfs server didn’t work as I expected. It created a complete Incus structure with sub datasets instead of using the zfs structure I already had.

As for @stgraber suggestion, I wanted just to avoid TrueNAS and similar solutions. The idea was just to run a light vm with smb and nfs that wouldn’t be as much resource intensive as a complete tool like TrueNAS or omv. I have a N100 based chinese motheboard, not much RAM available, and that’s exactly the reason I was looking for lighter alternatives to Proxmox. In fact, I’m loving Incus so far and I believe I’ve found the definitive tool to use in my homelab. :slight_smile:

So I took the easier path. Run NFS on the host, make all clients (both Incus vms/containers and also my other LAN machines) use that shares and call it a day.

I really appreciate all the help I’m getting here.

Sounds a bit confusing / wired to me. Just passing a ZFS dataset to a vm is as simple as:

incus storage volume attach default mydata myinstance disk /mnt

This is attaching an Incus storage volume to your VM or use

incus config device add <instance_name> mydisk disk source=/host/path path=/mnt

Which will mount a host local path into your container VM. There is no huge difference how it works between a container or VM. In case of sharing NFS it requires a VM as NFS isn’t namespace aware to run in a container.

Anyway, just wanted to point out that you can share host folders with containers / VMs. There is a learning curve to take to operate Incus and it requires some time until you know how things are working.

Hey @osch and @stgraber

Just to let you know, I believe I’ve finally found the culprit. Apparently it was the motherboard ([CW-NAS-ADLN-K](CWWK 12th Gen i3-N305 N100 2*Intel i226-V 2.5G NAS Motherboard 6*SATA3.0 6-Bay Soft Rout 1*DDR5 4800MHz Firewall ITX Mainboard – cwwk), a chinese motherboard I’ve been using for almost a year.

Everything was configured right from the beginning and the only additional step in my case (as I use Arch and the ahcidriver I want to passthrough is built in the kernel) was to add pci-stub.ids= in grub along with the other configs I had already done with your help. As soon as I used another mobo (an AsRock this time) SATA passthrough worked immediately.

I do appreciate you taking the time to help me.

There seems to be a bug on some asm1166 firmware versions, which is triggered when OVMF executes option rom from the asm1166.

There are multiple workarounds:

  • boot vm in legacy mode (can’t do this with incus)
  • update asm1166 firmware with one which doesn’t have this bug (see Guide on how to setup ASMedia 1166 Passthrough in Proxmox · GitHub )
  • disable rombar on the PCI passthrough device by adding firmware=false to the incus device spec.
    this was added in incus 6.19, but unfortunately the feature is broken and doesn’t really work right now.
    The fix was just merged upstream and maybe upcoming incus 6.21 would finally allow pci-e passthrough for such broken devices.

@alscx do you mind sharing firmware version you have on AsRock mobo if it has the same asm1166 controller? You can get it like this from linux:

sudo xxd -p -s 240 -l 6 "/sys/bus/pci/devices/$(lspci -mD -d 1b21:1166 | cut -d' ' -f1)/config"

mine is 241224000000 which is known to fix some issues with power states on amd – but causes other issues with PCI-E passthrough into UEFI VMs apparently.