Finding a bad physical disk on IncusOS

Hi!

My IncusOS host has slow pool performance and I can hear one of the disks making unusual noise. The disk probably needs replacing. Through IncusOS, how do I determine the serial number for the drive affected by this problem?

The logs have these abnormal messages and they indicate ata4.00 is where the problem is:

[2026/06/25 18:12:13 EDT] kernel: ata4.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 22
         res 40/00:0c:78:28:00/00:00:02:00:00/40 Emask 0x50 (ATA bus error)
[2026/06/25 18:12:13 EDT] kernel: ata4.00: status: { DRDY }
[2026/06/25 18:12:13 EDT] kernel: ata4: hard resetting link
[2026/06/25 18:12:19 EDT] kernel: ata4: link is slow to respond, please be patient (ready=0)
[2026/06/25 18:12:20 EDT] kernel: ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
[2026/06/25 18:12:20 EDT] kernel: ata4.00: configured for UDMA/133
[2026/06/25 18:12:20 EDT] kernel: ata4.00: retrying FLUSH 0xea Emask 0x50
[2026/06/25 18:12:20 EDT] kernel: ata4: EH complete
[2026/06/25 18:12:35 EDT] kernel: ata4: link is slow to respond, please be patient (ready=0)
[2026/06/25 18:12:36 EDT] kernel: ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
[2026/06/25 18:12:36 EDT] kernel: ata4.00: configured for UDMA/133
[2026/06/25 18:14:11 EDT] kernel: ata4.00: exception Emask 0x50 SAct 0x0 SErr 0x4070802 action 0xe frozen
[2026/06/25 18:14:11 EDT] kernel: ata4.00: irq_stat 0x00000040, connection status changed
[2026/06/25 18:14:11 EDT] kernel: ata4: SError: { RecovComm HostInt PHYRdyChg PHYInt CommWake DevExch }
[2026/06/25 18:14:11 EDT] kernel: ata4.00: failed command: FLUSH CACHE EXT
[2026/06/25 18:14:11 EDT] kernel: ata4.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 12
         res 40/00:7c:10:29:00/00:00:02:00:00/40 Emask 0x50 (ATA bus error)
[2026/06/25 18:14:11 EDT] kernel: ata4.00: status: { DRDY }
[2026/06/25 18:14:11 EDT] kernel: ata4: hard resetting link
[2026/06/25 18:14:17 EDT] kernel: ata4: link is slow to respond, please be patient (ready=0)
[2026/06/25 18:14:19 EDT] kernel: ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
[2026/06/25 18:14:19 EDT] kernel: ata4.00: configured for UDMA/133
[2026/06/25 18:14:19 EDT] kernel: ata4.00: retrying FLUSH 0xea Emask 0x50
[2026/06/25 18:14:19 EDT] kernel: ata4: EH complete
[2026/06/25 18:14:27 EDT] kernel: ata4: link is slow to respond, please be patient (ready=0)
[2026/06/25 18:14:30 EDT] kernel: ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
[2026/06/25 18:14:30 EDT] kernel: ata4.00: configured for UDMA/133
[2026/06/25 18:14:32 EDT] kernel: ata4.00: limiting speed to UDMA/100:PIO4
[2026/06/25 18:14:33 EDT] kernel: ata4: found unknown device (class 0)
[2026/06/25 18:14:38 EDT] kernel: ata4: link is slow to respond, please be patient (ready=0)
[2026/06/25 18:14:40 EDT] kernel: ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
[2026/06/25 18:14:40 EDT] kernel: ata4.00: configured for UDMA/100
[2026/06/25 18:15:16 EDT] kernel: ata4.00: exception Emask 0x50 SAct 0x0 SErr 0x4070802 action 0xe frozen
[2026/06/25 18:15:17 EDT] kernel: ata4.00: irq_stat 0x00000040, connection status changed
[2026/06/25 18:15:17 EDT] kernel: ata4: SError: { RecovComm HostInt PHYRdyChg PHYInt CommWake DevExch }
[2026/06/25 18:15:17 EDT] kernel: ata4.00: failed command: FLUSH CACHE EXT

Despite the obvious problem, the ZFS pool is ONLINE, however, the scrub seems to be impacted by the slow disk problem.

    - devices:
        - /dev/disk/by-id/ata-WDC_WD2003FYYS-02W0B0_WD-WMAY02888358
        - /dev/disk/by-id/ata-WDC_WD2003FYYS-02W0B1_WD-WMAY04244775
        - /dev/disk/by-id/ata-WDC_WD2003FYYS-02W0B1_WD-WMAY05081779
        - /dev/disk/by-id/ata-WDC_WD2003FYYS-02W0B1_WD-WMAY05086641
      encryption_key_status: available
      last_scrub:
        end_time: "1970-01-01T00:00:00Z"
        errors: 0
        progress: 100.00%
        start_time: "2026-06-25T22:58:04Z"
        state: IN_PROGRESS
      log:
        - /dev/disk/by-id/nvme-SHPP41-500GM_AND1N597710202203
      name: incus02pool01
      pool_allocated_space_in_bytes: 2.95501824e+09
      raw_pool_size_in_bytes: 3.985729650688e+12
      state: ONLINE
      type: zfs-raid10
      usable_pool_size_in_bytes: 3.985729650688e+12
      volumes:
        - name: main
          quota_in_bytes: 0
          usage_in_bytes: 2.952671232e+09
          use: incus

All drives have passed SMART tests and there are no unusual errors, so based on this output here, I can’t figure out which one would be associated with ata4.00. Here’s an example of what one looks like:

    - boot: false
      bus: sata
      capacity_in_bytes: 2.000398934016e+12
      id: /dev/disk/by-id/ata-WDC_WD2003FYYS-02W0B1_WD-WMAY05081779
      member_pool: incus02pool01
      model_family: Western Digital RE4
      model_name: WDC WD2003FYYS-02W0B1
      multipath: false
      remote: false
      removable: false
      serial_number: WD-WMAY05081779
      smart:
        enabled: true
        passed: true
        power_on_hours: 27156
      wwn: "0x50014ee206f554de"
      wwn_id: /dev/disk/by-id/wwn-0x50014ee206f554de

Thanks for reading my post. Hopefully there are some ways to figure out within Incus which drives have the errors.

You may be able to figure it out if you look at the early kernel boot messages when the drives were first probed. That should show you the ATA port coming online and then the drive in question appearing, hopefully with its serial or other useful characteristic.

1 Like

I was hoping that might work. The messages don’t include a serial number. They also appear asynchronously - maybe because of the faulty drive?

I’ll fire up a generic linux distribution and debug it that way. No problem overall.

Kernel boot messages for reference:

[2026/06/25 18:04:22 EDT] kernel: ata1: SATA link down (SStatus 0 SControl 330)
[2026/06/25 18:04:22 EDT] kernel: usb 3-2.4.1: New USB device found, idVendor=1a86, idProduct=e329, bcdDevice=38.00
[2026/06/25 18:04:22 EDT] kernel: usb 3-2.4.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[2026/06/25 18:04:22 EDT] kernel: usb 3-2.4.1: Product: WCH UART TO KB-MS_V1.8
[2026/06/25 18:04:22 EDT] kernel: usb 3-2.4.1: Manufacturer: WWW.WCH.CN
[2026/06/25 18:04:22 EDT] kernel: usb 3-2.4.1: SerialNumber: 2019915EF09D
[2026/06/25 18:04:23 EDT] kernel: ata2: SATA link down (SStatus 0 SControl 330)
[2026/06/25 18:04:23 EDT] kernel: usbcore: registered new interface driver usbhid
[2026/06/25 18:04:23 EDT] kernel: usbhid: USB HID core driver
[2026/06/25 18:04:23 EDT] kernel: ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[2026/06/25 18:04:23 EDT] kernel: ata3.00: ATA-8: WDC WD2003FYYS-02W0B1, 01.01D02, max UDMA/133
[2026/06/25 18:04:23 EDT] kernel: ata3.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 32), AA
[2026/06/25 18:04:23 EDT] kernel: ata3.00: configured for UDMA/133
[2026/06/25 18:04:23 EDT] kernel: scsi 6:0:0:0: Direct-Access              USB DISK 3.0     PMAP PQ: 0 ANSI: 6
[2026/06/25 18:04:23 EDT] kernel: sd 6:0:0:0: Attached scsi generic sg0 type 0
[2026/06/25 18:04:23 EDT] kernel: sd 6:0:0:0: [sda] 121145344 512-byte logical blocks: (62.0 GB/57.8 GiB)
[2026/06/25 18:04:23 EDT] kernel: sd 6:0:0:0: [sda] Write Protect is off
[2026/06/25 18:04:23 EDT] kernel: sd 6:0:0:0: [sda] Mode Sense: 45 00 00 00
[2026/06/25 18:04:23 EDT] kernel: sd 6:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[2026/06/25 18:04:23 EDT] kernel: scsi 2:0:0:0: Direct-Access     ATA      WDC WD2003FYYS-0 1D02 PQ: 0 ANSI: 5
[2026/06/25 18:04:23 EDT] kernel: sd 2:0:0:0: Attached scsi generic sg1 type 0
[2026/06/25 18:04:23 EDT] kernel: sd 2:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[2026/06/25 18:04:23 EDT] kernel: sd 2:0:0:0: [sdb] Write Protect is off
[2026/06/25 18:04:23 EDT] kernel: sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[2026/06/25 18:04:23 EDT] kernel: sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[2026/06/25 18:04:23 EDT] kernel: sd 2:0:0:0: [sdb] Preferred minimum I/O size 512 bytes
[2026/06/25 18:04:23 EDT] kernel:  sda: sda1 sda2 sda3 sda4 sda5 sda6 sda7 sda8 sda9 sda10 sda11
[2026/06/25 18:04:23 EDT] kernel: sd 6:0:0:0: [sda] Attached SCSI removable disk
[2026/06/25 18:04:23 EDT] kernel: sd 2:0:0:0: [sdb] Attached SCSI disk
[2026/06/25 18:04:29 EDT] kernel: ata4: link is slow to respond, please be patient (ready=0)
[2026/06/25 18:04:30 EDT] kernel: ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[2026/06/25 18:04:30 EDT] kernel: ata4.00: ATA-8: WDC WD2003FYYS-02W0B1, 01.01D02, max UDMA/133
[2026/06/25 18:04:30 EDT] kernel: ata4.00: 3907029168 sectors, multi 0: LBA48 NCQ (depth 32), AA
[2026/06/25 18:04:30 EDT] kernel: ata4.00: configured for UDMA/133
[2026/06/25 18:04:30 EDT] kernel: scsi 3:0:0:0: Direct-Access     ATA      WDC WD2003FYYS-0 1D02 PQ: 0 ANSI: 5
[2026/06/25 18:04:30 EDT] kernel: sd 3:0:0:0: Attached scsi generic sg2 type 0
[2026/06/25 18:04:30 EDT] kernel: sd 3:0:0:0: [sdc] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[2026/06/25 18:04:30 EDT] kernel: sd 3:0:0:0: [sdc] Write Protect is off
[2026/06/25 18:04:30 EDT] kernel: sd 3:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[2026/06/25 18:04:30 EDT] kernel: sd 3:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[2026/06/25 18:04:30 EDT] kernel: sd 3:0:0:0: [sdc] Preferred minimum I/O size 512 bytes
[2026/06/25 18:04:30 EDT] kernel: sd 3:0:0:0: [sdc] Attached SCSI disk
[2026/06/25 18:04:31 EDT] kernel: ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[2026/06/25 18:04:31 EDT] kernel: ata5.00: ATA-8: WDC WD2003FYYS-02W0B0, 01.01D01, max UDMA/133
[2026/06/25 18:04:31 EDT] kernel: ata5.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 32), AA
[2026/06/25 18:04:31 EDT] kernel: ata5.00: configured for UDMA/133
[2026/06/25 18:04:31 EDT] kernel: scsi 4:0:0:0: Direct-Access     ATA      WDC WD2003FYYS-0 1D01 PQ: 0 ANSI: 5
[2026/06/25 18:04:31 EDT] kernel: sd 4:0:0:0: Attached scsi generic sg3 type 0
[2026/06/25 18:04:31 EDT] kernel: sd 4:0:0:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[2026/06/25 18:04:31 EDT] kernel: sd 4:0:0:0: [sdd] Write Protect is off
[2026/06/25 18:04:31 EDT] kernel: sd 4:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[2026/06/25 18:04:31 EDT] kernel: sd 4:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[2026/06/25 18:04:31 EDT] kernel: sd 4:0:0:0: [sdd] Preferred minimum I/O size 512 bytes
[2026/06/25 18:04:31 EDT] kernel: sd 4:0:0:0: [sdd] Attached SCSI disk
[2026/06/25 18:04:31 EDT] kernel: ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[2026/06/25 18:04:31 EDT] kernel: ata6.00: ATA-8: WDC WD2003FYYS-02W0B1, 01.01D02, max UDMA/133
[2026/06/25 18:04:31 EDT] kernel: ata6.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 32), AA
[2026/06/25 18:04:31 EDT] kernel: ata6.00: configured for UDMA/133
[2026/06/25 18:04:31 EDT] kernel: scsi 5:0:0:0: Direct-Access     ATA      WDC WD2003FYYS-0 1D02 PQ: 0 ANSI: 5
[2026/06/25 18:04:31 EDT] kernel: sd 5:0:0:0: Attached scsi generic sg4 type 0
[2026/06/25 18:04:31 EDT] kernel: sd 5:0:0:0: [sde] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[2026/06/25 18:04:31 EDT] kernel: sd 5:0:0:0: [sde] Write Protect is off
[2026/06/25 18:04:31 EDT] kernel: sd 5:0:0:0: [sde] Mode Sense: 00 3a 00 00
[2026/06/25 18:04:31 EDT] kernel: sd 5:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[2026/06/25 18:04:31 EDT] kernel: sd 5:0:0:0: [sde] Preferred minimum I/O size 512 bytes
[2026/06/25 18:04:31 EDT] kernel: sd 5:0:0:0: [sde] Attached SCSI disk

Anything useful in incus admin os system resources show?

Yes! Searched for ata and eventually found this entry for ata-4.0 under storage.disks:

    - block_size: 512
      device: "8:32"
      device_id: wwn-0x50014ee206f4e410
      device_path: pci-0000:05:00.1-ata-4.0
      id: sdc
      model: WDC WD2003FYYS-0
      numa_node: 0
      partitions:
        - device: "8:33"
          id: sdc1
          partition: 1
          read_only: false
          size: 2.00038940672e+12
        - device: "8:41"
          id: sdc9
          partition: 9
          read_only: false
          size: 8.388608e+06
      read_only: false
      removable: false
      rpm: 1
      serial: WD-WMAY05086641
      size: 2.000398934016e+12
      type: scsi

That’s exactly what I needed. Thank you.