Depending on the system, if you can have a dedicated PCIe card for USB and then pass the whole card into the VM, you’ll definitely have a more reliable and faster experience.
As your devices aren’t marked as optional=true, they must have been present and working for the VM to start at all. So my guess is that something happened shortly afterwards, quite likely a kernel driver loading on the host which then caused that device to be reset and drop out of the VM.
Maybe look at the journalctl or dmesg output to see if there isn’t anything related to those devices happening after the VM is started.