Fourth-year veterinary students matched - and experienced clinicians outperformed - an AI-enabled digital stethoscope in diagnosing heart murmurs and arrhythmias in cats and dogs, according to a new study from North Carolina State University. The study emphasizes the importance of human expertise when using AI-assisted technology, particularly with instruments that are designed for "universal" usage.
"Stethoscopes are universal instruments - they're designed to aid in auscultation, or listening to the heartbeat," says Kursten Pierce, assistant professor of clinical sciences and board-certified cardiologist at NC State's College of Veterinary Medicine. "Veterinarians use the same stethoscope as a human physician uses. These new AI-enabled stethoscopes are being adopted by many veterinarians, because they have a lot of very useful technical features, such as recording a heart murmur or an EKG.
"However, the diagnostic AI for these stethoscopes is trained on human data, not dog or cat data, and we were hearing from veterinarians who were second-guessing themselves based upon the stethoscope's findings. So, we decided to look at an AI-enabled stethoscope's diagnostic accuracy in dogs and cats."
The research team performed auscultation on 105 companion animals: 54 dogs and 51 cats. Each animal was examined by a board-certified veterinary cardiologist, a cardiology resident, and a fourth-year veterinary student. The exams were conducted with an AI-enabled stethoscope, and the findings from the human experts were compared to the stethoscope's findings.
In dogs, doctor assessment found that 38 (70%) had a murmur and 24 (44%) had an arrhythmia (abnormal heartbeat). Of the 38 dogs with a murmur, the AI software correctly identified 33 (87%). Fourth-year students matched the stethoscope's performance exactly.
Arrhythmia detection told a very different story: the AI stethoscope never classified a single dog as free of an arrhythmia, and while it correctly flagged all 6 true cases of atrial fibrillation, it also mistakenly labeled 22 additional dogs as having atrial fibrillation when they did not.
In cats, doctor assessment found that 22 (43%) had a murmur and only one (2%) had an arrhythmia. The AI stethoscope diagnosed a murmur in only 2 of 51 cats – missing 20 of the true murmurs. Veterinary students performed substantially better in cats, correctly identifying murmurs in nearly two-thirds of affected cats.
"The most clinically concerning result to me was cats," says Jake Johnson, cardiology resident at NC State and the first author of the research. "This device essentially couldn't find a murmur, so a tool vets lean on for reassurance could let real disease go undetected - you wouldn't necessarily think about the fact that the diagnostic algorithms are designed based on humans.
"We saw a similar pattern in dogs with arrhythmias: the stethoscope never once called a dog's rhythm normal, and its atrial fibrillation calls were wrong three out of four times. That said, the ECG tracing it captures is genuinely good quality. This tool works best as an adjunct - a quick ECG or a flag worth a second look - not a stand-alone diagnosis."
The researchers hope that their results will be used to encourage veterinarians and veterinary students to be better informed about the limitations of these stethoscopes in veterinary care.
"We want to encourage veterinarians to rely on the expertise they've gained through their training and to understand what this tool can and cannot provide in a veterinary setting, so that we continue to provide the best possible care to people and their companion animals," Pierce says.
The study appears in the Journal of the American Veterinary Association. Pierce is the corresponding author. Other NC State contributors are Joshua Stern, veterinary cardiologist and associate dean for research graduate studies and Teresa DeFrancesco, professor of clinical sciences.
-peake-
