Not all radiologists are buying what AI is selling, even when the technology outperforms humans
New data signal hesitancy among many radiologists when it comes to considering algorithms’ interpretations of breast cancer screening exams.
Recently published research in the journal Radiology details how radiologists interact with AI computer aided detection (CAD) programs in clinical settings. The results revealed that providers are up to three times more likely to trust the opinion of humans over AI, even when the algorithm is proven to detect more cancerous lesions than radiologists.
The trial compared cancer detection rates among an AI CAD tool (Lunit INSIGHT MMG) and human radiologists tasked with interpreting the mammograms of nearly 55,000 women from April 2021 to June 2022. After reviewing the exams, each reader (human and artificial) forwarded those which they deemed positive for malignancy to a consensus discussion among radiologists, at which time they determined who should and should not be recalled for additional assessment.
Researchers used patients' histological analyses to confirm or rule out cancer, comparing the results to the exams flagged by both AI and radiologists. Positive predictive value was calculated by dividing the number of pathologically verified cancers by the number of positive exams.
Of the 54,991 women included in the trial, 5,489 were flagged for consensus discussion and 1,348 were recalled. Following consensus discussion, 14.2% of the exams flagged by radiologists were recalled, compared to just 4.6% of the studies flagged by CAD—more than a threefold difference. When two radiologists flagged an exam, recall rates rose to 57.2%, but when one radiologist and the AI flagged a study, recall rates again lagged behind at 38.6%
However, radiologists’ positive predictive values were significantly lower than that of CAD, at 3.4% compared to 22%. Cancer yield was the highest when exams were flagged by one radiologist and the AI, at 25%.
The results suggest that providers are less likely to trust AI’s interpretations, despite CAD having higher PPV than human radiologists.
“This isn’t a question of whether AI can detect cancer,” lead author Karen Dembrower, MD, a radiologist at the Karolinska Institute in Sweden, and colleagues note. “It’s about how AI findings are interpreted and acted on by the people making clinical decisions.”
The study abstract is available here.
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.