'Too many cooks in the kitchen': How AI is both helpful and a hindrance to radiologists
As the integration of artificial intelligence-enabled devices and software into clinical practice is gaining momentum, a new paper digs into whether AI is living up to its reputation as a reliable aid for radiologists.
Published in the American Journal of Roentgenology, authors of the follow-up paper to a prior study in Nature Medicine affirmed that AI is both a helping hand and a hindrance for radiologists.
“Enthusiasm for AI assistance in radiologists’ everyday workflow is on the rise. Prior work suggests that AI-aided detection has great value in CXR interpretation,” noted co-authors Angela Udongwo, MPH, with the Lewis Katz School of Medicine at Temple University and Farouk Dako, MD, MPH, with the department of radiology at Perelman School of Medicine Hospital of the University of Pennsylvania. “This study highlights the limitations of AI in improving radiologist performance in CXR interpretation, most notably the variability and unpredictability of its impact.”
In the original study, researchers analyzed the performances of 140 radiologists interpreting chest radiographs with and without the help of AI. Factors like experience and skill level, AI accuracy, subspecialty and a history of using AI tools were all considered when grading the radiologists’ performance on 15 different diagnostic tasks relative to interpreting chest X-rays.
Experts concluded that in many cases radiologist performance only improved minimally with AI assistance. Further, the benefits of using AI were inconsistent across all readers; The radiologists with the least experience with AI did not appear to benefit from its assistance any more than those who had a history of AI utilization.
The only factor that reliably predicted radiologist performance was the accuracy of the AI software itself.
Chest X-rays are perhaps the most common exam on radiologists’ work lists. These exams have been a popular target for AI developers, as offloading the initial reads of chest X-rays to AI has great potential to reduce radiologist workloads. However, multiple studies have revealed that the benefits of AI assistance are not consistent across all users, the authors noted.
“Ultimately, AI interpretation in tandem with radiologists in principle sounds like a prudent approach, although practically it might be a case of too many cooks in the kitchen at once,” the authors suggested. “We agree with the authors’ conclusion: Future studies need to examine individualized approaches to AI assistance, with an emphasis on clinician heterogeneity in experience, practice and AI background.”
Of note, in the original study, the AI software used did not provide the reasoning behind its predictions. The authors suggested that increasing transparency and explainability could increase trust while also decreasing variability among users. This should be an area of focus in future AI development, the authors suggested.
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.