AI model identifies radiologist-recommended follow-up imaging in reports, has potential for widespread use
An artificial intelligence-based model has the potential to improve follow-up adherence for radiologist-recommended additional imaging (RAI).
That’s according to new data published April 19 in the American Journal of Roentgenology that details the performance of a deep learning model known as BERT, short for Bidirectional Encoder Representations from Transformers [1]. The BERT-based model, which is trained to understand language context, recently yielded a significantly improved performance in identifying radiology reports with RAI in comparison to a traditional machine learning model (TLM).
The AI model could have great utility in ensuring patients undergo radiologist-recommended follow-up imaging, experts involved in the study suggested.
“The model could potentially be applied for real-time EHR monitoring for RAI, or other improvement initiatives, to help ensure timely performance of clinically necessary recommended follow-up,” corresponding author of the study Ramin Khorasani, MD, MPH, with the Department of Radiology at Brigham and Women's Hospital, and colleagues explained.
The BERT-based model was trained and tested on 6,300 radiology reports and externally validated on an additional 1,260. Its performance in identifying RAI from within the reports was compared to that of a previously developed TLM.
Out of the entire batch, 10% of the reports contained RAI. In the test set, the BERT-based model achieved precision of 94%, recall of 98% and F1 score of 96%. In comparison, the TML model showed precision of 69%, recall of 65% and F1 score of 67%. The BERT-based model also outperformed TML in accuracy on both the test set and the external validation set, yielding 99% in each.
The model’s performance in the external validation set is of particular importance, as it indicates its potential for widespread use, the experts suggested.
“High performance in the external validation set suggests the potential for other health systems to adapt the model without requiring institution-specific training,” the team noted.
The study abstract is available here, and an accompanying editorial can be found here (paywall).
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.