Can natural language processing predict downstream radiology utilization?
Natural language processing (NLP) models can help radiologists anticipate required resources, reported authors of a March 2 study published in the Journal of the American College of Radiology. Such models can be used to improve decision making and reduce costs.
Patients under surveillance for hepatocellular carcinoma (HCC) consume regular and consistent imaging resources, and when cancer is detected, their radiology needs surge, wrote A.D. Brown, MD, MBA, of the department of medical imaging at Toronto General Hospital in Toronto, Ontario, Canada, and colleagues.
To that end, the researchers looked at data from more than 2,500 free-text radiology reports of patients undergoing HCC surveillance to determine if an NLP approach could extract clinical data and predict downstream utilization of resources. They combined two open-source NLP models to analyze free text—bag-of-words and term frequency-inverse document frequency (TF-IDF) models—with three machine learning models—logistic regression (LR), support vector machine (SVM) and random forest.
Brown et al. found TF-IDF feature extraction approach slightly edged out the bag-of-words models. A combined TF-IDF, SVM model beat all others, notching a 92 percent accuracy, 83 percent sensitivity, 96 percent specificity and AUC of 0.971. The next closest model, TF-IDF plus LR, achieved a 91 percent accuracy, 79 percent sensitivity, 96 percent specificity and AUC of 0.969.
“These findings suggest that an algorithmic approach to text analysis could be used as a tool to help radiology administrators better predict changes in demand and proactively institute capacity management strategies to address fluctuations in demand,” Brown et al. wrote.
The results align with most NLP studies which have found TF-IDF to be superior to many tested methods.
In terms of false-positives, the TF-IDF just edged out the other models. Many of these false-positives were caused by extrahepatic findings, which were a source of confusion, the researchers wrote. Importantly, they added, the surveillance exams were done to identify signs of liver disease, however many other organs were also in the field of view, which may have affected the false-positive rate.
“In the future, these (NLP) techniques could be used to create systems that analyze finalized radiology reports in real time and provide administrators with updated predictions of radiology resource demand,” the authors concluded. “To provide robust forecasts of radiology demand, the NLP approaches used here could be combined with time series forecasting methods to provide forecasts for radiology demand across many different clinical indications and imaging modalities.”
Matt joined Chicago’s TriMed team in 2018 covering all areas of health imaging after two years reporting on the hospital field. He holds a bachelor’s in English from UIC, and enjoys a good cup of coffee and an interesting documentary.