AI trained to classify unstructured musculoskeletal radiology reports
An international team of researchers trained a recurrent neural network (RNN) to automatically classify musculoskeletal findings in unstructured radiology reports, according to a Feb. 4 study published in the American Journal of Roentgenology. The technique could allow radiologists to analyze more data in less time, and potentially reduce imaging costs.
Electronic medical records (EMRs) contain mounds of valuable, but unformatted information, making it difficult to use as a source for research, wrote first author, Changhwan Lee, with the department of biomedical engineering at Hanyang University in Seoul, Korea, and colleagues.
“For more efficient use, EMR text data such as physician notes and radiology reports must be converted to outcome labels that contain specific information including type or extent of disease,” they added. “However, categorizing EMR text with key annotations is difficult because it contains ambiguous words and narrative sentences.”
After evaluating their system for the word error rate of the output sentences and for binary classification, the team found its three-layer model performed best. It achieved a 1.03 percent word-error rate, the highest precision (0.967), recall (0.967), accuracy (0.982) and F1 score (0.967).
Lee and colleagues believe their approach could be beneficial in managing imaging utilization in radiology clinical decision support. For example, the team noted, understanding the rate of exams that produce normal findings may be useful as an indirect marker of appropriate utilization. The team also suggested their approach could be used to screen participants for clinical trials.
To create their RNN Lee et al. took data from musculoskeletal x-rays performed from January 1 to Dec, 31 2016, at Hanyang University Medical Center. An orthopedic surgeon chose more than 3,000 sentences from the reports. Twenty-eight percent indicated a fracture, 72 percent indicated no fracture was absent. The RNN was trained using 75 percent of that data, and the remaining 25 percent was used to train the system.
One limitation of the approach, and a problem seen in many AI algorithms, was the team’s inability to test the RNN on data from different institutions. Lee et al. did note musculoskeletal x-ray reports tend to have “similar constrained language and content.” Despite this drawback, the team sees a large role for RNN-based approaches going forward.
“The application of these machine learning techniques to leverage data found in free-text imaging reports can aid health care providers, payers, or accountable care organizations as part of a larger strategy to optimize imaging utilization appropriateness and reduce overall imaging costs via medical data analysis even when applied to data originating from multiple institutions,” the authors concluded.
Matt joined Chicago’s TriMed team in 2018 covering all areas of health imaging after two years reporting on the hospital field. He holds a bachelor’s in English from UIC, and enjoys a good cup of coffee and an interesting documentary.