JACR: NLP shows reports dont measure up to PQRS requirements

A considerable portion of radiology reports do not include all components of the Center for Medicare & Medicaid Services’ Physician Quality Reporting System (PQRS) measure 10, with high levels of variability for individual physician adherence, according to a study published in the June issue of the Journal of the American College of Radiology.

PQRS measure 10 tracks the percentage of radiology reports for possible stroke that document the presence or absence of hemorrhage, mass and acute infarction, explained Luciano M. Prevedello, MD, MPH, and colleagues from Brigham and Women’s Hospital in Boston. “Reporting the presence or absence of hemorrhage, mass and infarction is believed to be important when an acute cerebrovascular event is clinically considered,” they wrote. “This is particularly important because in eligible patients, these findings may determine the initial treatment.”

Determining adherence to this measure, however, is difficult. As a means to that end, the researchers developed a natural language processing algorithm to automate measurement of the PQRS measure 10 adherence rate and identify variables that may affect this rate.

More than 4,000 reports from CT and MRI exams performed in the ED between January 2008 and October 2010 in patients with suspected stroke were analyzed. Results showed that 58.1 percent met the PQRS requirement by documenting all three components. After controlling for variables, there was more than a two-fold variation in individual PQRS adherence.

The presence of infarct increased the chance of PQRS adherence, reported the authors, while the existence of hemorrhage reduced adherence. Reports with trainee participation had an adherence rate of 62 percent, compared with a rate of 47 percent for reports without trainee participation.

Prevedello and colleagues speculated that the presence of hemorrhage lowered confidence in reporting infarction, which is why existence of hemorrhage reduced adherence. “Another possible explanation is that there may be a misconception that reporting the presence or absence of infarction is less valuable in the setting of hemorrhage because therapeutic options would be limited,” they wrote, clarifying that the identification of infarction in this scenario is, in fact, able to aid in diagnosis while also adding prognostic value.

As to why the data showed that reports with trainee participation had higher adherence, the authors said this is possibly due to the nature of trainees describing more negative findings in their reports compared with attending physicians. The use of standardized reporting templates also may be higher among trainees.

The authors stressed that more standardization in imaging will be needed, and automating measurement of PQRS performance can make tracking adherence rates more practical and useful. “The automated computerized approach described here allows continued monitoring of the radiology reports for specific content. Coupled with business intelligence tools, the data extracted by the [natural language processing] algorithm can be used to create performance quality reports at the departmental or individual levels so that data-driven decisions and targeted interventions can be performed if needed.”