Radiologist skill level, not preference, to blame for varied diagnoses, new study suggests
It isn’t unheard of for radiologists to arrive at different diagnostic conclusions based on imaging from the same patient. While many in the field have suggested this is due to differing preferences, a new Stanford study suggests that differing skill sets might actually be to blame.
Experts arrived at this conclusion after analyzing 4.67 million chest radiographs on patients with suspected pneumonia. They found that radiologist skill level impacted variations 39% of the time in positive diagnosis and 78% of the time in missed diagnoses. Authors of the study noted that disregarding the role of skill level in varying diagnostic conclusions not only negatively impacts patients care, it can produce misleading results in research settings as well.
“Physicians differ in their propensity to choose aggressive treatments or order expensive tests, even when facing observably similar patients,” David C Chan, of Stanford University, Department of Veterans Affairs, and the National Bureau of Economic Research, and co-authors Matthew Gentzkow and Chuan Yu wrote.
“Such variation is of interest because it implies differences in resource allocation across similar cases and because it has increasingly been exploited in research designs using agent assignments as a source of quasi-random variation.”
Researchers evaluated the relationship between diagnosis and miss rates, and true and false positive rates. These findings were calculated into an equation of radiologist skill level, which researchers defined based on the Blackwell (1953) informativeness of their signals.
For positive diagnostic rates, the researchers found that when patients were reassigned from radiologists with skill sets measured to be in the 10th percentile to those grouped in the 90th percentile, their likelihood of receiving a positive diagnosis increased from 8.9% to 12.3%. Similarly, for missed diagnoses, the same skill level variation resulted in patients’ probability of receiving a false negative from 0.2% to 1.8%.
“Our estimated model suggests that radiologists view failing to diagnose a patient with pneumonia as more costly than incorrectly diagnosing one without, and that this leads less skilled radiologists to optimally choose lower diagnostic thresholds,” the authors wrote.
The radiologists who were the most skilled tended to be older and more experienced. Experts found that they spent more time on their reports as well.
“Policies that improve skill perform better than uniform decision guidelines. Failing to account for skill variation can lead to highly misleading results in research designs that use agent assignments as instruments.”
The detailed study can be viewed in the Quarterly Journal of Economics.
Related radiology reporting content:
Study urges radiologists to report CAC findings on all chest CTs, regardless of clinical indication
ARRS 2022 discusses pitfalls of radiologist 'tunnel vision'
Immediate radiology report access leads to influx of patient questions for 78% of providers
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.