Missing the mark: CAD systems can create false sense of certainty
 |
Observers explored images less completely in the presence of a CAD system, leading study authors Trafton Drew, PhD, of Brigham & Women’s Hospital in Boston, and colleagues to conclude that such systems can “engender a sense of certainty” in observers, causing them to miss potentially important findings.
“These results have implications for both how CAD is currently used and how to design more effective CAD systems,” wrote the authors.
Findings were based on two studies involving the interpretation of simulated radiologic images with spatial frequencies comparable to mammograms. A total of 47 naïve observers were used, half of whom completed the experiments without a CAD system, while the other half used a CAD system that marked 75 percent of all targets and 10 percent of non-targets. In experiment 1, the CAD system primarily aided target detection. In experiment 2, target objects and distractors were made to appear more similar to one another, and the CAD system was primarily used to aid target diagnosis. Observers’ eye-movements were tracked as they interpreted the images.
Results showed the benefit of CAD was mixed in experiment 1. Observer sensitivity for marked targets was 97 percent, significantly higher than the 81 percent sensitivity for finding targets without the use of CAD. However, targets that were not marked by the CAD system were missed more frequently than in the no-CAD group, with a sensitivity of 56 percent.
CAD’s effect in experiment 2 was more muted, with only a marginally significant increase in sensitivity from the no-CAD to CAD group, according to the authors.
In both experiments, observers using CAD examined a lower total percentage of the search area than observers who didn’t use CAD.
The authors wrote that the results suggest “CAD steers observers away from searching exhaustively through empty space. This could lead to an increased miss rate for unmarked targets when the target is missed by the CAD system, but in our experiments this effect was only clearly observed when the CAD system's primary role was detection rather than diagnosis.” They presented several possible explanations, including observers placing too much trust in CAD marks, a false sense of security in the absence of marks and observers simply ignoring CAD prompts.
“Although it is imperative to continue to improve performance by CAD systems, we believe that understanding how observers are influenced by the presence or absence of CAD is equally important,” wrote Drew and colleagues. They noted that CAD is currently used on nearly three-quarters of all mammograms in the U.S.
Evan Godt, Writer
Evan joined TriMed in 2011, writing primarily for Health Imaging. Prior to diving into medical journalism, Evan worked for the Nine Network of Public Media in St. Louis. He also has worked in public relations and education. Evan studied journalism at the University of Missouri, with an emphasis on broadcast media.