CAD—Closing the Gap on Sensitivity, Specificity: Hurdling PACS Integration & System Learning Curves
 | |
| The Visia CT Lung System utilizes CAD algorithms to automatically detect actionable lung nodules based on size, shape, density, and anatomical context. Source: MeVis Medical Solutions |
“There is a huge learning curve no matter what [CAD] system you use because all have the potential to increase the sensitivity with an associated risk to lose specificity,” says Cornelia Schaefer-Prokop, MD, associate professor at the University of Amsterdam Academic Medical Center in the Netherlands and a user of chest CAD systems.
Since the majority of systems on the market today don’t approach a specificity of 90 to 100 percent yet, as a radiologist, the challenge is in distinguishing between the true positives and the false positives—something that is difficult to quantify and introduce into routine work and seems to be dependant on reader experience.
Schaefer-Prokop, who is familiar with chest CAD systems from EDDA Technologies and Riverain Medical, says it is difficult to quantify, in terms of economic outcome, the eventual impact of these systems in clinical routine because it is easier to quantify the positive or negative impact for a selected group of patients—such as 150 patient patients with CT-proven lesions compared to 100 negative patients without lesions, and have six radiologists read in a controlled, study-like environment. Determining whether there will be an increase in sensitivity or specificity in relation to a radiologist’s experience level is easy in this scenario. Schaefer-Prokop maintains that the success of chest CAD systems will depend upon how they fare in routine clinical practice. This means the systems have to run in the background, be integrated with PACS, be easy to handle so as to not interfere or impede workflow and show an image with complete resolution in the same size as the original.
But she doesn’t dispute the potential of CAD. “It is an undoubtable fact that radiologists tend to miss lesions,” she says “and the potential of CAD is to help radiologists find these lesions, increase [diagnostic] confidence or decrease reading time.”
Lung CAD: On the path to PACS integration
True PACS integration is a step that will help increase routine utilization, says Jin Mo Goo, MD, from Seoul National University Hospital, a researcher in thoracic imaging and clinical applications of CAD, who recently presented his latest research on CAD [EDDA Technologies’ IQQA-Chest CAD] at the 2nd World Congress of Thoracic Imaging in Spain.
“For these CAD tools to be more clinically relevant, these applications should be incorporated into PACS,” Mo Goo says. “Because the ultimate goal of nodule evaluation is to determine nodule malignancy, tools to characterize lesions should be developed.”
He notes that CAD application in lung nodule detection on chest CT can enhance performance regardless of experience level, however, lung nodule detection on chest radiography may be dependant upon a reader’s experience because “the rejection of false-positive CAD marks on those images is not as easy as chest CT. In addition, readers need to understand that there could be considerable variability in the measurement results provided by CAD applications, such as nodule volumetry according to the scan and reconstruction parameters of CT and the software itself.”
Radiologists at Waukesha Memorial Hosptial in Waukesha, Wis., are using a CAD system (MeVis Visia CT CAD, MeVis Medical Solutions) as a second reader to aid in detecting lung nodules and calling out potential abnormalities for roughly 500 high-risk patients screened annually with low-dose, high-resolution CT as part of the International Early Lung Cancer Action Program (I-ELCAP). Comprised of a group of 48 institutions in nine countries, I-ELCAP is dedicated to studying the benefits associated with early lung cancer detection by CT screening.
“CAD’s very good at picking up nodules close to pulmonary arteries or veins, which are sometimes difficult to distinguish,” says radiologist M. Kristin M. Thorsen, MD. Visia is currently only used for lung cancer screening patients, but will use it routinely once it is integrated into their PACS workstation. Thorsen noted that she can send CAD results to a PACS archive, regardless of PACS vendor, allowing access to the results from all PACS workstations.
Optimizing breast imaging workflow
Screening and detection aids are certainly at the top of radiologists’ wish lists when it comes to breast imaging exams. And although the sensitivity of CAD has helped to detect up to 20 percent more cancers and nearly 75 percent of actionable, missed cancers, not all breast CAD is created equal.
Mammography CAD is already a step ahead of the game, with broad acceptance for calcifications, but less acceptance for soft-tissue lesions, which rely on more interpretation skills, says Schaefer-Prokop. “[CAD for] soft-tissue lesions in mammography is closer to a diagnostic and detection than for a chest radiograph, which suffers from a lot of projection effects, thereby requiring more of a learning curve to distinguish between a projection effect and a true lesion,” she notes.
While CAD for mammography is being used as a second reader with increased sensitivity, it is lacking of specificity. Breast MRI CAD, which serves more as a computer-aided evaluation and visualization tool, may shorten the time needed to interpret images, but evidence is lacking still on its level of specificity. It generally has a higher false-positive rate because of the difficulty in distinguishing between benign and malignant lesions.
“I don’t think you can do a breast MRI without CAD,” says Keith Wilson, MD, section head of MRI at Northwest Ohio Dedicated Breast MRI Center in Toledo at ProMedica Toledo Hospital. “We see lesions commonly as small as 4 mm or 5 mm due to the sensitivity of CAD.”
For Wilson, the reason for applying CAD to breast MRI is to provide the radiologist with information to make a more rapid interpretation to determine diagnosis.
As part of the software package available on the center’s Aurora 1.5T dedicated breast MRI system, AuroraCAD presents simultaneous axial, sagittal and coronal views of any acquisition or post-processed image set using multi-planar reconstruction (MPR). Using CAD, he can perform side-by-side comparison of pre-and post-contrast images, subtractions, 3D projection images and enhancement curves.
“As you page through about 160 images per MRI sequence, if you see something, click on MPR to see thin slices of the breast in axial, coronal and sagittal, you can page through those in 3D to rapidly analyze, divisionalize problem areas to display the most important information to help accurately determine diagnosis,” Wilson adds.
Colon CAD gains ground
The CAD market for colon imaging largely mirrors CT colonoscopy (CTC) in general—there is enormous potential that has yet to be fully realized. When more widespread reimbursement is in place and all the FDA-related issues have been worked out, the CAD market should see impressive growth, according to Perry J. Pickhardt, MD, associate professor of radiology, section of abdominal imaging, University of Wisconsin Medical School in Madison. “In our recent experience, we have found that CAD adds significantly even to expert reads. In addition, automated measurement tools for linear polyp size and polyp volume further enhance the value of CAD. Tools that limit a decrease in specificity also will be important additions in the future,” he says. And as screening CTC is implemented, the goal is to make sure the review is as accurate and sensitive as possible.
“Even a small increase in sensitivity related to the use of CAD could be important when you think about a colon screening that might not be looked at again in five to 10 years,” says Abraham
Dachman, MD, professor of radiology at the University of Chicago. “Even missing a small lesion could be important.”
Dachman, who is also primary investigator on iCAD’s clinical trial for its VeraLook CAD for CTC, says that CAD has the potential to reduce oversight errors that could occur during the review of a CTC exam, which generates a large number of images. Despite a recent decision from the Centers for Medicare & Medicaid Services (CMS) that determined that evidence was inadequate to reimburse for CTC as a screening test, reimbursement for diagnostic CTC procedures is currently available in the U.S. from many private insurers and 26 states currently mandate that patients with private healthcare coverage are ensured access to CTC.
“I think there is every reason to believe CAD will be cost-effective,” says Dachman “but in the U.S., it is going to depend on what CAD programs gets approved, what their sensitivity is, what their false positive rate is and what the cost is.”
According to research from Pickhardt and colleagues published in the February 2009 issue of Radiology, the addition of CAD [ColonCAD, Medicsight] to CTC screening improves the colorectal cancer screening rate, resulting in advantageous cost-effectiveness for screening. “In our recent experience, CAD has performed at a high level for all relevant polyps 6 mm or larger. Sensitivity for large lesions [10 mm and larger] is even higher, and detects some lesions that are missed by human readers,” Pickhardt says.
In this cost-concious era of evidence-based medicine, demonstrating the cost-effectiveness of both CTC and CAD are important, and Pickhardt says he hopes that the findings will contribute towards a logical reversal of the recent CMS decision. “CAD sensitivity for relevant colorectal polyps will continue to improve and the number of false-positive markes will continue to decrease,” he says. “This will ultimately lead to better patient care and hopefully dereased mortality from colorectal cancer. CAD tools that limit any decreases in specificity will be very welcome.”
The future
There has been considerable improvement over the last several years to increase the sensitivity and maintain or even improve the specificity of CAD systems across all applications. “There is always a trade-off between the sensitivity and the specificity,” says Schaefer-Prokop. “You can either go for the highest sensitivity, but that will go unavoidably with a higher false positive rate, or you go for a higher specificity to decrease the lower false positive rate, but then the level of detection also will go down.” This is an ongoing debate—what is the best trade off between sensitivity and specificity in routine work—that has yet to be resolved.
What is clear is that CAD is gaining traction as physicians continue to recognize the value of a tool that can not only highlight and quantify cancerous lesions physicians can see on their own, but more importantly, help them detect suspected lesions that might be initially overlooked. Increased physician awareness, coupled with improvements in technology, will continue to propel CAD into mainstream, everyday use.