How to track inconsistencies in CT protocol usage

The CT Protocol team at the University of Wisconsin (UW) have created a methodology for measuring organizational adherence to standardized CT protocols, published in the Journal of the American College of Radiology. Based on finding outliers in a large pool of scanning data, the workflows can be scaled to any size practice and are intended to help imaging centers identify problems.

The UW Department of Radiology has a strong protocoling culture, exemplified by their close relationship with GE Healthcare. An inter-disciplinary team of faculty, scientists and technologists has been optimizing GE’s CT protocols since 2011, covering nearly all indications and body types. Termed a “master protocol” system, this exhaustive list of specific scans means technologists aren’t tweaking settings with a patient on the table—but some within UW wanted to be sure.

“We wanted to create a methodology for checking something that everybody should be interested in knowing,” said Assistant Professor and CT Protocol Team physicist Tim Szczykutowicz, PhD. “You go through a lot of effort to make these protocols—lots of man-hours, lots of money—and if people aren’t executing what’s written, it’s a waste of your time.”

Szczykutowicz and his colleagues analyzed a year’s worth of scan data from a single CT scanner and created an algorithm to identify “one-off” exams, finding 144 unique irradiation events out of 132,000 recorded exams. While some one-off exams are acceptable due to unusual patient position or device maintenance, technologists made incorrect changes on 31% of the unique scans—a mere 0.03% of all exams over the course of the year.  

This extremely low level of non-adherence was expected, said Szczykutowicz, and he believes that the methodology is easily generalizable.

“Any institution can run this. If you did it scanner by scanner, all the protocols should be same—and if they’re not being changed at scan time, you could run our methodology and it should work everywhere,” he said. “There are some sites that have their technologists make changes, but they usually have a workflow for those changes, so you just have to make sure those one-offs follow the rules they’ve set up.”

Another caveat with this procedure is the cost, according to Szczykutowicz. Practices need a technologist, a physicist, and radiologists to come together and work through the scanner data, taking away time from their clinical duties. Freeing up all three groups requires a sizeable investment of time and money, but there are workarounds.

“This doesn’t need to be done on every scanner on every protocol, because you’re looking at technologist behavior,” said Szczykutowicz. “You could carve out just a 3-month period for a pool of technologists, invest a little man hours and do it once every other year.”

There is also potential for automation in the procedure. A digital repository of protocols would make comparisons simple; all it would require is a function checking how the protocol was scanned against how it actually was utilized. Much of the manual labor in the paper was classifying events, according to Szczykutowicz, and

Other opportunities for automation are presented when analyzing individual technologist behavior.

“You could have machine learning analyze trends to see how a given tech adapts protocol for patient size and indication and I’m sure you could teach a computer whether or not that change was correct or not, or compare their behavior to other techs and flag it if it appears anomalous,” said Szczykutowicz.

However, the most important piece of advice is simply to get started.

“Before you can try to optimize protocols, you need to document what you have now,” he said. “Whether it’s a little site that makes word documents, or a large academic center with complex operations, that is my number one piece of advice.”

If a site expects technologists to discuss breathing instructions and inspiration during a chest exam, it should be written down, according to Szczykutowicz. If a center wants their technologists to practice a patient maneuver for hernia imaging, there should be instructions on how to perform the maneuver and expectations for how often they practice.

“I think it would be an eye opening exercise for a lot of places when they actually sit down and document protocols, they’re going to notice things that don’t make sense or things they never thought about,” he said. “The documentation process inherently has some optimization, and once you’ve written it down it will spark questions to radiologists and technologists."