Can crowd-sourcing AI algorithms work in radiation oncology?
The supply of radiation oncologists hasn’t kept up with the global demand for radiation therapy. But could experts from across the world help create an AI algorithm capable of closing that gap?
A team of Massachusetts-based researchers tested that notion, exploring whether a crowd-sourced contest could rapidly produce an AI solution capable of segmenting lung tumors for radiation therapy as accurate as a trained expert. First author Raymond H. Mak, MD, published their results April 18 in JAMA Oncology.
“Although approximately 58% of lung cancer cases occur in less developed countries, these countries have a staggering shortage of radiation oncologists, with an estimated 23,952 radiation oncologists required in 84 low- and middle-income countries by 2020 yet only 11,803 were available in 2012,” the authors wrote. “We used a novel combined approach of crowd innovation and artificial intelligence (AI) to address this unmet need in global cancer care.”
The 10-week online contest included 77,942 CT scans, 8,144 of which included images with tumors segmented by an expert for clinical care. A total of 564 contestants from 62 countries registered for the three-phase challenge. The prizes totaled $55,000.
Contestants were provided a training set of 229 CTs with expert contours to develop their algorithm—feedback from a clinician was provided throughout the 10 weeks.
Overall, 34 contestants submitted AI algorithms. The top five—when combined using an ensemble model—achieved an accuracy within the benchmark (Dice coefficient of 0.79) of mean interobserver variation as measured between six human experts. In other words, the AI models segmented tumors as well as the trained experts.
Mak and co-authors noted the crowdsourced algorithms also beat existing commercially-available, semi-automated segmentation tools.
“The ability to rapidly develop high-performing AI algorithms for tumor segmentation via a cost-effective crowd innovation approach has the potential to substantially improve the quality of oncologic care globally,” the authors wrote. “Developing AI solutions for time-intensive tasks such as tumor segmentation can increase productivity and time with patients for busy clinicians by reducing computer-based work, and solve the known oncology workforce crisis (eg, number of trained radiation oncologists) in under-resourced health care systems worldwide.”
Matt joined Chicago’s TriMed team in 2018 covering all areas of health imaging after two years reporting on the hospital field. He holds a bachelor’s in English from UIC, and enjoys a good cup of coffee and an interesting documentary.