4 challenges machine learning must overcome to help clinicians
Machine learning has the potential to reshape the patient-doctor relationship, according to a new review published in the New England Journal of Medicine, but it must overcome a few challenges first.
“We expect a handful of early models and peer-reviewed publications of their (machine learning) results to appear in the next few years, which—along with the development of regulatory frameworks and economic incentives for value-based care—are reasons to be cautiously optimistic about machine learning in health care,” wrote Alvin Rajkomar, MD, of Google, and colleagues.
The researchers, who also included Jeffrey Dean, PhD, of Google and Isaac Kohane, MD, PhD, Harvard Medical School, pointed to four distinct challenges machine learning must overcome to successfully augment the work of clinicians:
Access to quality data
When building a machine learning model, it’s central to not only include large data sets, but also data that is both diverse, and representative of the format and quality of data expected to be used clinically.
It is generally preferred to have large datasets, even if they are noisy, to train a platform on complex statistical patterns, but to evaluate a model, smaller sets with curated labels is key, the authors wrote. For imaging platforms, a ground truth label settled on by multiple readers for each image is typically required.
“Machine-learning models generally perform best when they have access to large amounts of training data,” the researchers wrote. “Thus, a key issue for many uses of machine learning will be balancing privacy and regulatory requirements with the desire to leverage large and diverse data sets to improve the accuracy of machine-learning models.”
Learning from the past
Those who construct AI platforms—and its users—must be cognizant of how bias may affect its training.
Healthcare data often reveals disparities in care for vulnerable populations and a fee-for-service payment structure that does not align with the value-based care-oriented future of healthcare.
“The strength of machine learning, but also one of its vulnerabilities, is the ability of models to discern patterns in historical data that humans cannot find,” the researchers wrote.
Regulation, oversight and safety
For machine learning to have its greatest impact, it will need to become widely adopted and thus will require regulatory oversight, legal framework and structured local practices, the team wrote. They pointed to the oversight and framework used to safely distribute pharmaceuticals as an example.
Additionally, the clinicians and patients who use such models must understand when a model is not providing the best advice or if its results are not generalizable. This can be achieved by using real-world clinical evaluation and avoiding the retrospective variety.
Research and publications
Clinicians may not be aware of the websites that publish the results of machine learning studies.
“Manuscripts are often posted online at preprint services such as arXiv and bioRxiv, and the source code of many models exists in repositories such as GitHub,” the authors wrote. “Moreover, many peer-reviewed computer science manuscripts are not published by traditional journals…”
Matt joined Chicago’s TriMed team in 2018 covering all areas of health imaging after two years reporting on the hospital field. He holds a bachelor’s in English from UIC, and enjoys a good cup of coffee and an interesting documentary.