Clinical evidence is limited in many AI product promotions, analysis shows
Companies offering various artificial intelligence applications to radiology practices are quick to tout the benefits of their products, but a new analysis suggests that the scientific evidence backing their actual effectiveness might be lacking.
The AI vendor assessment was published recently in European Radiology [1]. In it, experts shared that the amount of new AI applications that have entered the market over the last two years has increased by 43%. That’s in addition to a 34% increase in companies offering them. With AI applications growing at such a rapid pace, experts sought to understand the tactics companies use to legitimize the value of their offerings.
“Given that users’ trust in AI applications is one of the key drivers of adopting them, it is important to analyze how these companies try to assure users about the credibility of their applications,” corresponding author Mohammad Rezazade Mehrizi of the KIN Center for Digital Innovation and School of Business and Economics at Vrije University in Amsterdam and co-authors explained.
For their work, the experts analyzed 393 different AI applications geared towards optimizing radiology workflow. This resulted in the review of 1,250 documents pertaining to the products from the companies offering them.
They found that these companies most frequently (31% of the time) focused on the quality of the medical services offered by their product by enhancing clinical decisions, patient care or both. The efficiency of radiology workflow was referenced in 18% of cases.
To promote the legitimacy of their products, companies most often touted their partnerships with medical and academic institutions (75%), followed by their applications’ legal approvals (72%). While the vendors did reference the expertise of their development teams, in addition to examples displaying the benefits of their products in clinical practice, it was significantly less so and both took a back seat to partnerships and legality.
In the future, the experts warn, more scientific evidence is crucial and should perhaps take center stage when developing and promoting AI applications:
Systematic evidence showing the performance and effectiveness of these solutions on the users’ side, such as clear evidence on the reduction of the time or costs or increasing the quality, is limited. This is a crucial step to ensure that AI applications offer clinically relevant and practically viable value.”
The study details can be viewed here.
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.