Literature review suggests most fMRI findings have been unreliable at best
A new analysis of the literature on fMRI suggests that a shortcoming in the most popular software systems used to evaluate the imaging data—SPM, FSL and AFNI—may have caused all three to produce false positive findings at a 70 percent clip from 1992 to 2015.
The team behind the analysis claims its discovery could turn up to 40,000 published studies on their collective head.
Their work is running in the Proceedings of the National Academy of Sciences.
Anders Eklund, PhD, MSc, of Linköping University in Sweden and colleagues tested the software by using it to drill down into anonymized, publicly available fMRI data on 499 healthy, resting-state control cases.
They split the cases into small groupings and measured the fMRI findings against each other to come up with three million random comparisons.
“Using mass empirical analyses with task-free fMRI data, we have found that the parametric statistical methods used for group fMRI analysis with the packages SPM, FSL and AFNI can produce [familywise error]-corrected cluster P values that are erroneous, being spuriously low and inflating statistical significance,” Eklund et al. write in their discussion. “This calls into question the validity of countless published fMRI studies based on parametric clusterwise inference.”
The authors emphasize that they focused on inferences corrected for multiple comparisons in each analysis they performed on the small groupings—yet “some 40 percent of a sample of 241 recent fMRI papers did not report correcting for multiple comparisons, meaning that many group results in the fMRI literature suffer even worse false-positive rates than found here.”
PNAS has posted the full report.
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.