JACR editorial exposes bias in peer-reviewed journals

“The marketplace of ideas in peer-reviewed published literature is not a venue that should rely on the principle caveat emptor,” wrote R. James Brenner, MD, JD, in an editorial in the July edition of the Journal of the American College of Radiology. In reality, bias in scientific literature obliges buyers to beware.

Bias creeps into scientific literature in multiple ways, wrote Brenner. The bias problem will grow as print and online research journals proliferate, and authors re-cite information without scrutinizing the original data. “[It] may be predictable folly to believe that many readers with limited time and background in experimental designs will review the methodology,” wrote Brenner.

Careless use of adjectives and adverbs comprises the first bias sin, Brenner noted. Words like "unequivocally" and "undoubtedly" espouse a single view, leaving no room for interpretation, he explained, adding that terms like "compelling" and "persuasive" lack objectivity yet nudge the reader toward a conclusion.

Brenner suggested more journals follow Radiology’s lead and reject terms like "significant" except in the case of statistical implications. Ultimately, qualifiers should support balanced reporting of data, and the data should inform the reader/consumer and conclusion.

Data omissions are rampant in abstracts, summaries and editorials with authors inflating the importance of some findings while downplaying others, potentially misleading casual readers. “[R]eaders need to critically evaluate an entire article instead of relying on the summary information provided in the abstract,” wrote Brenner.

Advancing clinical knowledge depends on relationships between researchers and vendors, and although financial disclosures offer a simple safeguard they are not a panacea because financial disclosure requirements aren’t always enforced. In addition, initial studies that support a new approach should be validated by unbiased researchers without a financial tie to the company, noted Brenner.

Finally, reviewers of scientific journals need to remain cognizant of their own biases and limitations. New England Journal of Medicine reviewers and radiology journal reviewers reached very different conclusions about the results of the ACRIN Digital Mammography Imaging Screening Trial, he exemplified. “If one considers that reimbursement schedules are predicated on outcome analyses, then the consequences of different results are substantive,” wrote Brenner.

Bias can’t be eradicated, but it can be controlled, he concluded. Both consumers and producers of scientific literature need to be vigilant, particularly as healthcare policymakers use evidence presented in peer reviewed journals to inform decisions.