Not All Research Is Good Research… Especially during a Pandemic
In an editorial for The BMJ, PCHSS researcher Professor Paul Glasziou and colleagues highlight the “deluge of poor quality research” that is undermining an evidence-based response to COVID-19. In “Waste in COVID-19 research”, the authors highlight the ways in which, amidst the unprecedented attention that the coronavirus is receiving, the rush to generate new findings seems to be leading to the cutting of corners.
Pointing to a pre-COVID-19 estimate that up to 85% of research was wasted as a result of “poor questions, poor study design, inefficiency of regulation and conduct, and non or poor reporting of results”, they warn that the pandemic is exacerbating these already serious quality challenges.
On the subject of clinical trials, they mention that well over a thousand COVID-19 studies have been registered in the National Library of Medicine ClinicalTrials.gov registry, but many are too poorly designed and too small to do anything but add to the research “noise” on the subject. For instance, out of 145 registered trials of hydroxychloroquine, 32 have a planned sample size of 100 or less, 10 have no control group, and 12 are comparative but non-randomised.
The pandemic has also led to massive increases in the release of preprints, with MedRxiv increasing its preprints by more than 400% – and at the same time experiencing a 100-fold increase in views and downloads. But preprinting appears to be leading to the diffusion of low-quality research that might ordinarily be corrected before publication. The authors signal that open access to preprints has “led to irresponsible dissemination as flawed studies are picked up by the media.” On this subject, they mention this striking example, again related to hydroxychloroquine:
The preprint of the first reported study of hydroxychloroquine on 20 March 2020—a non-randomised study of 46 patients with inappropriate analyses—has been cited 520 times, while a larger, randomised trial of hydroxychloroquine posted on MedRxiv on 14 April showing no benefits has received far less attention.
Finally, the authors note that waste and duplication in medical research has become a major issue during the pandemic. Stating that while some replication of studies is productive, the lack of a centralised, accessible infrastructure to navigate and coordinate research activity across the globe is leading to unnecessary repetition. Such coordinating bodies are needed for both drug based and non-drug based interventions – the latter of which, the authors highlight, “have so far been the only effective means of control.”
While noting the many positive developments that COVID-19 has brought about in medical research – including increased collaboration, greater accessibility to studies, and expedited approvals – the authors stress the need for the medical research community to continue to strive to be systematic in its activities, even in a time of crisis.