While no study is ever perfect, randomized controlled trials (RCTs) are the most stringent way of assessing the outcomes of a treatment or intervention. A recent study examined the overall quality or RCTs published in high impact journals over the past three decades.
Randomized controlled trials (RCTs) are a type of clinical study where the participants are assigned strictly by chance to one of two groups: a treatment group or a control group. The treatment group receives the intervention while the control group receives either the conventional intervention, a placebo, or no intervention at all. Relevant assessments and evaluations are made at the beginning of the study and repeated during and after the life of the study. The researchers then look to see if there are any outcome differences between groups.
Randomized controlled trials are considered to be well designed when:
- They have a clear hypothesis to test (the researchers are not trawling to find a difference)
- The treatment or intervention will allow researchers to appropriately test their hypothesis
- Participants are screened for eligibility and then randomly assigned to treatment or control groups
- Participants do not know to which group they are assigned
- The participants in the treatment and control groups are identical in all respects except for the intervention being tested, so that different outcomes can be attributed to the intervention and not confounding factors
- Until the clinical trial is over and all data has been analyzed, the researchers do not know which group a participant was assigned to. This is done to minimize researcher bias during the collection and analysis of the data.
Carefully planned and well-executed RCTs provide evidence for efficacy of healthcare interventions. As a result, they are often relied upon to guide clinical decision-making. Yet, with respect to methodological quality, not all RCTs are equal. An international team of researchers (from Paris, France; Ottawa, Canada; Oxford, England; Baltimore and New York, United States) examined the overall quality of RCTs published in journals of impact over the past three decades.
Pulling studies from the Cochrane Reviews database, the research team examined 20, 920 articles on RTCs that were published in 3,136 journals over the past 30 years. These studies were then analyzed, in duplicate, by trained reviewers for their risk of bias:
- If assignment was truly random
- If researchers and the participants were blinded to group assignment
- If outcome measures were appropriate and performed in the same manner for all
- If researchers collecting the data were blind to which group was receiving treatment
- If researcher analyzing data were blind to which group received treatment
- If the groups were similar at baseline
They found that the quality of clinical trials data has improved over the past three decades for the studies published in reputable, peer-reviewed journals. However, they also found that studies published in journals with low-impact factors were more likely to be poorly reported and based upon inadequate RCT methodologies. Moreover, the authors report that many of the problems could have been corrected at the planning stage of the trial in often simple and inexpensive ways.
RCTs are the gold standard for assessing the efficacy of interventions. The public has the right to expect that information used to guide decisions regarding their healthcare is both complete and reliable. The authors of this study suggest RCT quality can be further improved by providing researchers with checklists to promote best research practices and providing constructive feedback to journals to clarify standards for good reporting.
Written By: Debra A. Kellen, PhD