Due to the constraints of an RCT, results may not be valid outside the trial framework. Pragmatic trials attempt to address this limition in external validity by testing an intervention under routine conditions in a heterogeneous patient population. These routine conditions may include a broad range of adjustments. First, pragmatic trials may have broad inclusion criteria; ideally trial patients are only excluded if they are not intervention candidates outside of the study. Second, the intervention may be compared to routine practice, and clinicians and patients may not be blinded. This approach accepts that placebo effect may augment intervention outcomes when used in routine practice. Third, pragmatic trials may use routine clinic staff rather than topic experts, and staff may be encouraged to adjust the medication or intervention as they would in routine practice. Fourth, patient-reported outcomes may be measured in addition to—or instead of—traditional outcomes. Finally, patients are usually analyzed according to their assigned intervention arm; this is also known as intention-to-treat analysis. Pragmatic trials may range anywhere along this spectrum: on one end, an otherwise traditional RCT may use an intention-to-treat analysis; on the other, investigators may aim to conduct the study under completely routine circumstances with the exception of intervention randomization. The investigators must determine what level of pragmatism is appropriate for their particular research question.

Pragmatic trials help determine medication or intervention effectiveness in a more realistic clinical setting. The adjustments that make pragmatic trials more realistic, however, also create limitations. Pragmatic studies are conducted under routine clinical circumstances, so an intervention that is effective in a large, well-funded private clinic may not be equally effective in a safety-net clinic. Therefore, clinicians must consider the study setting before instituting similar changes in their own practice. In addition, pragmatic trials include a broad range of eligible patients and consequently contain significant patient heterogeneity. This heterogeneity may dilute the treatment effect and necessitate large sample sizes and extended follow-up periods to achieve adequate statistical power. This may then inflate study cost and counterbalance any money saved by conducting the trial in a routine clinic with routine staff.

Cluster randomized trials (CRTs) are defined by the randomization of patients by group or practice site rather than by individual. Beyond group randomization, CRTs may use either traditional RCT techniques or pragmatic trial techniques. Group level randomization has multiple effects. Group contamination is uncommon since participants are less likely to know study participants from other sites. This makes CRTs ideally suited for interventions that are organizational, policy-based, or provider-directed. With these education or resource-based interventions, well-intentioned participants or physicians may distribute information to control-arm participants without the investigator’s knowledge. Risks of cross contamination are significantly lowered when participants from different study arms are separated by site and less likely to know one another. Group randomization also better simulates real-world practice since a single practice usually follows the same treatment protocol for most of its patients. Finally, physicians and clinic staff can be educated on the site-specific intervention and then care for patients under relatively routine circumstances. In some circumstances this may help to coordinate blinding, minimize paperwork, and reduce infrastructure and personnel demands.

Clustered patient randomization, however, introduces analytical barriers. Patients often choose clinics for a specific reason, so patient populations may differ more among clinics than within clinics. Furthermore, differences may not be easily measurable (e.g., patients may differ significantly in how much education they expect and receive prior to starting a new medication), so adjusting for these differences may be difficult during analysis. Consequently, cluster randomization requires hierarchical modeling to account for similarities within groups and differences between groups, but hierarchical modeling produces wider confidence intervals. As with pragmatic trials, this may require increases in subject number and follow-up time in order to detect clinically significant outcome differences. Unfortunately, individual participant enrollment is usually limited within each cluster, so increasing a trial’s statistical power usually requires enrollment of additional clusters.