How to Critically Appraise a Randomized Controlled Trial (RCT) Using the DDO Framework and Cochrane Tools
- Mayta

- 3 days ago
- 15 min read
Introduction
As clinicians, we constantly face questions such as “Is this drug effective?”, “Is that treatment truly better?”, or “This new study says it works — should we believe it?”These questions come from patients, colleagues, hospital administrators, and even from within our own decision-making as we choose the best treatment for the person in front of us.
Because of this, one of the most important skills for every physician is the ability to read, interpret, and judge the reliability of research findings — especially those from Randomized Controlled Trials (RCTs). RCTs are often considered the highest level of evidence for evaluating therapeutic interventions, and they frequently appear in medical licensing examinations as well as clinical guideline discussions.
However, not all RCTs are created equal. Some are rigorous and trustworthy; others may have subtle flaws that lead to misleading conclusions. Being able to critically appraise an RCT allows us not only to protect our patients from ineffective or harmful treatments, but also to practice medicine with confidence grounded in scientific reasoning.
This article provides a clear, structured framework to help clinicians understand how to read an RCT paper properly — how to judge its design, its credibility, its results, and ultimately, whether the findings should influence our clinical practice.
Below is the next section, written in article style, focusing on how to read the DDO framework, starting with Domain exactly as you requested.
1. Domain: Identifying the Source Population
The first step in reading any RCT is understanding who the study is actually talking about.This is the Domain — the real-world group of people from whom the trial’s participants were drawn.If the Domain does not match your patients, the results may not apply, no matter how strong the trial looks.
When examining the Domain, focus on the following elements:
1.1 Identifying the Source Population
Ask yourself:
Who were the people eligible to enter this trial?
What real clinical population do they represent?
This helps determine the trial’s external validity — how well the results generalize to your practice.A trial done in highly selected, perfectly healthy volunteers may not apply to patients with multiple comorbidities in everyday care.
1.2 Setting: Where Were Participants Recruited?
Understanding the setting helps uncover context-related biases.
Typical settings include:
Hospital-based trials
Often involve more severe cases
May have better monitoring and standardized care
Community or primary care trials
More reflective of everyday practice
Greater variability in adherence and co-interventions
Specialty centers
Patients may differ in expertise level, socioeconomic status, or access to technology
Each setting affects the type of patient, the intensity of care, and the quality of follow-up.
1.3 Country or Region: National vs. Multinational Trials
Geography influences:
Disease prevalence
Standard of care
Access to medications
Cultural or genetic differences
Health system variations
Consider:
National trials may reflect local practice well but lack diversity.
Multinational trials improve generalizability but may introduce heterogeneity in care quality.
Always ask:
“Are these patients similar to the ones I treat?”
If not, external validity becomes limited.
1.4 Any Concerns for Selection Bias?
Selection bias occurs when the participants included in the trial do not truly represent the underlying population.
Clues to detect risk of selection bias:
Recruitment during specific hours or from selective clinics
Heavy exclusion of common comorbidities
Highly motivated volunteers only
Very narrow age range
Trials conducted only in tertiary centers
Lack of clarity on how participants were approached
Even in an RCT, selection bias can occur before randomization, especially if entry into the study is influenced by clinician judgment or participant characteristics.
A biased Domain leads to a “clean” but unrealistic sample.
1.5 Screening Using Inclusion and Exclusion Criteria
Carefully examine:
Inclusion Criteria
What characteristics qualified a patient to enter?
Are they clinically sensible?
Do they reflect the disease you manage daily?
Exclusion Criteria
Excessive or unnecessary exclusions may:
Eliminate older or sicker patients
Remove those with mild disease
Create an artificially “ideal” population
This undermines real-world applicability.
Red flags include:
Excluding common comorbidities
Excluding patients on concomitant medications commonly used in practice
Excluding elderly patients without strong justification
The more restrictive the criteria → the lower the generalizability.
Summary of Domain Appraisal
When reading the Domain section of an RCT, clinicians should systematically evaluate:
Who were the participants?
Where were they recruited?
What country/health system context applies?
Were they selected fairly, or is selection bias likely?
Are the inclusion/exclusion criteria reasonable and clinically representative?
A strong trial with a poor Domain may have excellent internal validity but limited usefulness at the bedside.
2. Determinant: Understanding the Treatment and Its Delivery
After clarifying the Domain, the next critical step in appraising an RCT is examining the Determinant — the intervention and how it was assigned, delivered, and maintained.The Determinant represents the causal factor whose effect the trial seeks to measure.A well-defined and well-implemented Determinant is essential for internal validity.
This section helps you evaluate:
What treatment was given
How groups were formed
Whether allocation was unbiased
How treatment was initiated
Whether blinding and fidelity were preserved
How adherence was monitored
2.1 Treatment Groups
Index Arm (Experimental Treatment)
This is the intervention being tested—e.g., PRP injection, new medication, new surgical technique. Check whether:
The experimental treatment is described clearly
Dosage, schedule, and method are standardized
The rationale for the intervention is explained
A vague or inconsistently applied experimental arm weakens the causal inference.
Control Arm (Comparator)
Controls may include:
Placebo
Sham procedure
Standard of care
Active comparator
A valid control arm allows a fair assessment of treatment effect.Ensure the control:
Reflects ethical and clinical norms
Is similar in appearance, intensity, and patient experience
Unbalanced control conditions introduce performance bias.
2.2 Allocation Into Study Groups
Proper allocation ensures that treatment assignment is truly random and uninfluenced by expectations or clinical judgment.
Sequence Generation
Ask:
How was the randomization sequence created?
Computer-generated?
Random number table?
Block randomization?
Stratified randomization?
Non-random or poorly described sequences increase selection bias.
Allocation Concealment
Concealment prevents foreknowledge of upcoming assignments during enrollment.This is different from blinding.
Gold-standard concealment methods include:
Centralized web-based systems
Pharmacy-controlled randomization
Sequentially numbered, opaque, sealed envelopes (SNOSE)
If investigators could predict the next allocation, the trial is compromised—even if the sequence itself was random.
2.3 Treatment Initiation
Timing: Immediately After Randomization?
Causal validity requires that treatment begins after randomization, not before.If treatment started earlier, baseline imbalances can occur.
Was Blinding Maintained at Treatment Initiation?
The period right after randomization is especially vulnerable:
Did staff know which treatment was assigned?
Did the process expose group identity?
Did differences in preparation reveal group allocation?
Even brief unblinding at initiation can contaminate outcomes.
2.4 Follow-up During and After Treatment
Type of Blinding: Who Was Blinded?
Evaluate blinding at every level:
Patients
Clinicians
Outcome assessors
Data analysts
The more subjective the outcome → the more important blinding becomes.
Partial blinding or unclear blinding increases performance and detection bias.
How Was Treatment Implementation Guaranteed?
A credible RCT ensures that the assigned intervention was actually delivered as intended.
Check for:
Protocol manuals
Standardized treatment procedures
Training of clinicians
Monitoring of deviations
Documentation of adherence to treatment steps
If implementation varies among sites or clinicians, the treatment effect may be diluted or exaggerated.
Protocol-Defined Co-interventions
Ask:
Were other treatments allowed or restricted?
Did both arms receive equal background therapy?
Were rescue medications or supplements controlled?
Without clear rules, differences in co-interventions introduce confounding, even in randomized trials.
Adherence and Compliance Control
Proper adherence is central to interpreting treatment effect.
Look for:
Pill counts
Injection logs
Electronic monitoring
Scheduled follow-up checks
Run-in periods (if any)
Low adherence undermines the fidelity of the experimental contrast and affects ITT vs. PP interpretations.
Summary of Determinant Appraisal
When reading this part of an RCT, ensure:
Treatments are well-defined and fairly compared
Randomization and concealment were done correctly
Treatment initiation occurred only after randomization
Blinding was maintained
Treatment implementation was standardized
Co-interventions were controlled
Adherence was monitored
A strong Determinant ensures that any observed difference between groups is due to the intervention, not to bias, imbalance, or inconsistent practice.
3. Outcome: Defining What the Trial Truly Measures
After understanding who was studied (Domain) and what intervention was tested (Determinant), the next essential step is evaluating the Outcome — what the trial actually measured to determine whether the intervention worked.
Outcomes drive the entire interpretation of an RCT.If outcomes are poorly chosen, poorly measured, or poorly timed, the trial’s conclusions become unreliable, regardless of sample size or statistical significance.
3.1 Primary Outcome / Primary Endpoint
The primary outcome is the central question the trial is powered to answer.
When reading an RCT, ask:
What is the primary outcome?
Is it clinically meaningful?
Is it a patient-important outcome or a surrogate?
Was it pre-specified? (before data collection?)
Was sample size calculated based on this outcome?
A valid primary outcome should be:
Clearly defined
Measured consistently
Relevant to patient care
Not overly influenced by subjective interpretation (unless proper blinding was used)
Examples of strong primary outcomes:
Survival, mortality
Hospitalization rate
Pain reduction measured with validated scales
Disease remission based on standardized criteria
Red flags:
Primary outcome not stated
Surrogate outcomes without justification
Changing primary outcomes post hoc (selective reporting bias)
Vague definitions (“clinical improvement”) without metrics
When the primary outcome is weak, the entire trial becomes fragile.
3.2 Secondary Outcomes / Secondary Endpoints
Secondary outcomes provide supportive information but must never override or replace the primary outcome.
Evaluate:
Are they clinically relevant?
Are they exploratory or pre-specified?
Is there risk of multiplicity (too many outcomes increasing Type I error)?
Are safety outcomes included?
Are they clearly defined?
Secondary outcomes can:
Offer mechanistic insights
Describe broader treatment effects
Highlight benefits beyond the primary endpoint
Identify potential harms
However, positive secondary outcomes cannot justify a negative primary outcome.This is a common misinterpretation seen in lower-quality papers.
3.3 Timing: When Is the End of Treatment or End of Study?
Accurate outcome interpretation requires knowing when outcomes were measured.
Key questions:
At what week/month was the primary outcome assessed?
Is this timing appropriate for the natural course of the disease?
Is follow-up long enough to see the intended effect?
Is it too short, risking under-detection of benefits or harms?
Considerations:
Some interventions act quickly (e.g., analgesics) → short follow-up acceptable
Some require long biological effect time (e.g., PRP, immunotherapy) → short follow-up is inadequate
Safety outcomes may need longer monitoring than efficacy outcomes
Watch for:
Asynchronous timing between arms
Loss to follow-up clustering near the endpoint
Changing the outcome timing mid-study
Improper timing can distort the treatment effect even if the study is randomized.
Summary of Outcome Appraisal
When evaluating the Outcomes section of an RCT, ensure:
Primary outcome is clear, pre-specified, clinically meaningful, and appropriately powered
Secondary outcomes support but never substitute the primary endpoint
Outcome timing matches the disease and intervention biology
Outcome definitions are consistent, objective when possible, and measured using valid tools
Safety outcomes are included and adequately monitored
Well-designed outcomes ensure that the trial answers the real clinical question, not a convenient or biased one.
4. Analysis: How to Evaluate the Statistical and Reporting Integrity of an RCT
Once the Domain (who), Determinant (what), and Outcome (what was measured) are clear, the final step is understanding how the data were analyzed.Even a perfectly designed RCT can be rendered invalid by inappropriate analysis, selective reporting, or incomplete follow-up.
This section outlines the core principles clinicians must evaluate when reading the analysis portion of any RCT.
4.1 Sample Size Estimation: Was the Study Properly Powered?
A robust RCT must justify its sample size before the trial starts.
Key principles:
The calculation should be based on the primary outcome, not secondary outcomes or convenience numbers.
It must specify:
Expected effect size
Standard deviation or event rate
Type I error (usually α = 0.05)
Power (typically 80–90%)
Assumed drop-out rate
Why it matters:
Underpowered studies → false negatives (type II error)Overpowered studies → detect trivial, non-clinical differences
Sample size estimation ensures the study can answer the question it asked.
4.2 Type of Analysis: ITT vs Per-Protocol vs Others
Understanding which analysis strategy was used determines how trustworthy the causal conclusions are.
Intention-to-Treat (ITT)
Analyzes participants according to the group they were randomized to
Preserves randomization
Reflects real-world effectiveness
The preferred method for primary analysis
Per-Protocol (PP)
Includes only those who completed treatment as planned
Estimates efficacy, not effectiveness
Susceptible to selection bias
Should be secondary, not primary
As-Treated (AT)
Analyzes participants based on treatment actually received
Destroys randomization
Essentially observational
CACE (Complier Average Causal Effect)
Useful when compliance differs between groups
Provides a causal estimate among true compliers
What to look for:
Was the primary analysis ITT?
Are secondary analyses (PP, AT) interpreted appropriately?
Did the authors justify deviations from ITT?
The choice of analysis directly affects the credibility of the trial.
4.3 Flow of Patients: Understanding Who Made It Into the Final Analysis
A high-quality RCT clearly shows the flow of participants from enrollment to analysis, usually via a CONSORT diagram.
Concepts to evaluate:
Numbers randomized per group
Numbers analyzed per group
Dropouts and losses to follow-up
Reasons for exclusion after randomization
Symmetry of losses between arms
Why it matters:
High or imbalanced attrition → attrition bias
Exclusion after randomization → compromises allocation
Missing outcome data can mask or exaggerate treatment effects
A robust RCT must account for every participant from randomization through analysis.
4.4 Baseline Characteristics: Were Groups Comparable at Start?
Randomization should create similar groups. This section ensures balance.
Key variables often assessed (conceptually):
Demographics (age, sex, BMI)
Disease severity
Duration of illness
Important comorbidities
Biomarkers
Baseline values of primary outcomes
Any variable known to strongly affect prognosis
How to appraise:
Are baseline differences small and clinically unimportant?
If imbalances exist, did the authors adjust correctly (pre-specified covariates only)?
Was stratified randomization used?
Baseline comparability reassures us that post-treatment differences are due to the intervention, not pre-existing differences.
4.5 Outcome Analysis: Pre–Post Changes and Between-Group Differences
To interpret results properly, distinguish three layers of effect:
A. Within-group changes (Pre vs Post)
Reflect whether each group improved
Can occur due to placebo effect, natural recovery, or regression to the mean
Never used to claim treatment superiority
B. Between-group differences
This is the true treatment effect:
(Post – Pre in Treatment) – (Post – Pre in Control)
This comparison removes natural improvement, placebo effects, and time effects.
C. Confidence Intervals (CI) and P-values
Evaluate:
Does the CI cross zero?
Is the effect clinically meaningful (MCID)?
Are conclusions consistent with statistical evidence?
D. Clinical Meaningfulness (MCID)
Statistical significance ≠ clinical significance.MCID indicates whether the change is noticeable or important for patients.
The interpretation must integrate effect size + CI + MCID, not rely on p-values alone.
4.6 Timing of Outcome Measurement
Outcome interpretation requires correct timing.
Ask:
At what point (week/month) was the primary outcome assessed?
Was follow-up long enough for the treatment to exert its biological effect?
Was safety monitoring adequate?
Timing mismatches distort effect estimation.
Summary of Analysis Appraisal
When understanding the analysis of an RCT, ensure:
Sample size was calculated properly and justified
ITT was used as primary analysis
Flow of patients is transparent and complete
Baseline characteristics are balanced
Results use between-group differences, not just within-group changes
Effect sizes align with clinical relevance (MCID)
Outcome timing is appropriate
All findings are supported by proper statistical logic
A robust analysis section transforms raw data into meaningful clinical evidence.
5. Assessing Study Quality Using the Cochrane Risk of Bias Tool (RoB 1)
Even a well-designed RCT can be undermined by bias introduced during conduct or analysis.To evaluate the internal validity of an RCT, clinicians commonly use the Cochrane Risk of Bias Tool (RoB 1)—a structured framework that examines where hidden errors might distort the estimated treatment effect.
RoB 1 focuses on several key domains of bias. Understanding each domain allows clinicians to decide whether the trial’s findings are trustworthy.
Below is a practical way to apply the tool in routine appraisal.
5.1 Selection Bias
What it is:Bias arising from the way participants are assigned to groups before treatment begins.
What to check:
Random sequence generation
Was the allocation truly random?
Was a method like computer randomization or random number tables used?
Allocation concealment
Could clinicians predict or manipulate group assignment?
Were sealed envelopes, central randomization, or pharmacy-controlled allocation used?
Why it matters:If group assignment is predictable or influenced, the groups may differ systematically at baseline, undermining causal inference.
Red flags:
“Randomization” not described
Alternating assignment (“every other patient”)
Non-opaque envelopes
Recruiters aware of next assignment
5.2 Performance Bias
What it is:Bias related to differences in care, co-interventions, or patient behavior due to awareness of treatment assignment.
What to check:
Was blinding of participants performed?
Was blinding of treating clinicians maintained?
Were both groups treated equally aside from the intervention?
Was patient attention or monitoring balanced?
Why it matters:Knowledge of group assignment can influence:
Co-interventions
Adherence
Placebo effects
Clinician enthusiasm or caution
These can distort the estimated treatment effect.
Red flags:
Unblinded clinicians when outcomes are subjective
Unequal follow-up intensity
Different co-interventions allowed between arms
5.3 Detection Bias
What it is:Bias occurring if the outcome assessors know which treatment each patient received.
What to check:
Were outcome assessors blinded?
Were outcomes objective (e.g., mortality) or subjective (e.g., pain scores)?
Were assessment tools standardized and validated?
Why it matters:Unblinded assessors may unconsciously rate outcomes differently based on expectations.
Red flags:
Subjective outcomes without assessor blinding
“Assessors were not informed” stated without describing blinding procedures
Outcomes measured by treating clinicians who know the study arm
5.4 Attrition Bias
What it is:Bias from incomplete outcome data due to dropouts, withdrawal, or protocol deviations.
What to check:
Percentage of loss to follow-up in each group
Whether losses were balanced across arms
Reasons for discontinuation
Whether intention-to-treat (ITT) was used
Whether missing data were handled appropriately
Why it matters:High or uneven attrition can change the apparent treatment effect, especially if related to treatment tolerability or lack of improvement.
Red flags:
10% drop-out (moderate concern)
20% drop-out (high concern)
Many exclusions after randomization
Analysis restricted to “completers only”
No explanation of how missing data were handled
5.5 Overall Bias Judgment
Once each domain is evaluated, an overall risk of bias judgment reflects the confidence in the study’s internal validity.
High Overall Risk of Bias
One or more domains rated high risk
Multiple domains unclear
Serious concerns about randomization, blinding, or attrition
Low Overall Risk of Bias
All domains judged low risk
Study design and conduct strongly support internal validity
Unclear Risk
Insufficient information reported
Cannot determine whether bias occurred
Why overall bias matters:This final judgment determines whether clinicians can rely on the reported treatment effect—or whether the effect may be exaggerated or unreliable.
Summary: How to Use RoB 1 in Clinical Practice
When reading an RCT, clinicians should systematically evaluate:
Selection Bias: Was randomization and concealment sound?
Performance Bias: Were participants and clinicians blinded?
Detection Bias: Were assessors blinded and outcomes objectively measured?
Attrition Bias: Was follow-up complete, and was ITT used?
Overall Bias: Does the trial provide trustworthy estimates of treatment effect?
A trial with low risk of bias across domains provides high-confidence evidence for clinical decisions.A trial with high risk of bias should be interpreted cautiously, no matter how impressive the reported results appear.
6. Using the Updated Cochrane Risk of Bias Tool (RoB 2): A Modern Framework for Evaluating RCT Quality
The original Cochrane Risk of Bias Tool (RoB 1) remains widely used, but clinical research has evolved. To address limitations of the earlier version—especially issues around selective reporting, protocols, and deviations from intended interventions—the Cochrane Collaboration introduced the Risk of Bias 2 (RoB 2) framework.
RoB 2 is more structured, more outcome-specific, and better aligned with how modern RCTs are conducted and analyzed.Instead of rating “the study” globally, RoB 2 evaluates risk of bias for each outcome, acknowledging that different outcomes can have different biases.
This section introduces the five domains of RoB 2 and how clinicians should apply them in everyday appraisal.
6.1 Domain 1 — Bias Arising From the Randomization Process
This domain assesses the integrity of the allocation process.
What to evaluate:
Was the random sequence generation truly random?
Was allocation concealed from recruiters?
Were baseline differences between groups minor and compatible with chance?
Why it matters:
If randomization or concealment is compromised, groups may differ systematically at baseline, invalidating causal inference.
Signals of concern:
Non-random or poorly described randomization
Imbalances in key baseline variables
Predictable assignment processes
Lack of allocation concealment description
6.2 Domain 2 — Bias Due to Deviations From Intended Interventions
This replaces the RoB 1 “Performance Bias” domain and incorporates both adherence and protocol deviations.
RoB 2 distinguishes between:
Effect of assignment (intention-to-treat estimand)
Effect of adhering to protocol (per-protocol estimand)
What to evaluate:
Were participants and clinicians aware of group assignment?
Did deviations occur because of this awareness?
Were co-interventions balanced?
Was adherence measured and maintained?
Was ITT analysis performed properly?
Why it matters:
Deviations from intended intervention can dilute or exaggerate treatment effects, especially when outcomes are subjective.
Signals of concern:
Unblinded clinicians providing different co-interventions
Poor adherence monitoring
Significant crossover not corrected by appropriate analysis (e.g., CACE)
6.3 Domain 3 — Bias Due to Missing Outcome Data
This corresponds to the RoB 1 “Attrition Bias” but adds nuance about the mechanism of missingness.
What to evaluate:
Proportion of missing data
Whether missingness differs between groups
Whether reasons for missing data relate to true outcomes
Whether appropriate statistical handling (e.g., multiple imputation) was used
Whether ITT or modified ITT was executed properly
Why it matters:
Missing data can distort effect estimates if dropouts differ between groups or relate to prognosis.
Signals of concern:
High or imbalanced loss to follow-up
Missingness related to lack of efficacy or adverse effects
Analyses restricted to complete cases only
6.4 Domain 4 — Bias in Measurement of the Outcome
This is the RoB 2 revision of “Detection Bias.”
What to evaluate:
Were outcome assessors blinded?
Were measurement tools valid, reliable, and standardized?
Was the outcome vulnerable to subjective interpretation?
Was the timing of measurement appropriate?
Why it matters:
Assessor knowledge of group assignment can systematically skew outcome evaluation, especially subjective outcomes such as symptom scores, satisfaction, or pain.
Signals of concern:
Unblinded assessors for subjective endpoints
Tools not validated or inconsistently applied
Different assessment frequency between groups
6.5 Domain 5 — Bias in Selection of the Reported Result
This domain addresses selective reporting and analytical flexibility—problems not well captured in RoB 1.
What to evaluate:
Was the trial protocol pre-registered?
Was the primary outcome changed after study initiation?
Were multiple analyses run but only favorable ones reported?
Are subgroup analyses pre-specified or post hoc?
Why it matters:
Selective reporting can create false impressions of treatment benefit and distort evidence synthesis.
Signals of concern:
Primary outcome differs between protocol and publication
Unexplained switching of statistical methods
Emphasis on secondary outcomes when primary outcome is negative
Overuse of unplanned subgroup analyses
6.6 Overall Bias Judgment in RoB 2
RoB 2 produces an overall rating for each outcome:
Low Risk of Bias
All domains rated “low risk,” or
Only minor concerns exist
Some Concerns
Unclear reporting
One or more domains have issues that may affect validity but are not clearly high risk
High Risk of Bias
At least one domain has high risk
Multiple domains have “some concerns”
Analysis seriously deviates from intended estimand
Key difference from RoB 1:RoB 2 emphasizes estimand-specific bias and outcome-level judgement, not just study-level appraisal.
7. How RoB 1 vs RoB 2 Change Interpretation
Concept | RoB 1 | RoB 2 |
Unit of assessment | Whole study | Specific outcome |
Bias domains | 6 classic domains | 5 modern, mechanistic domains |
Reporting model | Checklists | Algorithm-driven pathways |
Selective reporting | Less explicit | Dedicated domain |
Deviations from intervention | Simpler | Estimand-based (ITT vs PP) |
Missing data | Proportion-focused | Mechanism-focused |
RoB 2 provides deeper causal logic, aligning with modern therapeutic trial methodology and better reflecting contemporary CONSORT reporting standards.
Summary: Using RoB 2 in Practice
When applying RoB 2 to any RCT:
Evaluate randomization and concealment integrity
Assess whether deviations from intended interventions biased the effect
Examine missing data patterns and handling
Determine whether outcome measurement was unbiased
Check for selective reporting against pre-registered protocols
A trial with few concerns across all domains provides strong, trustworthy evidence.A trial with high risk in any domain requires cautious interpretation, regardless of statistical significance.





Comments