Evaluating the Feasibility of Systematic Review and Meta-Analysis Questions
Introduction
Before launching a systematic review or meta-analysis, researchers must rigorously assess whether the review question is feasible. Feasibility is not merely a logistical concern—it is a scientific gatekeeper. Without sufficient and relevant primary studies, even the most well-crafted review protocol cannot yield meaningful conclusions. Evaluating feasibility ensures that a review question is necessary, novel, and answerable.
This article outlines the strategic checkpoints every reviewer must consider prior to committing resources to a full systematic review or meta-analysis. These checkpoints ensure that the review will be both methodologically sound and clinically valuable.
1. The Role of a “Trial Search” in Scoping Feasibility
What Is a Trial Search?
A trial search is an informal, unsystematic pre-review exploration of existing studies that may be eligible for inclusion in the planned review. The goal is not to build a complete dataset but to estimate the scope and density of relevant evidence.
Key trial search actions include:
- Searching databases using exploratory keywords.
- Manually scanning references of potentially eligible studies.
- Reviewing citations from existing reviews or related protocols.
Why It Matters
This early reconnaissance helps researchers:
- Avoid investing in review questions with sparse or redundant evidence.
- Recognize challenges such as poor indexing or inconsistent terminology.
- Gauge variability in study designs or populations that may impact pooling later.
Illustrative Example: Suppose you plan a review on the use of probiotics to prevent ventilator-associated pneumonia. A trial search might reveal only a few underpowered studies with diverse outcome definitions. This insight could prompt a refinement of the PICO question or even a pivot to a scoping review instead.
2. Assessing Prior Reviews for Redundancy and Gaps
Have Similar Reviews Already Been Conducted?
Before initiating a new review, researchers must determine whether their question has already been answered—fully or partially—by previous systematic reviews.
Key actions:
- Search databases like PubMed, Cochrane Library, and EMBASE for published reviews.
- Search PROSPERO (the international prospective register of systematic reviews) for ongoing or planned reviews.
Checklist for Appraising an Existing Review:
- Is the review older than five years?
- Has substantial new evidence emerged since publication?
- Was the prior review methodologically robust (e.g., had a pre-registered protocol, low risk of bias)?
- Did the review fully address your PICO question, or did it focus on a different population or comparator?
When to Proceed
If a prior review exists but is outdated, limited in scope, or methodologically weak, a new review may be justified—especially if the new one will include more recent studies, improved methods, or broader clinical questions.
3. Determining the Availability of Research Evidence
Minimum Study Requirement
At the heart of review feasibility is the question: Are there enough studies to synthesize?
- For a systematic review, the existence of multiple primary studies is essential to justify a structured synthesis.
- For a meta-analysis, you need at least two sufficiently similar studies whose data can be meaningfully pooled.
The Cochrane guidance confirms that two studies are a minimum requirement, provided they:
- Report comparable outcomes.
- Use similar study designs and populations.
- Are of sufficient quality to permit statistical combination.
Practical Considerations
The degree of similarity among studies affects feasibility more than the absolute number. Heterogeneity in outcome definitions, study populations, or intervention protocols can compromise the validity of pooled estimates.
Rule-of-Thumb Thresholds:
- ≥2 similar studies: Technically sufficient.
- ≥4 somewhat heterogeneous studies: Possible, but interpret with caution.
- ≥6 studies: More reliable synthesis, especially if subgroup analyses or sensitivity checks are planned.
Example: In reviewing the effect of exercise on depression in adolescents, finding only two small trials with different depression scales may limit feasibility. However, six trials using standardized tools would offer more analytic power and robustness.
4. Avoiding Hidden Pitfalls in Sparse Evidence Meta-Analysis
When working with small numbers of studies, particularly fewer than five, two critical risks emerge:
- Underestimation or overestimation of heterogeneity: Estimating variability between studies becomes statistically unstable.
- Misleading precision: Confidence intervals may appear narrower or wider than warranted, depending on model assumptions.
To mitigate these risks, advanced methods—such as the beta-binomial model or Bayesian meta-analytic techniques—can be employed, but they require expert statistical input and careful justification.
Conclusion
Evaluating the feasibility of a systematic review or meta-analysis is a methodological safeguard, not an optional prelude. By conducting an unsystematic trial search, reviewing existing literature critically, and ensuring the presence of adequate, relevant studies, researchers enhance the scientific integrity and practical utility of their reviews.
These early steps help prevent wasted effort, avoid research duplication, and ensure that published reviews are clinically impactful and methodologically defensible.
Key Takeaways
- A trial search helps assess if sufficient, relevant studies exist.
- Prior reviews must be scrutinized for recency, quality, and coverage.
- At least two similar studies are required for meta-analysis; more are better.
- Sparse meta-analyses pose statistical risks—use with caution.
- PROSPERO registration is essential to avoid duplication and promote transparency.