Within-Design and Between-Design Heterogeneity in Network Meta-Analysis

Mayta
4 days ago
3 min read

Introduction

When people first read about network meta-analysis (NMA), they often understand ideas like direct and indirect comparisons, but get stuck on two more technical terms:

Within-design heterogeneity
Between-design heterogeneity

These come from the Q statistic decomposition in NMA (often via the design-by-treatment interaction model). This article explains what they mean, why they exist, and how to interpret them in practice.

1. What Does “Design” Mean in This Context?

In this context, “design” does not mean RCT vs observational.

Instead, in NMA, “design” usually refers to the pattern of treatments compared in a study.

A 2-arm trial comparing A vs B → one design (AB)
A 2-arm trial comparing A vs placebo → another design (A–placebo)
A 3-arm trial comparing A, B, and placebo → design (A–B–placebo)

In simple networks, people often think in terms of comparisons (A–B, A–C, B–C), which is okay for intuition, but formally, the decomposition is by designs (which treatments are in the same study).

2. Within-Design Heterogeneity

“Measures variability within the same comparison (e.g., all A–B trials).This is analogous to heterogeneity in conventional meta-analysis.”

2.1 What it means

Within-design heterogeneity is about:

How much do effect sizes vary among studies that have the same design / comparison?

Think of all trials that directly compare A vs B. If those trials give very different effect estimates, you have high within-design heterogeneity for the A–B comparison.

This is directly analogous to heterogeneity in a standard (pairwise) meta-analysis of A vs B.

2.2 How it is calculated (conceptually)

For each design (e.g., all A–B trials):

Compute a pooled effect for that design (e.g., the mean effect of A vs B)
Calculate how far each study’s effect is from that mean
Weight them by precision (inverse variance)
Sum across all designs

Mathematically (simplified idea):

2.3 Interpretation

Low within-design heterogeneity→ Studies that directly compare the same treatments are reasonably consistent.
High within-design heterogeneity→ Even among studies with the same comparison (e.g., A–B), results differ a lot.→ Possible reasons:
- Differences in populations, doses, and follow-up time
- Methodological differences
- Measurement differences in outcome

This component is not specific to NMA—it’s the same idea as heterogeneity in ordinary meta-analysis, just now calculated for each design within a network.

3. Between-Design Heterogeneity

“Evaluates whether variability differs across comparisons within the network.”

3.1 What it means

While within-design heterogeneity looks inside each comparison, between-design heterogeneity is about:

How much variability comes from differences between designs / comparisons across the whole network?

It captures how the pattern of effects differs across designs.

In other words, it asks:

Do trials with design A–B show a systematically different pattern than trials with design A–placebo or B–placebo?
Is there evidence that the network structure itself (how treatments connect via different designs) contributes extra variability?

3.2 Connection to inconsistency

Between-design heterogeneity is closely related to inconsistency:

In a simple triangle A–B, A–placebo, B–placebo, you can get:
- A direct A–B effect (from A–B trials), and
- An indirect A–B effect (from A-placebo and B-placebo trials)

If these differ a lot, the designs disagree with each other, and between-design heterogeneity increases.

So, between-design heterogeneity often reflects:

Differences in effects that cannot be explained just by random variation within each design
Potential violation of consistency (i.e., disagreement between direct and indirect evidence)

3.3 Mathematical relationship

4. Intuitive Example

Imagine a simple network with three treatments: A, B, and placebo.

You have:

Several A–B trials
Several A–placebo trials
Several B–placebo trials

4.1 Within-design heterogeneity

Look only at all A–B trials. Are their effects similar or widely scattered?
Do the same for A–placebo and B–placebo separately.
Combine this information → within-design heterogeneity.

This answers:

“Are A–B trials internally consistent? Are A–placebo trials internally consistent? etc.”

4.2 Between-design heterogeneity

Now look at the network as a whole:

Compare the direct A–B effect (from A–B trials)to the indirect A–B effect (computed from A–placebo and B–placebo)

If these disagree, the designs (A–B vs {A–placebo + B–placebo}) are in conflict.

This extra conflict is captured by between-design heterogeneity.

It answers:

“Do the different designs in the network tell a coherent story, or do certain pathways disagree?”

5. Why These Two Quantities Matter

Readers often see “within-design heterogeneity” and “between-design heterogeneity” in NMA output without understanding what they mean.

In short:

Within-design heterogeneity→ “How noisy are the results inside each comparison, like normal meta-analysis?”
Between-design heterogeneity→ “How much extra variability comes from disagreement between different comparison patterns (designs) in the network?”

Understanding this decomposition helps you:

Distinguish ordinary heterogeneity (differences between studies with the same comparison)from
Network-level conflict (differences across designs/comparisons that may reflect inconsistency)