Pooling 95% Confidence Intervals and Effect Sizes: Fixed-effect, Random-effects, Bootstrap, or Rubin's Rules?

อ่านฉบับภาษาไทย (Thai version)

Abstract

Several effect estimates often arrive together, each with a 95% confidence interval. The first question is where they came from. A confidence interval is computed only after an estimate and a variance exist. What is combined is whatever built the interval, never the limits. The estimates can come from three places, and the first of them splits further by estimand: the quantity the analysis is estimating. Independent studies assumed to share one common effect call for fixed-effect meta-analysis, weighted by inverse variance. Independent studies whose true effects may differ call for random-effects meta-analysis, which adds a between-study variance. Resamples of one dataset are combined through the distribution of bootstrap replicates. Datasets built by filling in missing values are combined with Rubin's rules. The model is named from the estimand, before any heterogeneity statistic is seen. This article walks all four routes with a decision chart, two worked examples, a way to recover the standard error from a published interval, and a widget that raises the between-study variance from zero.

---

🔍 Click to enlarge Click anywhere to close

What pooling actually combines

Say you are holding several effect estimates, each with its own confidence interval, from several sources. The simplest thing to try is averaging every lower limit together, then averaging every upper limit together.

\[ \bar{L} = \frac{L_1 + L_2 + \cdots + L_k}{k} \quad \bar{U} = \frac{U_1 + U_2 + \cdots + U_k}{k} \]

This calculation always returns two numbers, but the pair is not a 95% interval for anything. The reason is weight. Each estimate carries a different precision, so a value with a small standard error should count for more than a value with a large one.

Averaging the limits gives every estimate the same weight automatically, so the different precisions vanish at the first step. The midpoint of the pair is then the unweighted mean of the estimates, which throws precision away. Its half width is the typical half width of one study, and it does not shrink as studies are added. With k studies of similar size the pair comes out roughly the square root of k times too wide, so it buries a real effect rather than inventing one.

The correct method goes back to the raw ingredients first: each effect estimate, and the variance or standard error that goes with it. A pooling model matching where those values came from is chosen next. The model returns a pooled estimate and a pooled standard error, and only then is the new confidence interval computed from those two numbers.

The origin of the estimates splits into three kinds: independent studies of the same question, repeated resamples of one dataset, and imputed datasets built from one dataset with missing data. The first of those three splits once more, not by origin but by estimand, into fixed-effect and random-effects. That is why four methods come out of three origins, and the rest of the article walks them one at a time.

One more situation deserves its own warning, because it is not on that list at all: several estimates that come from the same patients. Subgroups of one cohort, per-centre results from one trial, and several outcomes measured on the same people all look like several independent estimates. They are not independent, so inverse-variance pooling counts the same patients twice and returns an interval that is too narrow. The answer is one model on the combined individual data, or a method that carries the correlation.

⤢ click to enlarge

Where did the estimates come from?

Fixed-effect: one common effect, weighted by precision

Fixed-effect meta-analysis has a single estimand, which is the quantity the analysis is trying to estimate. Here that quantity is the one underlying effect every study is assumed to be measuring. Any numeric difference seen between studies is then explained by sampling error alone, the random variation from sampling a limited number of patients.

Some authors call this the common-effect model, and reserve the plural fixed-effects for a precision-weighted average of effects that may genuinely differ. The weights are the same in both, but the estimand is not.

When the estimand looks like this, it is only fair to let a more precise study speak louder. The standard way to do that is inverse-variance weighting, which gives each study a weight equal to the reciprocal of its variance. For binary outcomes with few events these weights behave poorly, and the Mantel-Haenszel method, which weights studies by their raw event counts instead, is preferred. The principle is unchanged, only the weights differ.

\[ w_i = \frac{1}{SE_i^{2}} \]

The pooled effect is simply the weighted average of all the estimates, using this set of weights.

\[ \hat{\theta}_{FE} = \frac{\sum_i w_i \hat{\theta}_i}{\sum_i w_i} \]

In practice the effect can be read straight off the width of the interval. A study with a narrow confidence interval has a small standard error, so it earns a large weight, while a study with a wide interval earns a small one. This is exactly the information that averaging the limits throws away.

For ratio measures such as the odds ratio, risk ratio, and hazard ratio, the raw scale is skewed. Every step, from finding the weights to building the interval, must run on the log scale, and only the pooled estimate and both limits are exponentiated back to the original scale at the end.

What the analysis needs from each study is therefore only two things: its effect estimate and its standard error, not the lower and upper limits alone. The Cochrane Handbook, chapter 10, describes fixed-effect meta-analysis on exactly this assumption, and calls for the same basic ingredients. Worked example 1 below walks through every step with three illustrative risk ratios.

Random-effects: a second source of variability

In clinical research, the assumption of one single true effect is often unrealistic. The effect of the same drug can differ by country and hospital, by age group, disease severity, dose or treatment protocol, follow-up time, and clinical setting.

A random-effects model therefore lets each study have its own underlying effect, treating those effects as if they were drawn from one shared distribution. Uncertainty now comes from two sources: variability within a study, and variability between studies.

\[ w_i^{*} = \frac{1}{SE_i^{2} + \tau^{2}} \]

The quantity $\tau^{2}$ is the between-study variance, which measures how much the true effect itself differs from study to study. When $\tau^{2}$ is greater than zero, three predictable things follow.

• smaller studies gain relative weight compared with the fixed-effect model
• the pooled confidence interval usually widens, because another source of variability is now counted
• the interpretation shifts from one single common effect to the mean of a distribution of effects

Random-effects does not mean the heterogeneity has been solved. The clinical and methodological reasons behind that variation still need attention on their own.

The caveat is that the confidence interval around a pooled random-effects estimate only describes uncertainty about that mean. It says nothing about how widely the true effect varies across settings.

When heterogeneity is clinically meaningful, reporting a prediction interval answers that question more directly, because it gives the range a new, similar study or setting's true effect is likely to fall in. This range is always wider than the confidence interval around the mean.

A prediction interval needs a reasonable number of studies before it can be trusted. It carries the uncertainty in the between-study variance as well as the uncertainty in the mean. With only a handful of studies it can come out spuriously wide or spuriously narrow, so it is reported with that caution or not at all.

One more caution belongs here. When the number of studies is small and the between-study variance is above zero, the usual normal-based interval around the random-effects mean is too narrow. The Cochrane Handbook now prompts authors towards the Hartung-Knapp-Sidik-Jonkman interval in that situation, which swaps the normal multiplier for a t value and comes out wider.

Several methods exist for estimating $\tau^{2}$. DerSimonian-Laird is the classic choice and is still used widely, but it should not be called the one standard method. The Cochrane Handbook, chapter 10, section 10.10.4, notes that other estimators have better statistical properties in many situations.

RevMan now uses restricted maximum likelihood, or REML, as the default estimator of between-study variance for inverse-variance random-effects analyses, while keeping DerSimonian-Laird available as an option. If the statistical method is set to Mantel-Haenszel instead, the estimator reverts to DerSimonian-Laird.

Choose the model from the estimand, not from I-squared

A common mistake is letting the value of I-squared choose the model instead of the researcher. I-squared is the share of the variability that comes from real differences between studies rather than from sampling error. The simple rule often quoted is to set an I-squared threshold, then use fixed-effect below it and random-effects above it.

This rule is too simple, because a threshold answers no scientific question at all. The question that must be answered first is whether the studies are believed to be estimating the same effect, or whether the mean of effects that can differ across settings is what matters.

The answer to that question is the estimand, and the estimand must be written into the analysis plan before the heterogeneity results are seen. Choosing the model after the numbers are already known lets that choice depend on the dataset, unintentionally.

There is also a supporting statistical reason. Cochrane Handbook chapter 10 warns that both I-squared and tau-squared carry high uncertainty when the number of studies is small. Applying a hard threshold to a value that is itself poorly estimated is a decision built on a number that is not certain.

Sometimes the right answer is not to pool at all. When the populations, interventions, or outcomes differ so much that combining them has no clinical meaning, reporting each study separately with an explanation is more honest.

A random-effects model cannot turn studies that should not be combined into studies that should be. It only changes the question the pooled result is answering, not the clinical differences underneath it.

The same judgement settles a case that neither model can fix: estimates that share patients. Several subgroups of one cohort, or several outcomes measured on the same people, are not independent evidence. Pooling them counts the same patients more than once, and no choice between fixed-effect and random-effects repairs that.

Bootstrap: many replicates, one interval

Bootstrap lives in a different world from meta-analysis, in that there is only one dataset, not several studies. What is wanted is the uncertainty of one statistic computed from that single dataset.

The method resamples observations from the original data with replacement, meaning one observation can be drawn more than once, then refits the same analysis every time. Doing this 10,000 times gives one large set of estimates, called bootstrap replicates, meaning one estimate computed from one resample.

A percentile bootstrap interval reads its value straight off the distribution of replicates. The replicates are sorted, the 2.5th percentile becomes the lower limit, and the 97.5th percentile becomes the upper limit. One interval, and it is done.

The common misunderstanding is thinking there are 10,000 intervals to average. There are only that many replicated estimates, and their distribution builds one confidence interval. This is the heart of the percentile bootstrap.

Say a machine's limits force the bootstrap to run as five batches of 2,000 replicates each. If every batch comes from the same original dataset, uses the same resampling scheme, computes the same statistic, and was drawn with a different random seed, the replicates can all be pooled together. The different seed is what makes the batches independent: five runs from one seed would stack five identical copies of the same replicates. The quantiles are then computed once, from the combined set.

What is being done here should be called combining bootstrap replicates, not pooling several bootstrap intervals. The name matters, because it says what is actually combined: replicates, not the ends of an interval.

Finally, percentile is not the only way to build a bootstrap interval. There is also the basic bootstrap interval, the studentized interval, the bias-corrected interval, and the BCa interval, which corrects for bias and for how the variance changes with the estimate. Each name says which correction it adds: reflecting the replicates around the estimate, dividing by each replicate's own standard error, or shifting the limits for bias.

BCa gives better coverage in many settings, meaning its intervals contain the true value as often as they claim to. Choosing percentile is therefore a choice, not a universal default. BCa is not automatically the safer option either, because it needs a large number of replicates and a reasonably smooth statistic.

Multiple imputation: Rubin's rules

Multiple imputation fills in missing values with values drawn at random from an imputation model, repeated many times until there are m imputed datasets. Each one is complete on its own, but differs in the values that were filled in. The rules that follow are valid only if that model is a reasonable one and the data can be treated as missing at random given the variables it contains. Missing at random means that whether a value is missing depends only on other values we did observe, not on the missing value itself.

The same analysis model is then fitted in every dataset, giving one estimate and one variance per dataset. Rubin's rules are the set of formulas that combine those pairs into a single answer.

The first step is the pooled estimate, which is the plain average of all m estimates, with no weighting, because every imputation comes from the same dataset and holds the same status.

The second step is the within-imputation variance, the average of all m variances. This part stands for the ordinary sampling error, the uncertainty that would exist even if the data were complete.

The third step is the between-imputation variance, which measures how much the estimate moves from dataset to dataset. This is the price of part of the data never having been observed, and the total variance is the sum of these two parts, plus a correction for having only a finite number of imputations.

\[ T = \bar{U} + \left(1 + \frac{1}{m}\right) B \]

The imputation model must also be proper, meaning each filled-in value is drawn with the extra random noise that reflects real uncertainty rather than a single best guess, and it must be compatible with the analysis model. An imputation model that leaves out the outcome, or leaves out an interaction the analysis model contains, gives a pooled variance that is biased. The imputation model is therefore built with the analysis model already in hand. If the missingness depends on the unobserved value itself, no pooling formula repairs it.

The pooled standard error is the square root of T, and the confidence interval is then built from the pooled estimate and this pooled standard error. Inference from multiple imputation also needs the right degrees of freedom, the number that sets how wide the t multiplier must be. Barnard and Rubin propose a small-sample form that matters when m is small or the proportion of missing data is high.

When the desired result is an odds ratio, always pool at the level of the regression coefficient, on the log odds scale, first. The log odds scale is simply the natural logarithm of the odds, which is what a logistic coefficient already reports. Only then are the pooled coefficient and both limits exponentiated back into an odds ratio.

Two things must never be done: reporting the result from a single imputed dataset, because it hides the uncertainty the imputation itself created, and averaging the confidence interval limits from several datasets. Worked example 2 walks through every step with numbers.

When all you have is an estimate and its interval

In many published articles there is no raw data and no reported standard error, only an effect estimate and the lower and upper limits of a confidence interval. Meta-analysis is still possible in this case, if one more assumption is accepted.

That assumption is that the normal approximation holds. The published interval was built symmetrically around the estimate, on the scale used for the analysis, using a critical value from the normal distribution. A critical value is the multiplier applied to a standard error to reach the interval limits.

When that assumption holds, the standard error can be recovered directly from the width of the interval. For additive effect measures, such as a mean difference or a risk difference, the formula is the width of the interval divided by twice the normal critical value, that is by 3.92.

\[ SE \approx \frac{U - L}{3.92} \]

For ratio measures such as the risk ratio, odds ratio, and hazard ratio, the same work must first move to the log scale, replacing $U$ and $L$ with $\ln(U)$ and $\ln(L)$ in the same formula. Step 2 of the worked example below applies this formula to all three studies.

Once the standard error is recovered, everything returns to the normal path: inverse-variance weighting, then a model chosen from the estimand. A published confidence interval already carries its standard error; it only needs to be read out on the right scale.

The caveat is that this formula is an approximation, and it fails in two ways. The interval may not be symmetric on the scale used, for example because it came from an exact method that builds limits from exact probabilities, not a normal curve. Or it may be symmetric but built on a critical value from the t distribution rather than 1.96, which is what a small trial does. Dividing by 3.92 then understates the standard error and gives that study too much weight.

The divisor also belongs to the 95% level alone, so a 90% interval is divided by 3.29 instead. When either problem applies, it is better to ask the original authors for the standard error or the raw data.

Five ways a pooled interval goes wrong

• Averaging the lower and upper limits

  The mean of several lower limits is not a lower limit of anything, because the studies carry different variances.

  Fix: Recover each standard error first, then weight by inverse variance.

• Pooling ratio measures on the original scale

  The sampling distribution of a ratio measure is skewed, so weights and intervals built on the raw ratio are wrong.

  Fix: Pool on the log scale, then exponentiate the pooled estimate and both limits.

• Letting I-squared choose the model

  A threshold on I-squared answers no scientific question, and I-squared is imprecise when studies are few.

  Fix: Choose from the estimand, before the heterogeneity statistics are seen.

• Pooling five bootstrap intervals

  Five batches of replicates give one distribution, not five intervals to average.

  Fix: Stack the replicates, then take the quantiles once.

• Reporting one imputed dataset

  One imputed dataset hides the uncertainty that the imputation itself created.

  Fix: Fit the model in every imputed dataset, then pool with Rubin's rules.

Worked example 1: three studies, fixed-effect and random-effects

This example uses three hypothetical studies from the source note. Every value is reported as a risk ratio with a 95% confidence interval, and none of them comes from a real study.

• Study A: RR 0.80 (0.70 to 0.91)
• Study B: RR 0.92 (0.71 to 1.19)
• Study C: RR 0.77 (0.72 to 0.83)

Study C has the narrowest interval, so it has the highest precision and will earn the largest weight under the fixed-effect model. Study B has the widest interval and will earn the smallest weight.

The steps below walk from the published risk ratios to the pooled estimate, under both fixed-effect and random-effects. Every line can be redone with a calculator; intermediate values are rounded to four decimal places, and to two decimal places once converted back to a risk ratio.

1. Step 1. Move to the log scale

   \[ \ln(0.80) = -0.2231, \; \ln(0.92) = -0.0834, \; \ln(0.77) = -0.2614 \]

   A risk ratio is skewed, so every step runs on the log scale and comes back at the end.

2. Step 2. Recover each standard error from its interval

   \[ SE_{\log} = \frac{\ln(U) - \ln(L)}{3.92} \Rightarrow 0.0669, \; 0.1317, \; 0.0363 \]

   The published interval already carries the standard error, once it is read on the log scale.

3. Step 3. Build the inverse-variance weights

   \[ w_i = \frac{1}{SE_i^2} \Rightarrow w_A = 223.43, \; w_B = 57.65, \; w_C = 758.90, \; \sum w_i = 1039.98 \]

   Study C holds 72.97 percent of the weight and study B only 5.54 percent. Each weight is computed from the standard error exactly as printed above, rounded to four decimal places. Carrying full precision moves the weights in the third digit and leaves the pooled risk ratio and its interval unchanged.

4. Step 4. Pool the log risk ratios

   \[ \hat{\theta}_{FE} = \frac{-253.0317}{1039.98} = -0.2433 \]

   The weighted mean of the three log risk ratios is the fixed-effect pooled estimate.

5. Step 5. Get the pooled standard error and interval

   \[ SE = \frac{1}{\sqrt{1039.98}} = 0.0310, \; -0.2433 \pm 1.96 \times 0.0310 = (-0.3041, \; -0.1825) \]

   The pooled standard error comes from the total weight, not from any single study.

6. Step 6. Back-transform the fixed-effect result

   \[ e^{-0.2433} = 0.78, \; e^{-0.3041} = 0.74, \; e^{-0.1825} = 0.83 \]

   Fixed-effect result: RR 0.78, 95% CI 0.74 to 0.83.

7. Step 7. Ask how much the studies really disagree

   \[ Q = 1.8138, \; df = 2 \Rightarrow \hat{\tau}^2_{DL} = 0 \]

   Q is Cochran's heterogeneity statistic, which compares the spread seen across studies with the spread sampling error alone would give. Q falls below its degrees of freedom here, so the classic estimator is truncated to a between-study variance of 0. That is a boundary value, not proof that the true effects agree. With only three studies a zero estimate is still compatible with real heterogeneity, and the model was chosen from the estimand before this number was seen.

8. Step 8. Set an illustrative between-study variance

   \[ w_i^{*} = \frac{1}{SE_i^2 + 0.01} \Rightarrow 69.08, \; 36.57, \; 88.36, \; \sum w_i^{*} = 194.01 \]

   Set tau-squared to 0.01 by hand to see what real heterogeneity would do to the weights.

9. Step 9. Pool again under random-effects

   \[ \hat{\theta}_{RE} = \frac{-41.5590}{194.01} = -0.2142, \; SE = 0.0718, \; -0.2142 \pm 1.96 \times 0.0718 = (-0.3549, \; -0.0735) \]

   Weight now spreads towards the smaller studies: C falls from 72.97 to 45.54 percent.

10. Step 10. Back-transform the random-effects result

    \[ e^{-0.2142} = 0.81, \; e^{-0.3549} = 0.70, \; e^{-0.0735} = 0.93 \]

    Random-effects result with that illustrative value: RR 0.81, 95% CI 0.70 to 0.93.

11. Step 11. A safer interval when the studies are few

    \[ SE_{HK} = \sqrt{\frac{1}{k-1} \cdot \frac{\sum_i w_i^{*} (\hat{\theta}_i - \hat{\theta}_{RE})^{2}}{\sum_i w_i^{*}}} = 0.0462, \; -0.2142 \pm 4.3027 \times 0.0462 = (-0.4130, \; -0.0154) \]

    With only three studies, a normal critical value makes the random-effects interval too narrow. The Hartung-Knapp-Sidik-Jonkman interval uses a t value on k minus 1 degrees of freedom instead. Back-transformed it gives RR 0.81, 95% CI 0.66 to 0.98, which is the safer report here.

Result: Fixed-effect gives RR 0.78, 95% CI 0.74 to 0.83. With tau-squared set to 0.01 the random-effects answer is RR 0.81, 95% CI 0.70 to 0.93, and the Hartung-Knapp-Sidik-Jonkman interval on the same numbers is RR 0.81, 95% CI 0.66 to 0.98.

The three risk ratios are illustrative studies from the source note, and the value 0.01 is chosen by hand, not estimated from data. The Hartung-Knapp figures are computed from the same rounded weights as the steps above.

Worked example 2: Rubin's rules on five imputed datasets (synthetic)

The numbers in this example are synthetic data, invented specifically for this article. None of them comes from any real dataset or study.

The setup is one dataset with missing data, filled in with multiple imputation to produce m equal to 5 imputed datasets. The same logistic regression is then fitted in every dataset, giving one regression coefficient on the log odds scale per dataset, along with its standard error.

The steps below walk from all five coefficients to the pooled odds ratio and its interval. Every line can be redone with a calculator, and the displayed values are rounded to the number of decimal places written at each step.

1. Step 1. Collect the estimate from each imputation

   \[ \beta_j = 0.42, \; 0.51, \; 0.38, \; 0.47, \; 0.44 \]

   Five imputed datasets give five log odds estimates from the same logistic model.

2. Step 2. Square each standard error

   \[ SE_j = 0.18, \; 0.19, \; 0.17, \; 0.20, \; 0.18 \Rightarrow U_j = 0.0324, \; 0.0361, \; 0.0289, \; 0.0400, \; 0.0324 \]

   Each imputation reports its own standard error, and its square is that imputation's variance.

3. Step 3. Average the estimates

   \[ \bar{Q} = \frac{0.42 + 0.51 + 0.38 + 0.47 + 0.44}{5} = 0.444 \]

   The pooled log odds is the plain average of the five estimates.

4. Step 4. Within-imputation variance

   \[ \bar{U} = \frac{0.1698}{5} = 0.03396 \]

   Within-imputation variance is the average of the five variances, the ordinary sampling error.

5. Step 5. Between-imputation variance

   \[ B = \frac{0.00972}{5 - 1} = 0.00243 \]

   Between-imputation variance measures how much the answer moved because the data were imputed.

6. Step 6. Total variance

   \[ T = 0.03396 + \left(1 + \frac{1}{5}\right) \times 0.00243 = 0.036876 \]

   The correction of 1 plus 1 over 5 pays for having only a finite number of imputations.

7. Step 7. Pooled standard error

   \[ SE_{pooled} = \sqrt{0.036876} = 0.1920 \]

   The pooled standard error is larger than the standard error of any single imputation.

8. Step 8. Relative increase in variance

   \[ r = \frac{\left(1 + \frac{1}{5}\right) \times 0.00243}{0.03396} = \frac{0.002916}{0.03396} = 0.0859 \]

   The relative increase in variance, written r, says how much the imputations inflated the variance. It is the between-imputation part, after its finite-m correction, divided by the within-imputation part. Rounded to three decimals it is the 0.086 quoted in the note below.

9. Step 9. Degrees of freedom for the t value

   \[ \mathrm{df} = (5 - 1)\left(1 + \frac{1}{0.0859}\right)^2 \approx 639 \]

   The classic Rubin formula for degrees of freedom uses only r and the number of imputations. Those 639 degrees of freedom are plenty, so the t value rounds to 1.96. The formula treats the complete-data degrees of freedom as large, and this synthetic example fixes no sample size. The Barnard and Rubin form is bounded by that figure, so it would give fewer and a slightly wider interval.

10. Step 10. Interval on the log odds scale

    \[ 0.444 \pm 1.96 \times 0.1920 = (0.0677, \; 0.8203) \]

    The interval is the pooled log odds plus and minus 1.96 pooled standard errors.

11. Step 11. Back-transform to the odds ratio

    \[ OR = e^{0.444} = 1.56, \; 95\% \text{ CI} = (e^{0.0677}, \; e^{0.8203}) = (1.07, \; 2.27) \]

    Exponentiate the pooled log odds and both limits, never the five odds ratios separately.

Result: Pooled OR 1.56, 95% CI 1.07 to 2.27, from within-imputation variance 0.03396 and between-imputation variance 0.00243.

These five estimates are synthetic, written for this post so that every step can be checked by hand. The relative increase in variance here is 0.086, so the fraction of missing information is about 8 percent and five imputations are enough. That fraction is the share of the answer's uncertainty created by the missing data. When that fraction reaches 30 or 40 percent, five is far too few, and a common rule is to set m to at least one hundred times the fraction.

Terms used above

estimand (ปริมาณเป้าหมายที่ต้องการประมาณ)

The quantity the analysis is meant to estimate, stated before any model is fitted.

sampling error (ความคลาดเคลื่อนจากการสุ่มตัวอย่าง)

The random variation in an estimate that comes from having sampled only a limited number of patients.

inverse-variance weighting (การถ่วงน้ำหนักด้วยส่วนกลับของความแปรปรวน)

Weighting each study by 1 divided by its squared standard error, so precise studies count for more.

tau-squared (ความแปรปรวนระหว่างการศึกษา)

The between-study variance: how much the true effect itself differs from study to study.

I-squared (สัดส่วนความแปรปรวนที่มาจาก heterogeneity)

The share of total variability that comes from between-study differences rather than sampling error.

REML (วิธีประมาณความแปรปรวนแบบ restricted maximum likelihood)

Restricted maximum likelihood, an estimator of between-study variance and the current RevMan default for inverse-variance random-effects analyses.

bootstrap replicate (ค่าประมาณจากการ resample หนึ่งรอบ)

One estimate computed from one resample of the original dataset, repeated thousands of times.

BCa interval (ช่วงความเชื่อมั่นแบบ bias-corrected and accelerated)

A bootstrap interval corrected for bias and for how the variance changes with the estimate.

missing at random (ข้อมูลหายแบบสุ่มเมื่อกำหนดค่าที่สังเกตได้แล้ว)

Whether a value is missing depends only on other values that were observed, not on the missing value itself.

Rubin's rules (กฎของ Rubin สำหรับรวมผลจาก multiple imputation)

The formulas that combine estimates across imputed datasets, adding within-imputation and between-imputation variance.

fraction of missing information (สัดส่วนของสารสนเทศที่สูญไปกับข้อมูลที่ขาดหาย)

The share of the uncertainty in the pooled answer that the missing data created, and the figure that decides how many imputations are enough.

prediction interval (ช่วงทำนายสำหรับการศึกษาใหม่)

The range a new similar study's true effect is expected to fall in, wider than the confidence interval and trustworthy only when the studies are not few.

References

1. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.5, chapter 10: analysing data and undertaking meta-analyses. Cochrane, 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-10
2. Cochrane. Pre-define analyses. RevMan Knowledge Base. https://documentation.cochrane.org/revman-kb/pre-define-analyses-260702251.html
3. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials 1986;7:177 to 188. https://doi.org/10.1016/0197-2456(86)90046-2
4. Veroniki AA, Jackson D, Viechtbauer W, et al. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Research Synthesis Methods 2016;7:55 to 79. https://doi.org/10.1002/jrsm.1164
5. IntHout J, Ioannidis JPA, Borm GF. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Medical Research Methodology 2014;14:25. https://doi.org/10.1186/1471-2288-14-25
6. Efron B. Better bootstrap confidence intervals. Journal of the American Statistical Association 1987;82:171 to 185. https://doi.org/10.1080/01621459.1987.10478410
7. Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York: Wiley, 1987. https://doi.org/10.1002/9780470316696
8. Barnard J, Rubin DB. Small-sample degrees of freedom with multiple imputation. Biometrika 1999;86:948 to 955. https://doi.org/10.1093/biomet/86.4.948

Key takeaways

• Pool effect estimates and their variances, never the ends of the confidence intervals.
• Name the estimand first: fixed-effect asks about one common effect, random-effects about an average across studies.
• A random-effects model does not repair heterogeneity, so check clinical and methodological differences separately.
• Bootstrap builds one interval from the distribution of replicates, so batches of replicates combine, not their intervals.
• After multiple imputation, Rubin's rules add within-imputation and between-imputation variance before any interval is drawn.

Related in the wiki: [[multiple-imputation-clinical-research]] [[robust-meta-analysis-methods]] [[standard-deviation-sd-standard-error-se-and-confidence-intervals-ci-in-clinical-research]] [[classic-mape-mean-absolute-prediction-error-and-bootstrap-internal-validation]]