Chance, Bias and Confounding: Why an Association Is Not Yet a Treatment Effect

อ่านฉบับภาษาไทย (Thai version)

Abstract

Chance, bias and confounding can separate an observed association from the treatment effect needed for an economic decision. Begin by defining the population, treatment strategies, outcome and follow-up. A causal diagram then shows which patient characteristics could influence both treatment and outcome. Our illustrative mortality example compares severe and mild patients under each strategy. Treatment looks beneficial within each severity group, yet the pooled comparison suggests harm because treated patients are sicker. Standardization compares both strategies using the same severity mix and restores the intended comparison. The resulting absolute risks provide the starting point for estimating deaths avoided before adding costs. Adjustment still depends on measured causes, comparable follow-up and adequate overlap between groups. Confidence intervals describe sampling uncertainty under stated assumptions; they cannot certify causal validity. Probabilistic sensitivity analysis propagates the uncertainty supplied to it. It cannot repair a biased treatment effect without an explicit model of that bias.

🔍 Click to enlarge Click anywhere to close

Start with the causal question

Ask what would happen if the same eligible population followed either treatment strategy over the same follow-up. This target quantity is the estimand. Specify the comparator, outcome and horizon before choosing an estimate. [[study-design-estimand-and-effect-measures]] develops that question.

Chance is sampling variation: another sample gives another estimate. Confidence intervals describe precision under the analysis assumptions. More data can improve precision while leaving bias untouched. A p-value measures incompatibility with a specified null model, not the probability that treatment works.

Bias is systematic error from selection, measurement or analysis. Confounding is one source: shared causes of treatment and outcome distort their comparison. Better measurement addresses information bias; a comparable study population addresses selection bias. Adjustment addresses confounding only under justified assumptions.

Draw causes before choosing covariates

A directed acyclic graph, or DAG, displays assumed causal directions without feedback loops. Baseline severity affects both prescribing and death. Confounding by indication occurs when sicker patients preferentially receive treatment. Select common causes using clinical knowledge, rather than small p-values.

A mediator carries part of the treatment effect; adjusting for it can remove part of the total effect. A collider is a shared consequence of other variables; conditioning on it can introduce an association. Choose adjustment variables for the causal question, rather than including every available measurement.

Watch the association reverse

These illustrative inputs describe deaths at a shared follow-up endpoint, with complete outcome measurement. More severe patients receive treatment. Pooling therefore compares different severity mixtures. Standardization averages each strategy over the same target mixture: here, equal shares of severe and mild patients.

The arithmetic isolates the effect of that mixture. It does not establish efficacy in actual patients. A causal interpretation additionally needs no remaining confounding, consistent treatment definitions and comparable follow-up.

Calculate the observed risks

# Writes: stdout only; illustrative deaths at a shared follow-up endpoint.
deaths_t <- c(120, 3)
patients_t <- c(400, 100)
deaths_c <- c(40, 16)
patients_c <- c(100, 400)
risk_t <- c(deaths_t / patients_t, sum(deaths_t) / sum(patients_t))
risk_c <- c(deaths_c / patients_c, sum(deaths_c) / sum(patients_c))
group <- c("Severe", "Mild", "Crude")
for (i in 1:3) cat(group[i], "treated risk:", formatC(100*risk_t[i], 2, format="f"), "%\n")
for (i in 1:3) cat(group[i], "untreated risk:", formatC(100*risk_c[i], 2, format="f"), "%\n")
for (i in 1:3) cat(group[i], "risk ratio:", formatC(risk_t[i]/risk_c[i], 3, format="f"), "\n")

Severe treated risk: 30.00 %
Mild treated risk: 3.00 %
Crude treated risk: 24.60 %
Severe untreated risk: 40.00 %
Mild untreated risk: 4.00 %
Crude untreated risk: 11.20 %
Severe risk ratio: 0.750
Mild risk ratio: 0.750
Crude risk ratio: 2.196

Compare the same severity mix

# Writes: stdout only; illustrative target with equal severe and mild shares.
treated_risks <- c(120/400, 3/100)
untreated_risks <- c(40/100, 16/400)
target_t <- mean(treated_risks)
target_c <- mean(untreated_risks)
reduction <- target_c - target_t
cat("Standardized treated risk:", formatC(100*target_t, 2, format="f"), "%\n")
cat("Standardized untreated risk:", formatC(100*target_c, 2, format="f"), "%\n")
cat("Standardized risk ratio:", formatC(target_t/target_c, 3, format="f"), "\n")
cat("Absolute reduction:", formatC(100*reduction, 2, format="f"), "percentage points\n")

Standardized treated risk: 16.50 %
Standardized untreated risk: 22.00 %
Standardized risk ratio: 0.750
Absolute reduction: 5.50 percentage points

Choose adjustment for the population you need

Restriction narrows the eligible population; stratification compares patients within measured groups. Regression can predict risks that standardization averages over a chosen population. Each approach depends on measured causes and appropriate model assumptions.

A propensity score is the probability of treatment given measured baseline characteristics. Inverse probability of treatment weighting (IPTW) uses it to rebalance those characteristics. Matching instead selects comparable patients and may target a different population. Choose the method for the estimand and available overlap, not a universal ranking.

Overlap means patients with similar characteristics can receive either strategy. Extreme weights indicate weak support for the comparison. Check standardized mean differences, which scale covariate gaps by their spread, rather than balance-test p-values. [[propensity-weighting-balance-check-and-model-revision]] explains these checks.

Emulate the trial before analyzing records

Target-trial emulation defines the trial that observational records should approximate. Align eligibility, treatment assignment and follow-up at the same starting instant, called time zero. Otherwise survival before treatment can be mistaken for treatment benefit.

When strategies allow delayed initiation, clone-censor-weight may help implement that comparison. It copies records into compatible strategies, ends follow-up when care deviates, and weights for that censoring. It still needs measured predictors and valid assumptions. [[target-trial-emulation-observational-research]] develops this design.

Carry absolute risks into the economic model

The standardized reduction of 5.50 percentage points is the mortality-benefit input for this illustrative population. Multiply the absolute reduction by the treated population to estimate deaths avoided over the same horizon. Costs and survival consequences come next. A quality-adjusted life-year calculation additionally needs survival duration and health-state values.

Using the crude association would reverse the mortality-benefit input. It would not locate a full cost-effectiveness result without the other model assumptions. See [[icer-dominance-ce-plane-and-thresholds]] for that comparison.

Give bias its own analysis

Residual confounding remains when causes are omitted, poorly measured or modeled incorrectly. Negative controls test associations that should be absent but may share the same bias mechanisms. An unexpected association warns of bias; a null finding does not prove its absence.

An E-value summarizes the strength of unmeasured confounding needed to explain away a risk ratio under stated assumptions. It is a sensitivity measure, not a corrected effect. Quantitative bias analysis instead specifies plausible bias parameters and recalculates effects.

Ordinary PSA propagates uncertainty from specified parameter distributions, including sampling uncertainty. Repeated draws around a confounded estimate leave its center confounded. Bias parameters can be included explicitly, but need their own evidence and assumptions. Use [[evidence-pyramid-versus-grade-certainty]] to assess the evidence before selecting an input.

The series, in learning order

1. Health Economics Series Guide: Read the Evidence Before the Costs
2. Incidence, Prevalence and the Right Denominator: The Epidemiology Beneath Cost-of-Illness and Markov Models [[incidence-prevalence-and-the-right-denominator]]
3. Design, Model and Estimand: What Each Study Design Can Actually Estimate [[study-design-estimand-and-effect-measures]]
4. Risk Ratio, Odds Ratio, Risk Difference and NNT: One 2×2 Table, Four Answers
5. The Hazard Ratio Is Not a Risk Ratio: Proportional Hazards, Restricted Mean Survival Time and Model Inputs
6. Chance, Bias and Confounding: Why an Association Is Not Yet a Treatment Effect (this article)
7. Beyond the Evidence Pyramid: GRADE Certainty and Choosing Effectiveness Inputs for Economic Models
8. From Sensitivity to Cost per Case Detected: Predictive Values, Prevalence and the Economics of Screening
9. Whose Cost, in Which Year? Perspective, Opportunity Cost, Inflation and Discounting
10. QALY, DALY and the Utilities Behind Them: What Is Actually Being Measured
11. ICER, Dominance and the Cost-Effectiveness Plane: Reading the Thai Threshold of 160,000 Baht per QALY
12. Decision Trees, Markov Cohorts and Probabilistic Sensitivity Analysis: How a Model Turns Evidence into a Decision
13. Cost-Effective Is Not Affordable, and Efficient Is Not Fair: Budget Impact and Equity

Related in the wiki: When IPTW Does Not Balance: Revise the Model or the Weights, Then Recheck Overlap and Balance · Emulating the Target Trial: A Framework of Strengthening Causal Inference in Observational Research