Understanding Regression: From Correlation to Clinical Modeling [Multiple imputation, MI]

Introduction

Regression analysis is foundational in clinical research, serving as a tool to examine and quantify relationships between variables. Unlike mere correlation, regression provides not only an estimate of strength but also the nature and direction of associations—empowering researchers to predict outcomes, identify risk factors, and model complex clinical scenarios.

This article introduces the essential types of regression—Gaussian and logistic—explains their assumptions, and shows how to navigate non-linearities and interaction effects. It provides a clear, structured foundation for health professionals and clinical researchers seeking to move from exploratory to explanatory modeling.

Correlation vs Regression: Two Distinct Purposes

Correlation: Measuring Strength of Association

Purpose: To quantify the degree of linear association between two variables.
Output: A single number, the correlation coefficient (r), ranging from -1 to 1.
Use Case: Often used in early exploratory data analysis. For example, a study might find that systolic blood pressure and body mass index (BMI) have an r = 0.6, indicating a moderate-to-strong linear association.

However, correlation lacks directionality or causal interpretation—it tells us nothing about which variable is driving the other or how to predict one from the other.

Regression: Modeling the Relationship

Purpose: To model the specific nature of the relationship between an independent (predictor) and dependent (outcome) variable.
Output: A full equation estimating the outcome based on predictors.
Use Case: In clinical research, regression is used to estimate the effect of age on systolic blood pressure while adjusting for comorbidities, enabling more informed decision-making.

Gaussian and Logistic Regression: Choosing the Right Tool

Gaussian (Linear) Regression

Outcome Variable (Y): Continuous and normally distributed (e.g., systolic blood pressure, hemoglobin level).
Equation:

Linear Equation

Y = a + b X

a is the intercept, and b is the beta coefficient (or slope), representing the mean change in Y for a one-unit increase in X.

Example: Predicting systolic BP:

SBP Equation

SBP = 95 + 0.7 \times Age

Each additional year increases SBP by 0.7 mmHg, on average.

Interpretation: The beta coefficient equals the mean difference in outcome for each unit difference in the predictor.

Logistic Regression

Outcome Variable (Y): Binary (e.g., survival vs death, disease vs no disease).
Equation:

Logistic Regression

logit (P) = a + b X

Where P is the probability of the event occurring. The output is in log-odds; the exponentiated beta coefficient (e^b) gives the odds ratio (OR).

Example: Estimating the odds of developing diabetes based on BMI:

Logistic Regression with BMI

\log (\frac{P}{1 - P}) = - 2.3 + 0.1 \times BMI

An odds ratio (OR) of 1.1 per unit of BMI means that each additional BMI point increases the odds of developing diabetes by 10%.

Strength: Logistic regression allows not just odds interpretation, but also the prediction of event probability.

Modeling Non-Linear Relationships

The Linearity Assumption

Standard regression assumes a straight-line relationship:

Y = a+bX

Yet in practice, many relationships curve or plateau. For instance, age may increase risk for stroke up to a point, after which the relationship flattens.

Solutions for Non-Linearity

Polynomial Terms: Add squared or cubed terms.
- Example:

Quadratic Regression Model

Y = a + b_{1} X + b_{2} X^{2}

Fractional Polynomials: More flexible transformations (e.g., √X, log(X)).
- Best for capturing subtle curves.
Splines: Piecewise linear models that “bend” at predefined knots.
- Ideal for complex, multi-phase relationships.

Application:

In a model predicting cholesterol based on age:

A linear model tends to underestimate risk at older ages.
Adding age² improves fit and interpretability.

Modeling Interactions: When Effects Depend on Context

Understanding Interactions

An interaction exists when the effect of one predictor depends on the level of another.

Example: The effect of a blood pressure medication (X) on stroke risk (Y) might differ by sex (Z).
- In men: OR = 0.7
- In women: OR = 1.3

This indicates a statistically significant interaction: the drug benefits men but not women.

Model Specification

Modeling Interactions

Y = a + b_{1} X + b_{2} Z + b_{3} X Z

To model interactions, include both main effects and the interaction term in the equation:

Y = a + b₁X + b₂Z + b₃XZ

In statistical software like Stata or R, use the syntax X*Z or X:Z to specify interaction terms.

Clinical Relevance

Interactions matter for personalized medicine. They help identify for whom treatments work, not just whether they work.

Conclusion

Regression is more than just a statistical tool—it's a language for expressing and testing clinical hypotheses. Whether exploring relationships, adjusting for confounding, or predicting patient outcomes, understanding how to use Gaussian and logistic regression appropriately—and how to account for non-linearities and interactions—enables more precise, meaningful clinical research.

Key Takeaways

Correlation describes strength; regression models the relationship.
Gaussian regression is used for continuous outcomes, while logistic regression is used for binary outcomes.
Non-linear relationships require modeling flexibility, such as splines, polynomials, or transformations.
Interaction terms reveal when effects are context-dependent, crucial for subgroup-specific insights.
Thoughtful modeling leads to better care, with more accurate predictions, clearer decisions, and improved outcomes.