Joint Model vs Stacking: Combining Answers vs Combining Processes
On this page
อ่านฉบับภาษาไทย (Thai version)
Abstract
Ensemble learning and joint modelling both assemble more than one component, and the shared vocabulary of "combining models" routinely leads clinician-researchers to conflate them. Stacking is a performance technique: several base models — logistic regression, random forest, XGBoost — each predict the same binary outcome, and a meta-model regresses on their predicted probabilities to produce one sharper prediction. A joint model answers a different question entirely, tying together several related but distinct processes about the same patient — a repeatedly measured biomarker such as creatinine and a survival endpoint such as time to death — through a shared latent trajectory or shared random effects, so that the two outcomes are estimated within one joint probability rather than one after another. The decisive contrast is what gets shared between components: stacking passes finished predictions between layers, whereas a joint model shares the patient's own latent state between processes. This article lines up their equations, tables, and clinical examples side by side to make the distinction unmistakable.
Joint Models · Part 2 of 2. Start with Joint Models: one likelihood for two processes [[joint-models-longitudinal-survival]].
Two ideas that sound alike
Ask a room of clinician-researchers what a joint model is and someone will answer: "It's when you combine several models." That description is exactly right for a completely different technique — stacking — and the overlap in wording is where the confusion starts. Both approaches assemble more than one component. Both are sold as ways to get more out of your data than a single model would. Yet they are built to answer different questions, and mistaking one for the other leads to using the wrong tool on the wrong problem.
The cleanest way to separate them is to look at what each one combines. Stacking combines answers — the outputs of several models that all predict the same thing. A joint model combines processes — several related but distinct things happening to the same patient. This article walks through both with concrete clinical examples, lines up their equations side by side, and shows why they are not two flavours of the same idea.
Stacking, concretely
Suppose we want to predict a single binary outcome: whether an emergency department visit will have a length of stay over four hours, $Y = \text{ED length of stay} > 4\text{ hours}$, from a set of predictors $X = \{\text{age, vital signs, triage level, arrival mode}\}$. We build several base models that all predict the same $Y$ from the same $X$:
$$X \rightarrow \text{Logistic regression} \rightarrow \hat p_{\text{LR}}$$
$$X \rightarrow \text{Random forest} \rightarrow \hat p_{\text{RF}}$$
$$X \rightarrow \text{XGBoost} \rightarrow \hat p_{\text{XGB}}$$
For a single patient the three models will usually disagree a little. A logistic regression might be cautious, a random forest more aggressive, a gradient-boosted tree somewhere in between — for one patient the three models might return the following illustrative probabilities:
| Model | Predicted probability |
|---|---|
| Logistic regression | 0.40 |
| Random forest | 0.65 |
| XGBoost | 0.55 |
Which number do we trust? Stacking refuses to pick. Instead it trains a small meta-model whose predictors are the base models' predicted probabilities themselves:
$$\operatorname{logit}\{P(Y=1)\} = \gamma_0 + \gamma_1 \hat p_{\text{LR}} + \gamma_2 \hat p_{\text{RF}} + \gamma_3 \hat p_{\text{XGB}}$$
The meta-model learns how much weight each base model deserves — perhaps the random forest earns a large coefficient because it is reliably sharp, while the logistic regression is down-weighted. The whole structure is a two-layer pipeline: $X \rightarrow \{\text{Model 1, 2, 3}\} \rightarrow \{\hat p_1, \hat p_2, \hat p_3\} \rightarrow \text{meta-model} \rightarrow \hat p_{\text{final}}$. The essence of stacking is that many models predict the same outcome, and a meta-model learns how to combine those predictions into one sharper answer.
The joint model asks a different question
A joint model does not start from "how do we combine predictions from many models?" It starts from a different observation entirely: we have several related data-generating processes, and we want to model them together. The classic clinical example uses two outcomes of fundamentally different kinds:
- a longitudinal outcome $Y_i(t)$ — repeated creatinine measurements over time;
- a time-to-event (survival) outcome $T_i$ — the time to death.
These are not two predictions of the same thing; they are two different things about the same patient, and they are related. Patients whose creatinine worsens faster tend to have a higher risk of death. A joint model therefore has a longitudinal submodel for the creatinine trajectory and a survival submodel for the hazard of death, and — crucially — it estimates them together rather than one after the other.
The two submodels, briefly
The longitudinal side is a linear mixed model. Each creatinine measurement $Y_{ij}$ for patient $i$ at time $t_{ij}$ is described by a population trend plus a patient-specific deviation plus noise:
$$Y_{ij} = \beta_0 + \beta_1 t_{ij} + b_{0i} + b_{1i} t_{ij} + \epsilon_{ij}$$
Stripping out the measurement error $\epsilon_{ij}$ leaves the patient's underlying trajectory:
$$m_i(t) = \beta_0 + \beta_1 t + b_{0i} + b_{1i} t$$
Here $\beta_0, \beta_1$ are fixed effects (the population baseline and slope) and $b_{0i}, b_{1i}$ are random effects (how far patient $i$ starts above or below average, and how much faster or slower their creatinine climbs). The survival side then places that underlying level directly inside the hazard:
$$h_i(t) = h_0(t)\,\exp\{\gamma X_i + \alpha\, m_i(t)\}$$
where $h_0(t)$ is the baseline hazard, $X_i$ are baseline predictors with effects $\gamma$ (here $\gamma$ denotes the survival model's baseline-covariate effects — unrelated to the meta-model weights $\gamma_0\ldots\gamma_3$ above), and $\alpha$ is the association parameter that ties the trajectory to the risk of death — for instance $\alpha = \log(1.5)$ gives a hazard ratio of about $1.5$ per one-unit increase in underlying creatinine (the reading depends on the measurement unit/scale and on the current-value structure). The distinction that matters most is how the two submodels are fitted. In a two-stage approach you would fit the mixed model first, freeze the estimated creatinine trajectory, and only then plug it into a Cox model — the survival outcome can never reach back and revise the trajectory. A joint model instead estimates the longitudinal and survival submodels simultaneously, so that $\text{Longitudinal process} \longleftrightarrow \text{Survival process}$: information flows both ways.
The pivot: predictions versus shared patient information
Now the two techniques can be placed next to each other, and the real difference becomes visible. It is not about the number of components — it is about what the components are linked by.
Stacking links by predictions. Every base model produces a finished output for the same outcome, $\hat p_1, \hat p_2, \hat p_3$, and the meta-model combines those final outputs: $(\hat p_1, \hat p_2, \hat p_3) \rightarrow \hat p_{\text{final}}$. What passes between the layers is a number that already means "my best guess at $Y$."
A joint model links by shared patient information. Its two outcomes are of different kinds, $Y_i(t)$ and $T_i$, and they are tied together not by any prediction but by a shared underlying quantity: the trajectory $m_i(t)$, or its slope $m_i'(t)$, or the random effects $b_i = (b_{0i}, b_{1i})$ themselves. In the shared-random-effects form the hazard depends on the patient's own intercept and slope directly:
$$h_i(t)=h_0(t)\exp\{\gamma X_i+\alpha_0 b_{0i}+\alpha_1 b_{1i}\}$$
What is shared here is the patient's latent trajectory and characteristics — not a final prediction handed to a meta-model. That single distinction is the whole difference: stacking passes answers between layers, a joint model shares the patient's own hidden state between processes.
The essence: one joint probability
The formal signature of a joint model is a single equation. For the shared-random-effects version, the joint probability of the longitudinal measurements and the event time is an integral over the shared random effects:
$$p(Y_i,T_i) = \int p(Y_i\mid b_i)\, p(T_i\mid b_i)\, p(b_i)\, db_i$$
Read it piece by piece. The term $p(Y_i \mid b_i)$ explains the creatinine measurements given the random effects; $p(T_i \mid b_i)$ explains the event time given the same random effects; and $p(b_i)$ is the population distribution of those random effects. Because both outcomes are conditioned on the same $b_i$, they are not independent — the model defines a genuine joint probability structure rather than two separate calculations that happen to run on the same dataset. Stacking has no equation like this. There is no shared latent variable being integrated out; there is only a meta-regression on top of finished predictions.
Side by side
Laying every axis of comparison in one table makes the contrast concrete:
| Aspect | Stacking | Joint model |
|---|---|---|
| Goal | Boost predictive performance | Analyse related outcomes/processes together |
| Number of outcomes | Usually one | Usually at least two processes |
| What is combined | Several predictive models | Several related outcome processes |
| Linkage | Predictions of base models | Current value, slope, or random effects |
| Steps | Base models first, then meta-model | Submodels estimated jointly |
| Example | LR + RF + XGB predict ED-LOS > 4h | Creatinine trajectory + time to death |
| Final result | One combined prediction | Trajectory, association, and event risk |
| Main question | How much weight per model? | How does change in one outcome relate to the other? |
What they share, and why the confusion persists
Both methods do have several moving parts — that shared surface feature is precisely why they get muddled. But the parts play different roles. In stacking, several models compete and complement each other to predict one outcome: $X \rightarrow \hat Y_1, \hat Y_2, \hat Y_3$, and every arrow points at the same target. In a joint model, each submodel explains a different outcome process — $\text{Longitudinal} \rightarrow Y_i(t)$ and $\text{Survival} \rightarrow T_i$ — and they are then bound together by a shared structure. Stacking is many roads to one destination; a joint model is several distinct roads held together because the same traveller is on all of them.
The misconception worth correcting head-on is that a joint model is just "several models running in parallel." It is not. It defines a joint probability so that several outcome processes are estimated together, and in doing so it earns real benefits a parallel stack cannot — for example, it correctly handles informative dropout, where the sickest patients' creatinine rises fast and then stops being measured because they have died. A longitudinal-only analysis silently loses those patients and reports a trend that looks too healthy; the joint model uses the death times to keep them in the picture.
When to use which
Reach for stacking when you have one clear prediction target, several plausible modelling strategies, and you simply want the best combined predictive performance you can get — it is a performance tool. Reach for a joint model when you have two or more related processes about the same subjects — a biomarker measured repeatedly and a survival endpoint being the archetype — and the scientific question is about how change in one relates to the other, or when dropout is driven by the event itself. The two boxed summaries capture the divide:
$$\boxed{\text{Stacking} = \text{many models predict one } Y}$$
$$\boxed{\text{Joint model} = \text{many related outcome processes estimated together}}$$
- Stacking is a two-layer pipeline: many predictions of one outcome flow into a meta-model that outputs one final prediction.
- A joint model links different-kind outcomes through a shared trajectory or shared random effects: $Y_i(t) \longleftrightarrow b_i \longleftrightarrow T_i$.
- The tell is what is shared: stacking shares predictions between layers; a joint model shares the patient's latent state between processes, defining one joint probability.
- The sharpest sentence: stacking combines answers from many models; a joint model combines information from many related processes.