Decision Trees and Markov Models in Health Economic Evaluation: Principles, Construction, and Applications

Introduction

Healthcare decisions—whether at the bedside or in national policy—often require a systematic way to weigh options under uncertainty. Especially when choices involve both clinical outcomes and cost consequences, decision-analytic modeling becomes indispensable. Two foundational tools in this domain are the Decision Tree and the Markov Model. These frameworks allow analysts to simulate the trajectory of diseases, evaluate interventions, and predict long-term outcomes, which are essential for clinical guideline development and health technology assessment.

This article unpacks the logic, components, and use cases of these two modeling methods, offering a robust understanding of how each operates and when to apply them.

---

I. Decision Trees: Quantifying Short-Term Choices

1. Core Logic

A decision tree is a graphical and quantitative model used to analyze a sequence of choices and their associated outcomes. Each path represents a clinical scenario, incorporating:

• Decisions (e.g., to treat or not)
• Uncertain events (e.g., treatment success or adverse effects)
• End states with assigned values (e.g., survival, cost, utility)

2. Structural Anatomy

A well-designed decision tree includes:

• Decision Nodes (typically squares): Represent choice points.
• Chance Nodes (typically circles): Represent probabilistic outcomes.
• Branches: Link nodes, denoting options or events.
• Terminal Nodes (typically triangles): Denote outcomes with defined consequences.

3. Analytical Workflow

To build a valid decision tree:

1. Define the problem: Clearly state the clinical or economic question.
2. Map the structure: Lay out decision and chance nodes sequentially.
3. Input data: Populate branches with probabilities and assign outcomes utilities, life-years, or costs.
4. Calculate expected values:
   • Multiply each outcome’s value by its probability.
   • “Fold back” the tree to compare average outcomes of different strategies.

4. Example in Practice

Imagine a treatment for a short-duration condition like acute bacterial sinusitis. A decision tree might compare:

• Immediate antibiotic therapy
• Watchful waitingEach path would include branches for clinical cure, treatment failure, adverse drug events, and associated costs.

5. Limitations

Despite their clarity, decision trees have structural limits:

• They assume a one-way progression through time (no loops).
• Complexity grows exponentially with multiple time points or health states.
• They are best suited for acute conditions or interventions with short follow-up windows.

---

II. Markov Models: Capturing Chronic Complexity

1. Rationale for Use

Chronic conditions, such as diabetes, cancer, or stroke, require models that reflect ongoing risk, recurrence, and time-dependent transitions. Markov models solve this by representing disease processes as cycles between health states.

2. Conceptual Foundation

A Markov model simulates patient movement across mutually exclusive states over repeated time intervals, or "cycles." At each cycle, the patient can:

• Remain in the current state
• Transition to another health state
• Reach an absorbing state (e.g., death)

This recursive logic handles conditions where events can repeat (e.g., second stroke) or where past history affects risk (handled via state refinement or Monte Carlo simulation).

3. Structural Components

A basic Markov model includes:

• Health States: e.g., Healthy, Sick, Dead.
• Transition Probabilities: Likelihood of moving from one state to another in a given cycle.
• Cycle Length: Time interval (e.g., 1 year) during which transitions occur.
• Utilities and Costs: Assigned to each state per cycle.

4. Operational Steps

To build a Markov model:

1. Define health states precisely.
2. Determine appropriate cycle length (depends on disease natural history).
3. Source transition probabilities from literature or datasets.
4. Assign outcomes to each state (QALYs, costs).
5. Run the model over multiple cycles to simulate long-term impact.

5. Applied Illustration

Consider stroke prevention in atrial fibrillation:

• States: Well, Ischemic Stroke, Hemorrhage, Death.
• Transitions: Modeled annually, with probabilities derived from cohort studies.
• Objective: Evaluate cost-effectiveness of anticoagulants vs no therapy over 10 years.

6. Advantages

• Handles Recurrence: Ideal for diseases with multiple relapses/remissions.
• Supports Time-Dependent Analysis: Survival modeling, discounting, and cumulative cost/QALY tracking.
• Compatible with Sensitivity Analysis: To test robustness of assumptions.

---

III. When and How to Choose Between Models

---

IV. Addressing Markov Limitations: Simulation Enhancements

Markov models assume "memorylessness": transition probabilities depend only on the current state, not the path taken to arrive there. This can be unrealistic in many clinical scenarios. To overcome this:

• Monte Carlo Simulation (First-order): Simulates individual patient histories across cycles.
• Tracker Variables: Capture events like "number of strokes" to influence future transitions.
• Discrete Event Simulation: Allows timing of specific events to vary between patients.

These enhancements bring models closer to real-life disease dynamics while retaining analytic transparency.

---

V. Software and Practical Construction

Models can be built using:

• Excel: Flexible and transparent but time-consuming for large models.
• TreeAge Pro, R packages (heemod, msm), or specialized tools: Provide streamlined computation and visualization, though some operate as "black boxes" with reduced interpretability for non-technical stakeholders.

---

Conclusion

Decision trees and Markov models are foundational tools in health economics, each with distinct strengths. Decision trees excel in modeling one-time choices with near-term outcomes, while Markov models dominate when simulating chronic disease trajectories and long-term cost-utility profiles. Their appropriate use ensures that healthcare policies and clinical guidelines are grounded in not only evidence but also economic logic and ethical stewardship.

Key Takeaways

• Decision trees quantify choices with short-term, linear outcomes.
• Markov models simulate recurring events and time-dependent disease processes.
• Choice of model depends on clinical context, disease nature, and analytic purpose.
• Both tools support cost-effectiveness, resource allocation, and patient-centered care decisions.
• Simulations enhance realism, especially when modeling history-dependent transitions.