Health Economic Modelling Series Guide: From Evidence to a Decision
On this page
อ่านฉบับภาษาไทย (Thai version)
Abstract
A cost-effectiveness answer is the last line of a model, and this series opens the model behind it. This guide follows one illustrative decision through every article. A district hospital committee must decide whether to pay for a new preventive treatment, C, instead of usual care, A. In a trial, 8 of 66 patients on C and 13 of 60 on A had a serious complication, and C costs 150,000 THB more per patient. The analysts report that C buys 1.00 extra year in full health, and the committee pays up to 160,000 THB for one. Four articles build the inputs: the model's shape, a table you can trace, sources that fit and time units that agree. Four build and compare the model and count the yearly bill. Three test the answer for uncertainty, errors and the value of better evidence. Value, policy and budget stay separate questions.
One decision, and the line that ends it
A district hospital committee must decide whether to pay for a new preventive treatment, C, for adults newly diagnosed with a chronic condition. The alternative is usual care, A. In a trial, 8 of 66 patients on C and 13 of 60 on A had a serious complication. C costs 150,000 THB more per patient than A.
The analysts' report ends in one line. C buys 1.00 extra year in full health per patient, and the committee will pay up to 160,000 THB for such a year. A quick reader stops there and says yes.
That 1.00 is not in the trial. The trial counted complications over a short follow-up. A model turned those counts into years: it followed the patients over time, weighed each year by its quality and added up the costs.
A common belief is that a model is picked from a software menu and that its result is the answer. Here a model is an explicit argument. It states who the patients are, which alternatives compete, which path patients walk and which number fills each step. A reader can then check every link (Briggs and colleagues).
Each alternative is a strategy, also called an option: a whole path of care, such as A or C, with everything that follows the first treatment. After this guide, four articles build the inputs, four build and compare the model, and three test the answer. This guide walks the committee's case through each stage once.
The numbers on the committee's line
The extra year in full health has a name. A QALY, a quality-adjusted life year, is a year of life weighted by its quality, from 1 for full health to 0 for death. Over 10 years, a health state valued at 0.70 counts as 7.0 QALYs, so restoring full health gains 3.0; the full lesson is in QALY, DALY and the Utilities Behind Them: What Is Actually Being Measured.
The committee's model adds up such weighted years over its horizon, the period it follows the patients, and that is where its totals come from. Each patient on A gets 8.00 QALYs and costs 50,000 THB; each patient on C gets 9.00 QALYs and costs 200,000 THB. The committee's threshold, 160,000 THB, is the most it will pay for one QALY.
The incremental cost-effectiveness ratio (ICER) is the extra cost divided by the extra health gained against the next best option. Option C costs 150,000 THB more than A for 1.00 extra QALY, an ICER of 150,000 THB per QALY against a threshold of 160,000; the full lesson is in the article on the ICER and thresholds.
The same comparison can be stated in money. Net monetary benefit (NMB) is health valued at the threshold minus cost, so a positive incremental value (INMB) means the extra cost is worth paying. At 160,000 THB per QALY, 1.00 extra QALY for 150,000 THB more gives an INMB of 10,000 THB; the full lesson is in the article on the ICER and thresholds. Strategy by strategy, A's NMB is 8.00 × 160,000 - 50,000 = 1,230,000 THB and C's is 1,240,000 THB, and the INMB is their difference.
Worked example: the committee's line from the model's totals
Each step uses only the model's totals per patient and the threshold, so you can check every result with a calculator.
-
Read the model's totals
Per patient over the model's horizon, A costs 50,000 THB and gives 8.00 QALYs; C costs 200,000 THB and gives 9.00 QALYs.
-
Subtract A from C
\[ 200000-50000=150000,\quad 9.00-8.00=1.00 \]
C costs 150,000 THB more and gives 1.00 QALY more per patient.
-
Divide the extra cost by the extra health
\[ \frac{150000}{1.00}=150000 \]
150,000 THB per QALY, under the threshold of 160,000.
-
Value the health, then subtract the cost
\[ 160000\times 1.00-150000=10000 \]
An INMB of 10,000 THB per patient: positive, so the extra cost is worth paying.
-
Try two lower thresholds
\[ 150000\times 1.00-150000=0,\quad 140000\times 1.00-150000=-10000 \]
At 150,000 THB per QALY, C breaks even; at 140,000, its INMB is -10,000 THB.
Result: At 160,000 THB per QALY, C is worth 10,000 THB more per patient than it costs. The answer turns to no once the threshold falls below 150,000, the ICER.
Trace the committee's line in R
# The committee's case per patient over the model's horizon: usual care A against treatment C.
nice <- function(value, places = 0) formatC(value, format = "f", digits = places, big.mark = ",")
cost <- c(50000, 200000)
qaly <- c(8, 9)
threshold <- c(160000, 150000, 140000)
gap <- c(diff(cost), diff(qaly))
cat("Cost per patient on A and on C (THB):", nice(cost), "\n")
cat("QALYs per patient on A and on C:", nice(qaly, 2), "\n")
cat("NMB of A and of C at 160,000 THB per QALY (THB):", nice(threshold[1] * qaly - cost), "\n")
cat("Extra cost (THB) and extra QALYs of C:", nice(gap[1]), nice(gap[2], 2), "\n")
cat("Cost per extra QALY (THB):", nice(gap[1] / gap[2]), "\n")
cat("INMB of C at 160,000, 150,000 and 140,000 (THB):", nice(threshold * gap[2] - gap[1]), "\n")Cost per patient on A and on C (THB): 50,000 200,000
QALYs per patient on A and on C: 8.00 9.00
NMB of A and of C at 160,000 THB per QALY (THB): 1,230,000 1,240,000
Extra cost (THB) and extra QALYs of C: 150,000 1.00
Cost per extra QALY (THB): 150,000
INMB of C at 160,000, 150,000 and 140,000 (THB): 10,000 0 -10,000Requires R ≥ 4.5.2 · tibble 3.3.1 · dplyr 1.2.1 · ggplot2 4.0.3 · works in R 4.6.1 · Download health-economic-modelling-series-guide.R
Full tidy script
# Health Economic Modelling Series Guide: From Evidence to a Decision
# คู่มือชุดบทความ: ติดตามกรณีของคณะกรรมการ ตั้งแต่ข้อมูลนำเข้า แบบจำลอง จนถึงการทดสอบคำตอบ
# Requires: R >= 4.5.2
# Tested: works in R 4.6.1
# tibble 3.3.1 (stable)
# dplyr 1.2.1 (stable)
# ggplot2 4.0.3 (stable)
# install.packages(c("tibble", "dplyr", "ggplot2"))
# update.packages(oldPkgs = c("tibble", "dplyr", "ggplot2"))
# remotes::install_version("tibble", "3.3.1")
# remotes::install_version("dplyr", "1.2.1")
# remotes::install_version("ggplot2", "4.0.3")
library(tibble) # Write the two strategies as one explicit input table.
library(dplyr) # Add each strategy's net benefit at the threshold.
library(ggplot2) # Display the two strategies without saving files.
stopifnot(getRversion() >= "4.5.2")
#%%@note 1. The committee's case
# Per patient over the model's horizon: cost and QALYs of usual care A and of treatment C.
# ต้นทุนและ QALY ต่อผู้ป่วยตลอดช่วงเวลาของแบบจำลอง ของการดูแลตามปกติ A และการรักษา C
nice <- function(value, places = 0) formatC(value, format = "f", digits = places, big.mark = ",")
cost <- c(50000, 200000)
qaly <- c(8, 9)
threshold <- c(160000, 150000, 140000)
gap <- c(diff(cost), diff(qaly))
strategies <- tibble::tibble(strategy = c("A", "C"), cost = cost, qaly = qaly) |>
dplyr::mutate(net_benefit = threshold[1] * qaly - cost)
stopifnot(abs(diff(strategies$net_benefit) - (threshold[1] * gap[2] - gap[1])) < 1e-9)
#%%@note 2. Inputs the model needs
# The trial's two risks, the weight behind a QALY, and one yearly rate as a cycle's chance.
# ความเสี่ยงสองค่าจากการทดลอง ค่าน้ำหนักเบื้องหลัง QALY และอัตรารายปีที่แปลงเป็นโอกาสในหนึ่งรอบ
events <- c(8, 13)
treated <- c(66, 60)
weight <- 0.70
years <- 10
rate <- 0.12
cycle <- 0.25
#%%@note 3. The model and the yearly bill
# One cycle of the cohort, an option that costs more for less health, and the yearly bill.
# หนึ่งรอบของกลุ่มประชากร ทางเลือกที่มีต้นทุนมากกว่าแต่ได้สุขภาพน้อยกว่า และค่าใช้จ่ายต่อปี
start <- 1000
move <- c(0.80, 0.15, 0.05)
d_cost <- 20000
d_loss <- 0.10
new_patients <- 100
extra_cost <- 150000
stopifnot(extra_cost == gap[1], abs(sum(move) - 1) < 1e-9)
#%%@note 4. Testing the answer
# How often C wins when its INMB is uncertain, and the value of knowing the true state first.
# C ชนะบ่อยเพียงใดเมื่อ INMB ไม่แน่นอน และมูลค่าของการรู้สถานการณ์จริงก่อนเลือก
mean_inmb <- 10000
sd_inmb <- 40000
runs <- 1000
share <- stats::pnorm(mean_inmb / sd_inmb)
option_1 <- c(100, 140, 90, 130)
option_2 <- c(120, 110, 150, 100)
known <- mean(pmax(option_1, option_2))
single <- max(mean(option_1), mean(option_2))
stopifnot(mean_inmb == threshold[1] * gap[2] - gap[1])
#%%@note 5. Display the two strategies
# Plot cost against QALYs for A and C; the script saves no graphics file.
# แสดงต้นทุนเทียบกับ QALY ของ A และ C โดยไม่บันทึกไฟล์ภาพ
print(ggplot2::ggplot(strategies, ggplot2::aes(x = qaly, y = cost, label = strategy)) +
ggplot2::geom_point() + ggplot2::geom_text(nudge_y = 8000))
#%%@note 6. Numbers the article quotes
# Print the four short boxes' lines in their reading order.
# พิมพ์ผลลัพธ์ตรงกับกล่องสั้นทั้งสี่ตามลำดับในบทความ
cat("Cost per patient on A and on C (THB):", nice(cost), "\n")
cat("QALYs per patient on A and on C:", nice(qaly, 2), "\n")
cat("NMB of A and of C at 160,000 THB per QALY (THB):", nice(threshold[1] * qaly - cost), "\n")
cat("Extra cost (THB) and extra QALYs of C:", nice(gap[1]), nice(gap[2], 2), "\n")
cat("Cost per extra QALY (THB):", nice(gap[1] / gap[2]), "\n")
cat("INMB of C at 160,000, 150,000 and 140,000 (THB):", nice(threshold * gap[2] - gap[1]), "\n")
cat("Complication risk on C and on A:", paste0(nice(100 * events / treated, 2), "%"), "\n")
cat("QALYs over 10 years at 0.70, and the gain to full health:", nice(weight * years, 1),
nice(years * (1 - weight), 1), "\n")
cat("Chance of moving in one cycle, exact and shortcut:", nice(1 - exp(-rate * cycle), 4),
nice(rate * cycle, 4), "\n")
cat("Of 1,000 well people after one cycle (well, sick, dead):", start * move, "\n")
cat("D against C: extra cost (THB) and QALYs lost:", nice(d_cost), nice(d_loss, 2), "\n")
cat("D costs more and gives less health:", d_cost > 0 && d_loss > 0, "\n")
cat("Extra spending a year (THB):", nice(new_patients * extra_cost), "\n")
cat("Mean INMB over its spread:", formatC(mean_inmb / sd_inmb, format = "f", digits = 2), "\n")
cat("Share above zero, and the part past the centre:", round(c(share, share - 0.5), 4), "\n")
cat("Runs of 1,000 for C, and as a share:", round(runs * share), sprintf("%.2f", share), "\n")
cat("Best choice per state, best single choice, the gain:", known, single, known - single, "\n")
Stage one: build the inputs
Before any number, the model needs a shape. A state is where a patient is at a given time, such as well, sick or dead; an event, such as the complication, is something that happens on the way. The committee's patients meet the complication once, early, and then spend years moving between states. A one-off fork drawn once, a decision tree, suits the early event; for the years after, the model steps through time in cycles, each one fixed time step, and moves patients between states at every step (Siebert and colleagues).
The second article lets this pathway decide the structure Choosing a Decision-Analytic Model: Let the Clinical Pathway Decide. Then come the numbers that fill it. A parameter is one input the model needs, such as a complication risk, the quality weight of a sick year or the cost of one clinic visit. The third article puts each into a table with its value, its source and how sure we are of it Auditable Model Inputs: Build a Parameter Table You Can Trace.
Most inputs come from published studies, and the fourth article checks that each estimate answers the model's question Finding Model Inputs in the Literature: Match the Estimate to the Question. The trial gives the complication risks: 8 of 66 is 12.12% on C, and 13 of 60 is 21.67% on A. Before those enter the model, ask whether the trial's patients and follow-up match the district's adults.
The last input step is time, the subject of the fifth article. A transition probability is the chance of moving between two states within one cycle, converted from a rate so that time stays in the calculation. A rate of 0.12 per year over a cycle of 0.25 years gives 0.0296, not the shortcut 0.0300 of rate times cycle length; the full lesson is in Rates, Probabilities and Transition Probabilities: Keep Time in the Calculation. The same conversion, with the model's own cycle length, gives every chance the committee's model uses.
Trace the stage-one inputs in R
# Inputs: the trial's two risks, the weight behind a QALY, and a yearly rate as one cycle's chance.
nice <- function(value, places = 0) formatC(value, format = "f", digits = places, big.mark = ",")
events <- c(8, 13)
treated <- c(66, 60)
cat("Complication risk on C and on A:", paste0(nice(100 * events / treated, 2), "%"), "\n")
weight <- 0.70
years <- 10
cat("QALYs over 10 years at 0.70, and the gain to full health:", nice(weight * years, 1),
nice(years * (1 - weight), 1), "\n")
rate <- 0.12
cycle <- 0.25
cat("Chance of moving in one cycle, exact and shortcut:", nice(1 - exp(-rate * cycle), 4),
nice(rate * cycle, 4), "\n")Complication risk on C and on A: 12.12% 21.67%
QALYs over 10 years at 0.70, and the gain to full health: 7.0 3.0
Chance of moving in one cycle, exact and shortcut: 0.0296 0.0300Requires R ≥ 4.5.2 · tibble 3.3.1 · dplyr 1.2.1 · ggplot2 4.0.3 · works in R 4.6.1 · Download health-economic-modelling-series-guide.R
Full tidy script
# Health Economic Modelling Series Guide: From Evidence to a Decision
# คู่มือชุดบทความ: ติดตามกรณีของคณะกรรมการ ตั้งแต่ข้อมูลนำเข้า แบบจำลอง จนถึงการทดสอบคำตอบ
# Requires: R >= 4.5.2
# Tested: works in R 4.6.1
# tibble 3.3.1 (stable)
# dplyr 1.2.1 (stable)
# ggplot2 4.0.3 (stable)
# install.packages(c("tibble", "dplyr", "ggplot2"))
# update.packages(oldPkgs = c("tibble", "dplyr", "ggplot2"))
# remotes::install_version("tibble", "3.3.1")
# remotes::install_version("dplyr", "1.2.1")
# remotes::install_version("ggplot2", "4.0.3")
library(tibble) # Write the two strategies as one explicit input table.
library(dplyr) # Add each strategy's net benefit at the threshold.
library(ggplot2) # Display the two strategies without saving files.
stopifnot(getRversion() >= "4.5.2")
#%%@note 1. The committee's case
# Per patient over the model's horizon: cost and QALYs of usual care A and of treatment C.
# ต้นทุนและ QALY ต่อผู้ป่วยตลอดช่วงเวลาของแบบจำลอง ของการดูแลตามปกติ A และการรักษา C
nice <- function(value, places = 0) formatC(value, format = "f", digits = places, big.mark = ",")
cost <- c(50000, 200000)
qaly <- c(8, 9)
threshold <- c(160000, 150000, 140000)
gap <- c(diff(cost), diff(qaly))
strategies <- tibble::tibble(strategy = c("A", "C"), cost = cost, qaly = qaly) |>
dplyr::mutate(net_benefit = threshold[1] * qaly - cost)
stopifnot(abs(diff(strategies$net_benefit) - (threshold[1] * gap[2] - gap[1])) < 1e-9)
#%%@note 2. Inputs the model needs
# The trial's two risks, the weight behind a QALY, and one yearly rate as a cycle's chance.
# ความเสี่ยงสองค่าจากการทดลอง ค่าน้ำหนักเบื้องหลัง QALY และอัตรารายปีที่แปลงเป็นโอกาสในหนึ่งรอบ
events <- c(8, 13)
treated <- c(66, 60)
weight <- 0.70
years <- 10
rate <- 0.12
cycle <- 0.25
#%%@note 3. The model and the yearly bill
# One cycle of the cohort, an option that costs more for less health, and the yearly bill.
# หนึ่งรอบของกลุ่มประชากร ทางเลือกที่มีต้นทุนมากกว่าแต่ได้สุขภาพน้อยกว่า และค่าใช้จ่ายต่อปี
start <- 1000
move <- c(0.80, 0.15, 0.05)
d_cost <- 20000
d_loss <- 0.10
new_patients <- 100
extra_cost <- 150000
stopifnot(extra_cost == gap[1], abs(sum(move) - 1) < 1e-9)
#%%@note 4. Testing the answer
# How often C wins when its INMB is uncertain, and the value of knowing the true state first.
# C ชนะบ่อยเพียงใดเมื่อ INMB ไม่แน่นอน และมูลค่าของการรู้สถานการณ์จริงก่อนเลือก
mean_inmb <- 10000
sd_inmb <- 40000
runs <- 1000
share <- stats::pnorm(mean_inmb / sd_inmb)
option_1 <- c(100, 140, 90, 130)
option_2 <- c(120, 110, 150, 100)
known <- mean(pmax(option_1, option_2))
single <- max(mean(option_1), mean(option_2))
stopifnot(mean_inmb == threshold[1] * gap[2] - gap[1])
#%%@note 5. Display the two strategies
# Plot cost against QALYs for A and C; the script saves no graphics file.
# แสดงต้นทุนเทียบกับ QALY ของ A และ C โดยไม่บันทึกไฟล์ภาพ
print(ggplot2::ggplot(strategies, ggplot2::aes(x = qaly, y = cost, label = strategy)) +
ggplot2::geom_point() + ggplot2::geom_text(nudge_y = 8000))
#%%@note 6. Numbers the article quotes
# Print the four short boxes' lines in their reading order.
# พิมพ์ผลลัพธ์ตรงกับกล่องสั้นทั้งสี่ตามลำดับในบทความ
cat("Cost per patient on A and on C (THB):", nice(cost), "\n")
cat("QALYs per patient on A and on C:", nice(qaly, 2), "\n")
cat("NMB of A and of C at 160,000 THB per QALY (THB):", nice(threshold[1] * qaly - cost), "\n")
cat("Extra cost (THB) and extra QALYs of C:", nice(gap[1]), nice(gap[2], 2), "\n")
cat("Cost per extra QALY (THB):", nice(gap[1] / gap[2]), "\n")
cat("INMB of C at 160,000, 150,000 and 140,000 (THB):", nice(threshold * gap[2] - gap[1]), "\n")
cat("Complication risk on C and on A:", paste0(nice(100 * events / treated, 2), "%"), "\n")
cat("QALYs over 10 years at 0.70, and the gain to full health:", nice(weight * years, 1),
nice(years * (1 - weight), 1), "\n")
cat("Chance of moving in one cycle, exact and shortcut:", nice(1 - exp(-rate * cycle), 4),
nice(rate * cycle, 4), "\n")
cat("Of 1,000 well people after one cycle (well, sick, dead):", start * move, "\n")
cat("D against C: extra cost (THB) and QALYs lost:", nice(d_cost), nice(d_loss, 2), "\n")
cat("D costs more and gives less health:", d_cost > 0 && d_loss > 0, "\n")
cat("Extra spending a year (THB):", nice(new_patients * extra_cost), "\n")
cat("Mean INMB over its spread:", formatC(mean_inmb / sd_inmb, format = "f", digits = 2), "\n")
cat("Share above zero, and the part past the centre:", round(c(share, share - 0.5), 4), "\n")
cat("Runs of 1,000 for C, and as a share:", round(runs * share), sprintf("%.2f", share), "\n")
cat("Best choice per state, best single choice, the gain:", known, single, known - single, "\n")
Stage two: build and compare the model
The early complication fits a decision tree, the subject of the sixth article Building a Decision Tree Step by Step: Follow Outcomes, Costs and Health. On A, 21.67% of patients take the complication branch and the rest do not; on C, 12.12% do. The tree weighs each branch's cost and QALYs by those shares and adds them up.
The years after need states and cycles, the subject of the seventh article. A Markov model follows a group of people between a few health states, one cycle at a time. Of 1,000 healthy people, 800 stay healthy, 150 fall sick and 50 die in the first cycle; the full lesson is in Building a Markov Cohort Model Step by Step: Follow the Cohort Through Time. Repeating such cycles over the horizon and adding up each cycle's people, costs and quality gives A's 8.00 QALYs for 50,000 THB and C's 9.00 QALYs for 200,000 THB.
The eighth article counts people before money Budget Impact Analysis in Practice: Count People Before Counting Money. Suppose about 100 new patients a year start C, and each one's extra 150,000 THB is paid in the year they start. The district then spends 15,000,000 THB more a year; if one year's patients have their cost spread over several years, that group's yearly share is smaller but lasts longer. This yearly district bill answers a different question from value.
Real committees rarely weigh only two strategies, and the ninth article puts several in order of cost ICER and Net Monetary Benefit Across Strategies: Find the Relevant Frontier. Dominance is the rule that removes an option when another costs less and gives at least as much health. Suppose a third option, D, costs 20,000 THB more than C and gives 0.10 QALY less: C dominates it; the full lesson is in ICER, Dominance and the Cost-Effectiveness Plane: Reading the Thai Threshold of 160,000 Baht per QALY.
Trace the model and the yearly bill in R
# The model: one cycle of the cohort, an option that is beaten outright, and the yearly bill.
nice <- function(value, places = 0) formatC(value, format = "f", digits = places, big.mark = ",")
start <- 1000
move <- c(0.80, 0.15, 0.05)
cat("Of 1,000 well people after one cycle (well, sick, dead):", start * move, "\n")
d_cost <- 20000
d_loss <- 0.10
cat("D against C: extra cost (THB) and QALYs lost:", nice(d_cost), nice(d_loss, 2), "\n")
cat("D costs more and gives less health:", d_cost > 0 && d_loss > 0, "\n")
new_patients <- 100
extra_cost <- 150000
cat("Extra spending a year (THB):", nice(new_patients * extra_cost), "\n")Of 1,000 well people after one cycle (well, sick, dead): 800 150 50
D against C: extra cost (THB) and QALYs lost: 20,000 0.10
D costs more and gives less health: TRUE
Extra spending a year (THB): 15,000,000Requires R ≥ 4.5.2 · tibble 3.3.1 · dplyr 1.2.1 · ggplot2 4.0.3 · works in R 4.6.1 · Download health-economic-modelling-series-guide.R
Full tidy script
# Health Economic Modelling Series Guide: From Evidence to a Decision
# คู่มือชุดบทความ: ติดตามกรณีของคณะกรรมการ ตั้งแต่ข้อมูลนำเข้า แบบจำลอง จนถึงการทดสอบคำตอบ
# Requires: R >= 4.5.2
# Tested: works in R 4.6.1
# tibble 3.3.1 (stable)
# dplyr 1.2.1 (stable)
# ggplot2 4.0.3 (stable)
# install.packages(c("tibble", "dplyr", "ggplot2"))
# update.packages(oldPkgs = c("tibble", "dplyr", "ggplot2"))
# remotes::install_version("tibble", "3.3.1")
# remotes::install_version("dplyr", "1.2.1")
# remotes::install_version("ggplot2", "4.0.3")
library(tibble) # Write the two strategies as one explicit input table.
library(dplyr) # Add each strategy's net benefit at the threshold.
library(ggplot2) # Display the two strategies without saving files.
stopifnot(getRversion() >= "4.5.2")
#%%@note 1. The committee's case
# Per patient over the model's horizon: cost and QALYs of usual care A and of treatment C.
# ต้นทุนและ QALY ต่อผู้ป่วยตลอดช่วงเวลาของแบบจำลอง ของการดูแลตามปกติ A และการรักษา C
nice <- function(value, places = 0) formatC(value, format = "f", digits = places, big.mark = ",")
cost <- c(50000, 200000)
qaly <- c(8, 9)
threshold <- c(160000, 150000, 140000)
gap <- c(diff(cost), diff(qaly))
strategies <- tibble::tibble(strategy = c("A", "C"), cost = cost, qaly = qaly) |>
dplyr::mutate(net_benefit = threshold[1] * qaly - cost)
stopifnot(abs(diff(strategies$net_benefit) - (threshold[1] * gap[2] - gap[1])) < 1e-9)
#%%@note 2. Inputs the model needs
# The trial's two risks, the weight behind a QALY, and one yearly rate as a cycle's chance.
# ความเสี่ยงสองค่าจากการทดลอง ค่าน้ำหนักเบื้องหลัง QALY และอัตรารายปีที่แปลงเป็นโอกาสในหนึ่งรอบ
events <- c(8, 13)
treated <- c(66, 60)
weight <- 0.70
years <- 10
rate <- 0.12
cycle <- 0.25
#%%@note 3. The model and the yearly bill
# One cycle of the cohort, an option that costs more for less health, and the yearly bill.
# หนึ่งรอบของกลุ่มประชากร ทางเลือกที่มีต้นทุนมากกว่าแต่ได้สุขภาพน้อยกว่า และค่าใช้จ่ายต่อปี
start <- 1000
move <- c(0.80, 0.15, 0.05)
d_cost <- 20000
d_loss <- 0.10
new_patients <- 100
extra_cost <- 150000
stopifnot(extra_cost == gap[1], abs(sum(move) - 1) < 1e-9)
#%%@note 4. Testing the answer
# How often C wins when its INMB is uncertain, and the value of knowing the true state first.
# C ชนะบ่อยเพียงใดเมื่อ INMB ไม่แน่นอน และมูลค่าของการรู้สถานการณ์จริงก่อนเลือก
mean_inmb <- 10000
sd_inmb <- 40000
runs <- 1000
share <- stats::pnorm(mean_inmb / sd_inmb)
option_1 <- c(100, 140, 90, 130)
option_2 <- c(120, 110, 150, 100)
known <- mean(pmax(option_1, option_2))
single <- max(mean(option_1), mean(option_2))
stopifnot(mean_inmb == threshold[1] * gap[2] - gap[1])
#%%@note 5. Display the two strategies
# Plot cost against QALYs for A and C; the script saves no graphics file.
# แสดงต้นทุนเทียบกับ QALY ของ A และ C โดยไม่บันทึกไฟล์ภาพ
print(ggplot2::ggplot(strategies, ggplot2::aes(x = qaly, y = cost, label = strategy)) +
ggplot2::geom_point() + ggplot2::geom_text(nudge_y = 8000))
#%%@note 6. Numbers the article quotes
# Print the four short boxes' lines in their reading order.
# พิมพ์ผลลัพธ์ตรงกับกล่องสั้นทั้งสี่ตามลำดับในบทความ
cat("Cost per patient on A and on C (THB):", nice(cost), "\n")
cat("QALYs per patient on A and on C:", nice(qaly, 2), "\n")
cat("NMB of A and of C at 160,000 THB per QALY (THB):", nice(threshold[1] * qaly - cost), "\n")
cat("Extra cost (THB) and extra QALYs of C:", nice(gap[1]), nice(gap[2], 2), "\n")
cat("Cost per extra QALY (THB):", nice(gap[1] / gap[2]), "\n")
cat("INMB of C at 160,000, 150,000 and 140,000 (THB):", nice(threshold * gap[2] - gap[1]), "\n")
cat("Complication risk on C and on A:", paste0(nice(100 * events / treated, 2), "%"), "\n")
cat("QALYs over 10 years at 0.70, and the gain to full health:", nice(weight * years, 1),
nice(years * (1 - weight), 1), "\n")
cat("Chance of moving in one cycle, exact and shortcut:", nice(1 - exp(-rate * cycle), 4),
nice(rate * cycle, 4), "\n")
cat("Of 1,000 well people after one cycle (well, sick, dead):", start * move, "\n")
cat("D against C: extra cost (THB) and QALYs lost:", nice(d_cost), nice(d_loss, 2), "\n")
cat("D costs more and gives less health:", d_cost > 0 && d_loss > 0, "\n")
cat("Extra spending a year (THB):", nice(new_patients * extra_cost), "\n")
cat("Mean INMB over its spread:", formatC(mean_inmb / sd_inmb, format = "f", digits = 2), "\n")
cat("Share above zero, and the part past the centre:", round(c(share, share - 0.5), 4), "\n")
cat("Runs of 1,000 for C, and as a share:", round(runs * share), sprintf("%.2f", share), "\n")
cat("Best choice per state, best single choice, the gain:", known, single, known - single, "\n")
Stage three: test the answer
Every input above is uncertain, and a trial this small leaves the effect especially so; the tenth article deals with that. Probabilistic sensitivity analysis (PSA) reruns the model many times with every uncertain input drawn from its distribution, so the result is a share of runs rather than one number. When the extra net benefit averages 10,000 THB with a standard deviation of 40,000 THB, about 599 of 1,000 runs favour the treatment; the full lesson is in Probabilistic Sensitivity Analysis: From Parameter Distributions to CEAC.
Where does 599 come from? A standard deviation is the typical distance of a result from its average, and here the average, 10,000, sits 0.25 of those distances above zero (10,000 / 40,000). If the results follow a symmetric bell-shaped curve, 0.5987 of the curve lies above zero: the half above the average, plus the 0.0987 between zero and the average. That is a share of 0.60, about 599 of 1,000 runs.
A cost-effectiveness acceptability curve (CEAC) shows, at each threshold, the share of simulation runs in which a treatment is the best buy, meaning it has the higher net benefit. At 160,000 THB per QALY it reads 0.60, because about 599 of 1,000 runs favour the treatment; the curve is drawn in full in Probabilistic Sensitivity Analysis: From Parameter Distributions to CEAC.
A model can also be wrong in a more basic way, and the eleventh article tests two kinds of check Model Validation and Verification: Test the Question, Equations and Code. Verification asks whether the code computes what the equations say: 160,000 × 1.00 - 150,000 worked by hand must give the same 10,000 as the model. Validation asks whether the model is a believable picture of the real pathway. Does its structure make clinical sense, and do its predictions match what is observed, such as the 21.67% complication risk on usual care (Eddy and colleagues)?
The twelfth article asks whether better evidence is worth buying before deciding. The expected value of perfect information (EVPI) is the average gain from knowing the true state before choosing. Take a small made-up example on a net-benefit scale with four equally likely states: one option scores 100, 140, 90 and 130 and the other scores 120, 110, 150 and 100 in turn. Picking the better score in each state (120, 140, 150 and 130) averages 135 against 120 for the best single choice, the second option, so EVPI is 15; the full lesson is in Expected Value of Perfect Information: What Could Better Evidence Be Worth?.
Evidence that costs more than 15 on this scale cannot pay for itself.
Trace the tests of the answer in R
# Testing the answer: how often C wins when its INMB is uncertain; what perfect knowledge is worth.
mean_inmb <- 10000
sd_inmb <- 40000
runs <- 1000
share <- stats::pnorm(mean_inmb / sd_inmb)
cat("Mean INMB over its spread:", formatC(mean_inmb / sd_inmb, format = "f", digits = 2), "\n")
cat("Share above zero, and the part past the centre:", round(c(share, share - 0.5), 4), "\n")
cat("Runs of 1,000 for C, and as a share:", round(runs * share), sprintf("%.2f", share), "\n")
option_1 <- c(100, 140, 90, 130)
option_2 <- c(120, 110, 150, 100)
known <- mean(pmax(option_1, option_2))
single <- max(mean(option_1), mean(option_2))
cat("Best choice per state, best single choice, the gain:", known, single, known - single, "\n")Mean INMB over its spread: 0.25
Share above zero, and the part past the centre: 0.5987 0.0987
Runs of 1,000 for C, and as a share: 599 0.60
Best choice per state, best single choice, the gain: 135 120 15Requires R ≥ 4.5.2 · tibble 3.3.1 · dplyr 1.2.1 · ggplot2 4.0.3 · works in R 4.6.1 · Download health-economic-modelling-series-guide.R
Full tidy script
# Health Economic Modelling Series Guide: From Evidence to a Decision
# คู่มือชุดบทความ: ติดตามกรณีของคณะกรรมการ ตั้งแต่ข้อมูลนำเข้า แบบจำลอง จนถึงการทดสอบคำตอบ
# Requires: R >= 4.5.2
# Tested: works in R 4.6.1
# tibble 3.3.1 (stable)
# dplyr 1.2.1 (stable)
# ggplot2 4.0.3 (stable)
# install.packages(c("tibble", "dplyr", "ggplot2"))
# update.packages(oldPkgs = c("tibble", "dplyr", "ggplot2"))
# remotes::install_version("tibble", "3.3.1")
# remotes::install_version("dplyr", "1.2.1")
# remotes::install_version("ggplot2", "4.0.3")
library(tibble) # Write the two strategies as one explicit input table.
library(dplyr) # Add each strategy's net benefit at the threshold.
library(ggplot2) # Display the two strategies without saving files.
stopifnot(getRversion() >= "4.5.2")
#%%@note 1. The committee's case
# Per patient over the model's horizon: cost and QALYs of usual care A and of treatment C.
# ต้นทุนและ QALY ต่อผู้ป่วยตลอดช่วงเวลาของแบบจำลอง ของการดูแลตามปกติ A และการรักษา C
nice <- function(value, places = 0) formatC(value, format = "f", digits = places, big.mark = ",")
cost <- c(50000, 200000)
qaly <- c(8, 9)
threshold <- c(160000, 150000, 140000)
gap <- c(diff(cost), diff(qaly))
strategies <- tibble::tibble(strategy = c("A", "C"), cost = cost, qaly = qaly) |>
dplyr::mutate(net_benefit = threshold[1] * qaly - cost)
stopifnot(abs(diff(strategies$net_benefit) - (threshold[1] * gap[2] - gap[1])) < 1e-9)
#%%@note 2. Inputs the model needs
# The trial's two risks, the weight behind a QALY, and one yearly rate as a cycle's chance.
# ความเสี่ยงสองค่าจากการทดลอง ค่าน้ำหนักเบื้องหลัง QALY และอัตรารายปีที่แปลงเป็นโอกาสในหนึ่งรอบ
events <- c(8, 13)
treated <- c(66, 60)
weight <- 0.70
years <- 10
rate <- 0.12
cycle <- 0.25
#%%@note 3. The model and the yearly bill
# One cycle of the cohort, an option that costs more for less health, and the yearly bill.
# หนึ่งรอบของกลุ่มประชากร ทางเลือกที่มีต้นทุนมากกว่าแต่ได้สุขภาพน้อยกว่า และค่าใช้จ่ายต่อปี
start <- 1000
move <- c(0.80, 0.15, 0.05)
d_cost <- 20000
d_loss <- 0.10
new_patients <- 100
extra_cost <- 150000
stopifnot(extra_cost == gap[1], abs(sum(move) - 1) < 1e-9)
#%%@note 4. Testing the answer
# How often C wins when its INMB is uncertain, and the value of knowing the true state first.
# C ชนะบ่อยเพียงใดเมื่อ INMB ไม่แน่นอน และมูลค่าของการรู้สถานการณ์จริงก่อนเลือก
mean_inmb <- 10000
sd_inmb <- 40000
runs <- 1000
share <- stats::pnorm(mean_inmb / sd_inmb)
option_1 <- c(100, 140, 90, 130)
option_2 <- c(120, 110, 150, 100)
known <- mean(pmax(option_1, option_2))
single <- max(mean(option_1), mean(option_2))
stopifnot(mean_inmb == threshold[1] * gap[2] - gap[1])
#%%@note 5. Display the two strategies
# Plot cost against QALYs for A and C; the script saves no graphics file.
# แสดงต้นทุนเทียบกับ QALY ของ A และ C โดยไม่บันทึกไฟล์ภาพ
print(ggplot2::ggplot(strategies, ggplot2::aes(x = qaly, y = cost, label = strategy)) +
ggplot2::geom_point() + ggplot2::geom_text(nudge_y = 8000))
#%%@note 6. Numbers the article quotes
# Print the four short boxes' lines in their reading order.
# พิมพ์ผลลัพธ์ตรงกับกล่องสั้นทั้งสี่ตามลำดับในบทความ
cat("Cost per patient on A and on C (THB):", nice(cost), "\n")
cat("QALYs per patient on A and on C:", nice(qaly, 2), "\n")
cat("NMB of A and of C at 160,000 THB per QALY (THB):", nice(threshold[1] * qaly - cost), "\n")
cat("Extra cost (THB) and extra QALYs of C:", nice(gap[1]), nice(gap[2], 2), "\n")
cat("Cost per extra QALY (THB):", nice(gap[1] / gap[2]), "\n")
cat("INMB of C at 160,000, 150,000 and 140,000 (THB):", nice(threshold * gap[2] - gap[1]), "\n")
cat("Complication risk on C and on A:", paste0(nice(100 * events / treated, 2), "%"), "\n")
cat("QALYs over 10 years at 0.70, and the gain to full health:", nice(weight * years, 1),
nice(years * (1 - weight), 1), "\n")
cat("Chance of moving in one cycle, exact and shortcut:", nice(1 - exp(-rate * cycle), 4),
nice(rate * cycle, 4), "\n")
cat("Of 1,000 well people after one cycle (well, sick, dead):", start * move, "\n")
cat("D against C: extra cost (THB) and QALYs lost:", nice(d_cost), nice(d_loss, 2), "\n")
cat("D costs more and gives less health:", d_cost > 0 && d_loss > 0, "\n")
cat("Extra spending a year (THB):", nice(new_patients * extra_cost), "\n")
cat("Mean INMB over its spread:", formatC(mean_inmb / sd_inmb, format = "f", digits = 2), "\n")
cat("Share above zero, and the part past the centre:", round(c(share, share - 0.5), 4), "\n")
cat("Runs of 1,000 for C, and as a share:", round(runs * share), sprintf("%.2f", share), "\n")
cat("Best choice per state, best single choice, the gain:", known, single, known - single, "\n")
Keep value, policy and budget apart
The model's result is conditional. At 160,000 THB per QALY, C is worth 10,000 THB more per patient than it costs. That threshold is a policy choice, not a model output: at 150,000 the INMB is 0, and at 140,000 it is -10,000 THB, with the same patients and the same evidence.
The budget is a third question. A bill of 15,000,000 THB a year may be more than the district can spend, even though C is good value at a threshold of 160,000 THB per QALY. A brief that keeps value, policy and budget apart shows the committee which of the three it is deciding (Drummond and colleagues).
Where the foundation series fits
This series builds on the foundation series. Its article on decision trees, Markov models and PSA shows the three tools side by side, and its article on the ICER, dominance and thresholds teaches the committee's line in full. Read those first if a step above felt fast.
Terms as we use them
- Strategy (ทางเลือก)
- A whole path of care the committee could choose, such as A or C, with everything that follows it.
- Parameter (พารามิเตอร์)
- One input the model needs, such as the 12.12% risk on C or the quality weight 0.70 of a sick year.
- State and event (สถานะและเหตุการณ์)
- A state is where a patient is at a given time (well, sick, dead); an event, such as the complication, happens on the way.
- Cycle and horizon (รอบและช่วงเวลาของแบบจำลอง)
- A cycle is one fixed time step of the model; the horizon is the whole period the model follows patients.
- Threshold (เกณฑ์ความคุ้มค่า)
- The most the committee will pay for one QALY, here 160,000 THB: a policy choice, not a model result.
- Verification and validation (การตรวจสอบการคำนวณและการประเมินความสมเหตุสมผล)
- Verification checks that the code computes what the equations say; validation checks that the model is a believable picture of the real pathway.
Try it in the app
Open our economic evaluation app to change the inputs and watch the result update. It needs a login and a Pro or Max plan.
The app starts from its own example values, so its numbers will not match this article.
References
- Andrew Briggs, Karl Claxton, Mark Sculpher. Decision Modelling for Health Economic Evaluation (Handbooks for Health Economic Evaluation). Oxford University Press; 2006. ISBN 9780198526629. https://openlibrary.org/isbn/9780198526629
- Siebert Uwe, Alagoz Oguzhan, Bayoumi Ahmed M., Jahn Beate, Owens Douglas K., Cohen David J., Kuntz Karen M.. State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value in Health. 2012;15(6):812-820. doi:10.1016/j.jval.2012.06.014. https://doi.org/10.1016/j.jval.2012.06.014
- Eddy David M., Hollingworth William, Caro J. Jaime, Tsevat Joel, McDonald Kathryn M., Wong John B.. Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value in Health. 2012;15(6):843-850. doi:10.1016/j.jval.2012.04.012. https://doi.org/10.1016/j.jval.2012.04.012
- Michael F. Drummond, Mark J. Sculpher, Karl Claxton, Greg L. Stoddart, George W. Torrance. Methods for the Economic Evaluation of Health Care Programmes. Oxford University Press; 2015. ISBN 9780199665884. https://openlibrary.org/isbn/9780199665884
The series, in learning order
- Health Economic Modelling Series Guide: From Evidence to a Decision (this article)
- Choosing a Decision-Analytic Model: Let the Clinical Pathway Decide
- Auditable Model Inputs: Build a Parameter Table You Can Trace
- Finding Model Inputs in the Literature: Match the Estimate to the Question
- Rates, Probabilities and Transition Probabilities: Keep Time in the Calculation
- Building a Decision Tree Step by Step: Follow Outcomes, Costs and Health
- Building a Markov Cohort Model Step by Step: Follow the Cohort Through Time
- Budget Impact Analysis in Practice: Count People Before Counting Money
- ICER and Net Monetary Benefit Across Strategies: Find the Relevant Frontier
- Probabilistic Sensitivity Analysis: From Parameter Distributions to CEAC
- Model Validation and Verification: Test the Question, Equations and Code
- Expected Value of Perfect Information: What Could Better Evidence Be Worth?
Key takeaways
- A cost-effectiveness line is the end of a model: C's 1.00 QALY came from following patients for years, not from the trial alone.
- Build inputs before outputs: a shape the pathway decides, a table you can trace, sources that fit the question and chances per cycle that keep time in.
- Read the committee's line two ways: 150,000 THB per QALY against a threshold of 160,000, or an INMB of 10,000 THB per patient.
- Test the answer: about 599 of 1,000 reruns favour C, the code must reproduce its own numbers, and better evidence has a value worth knowing.
- Keep value, policy and budget apart: at 140,000 THB per QALY the same evidence says no, and 15,000,000 THB a year, if paid as patients start, is a separate question.
Related in the wiki: Decision Trees, Markov Cohorts and Probabilistic Sensitivity Analysis: How a Model Turns Evidence into a Decision · ICER, Dominance and the Cost-Effectiveness Plane: Reading the Thai Threshold of 160,000 Baht per QALY