Finding Model Inputs in the Literature: Match the Estimate to the Question
On this page
อ่านฉบับภาษาไทย (Thai version)
Abstract
A provincial hospital wants to know whether nurse phone calls after discharge are worth paying for. Its records show 200 readmissions within three months among 1,000 heart failure discharges, a risk of 0.20 under usual care. The model also needs the effect of the calls, and a literature search finds three studies that each report 0.70. They are not the same number. One trial reports a ratio of risks over three months, which can multiply the hospital's 0.20 directly: 0.20 × 0.70 = 0.14, so 60 fewer readmissions per 1,000 discharges. The other two report a ratio of odds and a ratio of how fast readmissions occur; converted under their own assumptions they give 51 and 55 fewer. Before searching, write the question the number must answer: whose patients, which comparison, which outcome, over what time. Then judge each study three ways: is it sound, does it fit these patients, and what must be assumed to convert it.
One hospital, one question, three studies
A provincial hospital discharges about 1,000 patients with heart failure each year. Its records show that 200 of them come back as an emergency admission within three months, a risk of 0.20 under usual care. The cardiology team proposes nurse phone calls in the first weeks after discharge. The hospital board wants a model before it pays for the nurses' time.
The model starts simply. Each discharged patient goes down one of two branches: readmitted within three months, or not. Under usual care the readmitted branch takes 0.20 of patients. With the calls it takes a smaller share, and that share is the number this article is about.
A colleague searches the literature and comes back with good news: three studies, and each one reports 0.70 for nurse follow-up. It is tempting to type 0.70 into the model and move on. Yet the three 0.70s mean different things, and only one of them can multiply the hospital's 0.20 as it stands.
Write the question before the search
The number the model needs has four parts. Whose patients: adults with heart failure discharged from this hospital. Which comparison: nurse phone calls against usual care. Which outcome, and over what time: an emergency readmission, within three months, counted as a share of patients, which is a risk.
Written down, those four parts are the parameter question, the exact quantity one model input must estimate. The question then shapes the search: terms for heart failure, for discharge, for telephone or nurse follow-up and for readmission, joined so that a study must touch all four ideas.
Record the databases, the full search strings, the date of the search and any limits, so another analyst could run it again and find the same studies (Briggs and colleagues).
Set the rules for keeping a study before reading any results: which designs, which comparisons, which outcome definitions and which follow-up times count. Apply the same rules to every study, and write down why each excluded study was dropped. In the hospital's search, three studies pass these rules, and each one reports 0.70.
Three studies, three kinds of 0.70
Trial A randomised patients to phone calls or usual care and counted readmissions within three months. Its 0.70 is a risk ratio: the risk in the called group divided by the risk under usual care, over the same three months. A risk ratio can multiply a risk, which is exactly what the model's branch needs.
Study B used hospital records, not randomisation, and reports an odds ratio from a regression. An odds ratio compares the odds of an event, the events divided by the non-events, between two groups. It drifts further from one than the risk ratio when events are common. With 8 of 66 treated and 13 of 60 control patients having the event, it is 0.50 while the risk ratio is 0.56; the full lesson is in Risk Ratio, Odds Ratio, Risk Difference and NNT: One 2×2 Table, Four Answers.
Trial C followed patients for a year and reports a hazard ratio from a survival analysis. A hazard ratio compares how fast events happen among people still event-free, moment by moment; it is not a ratio of risks over a period. Take a hazard ratio of 0.70 and a control survival of 0.60, meaning six in ten control patients still alive after five years. Then the five-year risks of death are 40.0% in the control group and 30.1% in the treated group, a risk ratio of 0.75 rather than 0.70; the full lesson is in The Hazard Ratio Is Not a Risk Ratio: Proportional Hazards, Restricted Mean Survival Time and Model Inputs.
So the three 0.70s are a ratio of risks, a ratio of odds and a ratio of speeds. Only Trial A's can multiply the hospital's 0.20 as it stands. The other two must be converted first, and each conversion rests on an assumption.
Check the two recap examples in R
# The two foundation examples behind the recaps: one trial's counts and one five-year survival.
# Writes: nothing to disk; the results print in the console.
events <- c(treated = 8, control = 13)
patients <- c(treated = 66, control = 60)
risks <- events / patients
odds <- events / (patients - events)
control_survival <- 0.60
hazard_ratio <- 0.70
five_year_risks <- 1 - control_survival^c(1, hazard_ratio)
cat("Events, then patients, treated and control:", events, patients, "\n")
cat("Risk ratio and odds ratio:", sprintf("%.2f", c(risks[1] / risks[2], odds[1] / odds[2])), "\n")
cat("Five-year risks, control and treated:", sprintf("%.1f%%", 100 * five_year_risks), "\n")
cat("Five-year risk ratio:", sprintf("%.2f", five_year_risks[2] / five_year_risks[1]), "\n")Events, then patients, treated and control: 8 13 66 60
Risk ratio and odds ratio: 0.56 0.50
Five-year risks, control and treated: 40.0% 30.1%
Five-year risk ratio: 0.75Requires R ≥ 4.5.2 · tibble 3.3.1 · dplyr 1.2.1 · ggplot2 4.0.3 · works in R 4.6.1 · Download finding-model-inputs-in-the-literature.R
Full tidy script
# Finding Model Inputs in the Literature: Match the Estimate to the Question
# Follow one hospital's readmission question from three candidate studies to its model input.
# ติดตามคำถามเรื่องการกลับมานอนโรงพยาบาลของโรงพยาบาลหนึ่ง จากการศึกษาสามเรื่องสู่ค่าที่แบบจำลองใช้
# Requires: R >= 4.5.2
# Tested: works in R 4.6.1
# tibble 3.3.1 (stable)
# dplyr 1.2.1 (stable)
# ggplot2 4.0.3 (stable)
# install.packages(c("tibble", "dplyr", "ggplot2"))
# update.packages(oldPkgs = c("tibble", "dplyr", "ggplot2"))
# remotes::install_version("tibble", "3.3.1")
# remotes::install_version("dplyr", "1.2.1")
# remotes::install_version("ggplot2", "4.0.3")
library(tibble) # Write the candidate studies and the readings as explicit tables.
library(dplyr) # Calculate the treated risk for each reading.
library(ggplot2) # Display the readings side by side without saving files.
stopifnot(getRversion() >= "4.5.2")
#%%@note 1. Define the illustrative inputs
# Every input is synthetic and written out once, in the order the article uses it.
# ค่าตั้งต้นทั้งหมดเป็นข้อมูลสมมติ เขียนไว้ครั้งเดียวตามลำดับที่บทความใช้
discharges <- 1000
baseline_risk <- 0.20
risk_ratio <- 0.70
baselines <- c(0.20, 0.60)
effect <- 0.70
trial_control_risk <- 0.25
events <- c(treated = 8, control = 13)
patients <- c(treated = 66, control = 60)
control_survival <- 0.60
hazard_ratio <- 0.70
#%%@note 2. Rework the two foundation examples
# One trial's counts give both ratios; one survival curve turns a rate ratio into risks.
# จำนวนจากการทดลองหนึ่งให้อัตราส่วนทั้งสองแบบ และเส้นการอยู่รอดหนึ่งเส้นแปลงอัตราส่วนเป็นความเสี่ยง
risks <- events / patients
odds <- events / (patients - events)
five_year_risks <- 1 - control_survival^c(1, hazard_ratio)
#%%@note 3. Apply the matching risk ratio to the local baseline
# Treated risk is the baseline times the ratio; the gap is the absolute reduction.
# ความเสี่ยงเมื่อมีการโทรติดตามคือความเสี่ยงพื้นฐานคูณอัตราส่วน ผลต่างคือการลดลงสัมบูรณ์
treated_risk <- baseline_risk * risk_ratio
absolute_reduction <- baseline_risk - treated_risk
per_thousand <- discharges * c(baseline_risk, treated_risk, absolute_reduction)
#%%@note 4. Read the same value through odds and through the share still free
# Each reading changes a different quantity, so each gives a different treated risk.
# แต่ละวิธีอ่านเปลี่ยนปริมาณคนละตัว จึงได้ความเสี่ยงเมื่อมีการโทรติดตามต่างกัน
base_odds <- baselines / (1 - baselines)
call_odds <- effect * base_odds
readings <- tibble::tibble(reading = rep(c("risk", "odds", "hazard"), times = 2),
baseline = rep(baselines, each = 3)) |>
dplyr::mutate(odds = effect * baseline / (1 - baseline),
treated = dplyr::case_when(reading == "risk" ~ baseline * effect,
reading == "odds" ~ odds / (1 + odds),
TRUE ~ 1 - (1 - baseline)^effect))
treated <- matrix(readings$treated, nrow = 3, dimnames = list(c("risk", "odds", "hazard"), NULL))
stopifnot(abs(treated["odds", ] - call_odds / (1 + call_odds)) < 1e-12)
shown <- sprintf("%.2f%%", 100 * treated)
#%%@note 5. Pair the trial's ratio with each baseline
# The ratio comes from the trial; the baseline should come from the hospital's own patients.
# อัตราส่วนมาจากการทดลอง ส่วนความเสี่ยงพื้นฐานควรมาจากผู้ป่วยของโรงพยาบาลเอง
baselines_compared <- c(baseline_risk, trial_control_risk)
avoided <- discharges * baselines_compared * (1 - risk_ratio)
trial_treated <- trial_control_risk * risk_ratio
#%%@note 6. Display the readings
# Plot the treated risk for each reading at both baselines; no graphics file is saved.
# แสดงความเสี่ยงเมื่อมีการโทรติดตามของแต่ละวิธีอ่านที่ความเสี่ยงพื้นฐานทั้งสองค่า ไม่บันทึกไฟล์ภาพ
print(ggplot2::ggplot(readings, ggplot2::aes(x = reading, y = 100 * treated)) +
ggplot2::geom_col() + ggplot2::facet_wrap(~baseline) +
ggplot2::labs(x = "How 0.70 is read", y = "Treated risk (%)"))
#%%@note 7. Numbers the article quotes
# Print exactly the short boxes in their reading order.
# พิมพ์ผลลัพธ์ตรงกับกล่องตัวอย่างตามลำดับในบทความ
cat("Events, then patients, treated and control:", events, patients, "\n")
cat("Risk ratio and odds ratio:", sprintf("%.2f", c(risks[1] / risks[2], odds[1] / odds[2])), "\n")
cat("Five-year risks, control and treated:", sprintf("%.1f%%", 100 * five_year_risks), "\n")
cat("Five-year risk ratio:", sprintf("%.2f", five_year_risks[2] / five_year_risks[1]), "\n")
cat("Treated risk as a share:", treated_risk, "\n")
cat("Treated risk (%):", round(100 * treated_risk, 2), "\n")
cat("Absolute reduction (percentage points):", round(100 * absolute_reduction, 2), "\n")
cat("Per 1,000 discharges, usual care, programme, avoided:", round(per_thousand), "\n")
cat("Odds at 0.20: usual, calls, calls + one:", base_odds[1], call_odds[1], 1 + call_odds[1], "\n")
cat("Still free at 0.20, usual, hazard:", round(1 - c(baselines[1], treated[3, 1]), 3), "\n")
cat("Treated risk at 0.20 read as risk, odds, hazard ratio:", shown[1:3], "\n")
cat("Avoided per 1,000 at 0.20:", round(1000 * (baselines[1] - treated[, 1])), "\n")
cat("Treated risk at 0.60 read as risk, odds, hazard ratio:", shown[4:6], "\n")
cat("Treated risk with the trial baseline (%):", round(100 * trial_treated, 2), "\n")
cat("Avoided per 1,000, local and trial baseline:", round(avoided), "\n")
Worked example: Trial A's 0.70 on the hospital's patients
Trial A's 0.70 is a risk ratio over three months, the same outcome and time the model needs. Assume for now that it carries over to this hospital's patients; a later section tests that assumption. The numbers are synthetic, for teaching, and come with no interval, so the example says nothing about precision.
-
Start from the hospital's own risk
\[ \frac{200}{1000} = 0.20 \]
Under usual care, 200 of 1,000 discharged patients are readmitted within three months: a risk of 0.20.
-
Multiply by the risk ratio
\[ 0.20 \times 0.70 = 0.14 \]
With the calls, the risk falls to 0.14, which is 14% of discharged patients.
-
Subtract to find the benefit
\[ 200 - 140 = 60 \]
The gap is 6 percentage points. Of 1,000 discharges, 140 come back rather than 200 under usual care, so the calls avoid 60 readmissions.
Result: Read as the risk ratio it is, Trial A's 0.70 gives 60 fewer readmissions per 1,000 discharges, provided it carries over to these patients.
Check the worked example in R
# One hospital's heart failure discharges: readmission within 90 days, synthetic teaching numbers.
# Writes: nothing to disk; the results print in the console.
discharges <- 1000
baseline_risk <- 0.20
risk_ratio <- 0.70
treated_risk <- baseline_risk * risk_ratio
absolute_reduction <- baseline_risk - treated_risk
per_thousand <- discharges * c(baseline_risk, treated_risk, absolute_reduction)
cat("Treated risk as a share:", treated_risk, "\n")
cat("Treated risk (%):", round(100 * treated_risk, 2), "\n")
cat("Absolute reduction (percentage points):", round(100 * absolute_reduction, 2), "\n")
cat("Per 1,000 discharges, usual care, programme, avoided:", round(per_thousand), "\n")Treated risk as a share: 0.14
Treated risk (%): 14
Absolute reduction (percentage points): 6
Per 1,000 discharges, usual care, programme, avoided: 200 140 60Requires R ≥ 4.5.2 · tibble 3.3.1 · dplyr 1.2.1 · ggplot2 4.0.3 · works in R 4.6.1 · Download finding-model-inputs-in-the-literature.R
Full tidy script
# Finding Model Inputs in the Literature: Match the Estimate to the Question
# Follow one hospital's readmission question from three candidate studies to its model input.
# ติดตามคำถามเรื่องการกลับมานอนโรงพยาบาลของโรงพยาบาลหนึ่ง จากการศึกษาสามเรื่องสู่ค่าที่แบบจำลองใช้
# Requires: R >= 4.5.2
# Tested: works in R 4.6.1
# tibble 3.3.1 (stable)
# dplyr 1.2.1 (stable)
# ggplot2 4.0.3 (stable)
# install.packages(c("tibble", "dplyr", "ggplot2"))
# update.packages(oldPkgs = c("tibble", "dplyr", "ggplot2"))
# remotes::install_version("tibble", "3.3.1")
# remotes::install_version("dplyr", "1.2.1")
# remotes::install_version("ggplot2", "4.0.3")
library(tibble) # Write the candidate studies and the readings as explicit tables.
library(dplyr) # Calculate the treated risk for each reading.
library(ggplot2) # Display the readings side by side without saving files.
stopifnot(getRversion() >= "4.5.2")
#%%@note 1. Define the illustrative inputs
# Every input is synthetic and written out once, in the order the article uses it.
# ค่าตั้งต้นทั้งหมดเป็นข้อมูลสมมติ เขียนไว้ครั้งเดียวตามลำดับที่บทความใช้
discharges <- 1000
baseline_risk <- 0.20
risk_ratio <- 0.70
baselines <- c(0.20, 0.60)
effect <- 0.70
trial_control_risk <- 0.25
events <- c(treated = 8, control = 13)
patients <- c(treated = 66, control = 60)
control_survival <- 0.60
hazard_ratio <- 0.70
#%%@note 2. Rework the two foundation examples
# One trial's counts give both ratios; one survival curve turns a rate ratio into risks.
# จำนวนจากการทดลองหนึ่งให้อัตราส่วนทั้งสองแบบ และเส้นการอยู่รอดหนึ่งเส้นแปลงอัตราส่วนเป็นความเสี่ยง
risks <- events / patients
odds <- events / (patients - events)
five_year_risks <- 1 - control_survival^c(1, hazard_ratio)
#%%@note 3. Apply the matching risk ratio to the local baseline
# Treated risk is the baseline times the ratio; the gap is the absolute reduction.
# ความเสี่ยงเมื่อมีการโทรติดตามคือความเสี่ยงพื้นฐานคูณอัตราส่วน ผลต่างคือการลดลงสัมบูรณ์
treated_risk <- baseline_risk * risk_ratio
absolute_reduction <- baseline_risk - treated_risk
per_thousand <- discharges * c(baseline_risk, treated_risk, absolute_reduction)
#%%@note 4. Read the same value through odds and through the share still free
# Each reading changes a different quantity, so each gives a different treated risk.
# แต่ละวิธีอ่านเปลี่ยนปริมาณคนละตัว จึงได้ความเสี่ยงเมื่อมีการโทรติดตามต่างกัน
base_odds <- baselines / (1 - baselines)
call_odds <- effect * base_odds
readings <- tibble::tibble(reading = rep(c("risk", "odds", "hazard"), times = 2),
baseline = rep(baselines, each = 3)) |>
dplyr::mutate(odds = effect * baseline / (1 - baseline),
treated = dplyr::case_when(reading == "risk" ~ baseline * effect,
reading == "odds" ~ odds / (1 + odds),
TRUE ~ 1 - (1 - baseline)^effect))
treated <- matrix(readings$treated, nrow = 3, dimnames = list(c("risk", "odds", "hazard"), NULL))
stopifnot(abs(treated["odds", ] - call_odds / (1 + call_odds)) < 1e-12)
shown <- sprintf("%.2f%%", 100 * treated)
#%%@note 5. Pair the trial's ratio with each baseline
# The ratio comes from the trial; the baseline should come from the hospital's own patients.
# อัตราส่วนมาจากการทดลอง ส่วนความเสี่ยงพื้นฐานควรมาจากผู้ป่วยของโรงพยาบาลเอง
baselines_compared <- c(baseline_risk, trial_control_risk)
avoided <- discharges * baselines_compared * (1 - risk_ratio)
trial_treated <- trial_control_risk * risk_ratio
#%%@note 6. Display the readings
# Plot the treated risk for each reading at both baselines; no graphics file is saved.
# แสดงความเสี่ยงเมื่อมีการโทรติดตามของแต่ละวิธีอ่านที่ความเสี่ยงพื้นฐานทั้งสองค่า ไม่บันทึกไฟล์ภาพ
print(ggplot2::ggplot(readings, ggplot2::aes(x = reading, y = 100 * treated)) +
ggplot2::geom_col() + ggplot2::facet_wrap(~baseline) +
ggplot2::labs(x = "How 0.70 is read", y = "Treated risk (%)"))
#%%@note 7. Numbers the article quotes
# Print exactly the short boxes in their reading order.
# พิมพ์ผลลัพธ์ตรงกับกล่องตัวอย่างตามลำดับในบทความ
cat("Events, then patients, treated and control:", events, patients, "\n")
cat("Risk ratio and odds ratio:", sprintf("%.2f", c(risks[1] / risks[2], odds[1] / odds[2])), "\n")
cat("Five-year risks, control and treated:", sprintf("%.1f%%", 100 * five_year_risks), "\n")
cat("Five-year risk ratio:", sprintf("%.2f", five_year_risks[2] / five_year_risks[1]), "\n")
cat("Treated risk as a share:", treated_risk, "\n")
cat("Treated risk (%):", round(100 * treated_risk, 2), "\n")
cat("Absolute reduction (percentage points):", round(100 * absolute_reduction, 2), "\n")
cat("Per 1,000 discharges, usual care, programme, avoided:", round(per_thousand), "\n")
cat("Odds at 0.20: usual, calls, calls + one:", base_odds[1], call_odds[1], 1 + call_odds[1], "\n")
cat("Still free at 0.20, usual, hazard:", round(1 - c(baselines[1], treated[3, 1]), 3), "\n")
cat("Treated risk at 0.20 read as risk, odds, hazard ratio:", shown[1:3], "\n")
cat("Avoided per 1,000 at 0.20:", round(1000 * (baselines[1] - treated[, 1])), "\n")
cat("Treated risk at 0.60 read as risk, odds, hazard ratio:", shown[4:6], "\n")
cat("Treated risk with the trial baseline (%):", round(100 * trial_treated, 2), "\n")
cat("Avoided per 1,000, local and trial baseline:", round(avoided), "\n")
Read the other two 0.70s correctly
Suppose Trial A did not exist and the team had only Study B, whose 0.70 multiplies odds, not risk. The hospital's odds of readmission divide the readmitted share by the rest, 0.20 / 0.80, which gives 0.25, and the calls lower them to 0.70 × 0.25 = 0.175. Dividing the odds by one plus the odds turns them back into a risk. Odds of 0.175 mean 0.175 readmitted for every one who is not, so the risk is 0.175 / 1.175, which is 14.89%.
Now suppose the team had only Trial C. Its 0.70 scales the speed of readmission, so it acts on the share of patients still free of readmission, which is 0.80 after three months under usual care. If the calls slow readmission by the same ratio at every moment, the treated share still free is 0.80 raised to the power 0.70, which is 0.855, so the treated risk is 14.46%.
A hazard ratio describes speed, not a total over a fixed period. That is why Trial C's one-year ratio can be applied to the hospital's three-month share, provided the speed ratio is the same at three months as over the whole year.
At the hospital's baseline the three readings sit close together: 14%, 14.89% and 14.46%. The benefit the model gives the calls moves more: 60, 51 or 55 readmissions avoided per 1,000 discharges, depending on which reading is right. In a sicker group with a usual-care risk of 0.60, the same three readings give 42%, 51.22% and 47.34%, more than nine points apart.
So never relabel a measure to make it fit. If only an odds ratio exists, convert it through the local odds, assuming the same odds ratio would hold at this hospital's baseline risk, and say so. If only a hazard ratio exists, convert it through the share still free, and state that the ratio is assumed to hold at every moment (Briggs and colleagues).
Check the three readings in R
# The same reported 0.70 read as each of three effect measures, at baselines 0.20 and 0.60.
# Writes: nothing to disk; the results print in the console.
baselines <- c(0.20, 0.60)
effect <- 0.70
base_odds <- baselines / (1 - baselines)
call_odds <- effect * base_odds
treated <- rbind(risk = baselines * effect, odds = call_odds / (1 + call_odds),
hazard = 1 - (1 - baselines)^effect)
shown <- sprintf("%.2f%%", 100 * treated)
cat("Odds at 0.20: usual, calls, calls + one:", base_odds[1], call_odds[1], 1 + call_odds[1], "\n")
cat("Still free at 0.20, usual, hazard:", round(1 - c(baselines[1], treated[3, 1]), 3), "\n")
cat("Treated risk at 0.20 read as risk, odds, hazard ratio:", shown[1:3], "\n")
cat("Avoided per 1,000 at 0.20:", round(1000 * (baselines[1] - treated[, 1])), "\n")
cat("Treated risk at 0.60 read as risk, odds, hazard ratio:", shown[4:6], "\n")Odds at 0.20: usual, calls, calls + one: 0.25 0.175 1.175
Still free at 0.20, usual, hazard: 0.8 0.855
Treated risk at 0.20 read as risk, odds, hazard ratio: 14.00% 14.89% 14.46%
Avoided per 1,000 at 0.20: 60 51 55
Treated risk at 0.60 read as risk, odds, hazard ratio: 42.00% 51.22% 47.34%Requires R ≥ 4.5.2 · tibble 3.3.1 · dplyr 1.2.1 · ggplot2 4.0.3 · works in R 4.6.1 · Download finding-model-inputs-in-the-literature.R
Full tidy script
# Finding Model Inputs in the Literature: Match the Estimate to the Question
# Follow one hospital's readmission question from three candidate studies to its model input.
# ติดตามคำถามเรื่องการกลับมานอนโรงพยาบาลของโรงพยาบาลหนึ่ง จากการศึกษาสามเรื่องสู่ค่าที่แบบจำลองใช้
# Requires: R >= 4.5.2
# Tested: works in R 4.6.1
# tibble 3.3.1 (stable)
# dplyr 1.2.1 (stable)
# ggplot2 4.0.3 (stable)
# install.packages(c("tibble", "dplyr", "ggplot2"))
# update.packages(oldPkgs = c("tibble", "dplyr", "ggplot2"))
# remotes::install_version("tibble", "3.3.1")
# remotes::install_version("dplyr", "1.2.1")
# remotes::install_version("ggplot2", "4.0.3")
library(tibble) # Write the candidate studies and the readings as explicit tables.
library(dplyr) # Calculate the treated risk for each reading.
library(ggplot2) # Display the readings side by side without saving files.
stopifnot(getRversion() >= "4.5.2")
#%%@note 1. Define the illustrative inputs
# Every input is synthetic and written out once, in the order the article uses it.
# ค่าตั้งต้นทั้งหมดเป็นข้อมูลสมมติ เขียนไว้ครั้งเดียวตามลำดับที่บทความใช้
discharges <- 1000
baseline_risk <- 0.20
risk_ratio <- 0.70
baselines <- c(0.20, 0.60)
effect <- 0.70
trial_control_risk <- 0.25
events <- c(treated = 8, control = 13)
patients <- c(treated = 66, control = 60)
control_survival <- 0.60
hazard_ratio <- 0.70
#%%@note 2. Rework the two foundation examples
# One trial's counts give both ratios; one survival curve turns a rate ratio into risks.
# จำนวนจากการทดลองหนึ่งให้อัตราส่วนทั้งสองแบบ และเส้นการอยู่รอดหนึ่งเส้นแปลงอัตราส่วนเป็นความเสี่ยง
risks <- events / patients
odds <- events / (patients - events)
five_year_risks <- 1 - control_survival^c(1, hazard_ratio)
#%%@note 3. Apply the matching risk ratio to the local baseline
# Treated risk is the baseline times the ratio; the gap is the absolute reduction.
# ความเสี่ยงเมื่อมีการโทรติดตามคือความเสี่ยงพื้นฐานคูณอัตราส่วน ผลต่างคือการลดลงสัมบูรณ์
treated_risk <- baseline_risk * risk_ratio
absolute_reduction <- baseline_risk - treated_risk
per_thousand <- discharges * c(baseline_risk, treated_risk, absolute_reduction)
#%%@note 4. Read the same value through odds and through the share still free
# Each reading changes a different quantity, so each gives a different treated risk.
# แต่ละวิธีอ่านเปลี่ยนปริมาณคนละตัว จึงได้ความเสี่ยงเมื่อมีการโทรติดตามต่างกัน
base_odds <- baselines / (1 - baselines)
call_odds <- effect * base_odds
readings <- tibble::tibble(reading = rep(c("risk", "odds", "hazard"), times = 2),
baseline = rep(baselines, each = 3)) |>
dplyr::mutate(odds = effect * baseline / (1 - baseline),
treated = dplyr::case_when(reading == "risk" ~ baseline * effect,
reading == "odds" ~ odds / (1 + odds),
TRUE ~ 1 - (1 - baseline)^effect))
treated <- matrix(readings$treated, nrow = 3, dimnames = list(c("risk", "odds", "hazard"), NULL))
stopifnot(abs(treated["odds", ] - call_odds / (1 + call_odds)) < 1e-12)
shown <- sprintf("%.2f%%", 100 * treated)
#%%@note 5. Pair the trial's ratio with each baseline
# The ratio comes from the trial; the baseline should come from the hospital's own patients.
# อัตราส่วนมาจากการทดลอง ส่วนความเสี่ยงพื้นฐานควรมาจากผู้ป่วยของโรงพยาบาลเอง
baselines_compared <- c(baseline_risk, trial_control_risk)
avoided <- discharges * baselines_compared * (1 - risk_ratio)
trial_treated <- trial_control_risk * risk_ratio
#%%@note 6. Display the readings
# Plot the treated risk for each reading at both baselines; no graphics file is saved.
# แสดงความเสี่ยงเมื่อมีการโทรติดตามของแต่ละวิธีอ่านที่ความเสี่ยงพื้นฐานทั้งสองค่า ไม่บันทึกไฟล์ภาพ
print(ggplot2::ggplot(readings, ggplot2::aes(x = reading, y = 100 * treated)) +
ggplot2::geom_col() + ggplot2::facet_wrap(~baseline) +
ggplot2::labs(x = "How 0.70 is read", y = "Treated risk (%)"))
#%%@note 7. Numbers the article quotes
# Print exactly the short boxes in their reading order.
# พิมพ์ผลลัพธ์ตรงกับกล่องตัวอย่างตามลำดับในบทความ
cat("Events, then patients, treated and control:", events, patients, "\n")
cat("Risk ratio and odds ratio:", sprintf("%.2f", c(risks[1] / risks[2], odds[1] / odds[2])), "\n")
cat("Five-year risks, control and treated:", sprintf("%.1f%%", 100 * five_year_risks), "\n")
cat("Five-year risk ratio:", sprintf("%.2f", five_year_risks[2] / five_year_risks[1]), "\n")
cat("Treated risk as a share:", treated_risk, "\n")
cat("Treated risk (%):", round(100 * treated_risk, 2), "\n")
cat("Absolute reduction (percentage points):", round(100 * absolute_reduction, 2), "\n")
cat("Per 1,000 discharges, usual care, programme, avoided:", round(per_thousand), "\n")
cat("Odds at 0.20: usual, calls, calls + one:", base_odds[1], call_odds[1], 1 + call_odds[1], "\n")
cat("Still free at 0.20, usual, hazard:", round(1 - c(baselines[1], treated[3, 1]), 3), "\n")
cat("Treated risk at 0.20 read as risk, odds, hazard ratio:", shown[1:3], "\n")
cat("Avoided per 1,000 at 0.20:", round(1000 * (baselines[1] - treated[, 1])), "\n")
cat("Treated risk at 0.60 read as risk, odds, hazard ratio:", shown[4:6], "\n")
cat("Treated risk with the trial baseline (%):", round(100 * trial_treated, 2), "\n")
cat("Avoided per 1,000, local and trial baseline:", round(avoided), "\n")
Pair the trial's ratio with your own baseline
Trial A also reports its own control group: 0.25 of usual-care patients were readmitted within three months. That trial ran in a health system with fewer outpatient clinics than this hospital's. Putting that 0.25 into the model would give a treated risk of 17.5% and 75 readmissions avoided per 1,000 discharges, against 60 with the hospital's own risk.
The trial answers how much the calls change the risk; the hospital's own records answer how high the risk is here. A ratio between two groups often carries over between settings better than a baseline risk does, but that is an assumption to check, not a rule. Ask whether the trial's patients, its usual care and its nurses resemble the hospital's (Briggs and colleagues).
How well an estimate applies to the model's patients and care setting is called its transferability. It is a different question from study quality, which asks whether bias threatens the estimate among the patients the study itself enrolled.
Study B shows the difference: its records may be complete, yet patients who agreed to the calls may have been healthier to begin with. Then part of its 0.70 reflects who chose the calls, a flaw inside the study that no transfer can repair.
Converting a number adds a third doubt. Reading Trial C's 0.70 through the share still free assumes the ratio holds at every moment; if the calls work mainly in the first weeks, the converted 14.46% is off. This doubt about the conversion itself is called transformation uncertainty. Record it apart from sampling uncertainty, the chance variation from studying only some patients, so a reader can tell data from assumption (Briggs and colleagues; Eddy and colleagues).
Check the two baselines in R
# Whose baseline goes with a trial's risk ratio: its own controls or the hospital's patients?
# Writes: nothing to disk; the results print in the console.
discharges <- 1000
risk_ratio <- 0.70
baseline_risk <- 0.20
trial_control_risk <- 0.25
baselines_compared <- c(baseline_risk, trial_control_risk)
avoided <- discharges * baselines_compared * (1 - risk_ratio)
trial_treated <- trial_control_risk * risk_ratio
cat("Treated risk with the trial baseline (%):", round(100 * trial_treated, 2), "\n")
cat("Avoided per 1,000, local and trial baseline:", round(avoided), "\n")Treated risk with the trial baseline (%): 17.5
Avoided per 1,000, local and trial baseline: 60 75Requires R ≥ 4.5.2 · tibble 3.3.1 · dplyr 1.2.1 · ggplot2 4.0.3 · works in R 4.6.1 · Download finding-model-inputs-in-the-literature.R
Full tidy script
# Finding Model Inputs in the Literature: Match the Estimate to the Question
# Follow one hospital's readmission question from three candidate studies to its model input.
# ติดตามคำถามเรื่องการกลับมานอนโรงพยาบาลของโรงพยาบาลหนึ่ง จากการศึกษาสามเรื่องสู่ค่าที่แบบจำลองใช้
# Requires: R >= 4.5.2
# Tested: works in R 4.6.1
# tibble 3.3.1 (stable)
# dplyr 1.2.1 (stable)
# ggplot2 4.0.3 (stable)
# install.packages(c("tibble", "dplyr", "ggplot2"))
# update.packages(oldPkgs = c("tibble", "dplyr", "ggplot2"))
# remotes::install_version("tibble", "3.3.1")
# remotes::install_version("dplyr", "1.2.1")
# remotes::install_version("ggplot2", "4.0.3")
library(tibble) # Write the candidate studies and the readings as explicit tables.
library(dplyr) # Calculate the treated risk for each reading.
library(ggplot2) # Display the readings side by side without saving files.
stopifnot(getRversion() >= "4.5.2")
#%%@note 1. Define the illustrative inputs
# Every input is synthetic and written out once, in the order the article uses it.
# ค่าตั้งต้นทั้งหมดเป็นข้อมูลสมมติ เขียนไว้ครั้งเดียวตามลำดับที่บทความใช้
discharges <- 1000
baseline_risk <- 0.20
risk_ratio <- 0.70
baselines <- c(0.20, 0.60)
effect <- 0.70
trial_control_risk <- 0.25
events <- c(treated = 8, control = 13)
patients <- c(treated = 66, control = 60)
control_survival <- 0.60
hazard_ratio <- 0.70
#%%@note 2. Rework the two foundation examples
# One trial's counts give both ratios; one survival curve turns a rate ratio into risks.
# จำนวนจากการทดลองหนึ่งให้อัตราส่วนทั้งสองแบบ และเส้นการอยู่รอดหนึ่งเส้นแปลงอัตราส่วนเป็นความเสี่ยง
risks <- events / patients
odds <- events / (patients - events)
five_year_risks <- 1 - control_survival^c(1, hazard_ratio)
#%%@note 3. Apply the matching risk ratio to the local baseline
# Treated risk is the baseline times the ratio; the gap is the absolute reduction.
# ความเสี่ยงเมื่อมีการโทรติดตามคือความเสี่ยงพื้นฐานคูณอัตราส่วน ผลต่างคือการลดลงสัมบูรณ์
treated_risk <- baseline_risk * risk_ratio
absolute_reduction <- baseline_risk - treated_risk
per_thousand <- discharges * c(baseline_risk, treated_risk, absolute_reduction)
#%%@note 4. Read the same value through odds and through the share still free
# Each reading changes a different quantity, so each gives a different treated risk.
# แต่ละวิธีอ่านเปลี่ยนปริมาณคนละตัว จึงได้ความเสี่ยงเมื่อมีการโทรติดตามต่างกัน
base_odds <- baselines / (1 - baselines)
call_odds <- effect * base_odds
readings <- tibble::tibble(reading = rep(c("risk", "odds", "hazard"), times = 2),
baseline = rep(baselines, each = 3)) |>
dplyr::mutate(odds = effect * baseline / (1 - baseline),
treated = dplyr::case_when(reading == "risk" ~ baseline * effect,
reading == "odds" ~ odds / (1 + odds),
TRUE ~ 1 - (1 - baseline)^effect))
treated <- matrix(readings$treated, nrow = 3, dimnames = list(c("risk", "odds", "hazard"), NULL))
stopifnot(abs(treated["odds", ] - call_odds / (1 + call_odds)) < 1e-12)
shown <- sprintf("%.2f%%", 100 * treated)
#%%@note 5. Pair the trial's ratio with each baseline
# The ratio comes from the trial; the baseline should come from the hospital's own patients.
# อัตราส่วนมาจากการทดลอง ส่วนความเสี่ยงพื้นฐานควรมาจากผู้ป่วยของโรงพยาบาลเอง
baselines_compared <- c(baseline_risk, trial_control_risk)
avoided <- discharges * baselines_compared * (1 - risk_ratio)
trial_treated <- trial_control_risk * risk_ratio
#%%@note 6. Display the readings
# Plot the treated risk for each reading at both baselines; no graphics file is saved.
# แสดงความเสี่ยงเมื่อมีการโทรติดตามของแต่ละวิธีอ่านที่ความเสี่ยงพื้นฐานทั้งสองค่า ไม่บันทึกไฟล์ภาพ
print(ggplot2::ggplot(readings, ggplot2::aes(x = reading, y = 100 * treated)) +
ggplot2::geom_col() + ggplot2::facet_wrap(~baseline) +
ggplot2::labs(x = "How 0.70 is read", y = "Treated risk (%)"))
#%%@note 7. Numbers the article quotes
# Print exactly the short boxes in their reading order.
# พิมพ์ผลลัพธ์ตรงกับกล่องตัวอย่างตามลำดับในบทความ
cat("Events, then patients, treated and control:", events, patients, "\n")
cat("Risk ratio and odds ratio:", sprintf("%.2f", c(risks[1] / risks[2], odds[1] / odds[2])), "\n")
cat("Five-year risks, control and treated:", sprintf("%.1f%%", 100 * five_year_risks), "\n")
cat("Five-year risk ratio:", sprintf("%.2f", five_year_risks[2] / five_year_risks[1]), "\n")
cat("Treated risk as a share:", treated_risk, "\n")
cat("Treated risk (%):", round(100 * treated_risk, 2), "\n")
cat("Absolute reduction (percentage points):", round(100 * absolute_reduction, 2), "\n")
cat("Per 1,000 discharges, usual care, programme, avoided:", round(per_thousand), "\n")
cat("Odds at 0.20: usual, calls, calls + one:", base_odds[1], call_odds[1], 1 + call_odds[1], "\n")
cat("Still free at 0.20, usual, hazard:", round(1 - c(baselines[1], treated[3, 1]), 3), "\n")
cat("Treated risk at 0.20 read as risk, odds, hazard ratio:", shown[1:3], "\n")
cat("Avoided per 1,000 at 0.20:", round(1000 * (baselines[1] - treated[, 1])), "\n")
cat("Treated risk at 0.60 read as risk, odds, hazard ratio:", shown[4:6], "\n")
cat("Treated risk with the trial baseline (%):", round(100 * trial_treated, 2), "\n")
cat("Avoided per 1,000, local and trial baseline:", round(avoided), "\n")
Close the evidence trail
Before the number enters the model, record the choice in the parameter table. Write down the source, the table or page where 0.70 was found and why Trial A was chosen. Add why Study B and Trial C were kept as checks rather than inputs, and any conversion you made.
Note what is missing too: in this synthetic example Trial A's 0.70 comes with no interval, so the model cannot yet say how uncertain the benefit is. If no study answers the question, record that gap and what it means for the decision (Eddy and colleagues). Do not invent an interval, a citation or a plausible-looking number to fill the table. A source catalogue helps you find candidate studies; it does not make their numbers interchangeable.
To practise on a real paper, read the economic evaluation of endovascular treatment for acute ischaemic stroke in Thailand by Rattanavipapong and colleagues. For each input, find its source and compare its patients, comparison, outcome and time with the model's question. This article does not reuse its numbers.
Terms to keep distinct
- Parameter question (คำถามของพารามิเตอร์)
- The exact quantity one model input must estimate: whose patients, which comparison, which outcome, over what time.
- Baseline risk (ความเสี่ยงพื้นฐาน)
- The risk of the event under usual care in the model's own patients; 0.20 here.
- Risk ratio (อัตราส่วนความเสี่ยง)
- Treated risk divided by untreated risk over the same period; it multiplies a risk directly.
- Odds ratio (อัตราส่วนออดส์)
- A ratio of odds, where odds are the events divided by the non-events; it multiplies odds, not risk.
- Hazard ratio (อัตราส่วนฮาซาร์ด)
- A ratio of how fast events happen among people still event-free; it acts on the share still free.
- Transferability (การนำไปใช้ข้ามบริบท)
- How well an estimate applies to the model's patients and care setting.
- Transformation uncertainty (ความไม่แน่นอนจากการแปลงค่า)
- Doubt that comes from the assumptions needed to convert an estimate, kept apart from sampling uncertainty.
Connect to the foundation articles
The foundation article on parameter sourcing and source catalogues shows where candidate numbers come from when local data are sparse. The one on risk ratios, odds ratios and risk differences works through the counts behind the odds-ratio example. The next part of this series keeps time inside a calculation, which a one-year result like Trial C's needs.
Try it in the app
Open our economic evaluation app to change the inputs and watch the result update. It needs a login and a Pro or Max plan.
The app starts from its own example values, so its numbers will not match this article.
References
- Briggs Andrew H., Weinstein Milton C., Fenwick Elisabeth A.L., Karnon Jonathan, Sculpher Mark J., Paltiel A. David. Model Parameter Estimation and Uncertainty: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6. Value in Health. 2012;15(6):835-842. doi:10.1016/j.jval.2012.04.014. https://doi.org/10.1016/j.jval.2012.04.014
- Rattanavipapong Waranya, Worakijthamrongchai Thanaboon, Soboon Budsadee, Luankongsomchit Vilawan, Kongmuangpuk Mananchaya, Isaranuwatchai Wanrudee, Teerawattananon Yot, Nilanont Yongchai. Economic evaluation of endovascular treatment for acute ischaemic stroke in Thailand. BMJ Open. 2022;12(9):e064403. doi:10.1136/bmjopen-2022-064403. https://doi.org/10.1136/bmjopen-2022-064403
- Eddy David M., Hollingworth William, Caro J. Jaime, Tsevat Joel, McDonald Kathryn M., Wong John B.. Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value in Health. 2012;15(6):843-850. doi:10.1016/j.jval.2012.04.012. https://doi.org/10.1016/j.jval.2012.04.012
The series, in learning order
- Health Economic Modelling Series Guide: From Evidence to a Decision
- Choosing a Decision-Analytic Model: Let the Clinical Pathway Decide
- Auditable Model Inputs: Build a Parameter Table You Can Trace
- Finding Model Inputs in the Literature: Match the Estimate to the Question (this article)
- Rates, Probabilities and Transition Probabilities: Keep Time in the Calculation
- Building a Decision Tree Step by Step: Follow Outcomes, Costs and Health
- Building a Markov Cohort Model Step by Step: Follow the Cohort Through Time
- Budget Impact Analysis in Practice: Count People Before Counting Money
- ICER and Net Monetary Benefit Across Strategies: Find the Relevant Frontier
- Probabilistic Sensitivity Analysis: From Parameter Distributions to CEAC
- Model Validation and Verification: Test the Question, Equations and Code
- Expected Value of Perfect Information: What Could Better Evidence Be Worth?
Key takeaways
- Write the parameter question first: whose patients, which comparison, which outcome and over what time, as a measure the model can use.
- Read what each 0.70 measures. As a risk ratio it multiplies the local 0.20 to give 0.14; read as an odds ratio or a hazard ratio it gives 14.89% or 14.46% once converted.
- Take the ratio from the best study and the baseline from your own patients: 0.20, not the trial's 0.25, so 60 readmissions avoided per 1,000 discharges, not 75.
- Judge study quality, transferability and any conversion separately, and record each choice so another analyst can follow it.
Related in the wiki: Parameter Sourcing, Reference Cost Lists and Utility Catalogues When Local Data Are Sparse · Risk Ratio, Odds Ratio, Risk Difference and NNT: One 2×2 Table, Four Answers