What Is Integrated Discrimination Improvement (IDI)? A Clear Guide with Example

🧪 What Is IDI?

While Net Reclassification Improvement (NRI) focuses on category shifts, IDI measures how much the new model improves average predicted probabilities for cases and non-cases across all thresholds—without relying on arbitrary cutoffs.

Key Idea:

IDI tells you how much better the new model separates diseased from non-diseased individuals.

📐 The IDI Formula

Unpacking the IDI Formula

$IDI = [\overline{P} new | D = 1 - \overline{P} old | D = 1] - [\overline{P} new | D = 0 - \overline{P} old | D = 0]$

Components Explained

$\overline{P} new | D = 1$ : Average predicted probability in cases (D=1) using the new model
$\overline{P} old | D = 1$ : Average predicted probability in cases using the old model
🔼 Should go up: Better discrimination among diseased

$\overline{P} new | D = 0$ : Average predicted probability in non-cases (D=0) using the new model
$\overline{P} old | D = 0$ : Average predicted probability in non-cases using the old model
🔽 Should go down: Better discrimination among non-diseased

🧪 Conceptual Analogy

Think of a scatter plot of predicted probabilities:

Good models spread cases (D=1) toward high predicted probabilities
And spread non-cases (D=0) toward low predicted probabilities

So:

IDI = "How much farther apart are the two clouds (D=1 vs D=0) in the new model compared to the old one?"

🔢 An Example in Numbers

You are comparing two diagnostic models for early liver fibrosis:

Model A: uses age, AST/ALT ratio
Model A+B: adds a novel serum fibrosis biomarker

Suppose we calculate:

	Mean Predicted Probability (Cases, D=1)	Mean Predicted Probability (Non-Cases, D=0)
Model A	0.42	0.21
Model A+B	0.58	0.18

🧮 Step-by-Step IDI Calculation

Step 1: Calculate improvement in cases

$Δ P cases = \overline{P} new | D = 1 - \overline{P} old | D = 1 = 0.58 - 0.42 = 0.16$

Step 2: Calculate improvement in non-cases

$Δ P non-cases = \overline{P} new | D = 0 - \overline{P} old | D = 0 = 0.18 - 0.21 = - 0.03$

(This is good — predicted risk in non-cases decreased.)

Step 3: Calculate IDI

$IDI = Δ P cases - Δ P non-cases = 0.16 - (- 0.03) = 0.19$

✔️ Interpretation: The new model increases the average separation between cases and non-cases by 0.19 — a substantial improvement in discrimination.

✅ Summary Table: Components of IDI

Step	What it means	Goal
$\overline{P} new D^{1} - \overline{P} old D^{1}$	Does the new model assign higher probabilities to true cases?	🔼 Increase
$\overline{P} new D^{0} - \overline{P} old D^{0}$	Does the new model assign lower probabilities to true non-cases?	🔽 Decrease
Subtract the two	The net gain in separation	→ IDI = ΔSeparation

💡 When to Use IDI?

Use IDI when:

You're comparing models, not just tests
You want to avoid arbitrary cutoff thresholds
You need a continuous, overall measure of improvement
You want a complement to AUC, NRI, and calibration metrics