ROC Analysis and Diagnostic Test Accuracy: From Discrimination to Cut-Point Selection in Stata (roctab & diagt)
- Mayta
- 10 hours ago
- 3 min read
Introduction
(When the outcome is disease status and the goal is test performance)
In diagnostic research, we are interested in how well a test distinguishes between patients with and without disease.
We are not only interested in whether a test is associated with disease, but how accurately it classifies patients, and whether this accuracy depends on the chosen cut-point.
1. ROC Analysis (Discrimination & Cut-point Exploration)
(Describe performance across all thresholds — no fixed cut-point yet)
Step 1: Define reference and index tests
Before any analysis, we must identify:
Reference standard (true disease status)
Index test (diagnostic test under evaluation)
tab patho
tab fna
📌 Requirements:
Reference test must be binary (0/1)
Index test must be ordinal or continuous
Higher values = higher disease risk
Step 2: ROC analysis (roctab)
roctab patho fna
What this does:
Evaluates overall discrimination
Computes the Area Under the ROC Curve (AUC)
Interpretation of AUC:
AUC value | Interpretation |
0.5 | No discrimination |
0.7–0.8 | Acceptable |
0.8–0.9 | Good |
>0.9 | Excellent |
📌 Conceptually:
“How well can the test separate diseased from non-diseased patients?”
Step 3: Explore sensitivity & specificity at each cut-point
roctab patho fna, detail
What this does:
Treats each possible threshold as a binary test
Shows, for each cut-point:
Sensitivity
Specificity
Likelihood ratios
📌 Conceptually:
“What happens to sensitivity and specificity if we change the cut-point?”
✅ This is the key step for choosing a cut-point
Step 4: ROC curve (visual assessment)
roctab patho fna, graph summary
Shows:
ROC curve (sensitivity vs 1 − specificity)
Reference line (no discrimination)
📌 Important:
Curve is data-driven
No assumptions
No covariate adjustment
Summary: ROC analysis
Method | Purpose |
roctab | Overall discrimination (AUC) |
roctab, detail | Sensitivity & specificity at each cut-point |
ROC curve | Visual trade-off between sensitivity & specificity |
2. Dichotomisation of the Index Test
(Fix the cut-point)
After inspecting ROC results and considering clinical relevance, the index test is converted into a binary test.
Example:High-risk FNA defined as C3–C5.
gen fna345 = fna
recode fna345 1/2=0 3/5=1
📌 Conceptually:
“We now fix a single clinically meaningful threshold.”
⚠ This step must come after ROC analysis.
3. Diagnostic Indices from 2×2 Tables (diagt)
(Final accuracy estimates) Step 5: Diagnostic accuracy estimation
diagt patho fna345
What this reports:
Sensitivity
Specificity
Positive predictive value (PPV)
Negative predictive value (NPV)
Likelihood ratios (LR+ and LR−)
Diagnostic odds ratio (DOR)
95% confidence intervals
📌 Conceptually:
“How accurate is the test at this chosen cut-point?”
Step 6: Predictive values and prevalence
By default:
PPV and NPV assume cohort-like sampling
If prevalence must be fixed:
diagt patho fna345, prev(10%)
📌 Important:
Sensitivity & specificity are prevalence-independent
PPV & NPV depend on prevalence
4. Subgroup (Heterogeneity) Analysis
(Does test performance change across groups?)
diagt patho fna345 if age40==0
diagt patho fna345 if age40==1
diagt patho fna345 if size2cm==0
diagt patho fna345 if size2cm==1
Interpretation:
If sensitivity or specificity differs meaningfully across strata, diagnostic performance is heterogeneous (effect modification).
Big Picture Summary (Very Important)
Level | Method | What it does |
Discrimination | roctab | How well the test separates disease vs no disease |
Cut-point exploration | roctab, detail | Sens/spec at each threshold |
Fixed accuracy | diagt | Final diagnostic indices at chosen cut-point |
Subgroup analysis | diagt if | Heterogeneity of performance |
Key takeaway
ROC analysis (roctab) evaluates discrimination and guides cut-point selection, whereas diagt estimates final diagnostic accuracy once a clinically relevant threshold is fixed.