ROC Analysis and Diagnostic Test Accuracy: From Discrimination to Cut-Point Selection in Stata (roctab & diagt)

Clinical Epidemiology ResearchUniqcret doctor knowledgesMethodology and Research DesignDiagnosis [Methodology]Data Analytics or StatisticsStata [Data Analytics]
On this page

Introduction

(When the outcome is disease status and the goal is test performance)

In diagnostic research, we are interested in how well a test distinguishes between patients with and without disease.

We are not only interested in whether a test is associated with disease, but how accurately it classifies patients, and whether this accuracy depends on the chosen cut-point.


1. ROC Analysis (Discrimination & Cut-point Exploration)

(Describe performance across all thresholds — no fixed cut-point yet)

Step 1: Define reference and index tests

Before any analysis, we must identify:

  • Reference standard (true disease status)
  • Index test (diagnostic test under evaluation)
tab patho
tab fna

📌 Requirements:

  • Reference test must be binary (0/1)
  • Index test must be ordinal or continuous
  • Higher values = higher disease risk

Step 2: ROC analysis (roctab)

roctab patho fna

What this does:

  • Evaluates overall discrimination
  • Computes the Area Under the ROC Curve (AUC)

Interpretation of AUC:

AUC valueInterpretation
0.5No discrimination
0.7–0.8Acceptable
0.8–0.9Good
>0.9Excellent

📌 Conceptually:

“How well can the test separate diseased from non-diseased patients?”


Step 3: Explore sensitivity & specificity at each cut-point

roctab patho fna, detail

What this does:

  • Treats each possible threshold as a binary test
  • Shows, for each cut-point:
    • Sensitivity
    • Specificity
    • Likelihood ratios

📌 Conceptually:

“What happens to sensitivity and specificity if we change the cut-point?”

✅ This is the key step for choosing a cut-point


Step 4: ROC curve (visual assessment)

roctab patho fna, graph summary

Shows:

  • ROC curve (sensitivity vs 1 − specificity)
  • Reference line (no discrimination)

📌 Important:

  • Curve is data-driven
  • No assumptions
  • No covariate adjustment

Summary: ROC analysis

MethodPurpose
roctabOverall discrimination (AUC)
roctab, detailSensitivity & specificity at each cut-point
ROC curveVisual trade-off between sensitivity & specificity

2. Dichotomisation of the Index Test

(Fix the cut-point)

After inspecting ROC results and considering clinical relevance, the index test is converted into a binary test.

Example:High-risk FNA defined as C3–C5.

gen fna345 = fna
recode fna345 1/2=0 3/5=1

📌 Conceptually:

“We now fix a single clinically meaningful threshold.”

⚠ This step must come after ROC analysis.


3. Diagnostic Indices from 2×2 Tables (diagt)

(Final accuracy estimates) Step 5: Diagnostic accuracy estimation

diagt patho fna345

What this reports:

  • Sensitivity
  • Specificity
  • Positive predictive value (PPV)
  • Negative predictive value (NPV)
  • Likelihood ratios (LR+ and LR−)
  • Diagnostic odds ratio (DOR)
  • 95% confidence intervals

📌 Conceptually:

“How accurate is the test at this chosen cut-point?”


Step 6: Predictive values and prevalence

By default:

  • PPV and NPV assume cohort-like sampling

If prevalence must be fixed:

diagt patho fna345, prev(10%)

📌 Important:

  • Sensitivity & specificity are prevalence-independent
  • PPV & NPV depend on prevalence

4. Subgroup (Heterogeneity) Analysis

(Does test performance change across groups?)

diagt patho fna345 if age40==0
diagt patho fna345 if age40==1

diagt patho fna345 if size2cm==0
diagt patho fna345 if size2cm==1

Interpretation:

If sensitivity or specificity differs meaningfully across strata, diagnostic performance is heterogeneous (effect modification).


Big Picture Summary (Very Important)

LevelMethodWhat it does
DiscriminationroctabHow well the test separates disease vs no disease
Cut-point explorationroctab, detailSens/spec at each threshold
Fixed accuracydiagtFinal diagnostic indices at chosen cut-point
Subgroup analysisdiagt ifHeterogeneity of performance


Key takeaway

ROC analysis (roctab) evaluates discrimination and guides cut-point selection, whereas diagt estimates final diagnostic accuracy once a clinically relevant threshold is fixed.

0
Message for International and Thai ReadersUnderstanding My Medical Context in ThailandRead more →Message for International and Thai ReadersUnderstanding My Broader Content Beyond MedicineRead more →

Comments

No comments yet. Be the first to share your thoughts.

Sign in to comment