AIC Akaike Information Criterion and BIC Bayesian Information Criterion in Logistic Regression

Your output:

Model |     N    ll(null)   ll(model)   df     AIC      BIC
--------------------------------------------------------------
.     |   3135   -1906.079   -1807.527   2   3619.054  3631.155

1. What Are AIC and BIC?

Both are information criteria used to compare models.

They answer:

Which model balances goodness-of-fit AND parsimony best?

They penalize complexity. AIC stands for:

Akaike Information Criterion

Named after: Hirotugu Akaike (1974)

BIC stands for:

Bayesian Information Criterion

Also called: Schwarz Criterion (Schwarz, 1978)

Quick Clinical Modeling Meaning

Term	Full Name	Purpose
AIC	Akaike Information Criterion	Model selection for prediction
BIC	Bayesian Information Criterion	Model selection with stronger penalty for complexity

In one sentence:

• AIC = goodness-of-fit + mild penalty

• BIC = goodness-of-fit + strong penalty

Lower value = better model (when comparing models on same data).

---

2. Mathematical Definition

Let:

• LL = log-likelihood

• k = number of parameters

• N = sample size

AIC

AIC=−2LL+2k

BIC

BIC=−2LL+kln(N)

Notice:

• AIC penalty = 2k

• BIC penalty = k ln(N)

With N = 3135:

ln(3135)≈8.05

So BIC penalizes ~4× stronger than AIC here.

---

3. What Do They Mean Clinically?

They are not hypothesis tests.

They are:

Relative model quality measures.

Lower value = better model.

You NEVER interpret AIC alone.You compare between models.

---

4. What Does Your Output Mean?

From your model:

ll(model) = -1807.527
df = 2
AIC = 3619.054
BIC = 3631.155

This is:

• Logistic model

• 2 parameters (intercept + Age)

Now imagine you compare:

Model	AIC
Linear Age	3619
Quadratic Age	3612
MFP Age	3608

Best model = lowest AIC

---

5. When Should You Use AIC vs BIC?

🔹 AIC

Use when:

• Goal = prediction

• Want best approximating model

• Clinical prediction model development

🔹 BIC

Use when:

• Goal = explanatory modeling

• Prefer simpler model

• Large sample size

• Theory-driven

---

6. Important Warning From Stata Manual

Your help file warns:

AIC/BIC not comparable if likelihoods are based on different events.

Example:

• stcox vs streg → not comparable

• Different datasets → not comparable

• Different outcome types → not comparable

But:

✅ Logistic vs logistic ✅ Linear vs quadratic logistic ✅ Logistic vs MFP logistic

→ perfectly valid comparison.

---

7. How You Should Use It in Your Age Modeling

Here is the proper workflow:

* Linear
logistic ED_LOS_ge4h c.Age
est store lin

* Quadratic
logistic ED_LOS_ge4h c.Age##c.Age
est store quad

* MFP
mfp: logistic ED_LOS_ge4h Age
est store mfp

* Compare all
estat ic

Or better:

estimates stats lin quad mfp

Then compare:

• AIC

• BIC

• Log-likelihood

---

8. Clinical Interpretation Example

Suppose:

Model	AIC	BIC
Linear	3619	3631
Quadratic	3613	3629
MFP	3610	3635

Interpretation:

• AIC prefers MFP

• BIC prefers Quadratic

• Quadratic more parsimonious

• MFP slightly better predictive fit

Decision depends on:

• Is this predictive model? → choose MFP

• Is this explanatory modeling? → choose quadratic

---

9. Key Concept: Why Not Use p-values Alone?

Because:

• Quadratic term may be non-significant

• But model still improves global fit

• p-value tests coefficient

• AIC evaluates whole model

This is why modeling uses AIC/BIC

---

10. Deep Insight (Very Important)

AIC does NOT tell you if model is correct.

It tells you:

Which model loses the least information relative to truth.

That is very different.

---

Summary

In Example ED LOS ≥ 4h model:

• Use LR test for nested comparison

• Use AIC/BIC for overall comparison

• Prefer AIC in prediction modeling

• Prefer BIC in explanatory modeling

• Always compare models fit on SAME dataset