The Calibration Slope: Overfitting, Extremeness & Shrinkage
On this page
Abstract
A prediction model can get the overall risk level exactly right and still be miscalibrated — if it is over-fit, its predictions are too extreme: the high risks are too high and the low risks too low. That pattern is measured by the calibration slope $B$, the slope of the observed outcome on the model's linear predictor. The ideal is $B=1$; a slope below 1 is the formal fingerprint of overfitting, and on a calibration plot it appears as a curve that is flatter than the 45° line and crosses it near the middle. This article isolates the slope from the level, estimates $B$ (here ≈ 0.57), and shows how shrinkage — $\text{logit}(p)=A+B\cdot\text{LP}$, which multiplies every coefficient by $B$ — rotates the calibration curve back onto the diagonal without changing how patients are ranked. An interactive slope dial and a line-by-line R/Python terminal let you watch it happen. This is Part 2 of a three-part series on calibration and recalibration.
Part 2 of 3 · Calibration & Recalibration. Before: Part 1 — calibration in the large [[calibration-in-the-large]]. Next: Part 3 — recalibration in practice [[recalibration-in-practice]].
Right on average, wrong in the spread
Part 1 fixed the overall level of risk with the intercept. But a model can pass the calibration-in-the-large check — mean predicted risk close to the observed rate — and still be wrong, because the level says nothing about the spread of predictions. An over-fit model has learned noise as if it were signal; its coefficients are too large, so it speaks with too much confidence. It pushes low-risk patients too low and high-risk patients too high. The average can still land in the right place while the extremes are systematically exaggerated.
A frozen model whose level is fine
In our simulated cohort of 3,000 patients the observed event rate is ~31% and the frozen model predicts an average of ~34% — a calibration-in-the-large ratio near 1.1, essentially on target. The level is not the problem here. The problem is that the model's effect for the predictor is too strong (a slope of ~1.6 where the truth is 1.0), so its predicted risks are spread too widely.
The picture: a curve that crosses the diagonal
Plot observed risk against predicted risk by tenths of predicted risk. An over-confident model produces a calibration curve that is flatter than the 45° line and crosses it: at the low end the observed rate is higher than predicted (the model under-called low risks), and at the high end the observed rate is lower than predicted (it over-called high risks). The predictions fan out further than reality warrants.
The metric: the calibration slope $B$
We quantify this with the calibration slope. Regress the observed outcome on the model's linear predictor and read off the slope:
$$\text{logit}\{P(Y=1)\}=A+B\cdot\text{LP},\qquad \text{LP}=\hat\beta_0+\hat\beta_1x_1+\cdots+\hat\beta_kx_k.$$
The ideal is $B=1$. Interpret it as:
- $B<1$ — predictions are too extreme (overfitting); high-risk predictions too high, low-risk too low.
- $B>1$ — predictions are not extreme enough.
In the simulation $B\approx0.57$ — well below 1, confirming the over-extreme spread. A slope of, say, $B=0.80$ would imply the predictor effects need roughly 20% shrinkage.
Turn the dial below to see it directly. At $B=1.6$ the model is over-confident; press Fit B to snap to the fitted slope that lands the curve on the diagonal, and Reset to return. The mean predicted risk is held on the observed rate throughout, so you are watching the slope alone.
The fix: shrinkage
Recalibrating the slope applies
$$\text{logit}(p_{\text{updated}})=A+B\cdot\text{LP},$$
which pulls every prediction toward the mean and rotates the calibration curve onto the diagonal. Because $B$ multiplies the whole linear predictor, it multiplies every original coefficient by the same factor:
$$\beta_{j,\text{shrunk}}=B\cdot\beta_j.$$
In the demonstration the predictor's coefficient shrinks from ~1.6 toward ~0.92, and the recalibrated model's own calibration slope becomes 1.00. Crucially, multiplying all predictions by a constant on the log-odds scale never re-orders patients — discrimination is unchanged. Shrinkage makes the model more modest, not better at ranking.
The catch — why you rarely touch the slope alone
There is a subtlety that Part 3 develops. Changing $B$ also changes the mean predicted risk, so a slope-only update generally disturbs calibration in the large — the very thing Part 1 fixed. Fixing $A=0$ while changing $B$ assumes the level stays correct after re-scaling, which is usually unjustified. So in practice, when the slope needs correcting, you estimate the intercept and slope together, $\text{logit}(p)=A+B\cdot\text{LP}$ — exactly the joint update covered in Part 3.
Run it yourself — R and Python
The terminal below runs the real analysis line by line on synthetic data: apply a frozen, over-confident model whose level is fine, confirm calibration in the large, estimate the calibration slope $B$, and watch the calibration plot rotate onto the diagonal after shrinkage. Type the highlighted line (or press Enter), Tab accepts the ghost, and switch the R / Python tabs. Nothing runs in your browser; every value and plot was computed beforehand.
โมเดลทำนายที่ให้ระดับความเสี่ยง โดยรวม ถูกต้อง ก็ยังพลาดได้ถ้ามันถูก over-fit จนทำนาย "สุดโต่งเกินไป" — สูงเกินไปสำหรับคนเสี่ยงสูง และต่ำเกินไปสำหรับคนเสี่ยงต่ำ เทอร์มินัลนี้รันการวิเคราะห์จริงทีละบรรทัดบนข้อมูล จำลอง 3,000 คน โมเดลเดิมที่ถูก "แช่แข็ง" มี intercept ที่ถูกต้อง (calibration in the large เกือบพอดี) แต่ slope ชันเกินไป เราวัดด้วย calibration slope (B) ได้ค่า ~0.57 (ต่ำกว่า 1 = ทำนายสุดโต่งเกิน) แล้วใช้ shrinkage คูณ linear predictor ด้วย B จนเส้น calibration ที่แบนเกินหมุนกลับมาแนบเส้นทแยง คุณเป็นคนพิมพ์เอง กด Enter เพื่อเติมและรัน กด Tab เพื่อรับคำที่ระบบเดา แผง Stage ด้านขวาจะโชว์ค่าและ กราฟ calibration ที่เปลี่ยนไป สลับแท็บ R / Python ได้ ไม่มีการคำนวณในเบราว์เซอร์ ทุกค่าและกราฟถูกคำนวณไว้ก่อนด้วย R และ Python
A prediction model can get the overall risk level right and still fail if it was over-fit — making predictions that are too extreme: too high for high-risk patients, too low for low-risk ones. This terminal runs the real analysis one line at a time on a simulated cohort of 3,000 patients. A frozen original model has the right intercept (calibration-in-the-large is about right) but too steep a slope. We measure it with the calibration slope (B) — here ~0.57, well below 1, the formal sign of too-extreme predictions — then apply shrinkage, multiplying the linear predictor by B until the too-flat calibration curve rotates back onto the 45° line. You drive it: press Enter to autofill and run, Tab to accept the autocomplete ghost, and the Stage on the right shows the numbers and the calibration plot changing. Switch between the R and Python tabs for the same analysis in either language. Nothing runs in your browser; every value and plot was computed in R and Python beforehand.
Key takeaways
- The calibration slope $B$ measures whether predictions are too extreme ($B<1$, overfitting) or too flat ($B>1$); ideal $B=1$.
- A model can be calibrated in the large yet have $B<1$ — the level is right but the spread is exaggerated; on the plot the curve crosses the diagonal.
- Shrinkage $\text{logit}(p)=A+B\cdot\text{LP}$ multiplies every coefficient by $B$, rotating the curve onto the diagonal without changing patient ranking.
- Slope-only recalibration is rarely justified because it disturbs the level — estimate $A$ and $B$ together (Part 3).
Back to Part 1 — calibration in the large [[calibration-in-the-large]], or on to Part 3 — recalibration in practice [[recalibration-in-practice]] for the joint update with honest validation.