Recalibration in Practice: Intercept, Slope & Honest Validation
On this page
Abstract
A model transported to a new setting can arrive wrong in two ways at once: the overall risk level is off (a calibration intercept problem) and the predictions are too extreme or too flat (a calibration slope problem). The general fix estimates both together, $\text{logit}(p)=A+B\cdot\text{LP}$, which folds cleanly back into updated coefficients: the slope multiplies every original coefficient and the intercept absorbs the leftover level. This closing article of the series shows the joint update, works a single patient from 81.8% down to 65.7%, and then asks the harder question — where does an honest slope come from? Re-using the development data gives an apparent slope of exactly 1, a mirage; a bootstrap reveals the optimism and a global shrinkage factor (here ≈ 0.76). We separate validation (apply the frozen model, report performance) from recalibration (update it), and give the clean development → internal-validation → external-validation → update sequence. Interactive A/B workbench and a line-by-line R/Python terminal included.
Part 3 of 3 · Calibration & Recalibration. Back to Part 1 — calibration in the large [[calibration-in-the-large]] and Part 2 — the calibration slope [[calibration-slope]].
When a model arrives wrong in two ways
Parts 1 and 2 treated one fault at a time — a wrong level (the intercept) or a wrong steepness (the slope). Transport a model to a genuinely different population and you often get both. On the calibration plot the curve both sits below the diagonal (over-prediction) and is too flat (predictions too extreme). In our simulated cohort of 3,000 patients the frozen model predicts an average risk of ~31% where only ~13% had the event, and its calibration slope is well below 1.
Update the intercept and slope together
The general recalibration fits both parameters at once by regressing the observed outcome on the original linear predictor:
$$\text{logit}\{P(Y=1)\}=A+B\cdot\text{LP}_{\text{original}}.$$
Here $A$ re-levels (as in Part 1) and $B$ re-scales the spread of predictions (as in Part 2). In the simulation the joint fit returns $A\approx-1.48$ and $B\approx0.63$; applying $\text{LP}_{\text{updated}}=A+B\cdot\text{LP}_{\text{original}}$ snaps the calibration curve onto the 45° line and brings the mean predicted risk back to ~13%, matching the observed rate.
Estimating the slope alone is possible but usually unjustified: changing $B$ also shifts the mean predicted risk, so it disturbs calibration in the large. When the slope needs correcting, estimate $A$ and $B$ jointly.
The slope multiplies every coefficient
Recalibration is not a black box bolted onto the output — it re-writes the model's coefficients. Substitute the linear predictor $\text{LP}=\beta_0+\beta_1x_1+\cdots+\beta_kx_k$ into $A+B\cdot\text{LP}$ and expand:
$$\text{LP}_{\text{updated}}=(A+B\beta_0)+(B\beta_1)x_1+\cdots+(B\beta_k)x_k.$$
So the updated intercept and each updated coefficient are
$$\beta_{0,\text{updated}}=A+B\beta_0,\qquad \beta_{j,\text{updated}}=B\beta_j.$$
The calibration slope $B$ multiplies every predictor coefficient by the same factor — that is exactly what "shrinkage" means — while the intercept mops up the overall level.
One patient, worked end to end
Take a patient with $\text{LP}_{\text{original}}=1.5$. The original model predicts
$$p_{\text{original}}=\frac{1}{1+e^{-1.5}}=0.818\;(81.8\%).$$
Suppose external updating gives $A=-0.40$ and $B=0.70$. Then
$$\text{LP}_{\text{updated}}=-0.40+0.70(1.5)=0.65,\qquad p_{\text{updated}}=\frac{1}{1+e^{-0.65}}=0.657\;(65.7\%).$$
The negative intercept lowers the overall level and the slope below 1 pulls this extreme prediction toward the middle — 81.8% becomes 65.7%. Try other linear predictors in the workbench below: Reset is the frozen model, Fit A,B lands the curve on the diagonal, and the worked-example readout updates live.
Where an honest slope comes from
You cannot read the correction off the development data. Apply a model to the same patients it was fitted on and the apparent calibration slope is ~1 and the apparent intercept ~0 by construction — an optimistic mirage. An honest optimism-corrected slope comes from internal validation by resampling. A bootstrap does this: repeatedly refit the whole modelling procedure on a resample, measure how much better it looks in that resample than in the original data (the optimism), average it, and correct. In the demonstration the apparent slope is 1.00 but the optimism-corrected slope — a global shrinkage factor — is about 0.76, i.e. the fitted coefficients should be shrunk by ~24%:
$$\beta_{j,\text{shrunk}}=0.76\,\beta_j,$$
with the intercept re-estimated afterwards so the mean predicted risk still matches the development event rate. K-fold cross-validation is an alternative; for regression models bootstrap is usually preferred for optimism and shrinkage.
Validation is not recalibration
Validation asks how the frozen model performs, unchanged — report discrimination, a calibration plot, the calibration intercept and slope, overall error, and clinical utility. Recalibration asks how to change the model for this population. The order matters, and one dataset cannot play both roles: once you use a dataset to estimate $A$ and $B$, it becomes a model-updating dataset, not an independent validation set for the updated model. The clean sequence is:
- Develop the original model.
- Internally validate by bootstrap or cross-validation (optimism-corrected slope / shrinkage).
- Apply the frozen model to new external data and report performance before any updating.
- Estimate $A$ and $B$ only if recalibration is needed.
- Validate the updated model again — ideally in yet another independent sample.
Run it yourself — R and Python
The terminal runs the whole arc on synthetic data: a frozen model wrong in both level and slope, the joint fit of $A$ and $B$, the calibration plot snapping onto the diagonal, the worked 81.8% → 65.7% example, and a bootstrap that exposes the shrinkage the development data hides. Type the highlighted line (or press Enter), Tab accepts the ghost, and switch R / Python tabs. Nothing runs in your browser.
เมื่อโมเดลทำนายถูกย้ายไปใช้กับประชากรใหม่ มันอาจผิดได้ สองอย่างพร้อมกัน — ระดับความเสี่ยงโดยรวมสูงเกิน (intercept) และการทำนายสุดโต่งเกินไป (slope ชันเกิน) เทอร์มินัลนี้รันการวิเคราะห์จริงทีละบรรทัดบนข้อมูล จำลอง 3,000 คน โมเดลเดิมที่ถูก "แช่แข็ง" ทำนายเฉลี่ย ~31% ทั้งที่เกิดจริงเพียง ~13% เราทำ full recalibration โดย fit A และ B พร้อมกัน — logit(p) = A + B·LP_original — จนเส้น calibration เด้งมาแนบเส้นทแยงทั้งระดับและความชัน ค่านี้พับกลับเป็นสัมประสิทธิ์ใหม่ (β0,updated = A + B·β0 และทุก βj,updated = B·βj) ต่อด้วยตัวอย่างเลขจริง (LP = 1.5 → 81.8% กลายเป็น 65.7%) แล้วดูว่า slope ที่ "ซื่อสัตย์" ต้องมาจาก bootstrap (internal validation) ไม่ใช่การ refit กับข้อมูลฝึกของตัวเอง และปิดท้ายด้วยหลักการ validation ≠ recalibration คุณเป็นคนพิมพ์เอง กด Enter เพื่อเติมและรัน กด Tab เพื่อรับคำที่ระบบเดา แผง Stage ด้านขวาจะโชว์ค่าและ กราฟ calibration ที่เปลี่ยนไป สลับแท็บ R / Python ได้ ไม่มีการคำนวณในเบราว์เซอร์ ทุกค่าและกราฟถูกคำนวณไว้ก่อนด้วย R และ Python
When a prediction model is moved to a new population it can be wrong in two ways at once — the overall risk level is too high (intercept) AND the predictions are too extreme (the slope is too steep). This terminal runs the real analysis one line at a time on a simulated cohort of 3,000 patients. A frozen original model predicts an average risk of ~31% when only ~13% actually had the event. We apply full recalibration, fitting A and B together — logit(p) = A + B·LP_original — until the calibration curve snaps onto the 45° line in both level and slope. That fix folds back into updated coefficients (β0,updated = A + B·β0 and every βj,updated = B·βj). A worked numeric example (LP = 1.5 → 81.8% becomes 65.7%) makes the arithmetic concrete; then we show that an honest slope must come from a bootstrap (internal validation), not from refitting the model to its own training data; and we close on the principle that validation ≠ recalibration. You drive it: press Enter to autofill and run, Tab to accept the autocomplete ghost, and the Stage on the right shows the numbers and the calibration plot changing. Switch between the R and Python tabs for the same analysis in either language. Nothing runs in your browser; every value and plot was computed in R and Python beforehand.
Key takeaways
- The general recalibration $\text{logit}(p)=A+B\cdot\text{LP}$ fixes level and spread together; estimate $A$ and $B$ jointly.
- It folds into the model: $\beta_{0,\text{updated}}=A+B\beta_0$ and $\beta_{j,\text{updated}}=B\beta_j$ — the slope shrinks every coefficient.
- An honest slope / shrinkage factor comes from bootstrap or cross-validation, never from the raw development data (apparent slope ≈ 1 is a mirage).
- Validation ≠ recalibration: a dataset used to estimate $A,B$ is no longer an independent validation set. Report the frozen model first.
Start the series at Part 1 — calibration in the large [[calibration-in-the-large]], or revisit Part 2 — the calibration slope [[calibration-slope]].