20. Prediction, Explanation, and Regularisation

Choosing Models for Different Scientific Goals

1 Introduction

Not all models are built for the same purpose. Some are built to explain mechanisms. Others are built to predict future outcomes accurately. These goals overlap, but they are not identical.

This chapter draws that distinction explicitly and introduces regularisation as one family of tools that becomes useful when prediction and stability matter more than coefficient-by-coefficient interpretation.

2 Key Concepts

This chapter depends on a few broad distinctions.

• Explanation and prediction are overlapping but different modelling goals.
• Out-of-sample performance matters most when prediction is the primary aim.
• Regularisation shrinks coefficients to improve stability and predictive performance.
• Interpretability can be traded against prediction depending on the scientific objective.
• Model choice should be justified by purpose, not only by convention.

3 Main Ideas

• Explanation prioritises interpretable coefficients and defensible causal language.
• Prediction prioritises out-of-sample performance.
• Regularisation helps stabilise models when predictors are numerous or collinear.

4 Source

This chapter will draw primarily on:

• 25-synthesis-prediction-explanation.qmd
• /Users/ajsmit/Documents/R_local/BCB_Stats/regularisation.qmd