12. Polynomial Regression

Curvature Within the Linear Model Framework

Author

A. J. Smit

Published

2026/03/19

1 Introduction

Polynomial regression belongs immediately after simple linear regression because it is still a linear model, even though the fitted relationship may look curved. It handles non-linearity by transforming the predictor into higher powers such as $X^2$ and $X^3$, while still estimating coefficients by linear least squares.

This makes polynomial regression a useful bridge chapter. It shows how to move beyond a straight line without yet leaving the ordinary linear modelling framework.

2 Key Concepts

Polynomial regression is still a linear model in its parameters.
Curvature is introduced by adding powers of the predictor, such as quadratic or cubic terms.
A polynomial fit is usually descriptive rather than mechanistic.
Higher-order polynomials can overfit very easily.
Interpretability usually declines as model order increases.

3 When This Method Is Appropriate

You should consider a polynomial model when:

the scatterplot suggests smooth curvature rather than a straight line;
a simple linear model leaves a clear curved pattern in the residuals;
you need a modestly flexible fit, but do not have a mechanistic nonlinear function in mind;
you still want to stay within the familiar lm() framework.

Polynomial regression is often appropriate as an intermediate step between simple linear regression and the more specialised approaches developed later in 20-generalised-additive-models.qmd and 21-nonlinear-regression.qmd.

4 Nature of the Data and Assumptions

Because polynomial regression is still fitted with lm(), the assumptions are the same as those for ordinary linear regression:

independence of observations;
approximately normal residuals;
homogeneous residual variance;
a response variable that is continuous.

The key difference is in the mean structure. Instead of assuming a straight-line relationship between $X$ and $Y$, the model allows curvature by including higher-order terms:

\[Y_i = \alpha + \beta_1 X_i + \beta_2 X_i^2 + \beta_3 X_i^3 + \epsilon_i\]

for a cubic model.

This is why polynomial regression may resemble nonlinear regression visually while still belonging to the linear-model family statistically.

5 R Functions

Polynomial regression is fit with lm():

lm(y ~ x + I(x^2), data = df)                 # quadratic
lm(y ~ x + I(x^2) + I(x^3), data = df)       # cubic
lm(y ~ poly(x, degree = 3), data = df)       # orthogonal polynomial form

The practical distinction is that I(x^2) and I(x^3) preserve the raw powers of the predictor, which can be easier to explain, while poly() creates orthogonal polynomial terms that are often more stable numerically.

6 Worked Example Placeholder

Polynomial regressions may resemble non-linear regression in terms of the visual appearance of the regression line, but they handle non-linearity by transforming the independent variable $X$ into higher powers. This makes them convenient when the relationship is bendy but there is no clear mechanistic model to justify a dedicated nonlinear function.

At present, this chapter serves as a structural placeholder in the sequence. The fuller worked example still needs to be developed, but the intended workflow is clear:

inspect the scatterplot and residuals from the straight-line model;
fit a quadratic or cubic extension with lm();
compare model adequacy and plausibility;
interpret cautiously, because the coefficients are not usually biologically transparent;
avoid pushing to unnecessarily high orders.

7 Practical Caution

It is worth noting that higher-order polynomials can lead to overfitting, where the model captures noise in the data rather than the underlying pattern. Overfitting becomes more likely as the order increases, and orders greater than 4 or 5 are rarely justified in introductory biological applications.

Another complication is that the biological interpretation of more complex polynomial models may be weak. In practice, polynomial regressions are often more useful for flexible description or short-range prediction than for mechanistic understanding.

8 Summary

Polynomial regression is the first structured way to model curvature after simple linear regression.
It remains within the ordinary linear model framework.
It is useful when the relationship is curved but not strongly mechanistic.
It should be used cautiously, because high-order polynomials overfit easily and are often hard to interpret.

The next chapter returns to the main modelling sequence by moving from one predictor to several predictors at once.

Reuse

CC BY-NC-SA 4.0

Citation

BibTeX citation:

@online{smit,_a._j.2026,
  author = {Smit, A. J., and J. Smit, A.},
  title = {12. {Polynomial} {Regression}},
  date = {2026-03-19},
  url = {http://tangledbank.netlify.app/BCB744/basic_stats/12-polynomial-regression.html},
  langid = {en}
}

For attribution, please cite this work as:

Smit, A. J., J. Smit A (2026) 12. Polynomial Regression. http://tangledbank.netlify.app/BCB744/basic_stats/12-polynomial-regression.html.

--- title: "12. Polynomial Regression" subtitle: "Curvature Within the Linear Model Framework" author: "A. J. Smit" date: last-modified date-format: "YYYY/MM/DD" reference-location: margin --- ```{r code-brewing-opts} #| echo: false knitr::opts_chunk$set( comment = "R>", warning = FALSE, message = FALSE, fig.width = 6.5, fig.height = 4.5, out.width = "88%", fig.asp = NULL, fig.align = "center", fig.retina = 2, dpi = 300 ) ``` # Introduction Polynomial regression belongs immediately after simple linear regression because it is still a linear model, even though the fitted relationship may look curved. It handles non-linearity by transforming the predictor into higher powers such as $X^2$ and $X^3$, while still estimating coefficients by linear least squares. This makes polynomial regression a useful bridge chapter. It shows how to move beyond a straight line without yet leaving the ordinary linear modelling framework. # Key Concepts - **Polynomial regression is still a linear model in its parameters.** - **Curvature is introduced by adding powers of the predictor**, such as quadratic or cubic terms. - **A polynomial fit is usually descriptive rather than mechanistic.** - **Higher-order polynomials can overfit very easily.** - **Interpretability usually declines as model order increases.** # When This Method Is Appropriate You should consider a polynomial model when: - the scatterplot suggests smooth curvature rather than a straight line; - a simple linear model leaves a clear curved pattern in the residuals; - you need a modestly flexible fit, but do not have a mechanistic nonlinear function in mind; - you still want to stay within the familiar `lm()` framework. Polynomial regression is often appropriate as an intermediate step between simple linear regression and the more specialised approaches developed later in [20-generalised-additive-models.qmd](20-generalised-additive-models.qmd) and [21-nonlinear-regression.qmd](21-nonlinear-regression.qmd). # Nature of the Data and Assumptions Because polynomial regression is still fitted with `lm()`, the assumptions are the same as those for ordinary linear regression: - independence of observations; - approximately normal residuals; - homogeneous residual variance; - a response variable that is continuous. The key difference is in the mean structure. Instead of assuming a straight-line relationship between $X$ and $Y$, the model allows curvature by including higher-order terms: $$Y_i = \alpha + \beta_1 X_i + \beta_2 X_i^2 + \beta_3 X_i^3 + \epsilon_i$$ for a cubic model. This is why polynomial regression may resemble nonlinear regression visually while still belonging to the linear-model family statistically. # R Functions Polynomial regression is fit with `lm()`: ```{r} #| eval: false lm(y ~ x + I(x^2), data = df) # quadratic lm(y ~ x + I(x^2) + I(x^3), data = df) # cubic lm(y ~ poly(x, degree = 3), data = df) # orthogonal polynomial form ``` The practical distinction is that `I(x^2)` and `I(x^3)` preserve the raw powers of the predictor, which can be easier to explain, while `poly()` creates orthogonal polynomial terms that are often more stable numerically. # Worked Example Placeholder Polynomial regressions may resemble non-linear regression in terms of the visual appearance of the regression line, but they handle non-linearity by transforming the independent variable $X$ into higher powers. This makes them convenient when the relationship is bendy but there is no clear mechanistic model to justify a dedicated nonlinear function. At present, this chapter serves as a structural placeholder in the sequence. The fuller worked example still needs to be developed, but the intended workflow is clear: 1. inspect the scatterplot and residuals from the straight-line model; 2. fit a quadratic or cubic extension with `lm()`; 3. compare model adequacy and plausibility; 4. interpret cautiously, because the coefficients are not usually biologically transparent; 5. avoid pushing to unnecessarily high orders. # Practical Caution It is worth noting that higher-order polynomials can lead to overfitting, where the model captures noise in the data rather than the underlying pattern. Overfitting becomes more likely as the order increases, and orders greater than 4 or 5 are rarely justified in introductory biological applications. Another complication is that the biological interpretation of more complex polynomial models may be weak. In practice, polynomial regressions are often more useful for flexible description or short-range prediction than for mechanistic understanding. # Summary - Polynomial regression is the first structured way to model curvature after simple linear regression. - It remains within the ordinary linear model framework. - It is useful when the relationship is curved but not strongly mechanistic. - It should be used cautiously, because high-order polynomials overfit easily and are often hard to interpret. The next chapter returns to the main modelling sequence by moving from one predictor to several predictors at once.