2.3. The Relationship Between Linear Regression and Polynomial Regression

There is a close relationship between linear regression models and polynomial regression models. Consider a multiple linear regression model which takes 3 input variables.

For example, predicting exam mark (\(y\)) using time spent studying (\(x_1\)), assignment mark (\(x_2\)) and attendance (\(x_3\)). This model would take the form:

\[y = \beta_0 + \beta_1 \textcolor{blue}{x_1} + \beta_2 \textcolor{blue}{x_2} + \beta_3 \textcolor{blue}{x_3}\]

A polynomial model of degree 3 would take the form:

\[y = \beta_0 + \beta_1 \textcolor{blue}{x} + \beta_2 \textcolor{blue}{x^2} + \beta+3 \textcolor{blue}{x^3}\]

The difference is that with the polynomial model, there is only one input variable, and we take that input variable and raise the values do a higher power.

For example, we could predict exam mark (\(y\)) using time spent studying (\(x\)), time spent studying squared (\(x^2\)) and time spent studying cubed (\(x^3\)).

We can build a polynomial regression model by adapting the code we used to build a multiple linear regression model, we just need to think about the x values we pass into the function .fit().

For a multiple linear regression model we would feed in a 2D array with \(n\) rows, one or each sample and then a column for each input variable. E.g.

\(x_1\)	\(x_2\)	\(x_3\)
Time Spent Studying	Assignment Mark	Attendance
4.5	73	93
8	89	100
1.5	65	74
3.5	66	88
5.5	67	84

[[4.5, 73,  93],
 [  8, 89, 100],
 [1.5, 65,  74],
 [3.5, 66,  88],
 [5.5, 67,  84]]

For a polynomial regression model we would feed in a 2D array with \(n\) rows, one or each sample and then a column raising the input variable to a power. E.g.

\(x\)	\(x^2\)	\(x^3\)
Time Spent Studying	Time Spent Studying Squared	Time Spent Studying Cubed
4.5	20.25	91.124
8	64	512
1.5	2.25	3.375
3.5	12.25	42.875
5.5	30.25	166.375

[[4.5, 20.25,  91.124],
 [  8,    64,     512],
 [1.5,  2.25,   3.375],
 [3.5, 12.25,  42.875],
 [5.5, 30.25, 166.375]]