Our orders are delivered strictly on time without delay
Our orders are delivered strictly on time without delay
What is polynomial regression analysis?
What is the benefit of polynomial regression models?
What is the difference between a linear, quadratic, and cubic regression analysis?
When looking at the SPSS output, how do you know what is the best fitting model?
What values do you need from the SPSS output in order to report the findings in the results section in APA style?
Polynomial regression analysis is a form of regression analysis in which the relationship between the independent variable (1$x$) and the dependent variable (2$y$) is modeled as an 3$n^{th}$ degree polynomial in 4$x$.5
The general equation for a polynomial regression model is:
Where:
$Y_i$ is the dependent variable.6
$X_i$ is the independent variable.7
$\beta_0, \beta_1, \dots, \beta_n$ are the coefficients (parameters) estimated from the data.8
$\epsilon_i$ is the random error term.9
Though it models a non-linear relationship between 10$X$ and 11$Y$, polynomial regression is considered a special case of multiple linear regression because the model is still linear in the parameters (12$\beta$'s) that are being estimated.13
The primary benefit of using polynomial regression is its flexibility in modeling non-linear relationships.14
Captures Curvilinearity: It allows the model to fit curves rather than just straight lines, which is essential when the underlying relationship between variables is not linear (e.g., performance increasing up to a certain point and then decreasing).15
Improved Accuracy: By capturing these complex, non-linear patterns, the model often achieves a better fit to the data, potentially leading to more accurate predictions in certain scenarios compared to simple linear regression.16
Relatively Simple: It provides a straightforward way to model complex non-linear trends while still using the familiar framework of linear regression techniques (like the method of least squares for estimation).17
Linear, quadratic, and cubic regression are simply polynomial regression models of different degrees (18$n$):19
| Type of Regression | Degree (n) | Equation | Shape of the Curve |
|---|---|---|---|
| Linear | 1 (First-degree) | $Y = \beta_0 + \beta_1X$ | A straight line |
| Quadratic | 2 (Second-degree) | $Y = \beta_0 + \beta_1X + \beta_2X^2$ | A single curve (parabola) with one "bend" or turning point (U or inverted U-shape) |
| Cubic | 3 (Third-degree) | $Y = \beta_0 + \beta_1X + \beta_2X^2 + \beta_3X^3$ | A curve with two "bends" or turning points (S-shape or inverted S-shape) |
In essence, increasing the degree of the polynomial adds more flexibility and allows the model to capture more intricate curves in the data.
When comparing multiple polynomial models (e.g., linear, quadratic, and cubic models for the same data), the goal is to find the most parsimonious model—the simplest model that adequately explains the variance. You typically look at the following:
Significance of the Highest-Order Term: Check the $p$-value (Sig.) for the highest-order polynomial term (e.g., $X^2$ in the quadratic model, $X^3$ in the cubic model) in the Coefficients table.
If the term's 20$p$-value is statistically significant (21$p < \alpha$, typically 22$p < .05$), this suggests the curve captured by that degree significantly improves the model's fit over the next lower-order model.23
$R^2$ and Adjusted $R^2$ Change: Examine the Model Summary table.
Look at the change in $R^2$ ($R$ Square Change) and its associated $F$ test to see if adding the higher-order term (e.g., moving from linear to quadratic) significantly increased the variance explained.
Check the Adjusted $R^2$. This value penalizes the model for having more predictors. The model with the highest Adjusted $R^2$ is generally preferred, as it balances explanatory power with complexity.
Principle of Parsimony: Choose the lowest-degree polynomial that is statistically significant and provides a substantial increase in $R^2$. While a cubic model might have a higher 24$R^2$ than a quadratic model, if the cubic term is not significant, the simpler quadratic model is usually preferred to avoid overfitting (fitting the sample data too closely, which harms generalizability).25
Visual Inspection: Plot the data and the fitted curves from each model to see which one visually best represents the overall trend without introducing excessive, unnatural wiggles.