Several Methods for Dealing with Independent Variable Collinearity in Multiple Linear Regression Det

Multicollinearity refers to the correlation between independent variables, that is, an independent variable can be expressed by the linear expression of one or several other independent variables. If multicollinearity exists, the matrix is irreversible when calculating the partial regression coefficients of the independent variables, resulting in an infinite number of solutions or no solutions.

In the process of using multiple linear regression to build a model, it is common for there to be multicollinearity between variables. So what do we do when we find that there is multicollinearity in a multilinear regression model?

This can be solved by:

1) Gradual regression.

The use of stepwise regression can be used to screen the variables that explain the greater variation of the response variables in the combinations of independent variables with multicollinearity, while the variables with the smaller explanations can be excluded from the model.

However, the disadvantage of this method is that when the collinearity is serious, the method of automatic variable screening cannot completely solve the problem.

2) Ridge regression.

Ridge regression is a biased estimate, but the standard error size of the regression coefficient can be effectively controlled.

3) Principal component regression.

Principal component analysis can be used to extract principal components from combinations of independent variables with multicollinearity, and then multiple linear regression can be performed with several principal components with large eigenvalues (e.g., greater than 1) and other independent variables. The obtained principal component regression coefficients are then inverted from the parameter estimation of the original independent variables based on the principal component expression.

In this method, part of the information is lost when extracting the principal components, and the stronger the multicollinearity between several independent variables, the less information is lost when extracting the principal components.

4) Path analysis.

If you have a clear understanding of the relationship between independent variables, you can consider establishing a path analysis model for more in-depth research.

Two things to know about multicollinearity:

In practice, multicollinearity is a matter of degree rather than of presence, and the meaningful distinction lies not in the presence and absence, but in the degree of multicollinearity. Multicollinearity is a feature of a sample rather than a population for a fixed explanatory variable.

Methods for eliminating multicollinearity:

1.Increase sample size.

2.Change with a priori information.

3.Remove unnecessary explanatory variables: Constraints on parameters.

4.Other methods: stepwise regression, ridgeregression, principal component analysis

These methods can be done by SPSS, and you can find the corresponding methods under the submenu of data analysis.

When removing unnecessary methods, it is better to use the stepwise regression method, which is more scientific.

The method of principal component analysis is relatively simple and scientific, and I mind using this method.