What are the fundamentals of least squares? What is the basic principle of least squares?

Least squares was proposed by Gauss in 1795 in his work on predicting the orbits of stars [1]. Later, least squares became the cornerstone of estimation theory. Because of the simple structure of least squares and the difficulty of programming, it is highly valued and widely used.

If the standard notation is used, the least-squares estimate can be expressed as:

ax=b (2-43)

The solution in the above equation is minimized, and the pseudo-inverse in the following equation can be obtained:

a'ax=a'b (2-44)

a'a)^(1)a'ax=(a'a)^(1)a'b (2-45) due to. a'a)^-1a'a=i (2-46) so there is. x=(a'a)^(1)a'b (2-47) This is the one-time completion algorithm of least squares, a modern recursive algorithm, which is more suitable for computer identification.

Least squares is one of the most basic identification methods, but it has two drawbacks [1]: first, when the model noise is colored noise, the least squares estimation is not an unbiased and consistent estimation; Second, as data grows, there will be the so-called "data saturation" phenomenon. In order to solve these two problems, corresponding identification algorithms have emerged, such as the forgetting factor method, the limited memory method, the deviation compensation method, the augmented least squares, the generalized least squares, the auxiliary variable method, the two-step method and the multi-level least squares method.

Encyclopedia: When we study the interrelationship between two variables (x,y), we usually get a series of pairs of data (x1,,y2...).xm,ym）；Plotting these data in the x-y Cartesian coordinate system, if you find that these points are near a straight line, you can make the equation for this straight line as follows (Eq. 1-1).

yj = a0 + a1 xi (Equation 1-1).

Where: a0, a1 are arbitrary real numbers.

In order to establish this linear equation, it is necessary to determine a0 and a1, and apply the principle of least squares, and take the sum of squares of the measured value yi and the dispersion (yi-yj) of the calculated value (yj=a0+a1x) (yi-yj)2 as the "optimization criterion".

Order: = (yi - yj)2 (Equation 1-2).

Substituting (Eq. 1-1) into (Eq. 1-2) yields:

(yi - a0 - a1xi)2 (Eq. 1-3).

When (yi-yj) is the smallest square, you can use the function to find the partial derivatives of a0 and a1 so that these two partial derivatives are equal to zero.

Eq. 1-4) Eq. 1-5).

That is: m a0 + xi ) a1 = yi (Eq. 1-6).

xi ) a0 + xi2 ) a1 = (xi,yi) (Eq. 1-7).

The two systems of equations with respect to a0 and a1 are unknowns are obtained, and the solution of these two systems of equations yields:

a0 = ( yi) m - a1( xi) m (Eq. 1-8).

a1 = [m xi yi - xi yi)] [m xi2 - xi)2 )] Eq. 1-9).

In this case, we substitute a0 and a1 into (Eq. 1-1), and (Eq. 1-1) is our regression metalinear equation, i.e., the mathematical model.

In the regression process, it is not possible to correlate the regression in its entirety through each regression data point (x1, y1 x2,y2...xm,ym), in order to judge the quality of the correlation formula, the correlation coefficient "r", the statistic "f", and the remaining standard deviation "s" can be used to judge; The closer the "r" is to 1, the better; The greater the absolute value of "f", the better; The closer the "s" is to 0, the better.

r = [ xiyi - m ( xi m)( yi m)] sqr (Eq. 1-10) *

In (Eq. 1-1), m is the sample size, i.e., the number of experiments; The values of x and y for any set of experiments in xi and yi respectively.

Least Squares:

The total dispersion cannot be the sum of n dispersions.

to denote it, usually in terms of the sum of the squares of the dispersion, ie.

As the total dispersion, and make it to the minimum, so that the regression line is the lowest value of q in all the straight lines, this method of making the "sum of squares of the dispersion minimum" is called least squares:

Since the absolute value makes the calculation constant, in practical applications, people prefer to use: q=(y1-bx1-a) +y2-bx-a)+yn-bxn-a).

In this way, the problem boils down to the fact that q is the smallest when a,b is taken, i.e., the "overall distance" to the point line y=bx+a is the smallest.

Finding a,b in a regressive linear equation using least squares has the following formula.

The principle of least squares: find a straight line so that the sum of squares of the difference in the ordinates of all the points on the graph is the smallest, which is actually the smallest variance.

Least Squares (also known as least square) is a mathematical optimization technique. It looks for the best function match for the data by minimizing the sum of squares of the error. Unknown data can be easily obtained by using the least squares method, and the sum of squares of the error between the repentance of the obtained data and the actual data is minimized.

Least Squares can also be used for curve fitting. Some other optimization problems can also be expressed by minimizing the energy of the most finger forward or maximizing the entropy by least squares.

The principle of least squares is to determine the position of a straight line as "minimum sum of squares of residuals". In addition to the convenience of calculation, the estimator obtained by the least squares method also has excellent characteristics. This approach is very sensitive to outliers.

The application of least squares in traffic and transportation science:

The purpose of traffic occurrence is to establish the quantitative relationship between the traffic volume generated by the zoning and the variables such as land use and socio-economic characteristics of the zoning, and to estimate the traffic volume generated by each subdivision in the planning year. Because there are two endpoints for a trip, we analyze the traffic generated and the traffic attracted by a district separately. There are usually two ways for traffic to happen**:

Regression analysis and cluster analysis.

Regression analysis is based on the statistical analysis of the dependent variable and one or more independent variables, to establish the relationship between the dependent variable and the independent variable, the simplest case is univariate regression analysis, the general formula is: y = +x where y is the dependent variable, x is the independent variable, and is the regression coefficient. If the traffic generation of the cell is generated using the above formula, all variables are marked with the following subscript i; If you use it to study zonal traffic attraction, label all variables with the following mark j.

The principle and positive derivation of the ordinary least squares method are as follows:

Least Squares is a very important method in statistics, and ordinary least squares (OLS) is one of the most basic and commonly used, the main idea is that the model is optimal when the distance from each point to the fitting model is the shortest (the least residuals).

However, if the distance is used to calculate directly, there will be a situation where the positive and negative are offset, and the calculation will be very cumbersome if the absolute value is calculated in the Min's line, so the sum of squares of the distance is used for calculation, so the least squares method can actually be translated as the least sum of squares method.

Least Squares is a very important method in statistics, and ordinary least squares (OLS) is the most basic and commonly used one-finger repentance, the main idea is that the model is optimal when the distance from each point to the fitting model is the shortest (the least residual).

However, if the distance is used to calculate directly, there will be a situation where the positive and negative are offset, and the calculation by using the absolute value will make the calculation very cumbersome, so the sum of squares of the distance is used for calculation, so the least squares method can actually be translated as the least sum of squares method.

Later generations researched it and finally believed that it was indeed Gauss who discovered the least squares method first, but it did not cause much repercussions at the time, and people did not recognize the importance of this method until the results of Legendre's research came out and Gauss helped astronomers successfully ** the orbit of Ceres through least squares.

It is only then that people really realize the importance of least squares. Although Gauss was the first to discover the method of least squares, it was Legendre who first systematically summarized it and attracted the attention of the mathematical community, and the two mathematicians are equally worthy of respect.

Least Squares is a mathematical optimization technique; It looks for the best function match for the data by minimizing the sum of squares of the error.

The relationship between y and x is fitted as a linear relationship, all the sample points are around this line, each point has a certain distance from this line, the sum of squares of all distances, and find the slope of the line corresponding to its minimum, that is, the least-squares estimate.

Determine the values of alpha and beta (parameters) such that the sum of squares of the residuals is minimized.

Least Squares (also known as least square) is a mathematical optimization technique. It looks for the best function match for the data by minimizing the sum of squares of the error. Unknown data can be easily obtained by using the least squares method, and the sum of squares of the errors between these calculated data and the actual data is minimized.

Least Squares can also be used for curve fitting. Some other optimization problems can also be expressed by minimizing energy or maximizing entropy using least squares.

When we study the interrelationship between two variables (x, y), we usually get a series of pairs of data (x1, y1, x2, y2...).xm , ym)；Plotting these data in the x-y Cartesian coordinate system, if you find that these points are near a straight line, you can make the equation for this straight line as follows (Eq. 1-1).