Characteristics of BP neural networks, disadvantages of BP neural networks

It is generally accepted that the thinking of the human brain is divided into three basic ways: abstract (logical) thinking, visual (intuitive) thinking, and inspired (epiphany) thinking.

Logical thinking refers to the process of reasoning according to logical rules; It first turns information into concepts and represents them with symbols, and then logically infers according to the symbolic operation in serial mode. This process can be written as a serial instruction for the computer to execute. Intuitive thinking, however, is the synthesis of distributed stored information, resulting in an idea or solution to a problem that suddenly arises. The fundamental point of this way of thinking lies in the following two points:

1.The information is stored on the network through the distribution of excitatory patterns on neurons; 2.Information processing is done through a dynamic process in which neurons interact with each other at the same time.

Artificial neural networks have preliminary adaptive and self-organizing capabilities. Synaptic weight values are changed during learning or training to suit the requirements of the surrounding environment. The same network can have different functions depending on the learning method and content.

An artificial neural network is a learning system that can develop knowledge beyond the original level of knowledge of the designer. In general, its learning and training methods can be divided into two types, one is supervised or tutored learning, which uses a given sample standard to classify or imitate; The other is unsupervised learning or untutored learning, in this case, only the learning method or some rules are prescribed, then the specific learning content varies with the environment in which the system is located (i.e., the input signal situation), and the system can automatically discover the characteristics and regularity of the environment, which has a function more similar to the human brain.

Popular Science China, Science Encyclopedia: BP Neural Network.

1) Local minimization problem: From a mathematical point of view, the traditional BP neural network is an optimization method for local search, which solves a complex hail complex nonlinearization problem, and the weight of the network is gradually adjusted along the direction of local improvement, which will make the algorithm sink into the local extremum, and the weight converges to the local minimum, resulting in network training failure. In addition, the BP neural network is very sensitive to the initial network weight, and the initialization of the network with different weights tends to converge to different local minimalities, which is also the fundamental reason why many scholars get different results each time they train.

2) BP neural network algorithms.

Slow convergence speed: Because the BP neural network algorithm is essentially a gradient descent method.

The objective function to be optimized is very complex, and as a result, there will inevitably be a "zigzag", which makes the BP algorithm inefficient; And because the optimized objective function is very complex, it will inevitably have some flat regions when the neuronal output is close to 0 or 1, and in these regions, the weight error changes very little, making the training process almost come to a standstill.

3) The choice of BP neural network structure is different: There is no unified and complete theoretical guidance for the selection of BP neural network structure, and it can only be selected by experience. If the network structure is too large, the efficiency in training is not high, and overfitting may occur.

If the selection is too small, the network may not converge. The structure of the network directly affects the approximation ability and promotion nature of the network. Therefore, how to choose the right network structure for the application is an important issue.

4) The contradiction between the application example and the network scale: It is difficult for the BP neural network to solve the contradiction between the instance scale and the network scale of the application problem, which involves the relationship between the possibility and feasibility of the network capacity, that is, the learning complexity problem.

5) The contradiction between the ability of BP neural network and the ability to train: the ability is also called the generalization ability or the generalization ability, and the training ability is also called the approximation ability or learning ability.

In general, when the training ability is poor, the ** ability is also poor.

back propagation

BP (Back Propagation) neural network, that is, the learning process of error backpropagation error backpropagation algorithm, consists of two processes: forward propagation of information and backpropagation of error. Each neuron in the input layer is responsible for receiving input information from the outside world and transmitting it to the neurons in the middle layer. The middle layer is the internal information processing layer, which is responsible for information transformation, and the middle layer (hidden layer) can be designed as a single hidden layer or multiple hidden layer structure according to the needs of information change ability; The information transmitted to each neuron in the output layer by the last hidden layer is further processed to complete the forward propagation processing process of a learning, and the information processing results are output from the output layer to the outside world.

There are many types of neural networks, and BP neural networks are the most commonly used.