Does SVM Support Vector Machine belong to the category of neural networks?

When the loss function is hinge loss and maximum margin, and f(x) is the linear function w*x+b (where w and b are the favorite SVM weights and biases), then the structure is a linear SVM

First of all, what is SVM? SVM is the abbreviation of "support vector machine", which refers to support vector machine, which is a common discrimination method. In the field of machine learning, it is a supervised learning model that is commonly used for pattern recognition, classification, and regression analysis.

So specifically, from a scientific point of view, the computational part of this linear SVM is the same as a single-layer neural network, which is a matrix product. The key to SVM is its hinge loss and the idea of maximum margin. In fact, this loss can also be used in neural networks (see the R-CNN method of object detection).

The question SVM (Support Vector Machine) belongs to the category of neural networks? For dealing with nonlinear data, SVM and neural networks take two different paths: neural networks achieve nonlinear functions through multiple hidden layers, which have some theoretical support (for example, neural networks with hidden layers can simulate any function), but they are not very complete at present; SVM uses the kernel trick method, which is theoretically complete (RKHS, simply a functional linear space).

Both have their own advantages and disadvantages, and the recent advantage of neural networks is that the network design can be very flexible, but it is always said that it is a great dancer; The theory of SVM is indeed beautiful, but the kernel design is not so easy, so it has not been so hot lately.

SVM - Support Vector Machine, commonly known as Support Vector Machine, is a supervised learning algorithm that belongs to the category of classification. In the application of data mining, it corresponds to and differs from clustering of unsupervised.

It is widely used in machine learning, computer vision and data mining.

Suppose you want to divide the solid circle and the hollow circle into two categories by the 38th line, then there are countless lines that can accomplish this task. In SVM, finding an optimal dividing line makes it the largest margin on both sides.

The SVM maps the vectors into a higher-dimensional space where a maximally spaced hyperplane is established. Two parallel hyperplanes are built on either side of the hyperplane that separates the data. Dividing the hyperplane maximizes the distance between two parallel hyperplanes.

It is assumed that the greater the distance or gap between parallel hyperplanes, the smaller the total error of the classifier.

It is a supervised learning method that is widely used in statistical classification as well as regression analysis.

Support vector machines can achieve global optimization, while neural networks are prone to fall into multiple local optimizations. Both libsvm and svmlite are very popular support vector machine tools, the e1071 package provides an implementation of libsvm, and the klap package provides an implementation of the latter.

The advantage of SVM is that it uses kernel functions for engineering problems, which can provide models with very high accuracy, and at the same time, with the help of regular terms, the model can avoid over-adaptation, and users do not have to worry about problems such as local optimization and multicollinearity, but the disadvantages are slow training and testing, and the model processing time is long, which is not suitable for large-scale datasets. Like neural networks, they are all black-box algorithms, and the results are more difficult to interpret. In addition, how to determine the appropriate kernel function is also a difficult point, and regularization is also a problem that needs to be considered.

The gamma function determines the shape of the separated hyperplane, defaulting to the reciprocal of the data dimension, and increasing its value usually increases the number of support vectors. Considering the cost function, the default value is usually 1, at which point the regular term is also constant, and the larger the regular term, the smaller the boundary.

The comparison of the two graphs shows that the penalty factor has a greater impact.

Extensions

In addition to selecting different eigensets and kernel functions, the performance of the SVM can be adjusted with the help of parameter gamma and penalty factors. Functions simplify this process.

The ten-fold cross is used to obtain the error deviation of each combination, and the best parameter combination with the lowest error is selected. Use this combination to train another support vector machine.

We generally think that neural networks are very high-tech things, and here we will learn about this "high-level" thing. In fact, deep learning should be at a high point in terms of technology, and neural networks should have been launched for many, many years.

The advantage of neurons is that they can detect nonlinear relationships, use the parallelization of algorithms to achieve efficient training of large data sets, and avoid errors in parameter estimation without parameters. The disadvantage is that it is easy to fall into the local optimum, and the algorithm training time is too long, and it may be overfitted.

The generalization weight in the figure is close to 0, indicating that the covariate has little effect on the classification result, and if the population variance is 1, the covariate has a nonlinear effect on the classification result.

The compute function can also get the output compute(network, testset[-5]) for each layer

This package provides a functional implementation of traditional feedforward backpropagation neural network algorithms, and the neuralnet package implements most of the neural network algorithms.

If type=class is not specified, the probability matrix is output by default.

What is a Support Vector Machine? SVM is an abbreviation for "Support Vector Machine" in English, and Support Vector Machine is a common identification method. In the field of machine learning, it is a supervised learning mode that is commonly used for pattern recognition, classification, and regression analysis.

In particular, the computational part of this linear support vector machine is the same as that of a single-layer neural network, which is just a matrix product. The key to SVM is its hinge loss and maximization concept. This loss can also be used for neural networks (see Object Detection Method for R-CNN).

Question: Do Support Vector Machines belong to the category of neural networks? To deal with nonlinear data, there are two different paths to support vector machines and neural networks: the hidden layer of nonlinear functional neural networks is implemented through a variety of methods, and there is some theoretical support (for example, hidden layer neural networks can simulate any function), but they are not perfect now; SVM uses a kernel deception method and is theoretically relatively complete (RKHS, which is just a linear space of a function function).

Both are good and bad, and the good thing about recent neural networks is that web design can be flexible, but the old man is said to be a great god. The theory of SVM is good, but the kernel design is not that easy, so it hasn't been too hot lately.

In addition, I want to say that no matter what scope of scientific and technological academic problems we study, we must construct a complete set of thinking logic from ourselves, list a thinking network, and go down from what is one, what is two, and what is three, so that it is more conducive to our thinking problems, and I hope mine will be helpful to you.