Question: Relationship between Perceptron and SVM: is it possible to see Perceptron as a kernel method?
Answer:
Perceptrons are also a supervised learning algorithm used for both regression and classification tasks. However, they differ from SVMs in several ways. Firstly, perceptrons are motivated by the desire to find the weights that best represent the data. The goal is to optimize a cost function, which captures the error between the input data and the desired output. Secondly, Perceptrons cannot handle non-linear data and thus cannot be used with the kernel trick.
Although Perceptrons and SVMs are both supervised learning algorithms, they are not interchangeable. However, it is possible to view Perceptrons as a kernel method by using a linear activation function. In this case, Perceptrons can be viewed as a special case of SVM with a linear kernel.
Question: Relations between Neural Networks and SVM: where is the ”kernel” in Neural Networks? What are the differences in terms of objective function between the two methods?
Answer:
The kernel in a Neural Network is the base of the network, which is responsible for the way the network processes data. It is the algorithm that determines how the network will be able to make decisions and how it will learn from its data. The kernel also defines how the weights of the network will be updated as the network learns.
The objective function of a Neural Network is to optimize the weights of the network in order to minimize the error between the output of the network and the desired output. The objective function of a kernel is to optimize the kernel parameters in order to maximize the accuracy of the network. The main difference between the two is that the objective function of the Neural Network is to minimize the error, while the objective function of the kernel is to maximize the accuracy.