Map > Data Science > Predicting the Future > Modeling > Classification/Regression > Artificial Neural Network > Perceptron
 

Artificial Neural Network - Perceptron

A single layer perceptron (SLP) is a feed-forward network based on a threshold transfer function. SLP is the simplest type of artificial neural networks and can only classify linearly separable cases with a binary target (1 , 0).
 

Algorithm
The single layer perceptron does not have a priori knowledge, so the initial weights are assigned randomly. SLP sums all the weighted inputs and if the sum is above the threshold (some predetermined value), SLP is said to be activated (output=1). 

The input values are presented to the perceptron, and if the predicted output is the same as the desired output, then the performance is considered satisfactory and no changes to the weights are made. However, if the output does not match the desired output, then the weights need to be changed to reduce the error. 

Because SLP is a linear classifier and if the cases are not linearly separable the learning process will never reach a point where all the cases are classified properly. The most famous example of the inability of perceptron to solve problems with linearly non-separable cases is the XOR problem.

However, a multi-layer perceptron using the backpropagation algorithm can successfully classify the XOR data.
 
Multi-layer Perceptron - Backpropagation algorithm
A multi-layer perceptron (MLP) has the same structure of a single layer perceptron with one or more hidden layers. The backpropagation algorithm consists of two phases: the forward phase where the activations are propagated from the input to the output layer, and the backward phase, where the error between the observed actual and the requested nominal value in the output layer is propagated backwards in order to modify the weights and bias values. 
 
Forward propagation:
Propagate inputs by adding all the weighted inputs and then computing outputs using sigmoid threshold.

Backward propagation:
Propagates the errors backward by apportioning them to each unit according to the amount of this error the unit is responsible for.

Example
A Neural Network in 11 lines of Python
 
ANN Interactive (only Windows)
 
Exercise