{"id":199,"date":"2018-01-08T00:15:00","date_gmt":"2018-01-08T00:15:00","guid":{"rendered":"https:\/\/kindsonthegenius.com\/blog\/2018\/01\/08\/basics-of-backpropagation-in-neural-networksmachine-learning\/"},"modified":"2018-01-08T00:15:00","modified_gmt":"2018-01-08T00:15:00","slug":"basics-of-backpropagation-in-neural-networksmachine-learning","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/blog\/basics-of-backpropagation-in-neural-networksmachine-learning\/","title":{"rendered":"Basics of Backpropagation in Neural Networks(Machine Learning)"},"content":{"rendered":"
\n
Backpropagation <\/b><\/span>is the learning algorithm used in neural networks and is a generalization of the least mean squares algorithm used in linear perceptron. Backpropagation requires a known and expected output value for each input value and therefore it is therefore a supervised learning method.<\/p>\n

How It Works<\/b>
Remember backpropagation is a learning algorithm. How does learning occur? Learning is done by changing the weights of the perceptron after each signal have been processed based on the calculated amount of error in the output compared to the expected result.<\/p>\n

Error = Output perceptron – expected result<\/i><\/span><\/span><\/b><\/div>\n

Calculating the Loss Function<\/b>
To understand the backpropagation algorithm, you need to understand the concept of the Loss Function which is also the Cost function. This is loosely the same as the error formula above, but this time we would need to formalize the definition a little. It would still be easy and clear.<\/div>\n

The loss function calculates the difference between the output of the network and the expected output.<\/div>\n
<\/div>\n
Let y, y’ be vectors in \u211dn<\/sup><\/span><\/div>\n
We select an error function E(y, y’) which give the difference between outputs y and y’ which is given by:<\/div>\n
<\/div>\n
<\/a><\/div>\n
<\/div>\n
<\/div>\n
The error function would be given by the square of the Euclidean distance between y and y’ as shown below:<\/div>\n
<\/div>\n
For n training examples, the average error can would be given by:<\/div>\n
<\/div>\n
<\/a><\/div>\n
<\/div>\n
The partial derivatives with respects to y and y’ would be given by:<\/div>\n
<\/div>\n
<\/a><\/div>\n
<\/div>\n
You have come a long way in understanding the backpropagation algorithm. Just take some time to get these formulas around your head because that is what would form the basis of understanding the backpagation algorithm<\/div>\n
<\/div>\n
The Backpropagation Algorithm<\/b><\/div>\n
The algorithm starts when an input vector x<\/b> is entered into the network. This input moves from the input layer through the hidden layers to the output layer and produces an output y. <\/div>\n
The loss function is used to compare this output with the expected(or actual) output to give an error value. The error value is calculated for each of the neurons in the output layer. <\/div>\n
These error values are then propagated backwards from the output layer, through the network.<\/div>\n


<\/span>Backpropagation uses these error values to calculate the gradient of the loss function as they are move back through the network. This gradient, is then used to update the weights of the nodes. The process is repeated again to get another output based on the updated weights. The process of backpropagation is repeated until the error function produces a minimum value.<\/span>

<\/span>Summary of the Backpropagation Algorithm<\/b><\/span><\/p>\n

    <\/span><\/p>\n
  1. Input vector x<\/b> is given to the network <\/span><\/li>\n

    <\/span><\/p>\n

  2. Input propagates forward to produce an output y in the output layer <\/span><\/li>\n

    <\/span><\/p>\n

  3. Error function is calculated <\/span><\/li>\n

    <\/span><\/p>\n

  4. Error is propagated backwards into the network <\/span><\/li>\n

    <\/span><\/p>\n

  5. Weights are adjusted accordingly <\/span><\/li>\n

    <\/span><\/p>\n

  6. Repeat the process until the final error is minimum <\/span><\/li>\n<\/ol>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

    Backpropagation is the learning algorithm used in neural networks and is a generalization of the least mean squares algorithm used in linear perceptron. Backpropagation requires … <\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[16,14],"tags":[],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/199"}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=199"}],"version-history":[{"count":0,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/199\/revisions"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=199"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=199"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=199"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}