{"id":200,"date":"2018-01-07T18:55:00","date_gmt":"2018-01-07T18:55:00","guid":{"rendered":"https:\/\/kindsonthegenius.com\/blog\/2018\/01\/07\/basics-of-multilayer-perceptron-a-simple-explanation-of-multilayer-perceptron\/"},"modified":"2018-01-07T18:55:00","modified_gmt":"2018-01-07T18:55:00","slug":"basics-of-multilayer-perceptron-a-simple-explanation-of-multilayer-perceptron","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/blog\/basics-of-multilayer-perceptron-a-simple-explanation-of-multilayer-perceptron\/","title":{"rendered":"Basics of Multilayer Perceptron &#8211; A Simple Explanation of Multilayer Perceptron"},"content":{"rendered":"<div style=\"color: #555555; font-size: 18px; line-height: 30px; text-align: justify;\">\n<div style=\"font-family: 'segoe ui';\">Today we will understand the concept of Multilayer Perceptron.<\/p>\n<p><b>Recap of Perceptron <\/b><br \/>You already know that the basic unit of a neural network is a network that has just a single node, and this is referred to as the <b>perceptron<\/b>.<br \/>The perceptron is made up of inputs x<sub>1<\/sub>, x<sub>2<\/sub>, &#8230;, x<sub>n<\/sub> their corresponding weights w<sub>1<\/sub>, w<sub>2<\/sub>, &#8230;, w<sub>n<\/sub>. A function known as activation function takes these inputs, multiplies them with their corresponding weights and produces an output y.<\/p>\n<table align=\"center\" cellpadding=\"0\" cellspacing=\"0\" style=\"margin-left: auto; margin-right: auto; text-align: center;\">\n<tbody>\n<tr>\n<td style=\"text-align: center;\"><a href=\"https:\/\/2.bp.blogspot.com\/-sLmcD8myRC4\/WjbfTsIFATI\/AAAAAAAAAhI\/qZCOLFRpUNkrlQ9n1_l5UoYgTdKMpZf3wCPcBGAYYCw\/s1600\/Perceptron.jpg\" style=\"margin-left: auto; margin-right: auto;\"><img decoding=\"async\" loading=\"lazy\" border=\"0\" data-original-height=\"357\" data-original-width=\"1030\" height=\"137\" src=\"https:\/\/2.bp.blogspot.com\/-sLmcD8myRC4\/WjbfTsIFATI\/AAAAAAAAAhI\/qZCOLFRpUNkrlQ9n1_l5UoYgTdKMpZf3wCPcBGAYYCw\/s400\/Perceptron.jpg\" width=\"400\" \/><\/a><\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: center;\">Figure 1: A Perceptron (&#8216;single-layer&#8217; perceptron)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><span style=\"font-size: large;\"><br \/><\/span><span style=\"font-size: large;\"><b>What is Multilayer Perceptron?<\/b><\/span><br \/>A multilayer perceptron is a class of neural network that is made up of at least 3 nodes. So now you can see the difference. Also, each of the node of the multilayer perceptron, except the input node is a neuron that uses a<i> non-linear activation function<\/i>.<br \/>The nodes of the multilayer perceptron are arranged in layers.<\/p>\n<ul>\n<li>The input layer<\/li>\n<li>The output layer<\/li>\n<li>Hidden layers: layers between the input and the output<\/li>\n<\/ul>\n<p>Also note the the learning algorithm for the multilayer perceptron is known as back propagation(explained here).<\/p>\n<p><b>How the Multilayer Perceptron Works<\/b><br \/>In MLP, the neurons use non-linear activation functions that is designed to model the behavior of the neurons in the human brain.<br \/>An multi-layer perceptron has a linear activation function in all its neuron and uses backpropagation for its training.<\/p>\n<p><b>About Activation Functions <\/b><\/p>\n<p>The activation function combines the input to the neuron with the weights and then adds a bias to produce the output. In other words, the activation function maps the weighted inputs to the output of the neuron.<br \/>One of such activation functions is the sigmoid function which is used to determine the output of the neuron. An example of a sigmoid function is the <span style=\"color: #990000;\"><b>logistic function<\/b><\/span> which is shown below<\/p>\n<div style=\"clear: both; text-align: center;\"><a href=\"https:\/\/1.bp.blogspot.com\/-Q8YMTSW3hhA\/Wk_t7R9nWTI\/AAAAAAAAAmA\/KIq3eWqKvxATYdMZPApHywWoU-O7ENVdwCPcBGAYYCw\/s1600\/Sigmoid-function.JPG\" style=\"margin-left: 1em; margin-right: 1em;\"><img decoding=\"async\" loading=\"lazy\" border=\"0\" data-original-height=\"220\" data-original-width=\"429\" height=\"164\" src=\"https:\/\/1.bp.blogspot.com\/-Q8YMTSW3hhA\/Wk_t7R9nWTI\/AAAAAAAAAmA\/KIq3eWqKvxATYdMZPApHywWoU-O7ENVdwCPcBGAYYCw\/s320\/Sigmoid-function.JPG\" width=\"320\" \/><\/a><\/div>\n<p>Another example of a sigmoid function, is the <span style=\"color: #990000;\"><b>hyperbolic tangent<\/b><\/span> activation function shown below which produces an output ranging between -1 and 1<\/p>\n<div style=\"clear: both; text-align: center;\"><a href=\"https:\/\/4.bp.blogspot.com\/-yFDcusHo-BM\/WlIwsZ96xqI\/AAAAAAAAAoY\/Q8OQoGYiGFIlPYZla4QyT_6O495Dxff5gCLcBGAs\/s1600\/Hyperbolic-tangent-function.JPG\" style=\"margin-left: 1em; margin-right: 1em;\"><img decoding=\"async\" loading=\"lazy\" border=\"0\" data-original-height=\"651\" data-original-width=\"849\" height=\"245\" src=\"https:\/\/4.bp.blogspot.com\/-yFDcusHo-BM\/WlIwsZ96xqI\/AAAAAAAAAoY\/Q8OQoGYiGFIlPYZla4QyT_6O495Dxff5gCLcBGAs\/s320\/Hyperbolic-tangent-function.JPG\" width=\"320\" \/><\/a><\/div>\n<\/div>\n<div style=\"font-family: 'segoe ui';\">Yet another type of activation function that can be used is the Rectified Linear Unit or ReLU which is said to have better performance than the logistic function and the hyperbolic tangent function.<\/p>\n<p><b>Applying Activation Function to MLP<\/b><br \/>With activation function, we can calculate the output of any neuron in the MLP. Assuming <b>w<\/b> denotes the vector of weights, <b>x<\/b> is the vector of inputs, <i>b<\/i> is the bias and <span style=\"font-size: large;\"><span style=\"color: black; font-family: &quot;calibri&quot;;\">\u03d5<\/span><\/span>&nbsp; is the activation function, the for the i<sup>th<\/sup>, neuron, the output <i>y<\/i> would be given by: <\/p>\n<div style=\"clear: both; text-align: center;\"><a href=\"https:\/\/2.bp.blogspot.com\/-gFK_cWUiq2g\/WlI7ryneEmI\/AAAAAAAAAoo\/aEhwEfYIHiMTldlmExJdqhr26f6GawF8wCLcBGAs\/s1600\/Multilayer-Perceptron-Activation-Formula.jpg\" style=\"margin-left: 1em; margin-right: 1em;\"><img decoding=\"async\" loading=\"lazy\" border=\"0\" data-original-height=\"138\" data-original-width=\"606\" height=\"72\" src=\"https:\/\/2.bp.blogspot.com\/-gFK_cWUiq2g\/WlI7ryneEmI\/AAAAAAAAAoo\/aEhwEfYIHiMTldlmExJdqhr26f6GawF8wCLcBGAs\/s320\/Multilayer-Perceptron-Activation-Formula.jpg\" width=\"320\" \/><\/a><\/div>\n<div style=\"clear: both; text-align: center;\"><\/div>\n<p>An MLP is made up of a set of nodes which forms the input layer, one or more hidden layers, and an output layer.<\/p>\n<p><b>Layers of Multilayer Perceptron(Hidden Layers)<\/b><br \/>Remember that from the definition of multilayer perceptron, there must be one or more hidden layers. This means that in general, the layers of an MLP should be a minimum of three layers, since we have also the input and the output layer. This is illustrated in the figure below.<\/p>\n<div style=\"clear: both; text-align: center;\"><a href=\"https:\/\/1.bp.blogspot.com\/-Xal8aZ5MDL8\/WlJm8dh1J9I\/AAAAAAAAAo4\/uCj6tt4T3T0HHUY4uexNuq2BXTUwcChqACLcBGAs\/s1600\/Multilayer-Perceptron.jpg\" style=\"margin-left: 1em; margin-right: 1em;\"><img decoding=\"async\" loading=\"lazy\" border=\"0\" data-original-height=\"583\" data-original-width=\"1024\" height=\"227\" src=\"https:\/\/1.bp.blogspot.com\/-Xal8aZ5MDL8\/WlJm8dh1J9I\/AAAAAAAAAo4\/uCj6tt4T3T0HHUY4uexNuq2BXTUwcChqACLcBGAs\/s400\/Multilayer-Perceptron.jpg\" width=\"400\" \/><\/a><\/div>\n<p>Also to note is that the function activating these hidden layers has to be non-linear function (activation function) as discussed in previous section\/<\/p>\n<p><b>Training\/Learning in Multilayer Perceptrons<\/b><br \/>The training process of the MLP occurs by continuous adjustment of the weights of the connections after each processing.&nbsp; This adjustment is based on the error in output(which is the different between the expected result and the output). This continuous adjustment of the weights is a supervised learning process called &#8216;backpropagation&#8217;.<br \/>The backpropagation algorithm consists of two parts:<\/p>\n<ul>\n<li>forward pass<\/li>\n<li>backward pass<\/li>\n<\/ul>\n<p>In the forward pass, the expect output corresponding to the given inputs are evaluated<br \/>In the backward pass, partial derivatives of the cost function with respects to the different parameters are propagated back through the network.<br \/>The process continues until the error is at the lowest value.<br \/>&nbsp;(A detailed lesson on backpropagation is found here.)<\/p>\n<p>Another learning method for the multilayer perceptron is the Stochastic Gradient Descent) which is explained in details in <a href=\"http:\/\/kindsonthegenius.blogspot.com\/2018\/01\/what-is-perceptron-how-perceptron-works.html\" target=\"_blank\">another tutorial.<\/a><\/p>\n<p>Thanks for reading and if you have any questions, do drop in the comment box below<\/p>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Today we will understand the concept of Multilayer Perceptron. Recap of Perceptron You already know that the basic unit of a neural network is a &hellip; <\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[14],"tags":[],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/200"}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=200"}],"version-history":[{"count":0,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/200\/revisions"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=200"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=200"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=200"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}