{"id":182,"date":"2018-01-15T00:35:00","date_gmt":"2018-01-15T00:35:00","guid":{"rendered":"https:\/\/kindsonthegenius.com\/blog\/2018\/01\/15\/linear-models-for-regressionlinear-basis-function-models\/"},"modified":"2018-01-15T00:35:00","modified_gmt":"2018-01-15T00:35:00","slug":"linear-models-for-regressionlinear-basis-function-models","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/blog\/linear-models-for-regressionlinear-basis-function-models\/","title":{"rendered":"Linear Models for Regression(Linear Basis Function Models)"},"content":{"rendered":"<div style=\"color: #555555; font-size: 18px; line-height: 30px; text-align: justify;\">\n<div style=\"font-family: 'segoe ui';\">\n<div style=\"text-align: center;\"><a href=\"https:\/\/youtu.be\/K_48DJEv0-o\" target=\"_blank\">Linear Models of Regression Video Updated <\/a><br \/><a href=\"https:\/\/youtu.be\/VU8DUyL6iqQ\" target=\"_blank\">Watch the Linear Regression Video Here&#8230; <\/a><\/div>\n<p>This article discusses the Linear Models of Regression with focus on Linear basis function model. <\/p>\n<p>&nbsp;<span style=\"font-size: large;\"><b><span style=\"color: #45818e;\">Content<\/span><\/b><\/span> <\/p>\n<ul>\n<li><span style=\"color: #990000;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"font-size: 14pt;\">Background of Linear Regression<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #990000;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"font-size: 14pt;\">The Regression Problem<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #990000;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"font-size: 14pt;\">Linear Function Model<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #990000;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"font-size: 14pt;\">Constructing the Basis Function<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #990000;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"font-size: 14pt;\">Introducing a non-linear function<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #990000;\"><span style=\"font-family: &quot;Trebuchet MS&quot;, sans-serif;\"><span style=\"font-size: 14pt;\">Final Notes<\/span><\/span><\/span><\/li>\n<\/ul>\n<p>Other discussions can be found in this link: <a href=\"https:\/\/www.blogger.com\/blogger.g?blogID=2304557215258226977#editor\/target=post;postID=6827122042926805344;onPublishedMenu=allposts;onClosedMenu=allposts;postNum=2;src=postname\">Best 23 Easy Tutorials on Machine Learning and Artificial Intelligence(Easy to Understand)<\/a><\/p>\n<div style=\"clear: both; text-align: center;\"><a href=\"https:\/\/1.bp.blogspot.com\/-RRisHiASzsE\/Wlv2PNWusNI\/AAAAAAAAAyw\/-LG241Tta-82BouktxyB-eP7qSJmO4C4gCLcBGAs\/s1600\/Linear-Models-of-Regression.jpg\" style=\"margin-left: 1em; margin-right: 1em;\"><img decoding=\"async\" loading=\"lazy\" border=\"0\" data-original-height=\"490\" data-original-width=\"1144\" height=\"137\" src=\"https:\/\/1.bp.blogspot.com\/-RRisHiASzsE\/Wlv2PNWusNI\/AAAAAAAAAyw\/-LG241Tta-82BouktxyB-eP7qSJmO4C4gCLcBGAs\/s320\/Linear-Models-of-Regression.jpg\" width=\"320\" \/><\/a><\/div>\n<p><ins data-ad-client=\"ca-pub-7041870931346451\" data-ad-format=\"fluid\" data-ad-layout=\"in-article\" data-ad-slot=\"4209786523\" style=\"display: block; text-align: center;\"><\/ins> <span style=\"color: #45818e;\"><span style=\"font-size: large;\"><b>Background<\/b><\/span><\/span><br \/>Remember that <a href=\"http:\/\/kindsonthegenius.blogspot.com\/2018\/01\/what-is-difference-between.html\" target=\"_blank\">Regression <\/a>is of the technique under <a href=\"http:\/\/kindsonthegenius.blogspot.com\/2018\/01\/what-is-difference-between-supervised.html\" target=\"_blank\">Supervised Learning<\/a>. The other is <a href=\"http:\/\/kindsonthegenius.blogspot.com\/2018\/01\/what-is-difference-between.html\" target=\"_blank\">Classification<\/a>. The objective of the regression model is to determine the value of one or more of a target variable t, given the value of a D-dimensional vector, x of input variables. In other words, you need to find the function that relates the input and the output. This can be done using Linear Models.<br \/>One of such modes is the polynomial curve fitting which gives a function that is a linear function of a particular parameter.<br \/>A better model is the <span style=\"color: #990000;\"><i>Linear Basis Function. This is discussed next.<\/i><\/span><\/p>\n<p><span style=\"font-size: large;\"><span style=\"color: #45818e;\"><b>The Linear Basis Function<\/b><\/span><\/span><br \/>Given a set of input dataset of N samples <i><span style=\"font-family: &quot;georgia&quot; , &quot;times new roman&quot; , serif;\">{x<sub>n<\/sub>}<\/span><\/i>, where <i><span style=\"font-family: &quot;georgia&quot; , &quot;times new roman&quot; , serif;\">n = 1, &#8230; , N<\/span><\/i>, as well as the corresponding target values {t<sub>n<\/sub>}, the goal is to deduce the value of t for new value of x. The set of input data set together with the corresponding target values t is known as the training data set.<\/div>\n<div style=\"font-family: 'segoe ui';\">On way to handle this is by constructing a function <span style=\"color: black;\"><i><span style=\"font-family: &quot;georgia&quot; , &quot;times new roman&quot; , serif;\">y(x) <\/span><\/i><\/span>that maps x to t such that:<\/div>\n<div style=\"text-align: center;\"><span style=\"color: black;\"><span style=\"font-family: &quot;georgia&quot; , &quot;times new roman&quot; , serif;\"><i>y(x) = t<\/i><\/span><\/span><\/div>\n<div style=\"font-family: 'segoe ui';\">for a new input value of <span style=\"font-family: &quot;georgia&quot; , &quot;times new roman&quot; , serif;\"><i>x<\/i><\/span>.<\/div>\n<div style=\"font-family: 'segoe ui';\">Then we can examine this model by finding the probability that the results are correct. This means that we need to examine the probability of t given x&nbsp;<\/div>\n<div style=\"text-align: center;\"><span style=\"color: black;\"><span style=\"font-family: &quot;georgia&quot; , &quot;times new roman&quot; , serif;\"><i>p(t|x)<\/i><\/span><\/span><\/div>\n<div style=\"font-family: 'segoe ui';\"><\/div>\n<p><ins data-ad-client=\"ca-pub-7041870931346451\" data-ad-format=\"fluid\" data-ad-layout=\"in-article\" data-ad-slot=\"4209786523\" style=\"display: block; text-align: center;\"><\/ins> <\/p>\n<div style=\"font-family: 'segoe ui';\"><span style=\"font-size: large;\"><span style=\"color: #45818e;\"><b>Constructing the Linear Basis Function<\/b><\/span><\/span><\/div>\n<div style=\"font-family: 'segoe ui';\">The basic linear model for regression is a&nbsp; model that involves a linear combination of the input variables:<\/p>\n<div style=\"text-align: center;\"><span style=\"color: black;\"><i><span style=\"font-family: &quot;times&quot; , &quot;times new roman&quot; , serif;\">y(w,x) = w<sub>o<\/sub> + w<sub>1<\/sub>x<sub>1<\/sub> + w<sub>2<\/sub>x<sub>2<\/sub> + &#8230; + w<sub>D<\/sub>x<sub>D<\/sub><\/span><\/i><\/span><\/div>\n<p>where <span style=\"color: black;\"><i><span style=\"font-family: &quot;times&quot; , &quot;times new roman&quot; , serif;\">x = (x<sub>1<\/sub>, x<sub>2<\/sub>, &#8230; ,x<sub>D<\/sub>)T<\/span><\/i><\/span><\/p>\n<p>This is what is generally known as<a href=\"http:\/\/kindsonthegenius.blogspot.com\/2018\/01\/what-is-difference-between.html\" target=\"_blank\"> <\/a><span style=\"color: #cc0000;\"><i><a href=\"http:\/\/kindsonthegenius.blogspot.com\/2018\/01\/what-is-difference-between.html\" target=\"_blank\">linear regression<\/a>.<\/i><\/span><br \/>The key attribute of this function is that it is a linear function of the parameters <span style=\"color: black;\"><span style=\"font-family: &quot;times&quot; , &quot;times new roman&quot; , serif;\"><i>w<sub>0<\/sub>, w<sub>1<\/sub>,&#8230;, w<sub>D<\/sub>.<\/i><\/span><\/span> It is also a linear function of the input variable <span style=\"color: black;\"><i><span style=\"font-family: &quot;georgia&quot; , &quot;times new roman&quot; , serif;\">x<\/span><\/i><\/span>. Being a linear&nbsp; function of the input variable x, limits the usefulness of the function. This is because most of the observations that may be encountered does not necessarily follow a linear relationship. To solve this problem consider modifying to model to be a combination of fixed non-linear functions of the input variable.<\/p>\n<p>If we assume that the non-linear function of the input variable is <span lang=\"grc\"><i><span style=\"font-family: &quot;georgia&quot; , &quot;times new roman&quot; , serif;\"><span style=\"color: black;\">\u03c6(x)<\/span>,<\/span><\/i> then we can re-write the original function as :<\/span><br \/><span lang=\"grc\"><br \/><\/span><span style=\"font-family: &quot;times&quot; , &quot;times new roman&quot; , serif;\"><span style=\"color: black;\"><i><span lang=\"grc\">y(x,w) = w0&nbsp;+ w<sub>1<\/sub><\/span><span lang=\"grc\"><span lang=\"grc\">\u03c6(x<sub>1<\/sub>)&nbsp;+ w<sub>2<\/sub><\/span><\/span><span lang=\"grc\">\u03c6(x<sub>2<\/sub>) + &#8230; + w<sub>D<\/sub><\/span><span lang=\"grc\"><span lang=\"grc\">\u03c6(x<sub>D<\/sub>)&nbsp;<\/span><\/span><\/i><\/span><\/span><\/p>\n<p><span lang=\"grc\"><span lang=\"grc\">Summing it up, we will have:<\/span><\/span><\/p>\n<div style=\"clear: both; text-align: center;\"><a href=\"https:\/\/2.bp.blogspot.com\/-e-JP4aQKWpk\/Wlvt5F-aRUI\/AAAAAAAAAyc\/rf6r4JiWWeYGU2ekfY4o75HfacVZ9nKwwCLcBGAs\/s1600\/Linear%2BBasis%2BFunction%2BModel.jpg\" style=\"margin-left: 1em; margin-right: 1em;\"><img decoding=\"async\" loading=\"lazy\" border=\"0\" data-original-height=\"147\" data-original-width=\"515\" height=\"89\" src=\"https:\/\/2.bp.blogspot.com\/-e-JP4aQKWpk\/Wlvt5F-aRUI\/AAAAAAAAAyc\/rf6r4JiWWeYGU2ekfY4o75HfacVZ9nKwwCLcBGAs\/s320\/Linear%2BBasis%2BFunction%2BModel.jpg\" width=\"320\" \/><\/a><\/div>\n<p><span lang=\"grc\"><span lang=\"grc\">where <\/span><\/span><span lang=\"grc\"><i>\u03c6(x)<\/i> are known as<i> basis functions<\/i>.<\/span><br \/><span lang=\"grc\"><br \/><\/span><span lang=\"grc\">The total number of parameters in this function will be <span style=\"font-family: &quot;georgia&quot; , &quot;times new roman&quot; , serif;\">M,<\/span> therefore the summation of terms is from<span style=\"font-family: &quot;georgia&quot; , &quot;times new roman&quot; , serif;\"><i> j = 1 to M.<\/i><\/span><\/span><br \/><span lang=\"grc\">The parameter w<sub>0<\/sub>&nbsp; is known as the bias parameter which allows for a fixed offset in the data.&nbsp;<\/span><\/p>\n<p><span lang=\"grc\">(<b><span style=\"color: #990000;\">Quiz<\/span><\/b>: Can you remember any other topic the word bias appears? Leave it in comment on the left of the page!)<\/span><br \/><span style=\"color: #45818e;\"><span lang=\"grc\" style=\"font-size: large;\"><br \/><\/span><\/span><span style=\"color: #45818e;\"><span style=\"font-size: large;\"><b><span lang=\"grc\">Final Notes<\/span><\/b><\/span><\/span><br \/><span lang=\"grc\">The topic for linear models of regression covers much more than what is presented here. But what we have discussed is the basics of the Linear Basis Function Model and I have decided to keep it simple and clear so that you can easily understand it and possibly pass an oral exam.<\/span><br \/><span lang=\"grc\">But if you&nbsp; are a Math student and want to delve further, I would recommend reading chapter three of the book. &#8216;<span style=\"color: #45818e;\"><i>Pattern Recognition and Machine Learning<\/i><\/span>&#8216;, by Christopher Bishop. <\/span> <\/div>\n<\/div>\n<p><ins data-ad-client=\"ca-pub-7041870931346451\" data-ad-format=\"fluid\" data-ad-layout=\"in-article\" data-ad-slot=\"4209786523\" style=\"display: block; text-align: center;\"><\/ins> <br \/><ins data-ad-client=\"ca-pub-7041870931346451\" data-ad-format=\"fluid\" data-ad-layout=\"in-article\" data-ad-slot=\"4209786523\" style=\"display: block; text-align: center;\"><\/ins><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Linear Models of Regression Video Updated Watch the Linear Regression Video Here&#8230; This article discusses the Linear Models of Regression with focus on Linear basis &hellip; <\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[139,392],"tags":[],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/182"}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=182"}],"version-history":[{"count":0,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/182\/revisions"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=182"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=182"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=182"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}