![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/Simple-Linear-Regression-in-Machine-Learning-300x212.jpg\")
<\/p>\nIn this tutorial we would cover Simple Linear Regression in a very easy-to-understand way. We are assuming you don’t have much knowledge of Machine Learning and maybe a little knowledge of statistics.<\/p>\n
<\/p>\n
We would examine the following topics:<\/p>\n
<\/p>\n
<\/p>\n
Simple linear regression <\/i>lives up to its name: it is a very straightforward approach for predicting a quantitative response Y <\/i>on the basis of a single predictor variable X<\/i>. It assumes that there is approximately a linear relationship between X <\/i>and Y <\/i><\/p>\n
Consider the dataset in table 1.0. The table shows the amount of Sales for\u00a0 a given amount spent on advertising.<\/p>\n
Year<\/strong><\/td>\n| Adverts (X)<\/strong><\/td>\n | Sales (Y)<\/strong><\/td>\n<\/tr>\n | 2008<\/td>\n | $60<\/td>\n | $1500<\/td>\n<\/tr>\n | 2009<\/td>\n | $75<\/td>\n | $2200<\/td>\n<\/tr>\n | 2010<\/td>\n | $77<\/td>\n | $3500<\/td>\n<\/tr>\n | 2011<\/td>\n | $89<\/td>\n | $4230<\/td>\n<\/tr>\n | 2012<\/td>\n | $93<\/td>\n | $5500<\/td>\n<\/tr>\n | 2014<\/td>\n | $101<\/td>\n | $5910<\/td>\n<\/tr>\n | 2015<\/td>\n | $104<\/td>\n | ????<\/span><\/td>\n<\/tr>\n | 2018<\/td>\n | $110<\/td>\n | ????<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n | Table 1.0<\/em>: Yearly Sales Values for Adverts<\/p>\n The goal of simple linear regression is to predict the Sales for given advert. In order words, we want to predict the response Y(dependent variable), based on the predictor variable X (independent variable).<\/p>\n In Simple Linear Regression, we make an assumption that there exists a linear relationship between the two variable.\u00a0 The easiest way to make this prediction is to find the function y = f(x) that relates the two variable. This we can do by:<\/p>\n The equation of a line is given by<\/p>\n y = mx + c<\/p>\n But in the language of regression we write it as<\/p>\n Y\u00a0\u2248 \u03b20<\/sub> +\u03b21<\/sub>X<\/p>\n Note that we did not use equal sign (=). The sign\u00a0\u2248 is a regression operator that says that Y is “modeled as” and not Y is equal to.<\/p>\n <\/p>\n <\/p>\n These are called the regression coefficients, model coefficients or model parameters which are unknown.<\/p>\n In Machine learning, the training data is used to determine values that are close to\u00a0\u03b20<\/sub> and\u00a0\u03b21<\/sub> but not exactly. If we call the coefficients produced by the training\u00a0\u03b20<\/sub>‘ and\u00a0\u03b21<\/sub>‘, then we can predict the values of future sales using the formula:<\/p>\n y’ = \u03b21<\/sub>‘ +\u03b20<\/sub>‘x<\/p>\n The ‘ in the variables indicate the the values are estimates of the unknown parameters.<\/p>\n <\/p>\n <\/p>\n To estimate the coefficients\u00a0\u03b20<\/sub> and\u00a0\u03b21<\/sub>, we need to use the data we have. In the Table 1.0, you can see that we have 6 known data points and 2 unknown points. We can represent out dataset as:<\/p>\n (x1<\/sub>,y1<\/sub>), (x2<\/sub>,y2<\/sub>), . . . , (xn<\/sub>, yn<\/sub>)<\/p>\n where n = 6.<\/p>\n For our dataset, (x1<\/sub>,y1<\/sub>) = (60, 1500). (x2<\/sub>, y2<\/sub>) = ( 75, 2200) and so on.<\/p>\n If we use the method of plotting the graph and making the line pass through each of the datapoint, we would get the exact values of\u00a0\u03b21<\/sub> and\u00a0\u03b20<\/sub>. We would use our training data set to\u00a0 obtain estimate\u00a0\u03b20<\/sub>‘ and\u00a0\u03b21<\/sub>‘<\/p>\n So if we want to make an estimate for the value of sales for 2018, we would, apply the coefficients such that:<\/p>\n y’ = \u03b20<\/sub>‘ +\u03b21<\/sub>‘ * 110<\/p>\n The objective is to find the estimated coefficietns\u00a0 \u03b20<\/sub>‘ and \u03b21<\/sub>‘ that is a close to\u00a0 \u03b20<\/sub> and \u03b21<\/sub> as possible.\u00a0 The difference between our estimated value and the real values is an error term which needs to be minimized. The method we are going to use to do this is called the Least Squares approach. Let’s see how it works<\/p>\n <\/p>\n <\/p>\n Let’s assume we find the value of\u00a0 \u00a0yi<\/sub>‘ = \u03b20<\/sub>‘ + \u03b21<\/sub>‘xi<\/sub> for the ith observation.<\/p>\n Then we can calculate the error ei<\/sub> = yi<\/sub> – yi<\/sub>‘. This value is called the ith residual<\/strong>. That is, the difference between the actual value of y and the estimated value of y predicted by the model.<\/p>\n Calculating the residual sum of squares (simply square the residual) for all the data, we have<\/p>\n Residual Sum of Squares (RSS) = e1<\/sub>2<\/sup> + e2<\/sub>2<\/sup> + . . . + en<\/sub>2<\/sup>.<\/p>\n Knowing that ei<\/sub> = yi<\/sub> – yi<\/sub>‘ and\u00a0\u00a0yi<\/sub>‘ = \u03b20<\/sub>‘ +\u03b21<\/sub>‘xi<\/sub>,<\/p>\n We can write the summation as:<\/p>\n RSS = (y1<\/sub> – \u03b20<\/sub>‘ – \u03b21<\/sub>‘x1)2<\/sup> + (y2<\/sub> –\u00a0\u03b20<\/sub>‘ – \u03b21<\/sub>‘x2)2<\/sup> + . . . + (yn<\/sub> –\u00a0\u03b20<\/sub>‘ – \u03b21<\/sub>‘xn<\/sub>)2<\/sup><\/p>\n The Method of Least Squares chooses\u00a0\u03b20<\/sub>‘ and\u00a0\u03b21<\/sub>‘ so as to minimize the RSS.<\/p>\n The values of\u00a0\u03b20<\/sub>‘\u00a0 and \u03b21<\/sub>‘ that would make the RSS minimum is given by the following equations:<\/p>\n The formulas above are actually very simple to understand.<\/p>\n x\u0304 is the sample mean of the x dataset\u00a0 \u00a0and is given by:<\/p>\n In the next Tutorial on the regression series, we would take a real dataset, calculate these values, calculate the regression coefficients and actually use the the coefficients to predict missing values in the dataset.<\/p>\n Or I could give this as an exercise: Use the formulas to predict the missing values in Table 1.0.<\/p>\n Do leave a comment if this have been informative for you.<\/p>\n","protected":false},"excerpt":{"rendered":" In this tutorial we would cover Simple Linear Regression in a very easy-to-understand way. We are assuming you don’t have much knowledge of Machine Learning … <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"pagelayer_contact_templates":[],"_pagelayer_content":"","footnotes":""},"categories":[16],"tags":[],"class_list":["post-1867","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/1867","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=1867"}],"version-history":[{"count":1,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/1867\/revisions"}],"predecessor-version":[{"id":2036,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/1867\/revisions\/2036"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=1867"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=1867"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=1867"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}} |
![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/RSS-min-1-300x71.jpg\")
<\/p>\n![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/RSS-min-2.jpg\")
<\/p>\n![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/Sample-Meana.jpg\")
<\/p>\n![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/ybar.jpg\")
\u00a0is the sample mean for the y dataset and is given by:<\/p>\n![\"\"](\"https:\/\/www.kindsonthegenius.com\/wp-content\/uploads\/2020\/09\/Sample-meany.jpg\")
<\/p>\n