{"id":192,"date":"2018-01-10T18:58:00","date_gmt":"2018-01-10T17:58:00","guid":{"rendered":"https:\/\/kindsonthegenius.com\/blog\/2018\/01\/10\/introduction-to-support-vector-machine-svm\/"},"modified":"2020-08-22T14:38:04","modified_gmt":"2020-08-22T12:38:04","slug":"introduction-to-support-vector-machine-svm","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/blog\/introduction-to-support-vector-machine-svm\/","title":{"rendered":"Introduction to Support Vector Machine (SVM)"},"content":{"rendered":"

Today we would give a clear and simple explanation of Support Vector Machines. We would discuss the basics of support vector machines in very clear terms.<\/p>\n

Video 1: Introduction to Support Vector Machines Video<\/a>
\nVideo 2: Support Vector Machines Tutorials<\/a><\/p>\n

\u00a0<\/a><\/div>\n

We would cover the following 6 topics in this lesson.<\/p>\n

    \n
  1. What are Support Vector Machines?<\/a><\/li>\n
  2. How does SVMs Work<\/a><\/li>\n
  3. Maximum-Margin Hyperplane <\/a><\/li>\n
  4. What are the advantages of SVM?<\/a><\/li>\n
  5. What are the Cons of SVM?\u00a0<\/a><\/li>\n
  6. Introduction to the Kernel Trick <\/a><\/li>\n
  7. Final Notes<\/a><\/li>\n<\/ol>\n

     <\/p>\n

    You need to note the following points<\/p>\n

      \n
    • Support Vector Machine is a binary classification algorithm that maximises the margin<\/li>\n
    • Support Vectors are points that are closest to each the classes while the maximum margin hyperplane is the hyperplane mid-point between these two. That is why it is called maximum margin hyperplane<\/li>\n
    • The support vectors determine the margin<\/li>\n
    • The distance between the planes is called a margin<\/li>\n
    • The SVM algorithm tries to find the largest margin between the hyperplanes<\/li>\n<\/ul>\n

       <\/p>\n

      1. What is a Support Vector Machine?<\/h3>\n

      A support vector machine(svm) also called support vector networks is a supervised learning method that is used for regression, classification and outlier detection.
      \nGiven a set of training data set, with each observation marked as belonging to either of two classes, as Support Vector Machine develops a model that assigns a new observation to one class or the other.<\/p>\n

       <\/p>\n

      2. How Does Support Vector Machines Work?<\/h3>\n

      SVM separates the two classes of data using a hyperplane?
      \nBut what is a Hyperplane anyway?<\/i><\/span>
      \nHyperplanes are planes in a lower dimensional space than the given input data that separetes the data into classes. If a space is a 3-dimensional space, then the hyperplanes are 2-dimensionas planes. If a space is 2-dimensional, then its hyperplane are 1-dimensional lines.
      \nNow that you understand what hyperplanes are, let’s continue our discussion on how SVM work.<\/p>\n

       <\/p>\n

      3. The Maximum-Margin HyperPlane<\/h3>\n

      Assuming a linearly separable dataset, we can then select two hyperplanes that seperate the two data classes such that the distance between them is as large as possible. The distance between these two hyperplanes is known as the ‘margin’ or decision boundary for each of the classes. This is illustrated in Figure 2 below.<\/p>\n\n\n\n\n
      <\/a><\/td>\n<\/tr>\n
      Figure 2:<\/b> Margin between the two HyperPlanes<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

      The maximum-margin hyperplane is one that lies at the midpoint between these two hyperplanes.<\/p>\n

      Now the two classes could be defined using the linear model given by:<\/p>\n

      y(x) = wT<\/sup>\u03c6(x<\/span><\/b>) + b<\/span><\/i><\/span><\/div>\n

      where \u03c6(x) is a transformation function and
      \nb is the bias<\/p>\n

      For the example above example in Figure 2, the parameters has to\u00a0 be chosen such that for the blue data points, y(x) < 0 and for the red data points y(x) > 0.
      \nSo we can now define the equations for the two classes as<\/p>\n

      For the blue class on the left<\/i><\/p>\n

      wT<\/sup>\u03c6(x<\/span><\/b>) + b = -1<\/span><\/i><\/span><\/div>\n

      For the red class on the right:<\/p>\n

      wT<\/sup>\u03c6(x<\/span><\/b><\/span>) + b = +1<\/span><\/i><\/span><\/div>\n

      This means that each data point must lie on the correct margin
      \nWe can now determine the maximum-margin hyperplane by fitting a line exactly half-way between the two hyperplanes. This is illustrated in Figure 3 below.<\/p>\n

       <\/p>\n\n\n\n\n
      <\/a><\/td>\n<\/tr>\n
      Figure 3:<\/b> Maximum-Margin Hyperplane<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

      <\/ins>
      \n<\/b><\/p>\n

      4. Advantage of SVM<\/h3>\n

       <\/p>\n

        \n
      • SVMs are very efficient in handling data in high-dimensional space<\/li>\n
      • It is memory efficient because it uses a subset of training points in the support vectors<\/li>\n
      • It is very effective when the number of dimensions is greater than the number of observations<\/li>\n<\/ul>\n

         <\/p>\n

        5. Cons of SVM<\/h3>\n

         <\/p>\n

          \n
        • It is not suitable for very large data sets<\/li>\n
        • It is not very efficient for data set have much outliers<\/li>\n
        • It doen’t directly provide an indication of probability estimation<\/li>\n<\/ul>\n

          <\/ins>
          \n<\/b><\/p>\n

          6. Introduction to the Kernel Trick<\/h3>\n

          When it comes to solving complex classification function e.g for very large data sets, then the simple SVM method may not be very effective. A method known as the kernel trick is employed.
          \nA function that takes as input, vectors in the original input space and returns the dot product of the vectors in the feature space is called a Kernel Function or the Kernel Trick. We would not discuss more on the kernel trick here since there is another complete lesson on the Kernel Trick<\/p>\n

           <\/p>\n

          7. Final Notes<\/h3>\n

          We have examined the basics of SVM or Introduction to Support Vector Machines. This lesson is meant to give you an overview of what the concept is. For indepth study of SVM, I would recommend getting a\u00a0 textbook such as ‘Pattern Recognition and Machine Learning<\/i>‘ by Bishop.<\/p>\n

          I would like to thank you for reading. For any observation, you can leave a comment on the form on the left side of the page.<\/p>\n

           <\/p>\n","protected":false},"excerpt":{"rendered":"

          Today we would give a clear and simple explanation of Support Vector Machines. We would discuss the basics of support vector machines in very clear … <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[11,16,395],"tags":[],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/192"}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=192"}],"version-history":[{"count":2,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/192\/revisions"}],"predecessor-version":[{"id":932,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/192\/revisions\/932"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=192"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=192"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=192"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}