{"id":178,"date":"2018-01-21T06:50:00","date_gmt":"2018-01-21T05:50:00","guid":{"rendered":"https:\/\/kindsonthegenius.com\/blog\/2018\/01\/21\/3-approaches-to-classification-in-machine-learning\/"},"modified":"2020-08-22T09:23:52","modified_gmt":"2020-08-22T07:23:52","slug":"3-approaches-to-classification-in-machine-learning","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/blog\/3-approaches-to-classification-in-machine-learning\/","title":{"rendered":"3 Approaches to Classification in Machine Learning"},"content":{"rendered":"
\n
In this lesson, we would examine 3 approaches to classification. The first 2 would be based on the a priori knowledge of the probabilities. The third one would be based on the formulation of a discriminant function.<\/p>\n
<\/a><\/div>\n
    \n
  1. Determination of the Class-Conditional Densities p(x | Ck<\/sub>)<\/span><\/i><\/span><\/li>\n

    <\/span><\/p>\n

  2. Determination of the a Posteriori Class Probabilities p(Ck<\/sub> | x)<\/span><\/i><\/span><\/li>\n

    <\/span><\/p>\n

  3. Use of a Discriminative Function<\/span> <\/li>\n<\/ol>\n

    Let’s begin with the first one.<\/p>\n

    1. Determination of the Class-Conditional Densities p(x | Ck<\/sub>)<\/i><\/span><\/span><\/b>
     The first step is to solve the inference problem of determining the class-conditional probability densities p(x|Ck<\/sub><\/span><\/span><\/i>) for each class Ck<\/sub><\/i><\/span><\/span> individually. Also separately infer the prior class probabilities p(Ck<\/sub>).<\/span><\/i><\/span>
    Then use Bayes theory of the form<\/p>\n\n\n\n\n
    <\/a><\/td>\n<\/tr>\n
    Bayes Theorem<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

    to find the posterior class probabilities p(Ck<\/sub> | x)<\/span><\/i><\/span>.
    We not that the expression p(x)<\/span><\/i><\/span> in the numerator can be found by the formula:<\/p>\n

    Having found the posterior probability p(Ck<\/sub> | x)<\/span><\/i><\/span>, we can use decision theory to determine the class membership. Such approaches that model the inputs as well as the outputs are called generative models, because they can be used to generate synthetic data points in the input space.<\/p>\n

    2. First Determine the Posterior Class Probabilities, p(Ck<\/sub> | x)<\/span><\/i><\/span><\/b>
    This approach first solves the inference problem of determining the posterior class probabilities p(Ck<\/sub> | x),<\/span><\/i><\/span> and then subsequently use decision theory to assign each new x to one of the available classes. Such approaches that model the posterior probabilities directly are called discriminative models.<\/p>\n

    3. Using a Discriminant Function<\/b>
    The third approach is to find a function f(x)<\/span><\/i><\/span>, called a discriminant function. This function maps each input x<\/i><\/span><\/span> directly to a class label.
    For example, in the case of the two-class problem, the function could have a binary output such that f(x) = 0<\/span><\/span><\/i> represents class C1<\/sub> while f(x) = 1<\/span><\/i><\/span> represents class C2<\/sub>. In the use of discriminant function, probability is not used. The binary output discriminant function is shown:<\/p>\n

    <\/a><\/div>\n

    Final Notes<\/b>
    The first approach is the most demanding  of the three. This is because it involves finding the joint distribution over both x and Ck<\/sub><\/i><\/span><\/span>. But this approach have the advantage of allowing for marginal dencity of data to be determined.
    The simplest approach is the last one as it does not require probability functions.<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

    In this lesson, we would examine 3 approaches to classification. The first 2 would be based on the a priori knowledge of the probabilities. The … <\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[11,16],"tags":[],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/178"}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=178"}],"version-history":[{"count":1,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/178\/revisions"}],"predecessor-version":[{"id":1443,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/178\/revisions\/1443"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=178"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=178"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=178"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}