{"id":177,"date":"2018-01-21T08:54:00","date_gmt":"2018-01-21T07:54:00","guid":{"rendered":"https:\/\/kindsonthegenius.com\/blog\/2018\/01\/21\/how-to-minimize-misclassifcation-rate-in-classification-machine-learning\/"},"modified":"2020-08-22T09:23:07","modified_gmt":"2020-08-22T07:23:07","slug":"how-to-minimize-misclassifcation-rate-in-classification-machine-learning","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/blog\/how-to-minimize-misclassifcation-rate-in-classification-machine-learning\/","title":{"rendered":"How to Minimize Misclassifcation Rate in Classification (Machine Learning)"},"content":{"rendered":"
\n
Remember that classification is a supervised learning concept that has to do with determining the the class a new input variable belongs.
In trying to assign input variables to classes, there is possibility that we may assign to a wrong class. This is called misclassification.<\/p>\n
<\/a><\/div>\n

By definition, misclassifcation occurs when an input variable is assigned to the wrong class.<\/p>\n

One of the goals of classification is to minimize the number of misclassifications. This is done by defining a rule that assigns input x to one of the available classes.
The approach is to divide the input space into regions Rk<\/sub><\/span><\/i><\/span> called decision regions, one region for each class.
Rk<\/sub><\/span><\/i><\/span> is assigned to class Ck<\/sub><\/i><\/span><\/span><\/p>\n

Consider the case of two classes C1<\/sub> and C2<\/sub>. A mistake occurs when an input vector belonging to R1<\/sub> is assigned to C2<\/sub> or vector x belonging to R2<\/sub> is assigned to C1<\/sub>.
We can represent this as follows:<\/p>\n

<\/div>\n

<\/p>\n

<\/a><\/div>\n
<\/div>\n

To minimize misclassification, we must choose to assign x to which of the classes has the smaller value of the integrand.<\/p>\n

So if p(x, C1<\/sub>)<\/i><\/span><\/span>  is greater than p(x, C2<\/sub>)<\/i><\/span><\/span>, then x would be assigned to C1<\/sub>.<\/p>\n

Using the product rule, we can  determine the posterior probability:<\/p>\n

p(x, Ck<\/sub>) = p(Ck<\/sub> | x)p(x)<\/span><\/i><\/span><\/div>\n

Note that the term p(Ck<\/sub> | x)<\/span><\/i><\/span> is known as the posterior probability and x should be assigned to the class having the largest posterior probability p(Ck<\/sub> | x).<\/span><\/i><\/span><\/p>\n

For the more general case of K classes, it would be a bit easier to maximize the probability of being correct and this is given by:<\/p>\n

<\/a><\/div>\n
<\/div>\n

This means the to minimize misclassification, we need to maximize this probability over the region Rk<\/sub><\/span><\/i><\/span>.
Using the product rule which states that:<\/p>\n

p(<\/span><\/i>x<\/span><\/b>, Ck<\/sub>) = p(Ck<\/sub> | <\/span><\/i>x<\/b><\/span>)p(x)<\/span><\/i><\/span><\/div>\n

We can see that each class has to be assigned to the class that have the highest posterior probability  p(Ck<\/sub> | <\/span><\/i>x<\/b><\/span>)<\/span><\/i><\/span> <\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Remember that classification is a supervised learning concept that has to do with determining the the class a new input variable belongs.In trying to assign … <\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[16],"tags":[],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/177"}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/comments?post=177"}],"version-history":[{"count":1,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/177\/revisions"}],"predecessor-version":[{"id":1442,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/posts\/177\/revisions\/1442"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/media?parent=177"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/categories?post=177"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/blog\/wp-json\/wp\/v2\/tags?post=177"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}